Pith. sign in

REVIEW 3 major objections 7 minor 50 references

RegisterBridgeMM: A Register-Centric Framework for RGB-Infrared Object Detection

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Frozen ViT register tokens beat dense RGB-IR fusion on four benchmarks.

desk verdict A genuinely new register-centric fusion design with a clean architecture, but the SOTA claim rests on thin cross-paper margins and no variance. read the letter →

arxiv 2608.04833 v1 pith:LVFF36Z3 submitted 2026-08-05 cs.CV

Zian Wang , Hangchuan Liang , Yuehua Chen , Changchun Li , Chaoyi Guo , Mingzhe Liu , Fangming Gu This is my paper · ORCID
classification cs.CV
keywords RGB-infraredobjectdetectionmultimodalfusionregistertokensfrozenvisiontransformerDINOv3parameter-efficientadaptationcross-modalattention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the register tokens of a frozen vision transformer can serve as the main communication channel between RGB and infrared streams, replacing dense patch-to-patch cross-attention. Its diagnostic claim is that on paired RGB-IR inputs, three of four DINOv3 register tokens encode nearly identical information across modalities while the fourth is modality-specific, a 3+1 structure that a register bridge can exploit. Around that observation the paper builds a three-stage lifecycle, Aggregate, Bridge, and Project, using bidirectional register-to-patch reading, a consensus/residual split, and spatially adaptive calibration. With both DINOv3 streams frozen and only 27.8 million trainable parameters, it reports the highest mAP50-95 among the evaluated methods on LLVIP, M3FD, DroneVehicle, and FLIR-Aligned. If those results hold, multimodal detection can be made parameter-efficient without fine-tuning the backbone.

What carries the argument

The central object is the register token, one of four extra tokens in each DINOv3 stream that pretraining shapes into compact global summaries of the image. The paper organizes their use into a three-stage lifecycle: Aggregate keeps the frozen per-modality register summaries from pretraining; Bridge inserts three bidirectional cross-attention modules at layers 2, 5, and 8, where RWPR updates only register tokens by querying the opposite modality's patches at O(kN) cost instead of O($N^{2}$), and RCRS decomposes the result into a consensus vector plus a scaled residual with a learnable gate; Project uses the deepest consensus and residual magnitudes as a summary, combined with a learned inter-modal gap map, to produce spatially adaptive scale-and-shift parameters that calibrate the patch feature pyramid before the RT-DETR detection head. All trainable modules are gated to be near-identity at initialization, preserving the frozen pretrained patch representation.

What would settle it

Re-run RegisterBridgeMM and the strongest baselines from the paper, such as WaveMamba and GM-DETR, on all four datasets under one shared training schedule, evaluation code, and at least three seeds; if any baseline matches or exceeds the reported mAP50-95 within one point, the headline performance claim is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that pretrained DINOv3 register tokens already organize paired RGB and infrared images into a 3+1 structure: three register positions whose embeddings are nearly identical across modalities (cosine similarity above 0.97) and one that is markedly different (about 0.63), and that this structure is a usable substrate for cross-modal fusion. Training the proposed bridge reshapes that pattern: the shared tokens become slightly less collapsed and the divergent token moves closer to consensus, while register-to-patch attention becomes increasingly selective with depth, from about 26 percent entropy ratio at layer 5 to under 1 percent at layer 11. The method routes all cross-modal interaction through four register tokens per stream: RWPR lets registers read opposite-modality patches at O(kN) cost, RCRS explicitly separates consensus from modality-specific residual with a learnable gate, and MAC projects the register summary into per-pixel scale-and-shift calibration of the feature pyramid. On the paper's four benchmark comparisons, this yields the best mAP50-95 in each table (70.5 on LLVIP, 64.9 on M3FD, 61.5 on DroneVehicle, 49.8 on FLIR-Aligned) with frozen backbones and 27.8 million trainable parameters.

Load-bearing premise

The headline comparison takes the mAP numbers of prior detectors from their published papers rather than re-running them under the same training and evaluation protocol, so the reported 0.3 to 1.0 point margins may be within run-to-run variation.

Editorial extensions

If this is right

  • Cross-modal fusion cost drops from O(N^2) patch-to-patch attention to O(kN) register-to-patch attention with four registers, making the approach cheaper at detection-scale resolutions.
  • Both backbone streams can remain frozen, so adapting a paired RGB-IR detector needs only 27.8M trainable parameters, far fewer than the 69M to 287M reported for several compared fusion methods.
  • Because the method improves both mAP50 and mAP50-95 on all four datasets, the gains appear to include localization quality rather than only classification confidence.
  • Bridge placement is load-bearing: injecting at layers 2, 5, and 8 beats adding a bridge at layer 11, since updated registers need subsequent self-attention to influence patch features.
  • The bridge transfers across pretraining versions: swapping DINOv3-B for DINOv2-B costs only 0.7 mAP50-95 on LLVIP, suggesting the register mechanism is not tied to one backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 3+1 register organization is a general property of pretrained vision transformers, the same frozen-backbone register bridge could be applied to other aligned modality pairs, such as RGB-depth or RGB-event data, without redesigning the fusion module.
  • The depth-growing modality-specific residual share reported in the paper suggests a testable hypothesis: register positions may be functionally specialized, and probing which positions stay shared and which diverge could predict where cross-modal bridges help most.
  • Because the headline margins over the strongest baselines are 0.3 to 1.0 mAP50-95 and those baselines were not re-run under a shared protocol, a recipe-controlled re-implementation with multiple seeds is the natural next check; the architecture's efficiency claim would survive even if the exact margins do not.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes RegisterBridgeMM, a fusion framework for RGB-infrared object detection that routes cross-modal communication through pretrained DINOv3 register tokens. With both backbone streams frozen, it introduces three stages: Aggregate (preserving pretrained register summarization), Bridge (RWPR bidirectional register-to-patch reading and RCRS consensus-residual regulation), and Project (MAC spatially adaptive calibration), followed by an RT-DETR head. The paper claims the highest mAP50-95 among the compared methods on LLVIP, M3FD, DroneVehicle, and FLIR-Aligned, with 27.8M trainable parameters, and supports the design with component ablations on LLVIP.

Significance. If the empirical claim holds, the paper demonstrates a compact and parameter-efficient fusion pathway that reuses pretrained register tokens rather than dense patch-level cross-attention, and the three-stage design is clean and internally consistent. The strengths include the monotone per-module gains in the component ablation (Table V), the Patch-Query control that supports registers over sampled patch tokens, and the frozen-versus-unfrozen comparison (Table VI) that supports the value of preserving pretrained registers. The main weakness is that the headline state-of-the-art claim rests on comparisons to published numbers under unshared training and evaluation protocols, with margins of 0.3 to 1.0 mAP50-95 points that are within typical run-to-run noise.

major comments (3)
  1. [IV-C, Tables I-IV] The headline claim that RegisterBridgeMM 'achieves the highest mAP50-95 among the evaluated methods' is not falsifiable from the reported tables, because all baseline numbers are taken from prior papers rather than produced under a common training and evaluation protocol. The margins over the strongest baseline are 0.3 (LLVIP vs GM-DETR), 0.5 (M3FD vs WaveMamba, FLIR-Aligned vs DAMSDet), and 1.0 (DroneVehicle vs WaveMamba), which are within run-to-run and recipe-induced variation for detection training. The paper itself cites [20] on exactly this protocol sensitivity, but does not apply that caution to its own tables. To support the claim, the authors should rerun the strongest baselines under their own protocol, or report multi-seed mean and standard deviation for their method and for the most competitive baselines under identical settings, and adjust the claim accordingly.
  2. [IV-B, IV-D] The implementation details omit the training protocol: epochs, batch size, optimizer, learning-rate schedule, input resolution, data augmentation, number of seeds, and hardware configuration are not specified. Since the ablations in Section IV-D are described as using 'the same training schedule', this undefined schedule makes the numbers in Tables V-VII impossible to reproduce or to compare meaningfully across methods. A complete protocol description and, ideally, code release are needed for the empirical claims to be verifiable.
  3. [IV-D, Tables V-VII] All ablations are reported as single numbers without variance or multiple seeds. The layer-placement conclusion in Table VI, for example, rests on differences such as 69.8 vs 70.5 (a 0.7-point gap) between {2,5,8,11} and {2,5,8}, which is within typical run-to-run noise for detection training. Similarly, the RCRS and MAC design comparisons in Table VII use single-run values. The authors should report mean and standard deviation over at least three seeds for the key ablations, or explicitly state that the differences are preliminary.
minor comments (7)
  1. [Fig. 1(a)] The diagnostic supporting the '3+1' register partition lacks error bars and control conditions; reporting the number of image pairs and adding same-modality or shuffled-pair controls would strengthen the claim that the pattern is specific to RGB-IR correspondence.
  2. [III-D, Eq. (7)] The notation is ambiguous: c⋆ is defined in Eq. (5) as a k×D tensor, but sCR is declared to be in R^{3D}. Clarify that the k-register dimension is averaged in each concatenated term.
  3. [Fig. 2] The label 'SPB' appears in the architecture figure but is never defined in the text; please define it or use a self-explanatory label.
  4. [Tables III and IV] The 'Params' column should state for every listed method whether the number is trainable or total; the current note ('Trainable params') explicitly applies only to RegisterBridgeMM, and several baselines may report total parameters.
  5. [IV-A, FLIR-Aligned] The alignment procedure for FLIR-Aligned is not described; the paper cites a URL ([33]) but should specify exactly how the aligned RGB-IR pairs were generated and which test split was used.
  6. [III-D] The sentence beginning 'ℓ ⋆ denote the deepest injection layer' is missing a verb; it should read 'Let ℓ⋆ denote...'.
  7. [Table V] The Patch-Query control is described in one sentence; specify whether the uniformly sampled patch tokens are fixed across iterations or resampled per forward pass, since this affects the comparison with register queries.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an externally measured benchmark outcome, and the register diagnostics are motivational and confirmatory rather than load-bearing derivations.

full rationale

RegisterBridgeMM's central claim is a measured benchmark outcome (Tables I-IV), not a derivation from first principles, so there is no input-to-output reduction to expose. The architecture is motivated by an empirical diagnostic on pretrained DINOv3 registers (Fig. 1a, blue), and the same diagnostic is re-measured after training (Fig. 1a, orange; Fig. 1b), but those measurements are confirmatory analyses of the trained bridge, not quantities from which the detection results are derived; the benchmark tables and ablations (Tables V-VII) stand independently as external evaluations. No load-bearing self-citation occurs: references to DINOv2, DINOv3, and register tokens [2]-[4] are external prior results, and the protocol-sensitivity reference [20] is cited without being used to compute any result. The possible weakness that SOTA margins of 0.3-1.0 mAP50-95 may be within run-to-run or protocol-induced noise, and that baseline numbers are copied from prior papers without retraining under a shared protocol, is a correctness and robustness risk in the empirical claim, not a circularity. Accordingly, no circular step meets the quoted-equation or self-citation threshold.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on four families of assumptions: that DINOv3 registers function as compact image summaries when frozen (Section III-B), that a 4-token register bottleneck can carry enough cross-modal spatial information for detection (Section III-C), that cosine agreement between corresponding RGB and IR register positions indicates a shared-versus-specific decomposition (Section I, Fig. 1a), and that published baseline numbers are protocol-comparable to the authors' runs (Section IV-C). None of these is independently verified outside the paper's own experiments, and the paper ships no code to check them. The hand-chosen design constants (injection layers {2,5,8}, gate initializations, k=4) are tuned on LLVIP and then reported on all four benchmarks, which is standard practice but means the headline numbers inherit the LLVIP tuning choice. RWPR, RCRS, and MAC are trainable modules with direct empirical handles (Tables V and VII), so they are architectural contributions rather than unfalsifiable postulates; they are not listed as invented entities.

free parameters (6)
  • Bridge injection layers = {2,5,8}
    Hand-chosen placement of the three RWPR+RCRS bridges; Table VI shows {8} alone gives 67.5 mAP50-95, {5,8} gives 69.0, {2,5,8} gives 70.5, and adding layer 11 hurts, so the headline result depends on this ablation-selected choice.
  • Register token count k = 4
    Inherited from the DINOv3 pretrained format; the efficiency argument (6kN versus N^2 pairwise terms) and the 3+1 diagnostic both depend on this fixed small count.
  • RWPR residual gate initialization = alpha = 0.1 (logistic, alpha_max = 1)
    Chosen so the bridge starts near-identity; the paper gives no sensitivity analysis, and the value shapes how much cross-modal signal enters early in training.
  • RCRS balance initialization = beta = 0.5 (learnable)
    Hand-set midpoint; Table VII shows beta=0 (full consensus) costs 0.4 mAP50-95, so the choice of learnable balance matters for the +1.2 point contribution attributed to RCRS.
  • MAC per-scale gates and zero-initialized affine head = alpha_s init 0.1; affine head zero-initialized
    Design choices that make MAC an identity at initialization; the spatially adaptive variant is selected by ablation (+0.9 over channel-only on LLVIP).
  • Multi-scale fusion level gates and lambda_s = learned per-scale sigmoid gates
    The top-down fusion weights in Eq. 11 are trained per benchmark; their contribution to the headline numbers is not ablated separately.
assumptions (5)
  • domain assumption DINOv3 register tokens act as compact global image summaries even when the backbone is frozen
    Section III-B (Aggregate) treats registers as k-slot digests without learning; the paper's only support is its own Fig. 1 diagnostic, which has no error bars or pair count.
  • domain assumption Four register tokens per stream are a sufficient cross-modal bottleneck for detection-grade information
    Section III-C routes all cross-modal exchange through k=4 register queries and no patch-to-patch attention; sufficiency is asserted and only indirectly tested by the component ablation in Table V.
  • domain assumption Cosine similarity between paired RGB and IR register embeddings reveals a modality-shared versus modality-specific content decomposition
    Section I and Fig. 1(a) interpret the 3+1 similarity pattern as shared-versus-specific structure; cosine agreement measures embedding alignment, not an identified partition of content, so the interpretation is plausible but unproven.
  • domain assumption The mAP numbers cited for prior methods are comparable to the authors' runs under a shared evaluation protocol
    Section IV-C builds the SOTA claim on published numbers; the paper cites [20] to note that training recipes and protocols shift multispectral comparisons but does not re-run any baseline or report variance.
  • domain assumption RT-DETR as the detection head with the three-level fused pyramid yields no material interaction with the fusion design
    Section III-A fixes the head from [29] for all variants; the ablations attribute gains to the bridge modules, implicitly assuming the head is a neutral consumer of both baseline and fused features.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RegisterBridgeMM: A Register-Centric Framework for RGB-Infrared Object Detection." pith.science (2026). https://pith.science/paper/LVFF36Z3

@misc{pith2026260804833,
  author       = {Pith},
  title        = {Pith review of: RegisterBridgeMM: A Register-Centric Framework for RGB-Infrared Object Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LVFF36Z3}},
  note         = {Machine review of arXiv:2608.04833}
}
read the original abstract

RGB-infrared (RGB-IR) object detection benefits from complementary visible and thermal cues, but effective fusion remains challenging under illumination changes, weather variation, and cluttered scenes. Existing RGB-IR fusion methods often trade expressive patch-level interaction for lighter but more constrained adaptation mechanisms. We empirically observe that pretrained register tokens contain both modality-shared and modality-specific information on paired RGB-IR inputs, suggesting that they can serve as a compact substrate for cross-modal communication. Building on this observation, we propose RegisterBridgeMM, a register-mediated fusion framework organized as a three-stage register lifecycle. Aggregate preserves per-modality register summarization inherited from pretraining; Bridge performs bidirectional register-to-patch reading with consensus-residual regulation; and Project translates the resulting register summary into spatially adaptive calibration of patch features. This register pathway avoids dense patch-to-patch cross-modal interaction while preserving the pretrained patch representation. With both backbone streams frozen, RegisterBridgeMM achieves the highest mAP50-95 among the evaluated methods on all four benchmarks: LLVIP, M3FD, DroneVehicle, and FLIR-Aligned.

Figures

Figures reproduced from arXiv: 2608.04833 by the authors.

Figure 1
Figure 1. A pretrained register diagnostic and the learned bridge behavior of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of RegisterBridgeMM. The three-stage register lifecycle: [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Modality-specific residual energy share of bridged registers on LLVIP [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative detection results on representative RGB–IR pairs. Rows show RGB input, IR input, the frozen simple-concatenation baseline, and [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Attention comparison between the frozen baseline and Register [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 41 canonical work pages

  1. [20]

    Optimizing Multispectral Object Detection: A Bag of Tricks and Comprehensive Benchmarks

    C. Zhou, P. Cheng, J. Fang, Y . Zhang, Y . Yan, X. Jia, Y . Xu, K. Wang, and X. Cao, “Optimizing multispectral object detection: A bag of tricks and comprehensive benchmarks,”arXiv preprint arXiv:2411.18288, 2024

    work page Pith review arXiv 2024

  2. [1]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov,et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” inProc. Int. Conf. Learn. Represent. (ICLR), 2021

    work page 2021

  3. [2]

    DINOv2: Learning robust visual features without supervi- sion,

    M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khali- dov,et al., “DINOv2: Learning robust visual features without supervi- sion,”Trans. Mach. Learn. Res., 2024

    work page 2024

  4. [3]

    Sim ´eoni, H

    O. Sim ´eoni, H. V . V o, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, et al., “DINOv3,”arXiv preprint arXiv:2508.10104, 2025

    arXiv 2025

  5. [4]

    Vision Transformers need registers,

    T. Darcet, M. Oquab, J. Mairal,et al., “Vision Transformers need registers,” inProc. Int. Conf. Learn. Represent. (ICLR), 2024

    work page 2024

  6. [5]

    Cross-modality fusion transformer for multispectral object detection,

    Q. Fang, D. Han, and Z. Wang, “Cross-modality fusion transformer for multispectral object detection,”arXiv preprint arXiv:2111.00273, 2021

    arXiv 2021

  7. [6]

    Confidence-aware fusion using Dempster-Shafer theory for multispectral pedestrian de- 10 tection,

    Q. Li, C. Zhang, Q. Hu, H. Fu, and P. Zhu, “Confidence-aware fusion using Dempster-Shafer theory for multispectral pedestrian de- 10 tection,”IEEE Trans. Multimedia, vol. 25, pp. 3420–3431, 2023, doi: 10.1109/TMM.2022.3160589

    arXiv 2023

  8. [7]

    Multiscale cross-modal homogeneity enhancement and confidence-aware fusion for multispectral pedestrian detection,

    R. Li, J. Xiang, F. Sun, Y . Yuan, L. Yuan, and S. Gou, “Multiscale cross-modal homogeneity enhancement and confidence-aware fusion for multispectral pedestrian detection,”IEEE Trans. Multimedia, vol. 26, pp. 852–863, 2024, doi: 10.1109/TMM.2023.3272471

    work page arXiv 2024

Show all 50 references
  1. [8]

    Improving multispectral pedestrian detection by addressing modality imbalance problems,

    K. Zhou, L. Chen, and X. Cao, “Improving multispectral pedestrian detection by addressing modality imbalance problems,” inProc. Eur. Conf. Comput. Vis. (ECCV), 2020

    2020

  2. [9]

    ICAFusion: Iterative cross-attention guided feature fusion for multispectral object detection,

    J. Shen, Y . Chen, Y . Liu, X. Zuo, H. Fan, and W. Yang, “ICAFusion: Iterative cross-attention guided feature fusion for multispectral object detection,”Pattern Recognition, vol. 145, p. 109913, 2024

    2024

  3. [10]

    IRDFusion: Iterative relation-map difference guided feature fusion for multispectral object detection,

    J. Shen, H. Zhan, X. Zuo, H. Fan, X. Yuan, J. Li, and W. Yang, “IRDFusion: Iterative relation-map difference guided feature fusion for multispectral object detection,”Pattern Recognition, vol. 176, p. 113189, 2026

    2026

  4. [11]

    TFDet: Target-aware fusion for RGB-T pedestrian detection,

    X. Zhang, X. Zhang, J. Wang, J. Ying, Z. Sheng, H. Yu, C. Li, and H.- L. Shen, “TFDet: Target-aware fusion for RGB-T pedestrian detection,” IEEE Trans. Neural Netw. Learn. Syst., vol. 36, no. 7, pp. 13276–13290, 2025

    2025

  5. [12]

    YOLO-Adaptor: A fast adaptive one- stage detector for non-aligned visible–infrared object detection,

    H. Fu, H. Liu, J. Yuan,et al., “YOLO-Adaptor: A fast adaptive one- stage detector for non-aligned visible–infrared object detection,”IEEE Trans. Intell. Veh., 2024, pp. 1–14

    2024

  6. [13]

    Cross-modality proposal-guided feature mining for unregistered RGB-thermal pedestrian detection,

    C. Tian, Z. Zhou, Y . Huang, G. Li, and Z. He, “Cross-modality proposal-guided feature mining for unregistered RGB-thermal pedestrian detection,”IEEE Trans. Multimedia, vol. 26, pp. 6449–6461, 2024, doi: 10.1109/TMM.2024.3350926

    2024

  7. [14]

    COFNet: Contrastive object-aware fusion using box-level masks for multispectral object detection,

    M. Zhou, Y . Li, G. Yang, X. Wei, H.-Y . Pu, J. Luo, and W. Jia, “COFNet: Contrastive object-aware fusion using box-level masks for multispectral object detection,”IEEE Trans. Multimedia, vol. 27, pp. 7444–7458, 2025, doi: 10.1109/TMM.2025.3599097

    2025

  8. [15]

    GM-DETR: Generalized multispectral DEtection TRansformer with efficient fusion encoder for visible–infrared detection,

    Y . Xiao, F. Meng, Q. Wu, L. Xu, M. He, and H. Li, “GM-DETR: Generalized multispectral DEtection TRansformer with efficient fusion encoder for visible–infrared detection,” inProc. IEEE/CVF Conf. Com- put. Vis. Pattern Recognit. Workshops (CVPRW), 2024, pp. 5541–5549

    2024

  9. [16]

    C 2Former: Calibrated and complementary transformer for RGB–infrared object detection,

    M. Yuan and X. Wei, “C 2Former: Calibrated and complementary transformer for RGB–infrared object detection,”IEEE Trans. Geosci. Remote Sens., vol. 62, pp. 1–12, 2024

    2024

  10. [17]

    Multimodal token fusion for vision transformers,

    Y . Wang, X. Chen, L. Cao, W. Huang, F. Sun, and Y . Wang, “Multimodal token fusion for vision transformers,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 12186–12195

    2022

  11. [18]

    Fusion-Mamba for cross-modality object detection,

    W. Dong, H. Zhu, S. Lin, X. Luo, Y . Shen, X. Liu, J. Zhang, G. Guo, and B. Zhang, “Fusion-Mamba for cross-modality object detection,”IEEE Trans. Multimedia, vol. 27, pp. 7392–7406, 2025

    2025

  12. [19]

    WaveMamba: Wavelet-driven Mamba fusion for RGB–infrared object detection,

    H. Zhu, W. Dong, L. Yang, H. Li, Y . Yang, Y . Ren, Q. Zhu, Z. Feng, C. Li, S. Lin,et al., “WaveMamba: Wavelet-driven Mamba fusion for RGB–infrared object detection,” inProc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2025, pp. 11219–11229

    2025

  13. [21]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” inProc. Int. Conf. Learn. Represent. (ICLR), 2022

    2022

  14. [22]

    FiLM: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. de Vries, V . Dumoulin, and A. Courville, “FiLM: Visual reasoning with a general conditioning layer,” inProc. AAAI Conf. Artif. Intell. (AAAI), 2018

    2018

  15. [23]

    Vision transformer adapter for dense predictions,

    Z. Chen, Y . Duan, W. Wang, J. He, T. Lu, J. Dai, and Y . Qiao, “Vision transformer adapter for dense predictions,” inProc. Int. Conf. Learn. Represent. (ICLR), 2023

    2023

  16. [24]

    UniRGB-IR: A unified framework for visible–infrared semantic tasks via adapter tuning,

    M. Yuan, B. Cui, T. Zhao, J. Wang, S. Fu, X. Yang, and X. Wei, “UniRGB-IR: A unified framework for visible–infrared semantic tasks via adapter tuning,” inProc. ACM Int. Conf. Multimedia (MM), 2025, pp. 2409–2418

    2025

  17. [25]

    SLGNet: Synergizing structural priors and language-guided modulation for multimodal object detection,

    X. Xiang, G. Zhou, Z. Wen, W. Li, B. Niu, F. Wang, L. Huang, Q. Wang, Y . Liu, Z. Pan, and Y . Hu, “SLGNet: Synergizing structural priors and language-guided modulation for multimodal object detection,”arXiv preprint arXiv:2601.02249, 2026

    2026

  18. [26]

    Leveraging registers in vision transformers for robust adaptation,

    S. Yellapragada, K. Thopalli, V . Narayanaswamy, W. Sakla, Y . Liu, Y . Mubarka, D. Samaras, and J. J. Thiagarajan, “Leveraging registers in vision transformers for robust adaptation,” inProc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), 2025

    2025

  19. [27]

    Vision transformers do not need trained registers,

    N. Jiang, A. Dravid, A. Efros, and Y . Gandelsman, “Vision transformers do not need trained registers,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2025

    2025

  20. [28]

    Efficient vision- language models by summarizing visual tokens into compact registers,

    Y . Wen, Q. Cao, Q. Fu, S. Mehta, and M. Najibi, “Efficient vision- language models by summarizing visual tokens into compact registers,” arXiv preprint arXiv:2410.14072, 2024

    2024 arXiv

  21. [29]

    DETRs beat YOLOs on real-time object detection,

    Y . Zhao, W. Lv, S. Xu, J. Wei, G. Wang, Q. Dang, Y . Liu, and J. Chen, “DETRs beat YOLOs on real-time object detection,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 16965–16974

    2024

  22. [30]

    LLVIP: A visible–infrared paired dataset for low-light vision,

    X. Jia, C. Zhu, M. Li,et al., “LLVIP: A visible–infrared paired dataset for low-light vision,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2021

    2021

  23. [31]

    Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection,

    J. Liu, X. Fan, Z. Huang,et al., “Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 5802–5811

    2022

  24. [32]

    Drone-based RGB-infrared cross- modality vehicle detection via uncertainty-aware learning,

    Y . Sun, B. Cao, P. Zhu, and Q. Hu, “Drone-based RGB-infrared cross- modality vehicle detection via uncertainty-aware learning,”IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 10, pp. 6700–6713, 2022

    2022

  25. [33]

    Free FLIR thermal dataset for algorithm train- ing,

    Teledyne FLIR, “Free FLIR thermal dataset for algorithm train- ing,” 2021. [Online]. Available: https://oem.flir.com/en-gb/solutions/ automotive/adas-dataset-form/

    2021

  26. [34]

    Guided attentive feature fusion for multispectral pedestrian detection,

    H. Zhang, E. Fromont, S. Lef `evre, and B. Avignon, “Guided attentive feature fusion for multispectral pedestrian detection,” inProc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), 2021, pp. 72–80

    2021

  27. [35]

    DAMSDet: Dynamic adaptive multispectral detection transformer with competitive query selection and adaptive feature fusion,

    J. Guo, C. Gao, F. Liu, D. Meng, and X. Gao, “DAMSDet: Dynamic adaptive multispectral detection transformer with competitive query selection and adaptive feature fusion,” inProc. Eur. Conf. Comput. Vis. (ECCV), 2024, pp. 464–481

    2024

  28. [36]

    CDDFuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,

    Z. Zhao, H. Bai, J. Zhang, Y . Zhang, S. Xu, Z. Lin, R. Timofte, and L. Van Gool, “CDDFuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 5906–5916

    2023

  29. [37]

    Learning a graph neural network with cross modality interaction for image fusion,

    J. Li, J. Chen, J. Liu, and H. Ma, “Learning a graph neural network with cross modality interaction for image fusion,” inProc. ACM Int. Conf. Multimedia (MM), 2023, pp. 4471–4479

    2023

  30. [38]

    Equivariant multi-modality image fusion,

    Z. Zhao, H. Bai, J. Zhang, Y . Zhang, K. Zhang, S. Xu, D. Chen, R. Timofte, and L. Van Gool, “Equivariant multi-modality image fusion,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 25912–25921

    2024

  31. [39]

    SuperFusion: A ver- satile image registration and fusion network with semantic awareness,

    L. Tang, Y . Deng, Y . Ma, J. Huang, and J. Ma, “SuperFusion: A ver- satile image registration and fusion network with semantic awareness,” IEEE/CAA J. Automatica Sinica, vol. 9, no. 12, pp. 2121–2137, 2022

    2022

  32. [40]

    KCDNet: Multimodal object detection in modal information imbalance scenes,

    H. Wang, S. Qu, Z. Qiao, and X. Liu, “KCDNet: Multimodal object detection in modal information imbalance scenes,”IEEE Trans. Instrum. Meas., vol. 73, pp. 1–13, 2024

    2024

  33. [41]

    Multi-modal feature pyramid transformer for RGB-infrared object detection,

    Y . Zhu, X. Sun, M. Wang, and H. Huang, “Multi-modal feature pyramid transformer for RGB-infrared object detection,”IEEE Trans. Intell. Transp. Syst., vol. 24, no. 9, pp. 9984–9995, 2023

    2023

  34. [42]

    LRAF- Net: Long-range attention fusion network for visible–infrared object detection,

    H. Fu, S. Wang, P. Duan, C. Xiao, R. Dian, S. Li, and Z. Li, “LRAF- Net: Long-range attention fusion network for visible–infrared object detection,”IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 10, pp. 13232–13245, 2023

    2023

  35. [43]

    SLBAF- Net: Super-lightweight bimodal adaptive fusion network for UA V detec- tion in low recognition environment,

    X. Cheng, K. Geng, Z. Wang, J. Wang, Y . Sun, and P. Ding, “SLBAF- Net: Super-lightweight bimodal adaptive fusion network for UA V detec- tion in low recognition environment,”Multimedia Tools Appl., vol. 82, no. 30, pp. 47773–47792, 2023

    2023

  36. [44]

    Translation, scale and rotation: Cross- modal alignment meets RGB-infrared vehicle detection,

    M. Yuan, Y . Wang, and X. Wei, “Translation, scale and rotation: Cross- modal alignment meets RGB-infrared vehicle detection,” inProc. Eur. Conf. Comput. Vis. (ECCV), 2022, pp. 509–525

    2022

  37. [45]

    CrossFormer: Cross-guided attention for multi-modal object detection,

    S. Lee, J. Park, and J. Park, “CrossFormer: Cross-guided attention for multi-modal object detection,”Pattern Recognition Lett., vol. 179, pp. 144–150, 2024

    2024

  38. [46]

    Lightweight modal-guided cross-attention fusion network for visible-infrared object detection,

    W. Wu, H. Zhang, X. Zhang, H. Yin, and Y . Zhang, “Lightweight modal-guided cross-attention fusion network for visible-infrared object detection,”Pattern Recognition, p. 113350, 2026

    2026

  39. [47]

    GLFNet: Global and local feature fusion network for multi-modal object detection,

    Y . Sun, H. Zhang, and Q. Wang, “GLFNet: Global and local feature fusion network for multi-modal object detection,”IEEE Geosci. Remote Sens. Lett., vol. 21, pp. 1–5, 2024

    2024

  40. [48]

    DMM: Disparity-guided multispectral mamba for oriented object detection in remote sensing,

    J. Zhou, Y . Liu, and L. Zhang, “DMM: Disparity-guided multispectral mamba for oriented object detection in remote sensing,”arXiv preprint arXiv:2407.08132, 2024

    2024 arXiv

  41. [49]

    IV-YOLO: A lightweight dual-modal object detector for infrared and visible images,

    M. Li, J. Chen, and W. Sun, “IV-YOLO: A lightweight dual-modal object detector for infrared and visible images,”Sensors, vol. 24, no. 19, p. 6181, 2024

    2024

  42. [50]

    Cross-modality selective optimization and modality-aware attention fusion for multispectral object detection,

    H. Wang, Y . Xu, and Z. Li, “Cross-modality selective optimization and modality-aware attention fusion for multispectral object detection,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) Workshops, 2024

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.