Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance

Chao Tian; Chao Yang; Guoqing Zhu; Qiang Wang; Zhenyu He

REVIEW 3 major objections 5 minor 41 references

Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A base-and-auxiliary training framework makes RGB-thermal detectors robust to extreme modality imbalance, cutting the KAIST log-average miss rate from 70.72 to 31.35 for RGB-only input (about 55 percent relative).

desk verdict A solid robustness-engineering paper for RGB-T detection whose 55% claim is the best-case cell, and whose training degradation doesn't actually sample the complete modality absence used in testing. read the letter →

arxiv 2505.22154 v1 pith:2RXLYRTD submitted 2025-05-28 cs.CV

Chao Tian , Chao Yang , Guoqing Zhu , Qiang Wang , Zhenyu He This is my paper

classification cs.CV

keywords RGB-Thermalobjectdetectionmodalityimbalancedegradationbase-and-auxiliarytrainingexponentialmovingaveragequality-awareinteractionpseudo-degradationKAISTbenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that an RGB-thermal object detector can be trained to keep working when one camera fails or its image degrades, rather than treating balanced two-modality data as the only viable regime. The proposed base-and-auxiliary scheme makes a teacher detector, updated by exponential moving average, enforce that a student detector give consistent output logits on clean and pseudo-degraded input pairs; a quality-aware modality interaction module reweights each modality before fusion. The paper reports that this lowers the log-average miss rate on KAIST from 70.72 to 31.35 for RGB-only input (about 55 percent relative) and from 98.3 to 60.64 for VFNet, and that the same framework also lifts SSD and handles Gaussian noise at test time. A sympathetic reader would care because real camera arrays lose or corrupt one sensor in the field, and the claim is that training-time synthetic corruption plus a consistency constraint transfers to test-time complete absence.

What carries the argument

The load-bearing mechanism is the base-and-auxiliary consistency constraint applied to output logits, implemented as $\min_F \|F_{\text{ema}}(x) - F(\hat{x})\|^2$, where $F_{\text{ema}}$ is the exponential-moving-average teacher and $\hat{x}$ is the pseudo-degraded pair. Around this sits a quality-aware modality interaction module: sigmoid attention masks $M_{\text{rgb}}$, $M_{\text{tir}}$ predicted from each stream's features reweight the complementary stream before multi-stage fusion, so a degraded modality is downweighted where it is uninformative. Pseudo-degradation models a failing modality as a global affine change $\hat{x}_m = b + c \cdot x_m$, with contrast multiplier $c \sim U(0, 0.7)$ and bias $b \sim N(127.45, 2440)$, applied to one modality with probability 0.3.

What would settle it

Train with the same framework but at test time replace the missing modality with the dataset's mean pixel value or with random noise rather than a black image; if LAMR jumps back toward the supervised baseline, the detector has latched onto the zero-input pattern rather than modality-invariant features. Alternatively, test on misaligned or saturated image pairs, which are not contrast-and-bias-like, and check whether the 31.35 RGB-only LAMR holds.

Watch

Extended reading notes

Core claim

The central claim, on the paper's own terms, is that invariance to modality degradation can be learned by direct constraint on the detector's output logits rather than by heavy augmentation that disturbs convergence. The authors pair two unshared networks: an auxiliary detector supervised with annotations and pseudo-degraded samples, and a base detector that is an exponential-moving-average of the auxiliary's previous states. An L2 consistency loss on the logits of the two networks pulls the degraded response toward the clean response, so the EMA teacher accumulates knowledge of both balanced and imbalanced inputs and is the model used at test time. With the per-location quality masks of the interaction module, this yields KAIST LAMR of 31.35 for RGB-only and 16.63 for TIR-only input, compared with 70.72 and 98.3 for the supervised baseline at zero contrast, and the paper shows the framework also improves VFNet and SSD detectors. The authors conclude that the smoother parameter updates of the EMA teacher explain why the base detector outperforms the auxiliary detector, which itself already beats the vanilla supervised base.

Load-bearing premise

The pseudo-degradation used in training only lowers contrast and shifts brightness of one modality, while the test removes an entire modality, so the whole framework rests on the assumption that learning from that mild corruption transfers to complete sensor absence.

Editorial extensions

If this is right

RGB-only and TIR-only inputs become usable in practice: on KAIST, LAMR falls from 70.72 to 31.35 with the RGB stream alone and to 16.63 with the thermal stream alone.
The training framework is not tied to one detector: VFNet's zero-contrast LAMR drops from 98.3 to 60.64, and SSD's from 100 to 66.77, so the scheme transfers.
Robustness extends beyond contrast: the same training keeps LAMR near the clean level when high-intensity Gaussian noise is added at test time, and adding Gaussian-noise degradation in training improves this further.
The EMA-updated base detector is the usable artifact: it beats the auxiliary detector on all three modality settings, so the final model keeps balanced-data competence while being robust to imbalance.
The interaction module alone hurts imbalanced performance because it exacerbates the out-of-distribution shift; the improvement comes from coupling it with pseudo-degradation and the consistency constraint.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper tests whole-modality removal at test time, but the introduction motivates local imbalance from field-of-view mismatch; a natural extension is training with region-wise pseudo-degradation and testing on partially overlapping images.
If the contrast-and-bias corruption transfers to true absence, the framework should also transfer to other failure modes of a similar family (e.g., lens fog, gain drop), which can be tested directly by applying each failure model at test time.
Because the framework improved RGB-only performance more than TIR-only performance on KAIST, one inference is that the interaction module learned to discount the degraded stream asymmetrically; measuring the learned masks could reveal whether the detector genuinely exploits both modalities or falls back to the stronger one.
If the EMA consistency argument is right, the replacement rate (set to $10^{-3}$ here) should interact with annotation noise: on a cleaner dataset the optimal rate might shift, a testable prediction the paper does not run.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look. The core idea—an EMA-updated base detector plus a quality-aware fusion module plus pseudo-degradation during training—is a reasonable, well-executed combination for handling modality imbalance in RGB-T detection. The gains are consistent across detectors and datasets: they show large LAMR improvements over strong baselines on KAIST when one modality is removed, and similar patterns on FLIR. That is real empirical value, and the method is simple enough to be broadly applicable.

But there are two soft spots that need attention. First, the headline 55% missing-rate reduction is cherry-picked. It comes from the single best cell in Table 6 (w/o TIR, contrast 0, All), not from an aggregate over imbalance conditions. The abstract says 'decreasing the Missing Rate by 55%' without that context; it should say 'up to 55%' and be clear it is a best-case result.

Second, the pseudo-degradation does not match the test-time condition. Eq. 6-7 use a contrast multiplier and a Gaussian bias with mean 127.45 and variance 2440. The c=0 limit is a constant mid-gray image, not a zeroed or absent modality. When you test by removing RGB or TIR entirely, the input is off the training manifold. The authors assume the interaction module learns to downweight empty features, but nothing in the training distribution forces that. The reported gains might come from the EMA/consistency framework rather than from pseudo-degradation transferring to complete absence. This is a load-bearing gap, but it is not fatal: the framework still works better than baselines empirically, and the transfer may be happening through the EMA teacher. Still, the paper needs to either train on actual zeroed modalities or explicitly justify the transfer.

The ablation table also has a prose-to-row mismatch. The text references states that don't cleanly map to the rows, and the interaction module alone (row 2) appears to worsen balanced R+T performance versus the no-Int baseline (row 1), which is never explained. Add the absence of error bars and no code/data release, and the paper has several correctable reporting issues.

Overall, this is a legitimate engineering contribution, not a break through. It deserves a serious referee and a careful revision. I would accept after the authors clarify the test protocol, fix the ablation table, and temper the headline claim.

Referee Report

3 major / 5 minor

Summary. The paper proposes a base-and-auxiliary training framework for RGB-thermal object detection under extreme modality imbalance. Two unshared detectors are used: a base detector updated by exponential moving average (EMA) from an auxiliary detector, and an auxiliary detector supervised by both a detection loss and a consistency loss with the base. A quality-aware modality interaction module reweights each modality's features before fusion. To simulate real-world degradation, the authors apply a pseudo-degradation transform b + c*x_m to one modality, with contrast multiplier c~U(0,0.7) and bias b~N(127.45,2440). Experiments on KAIST and FLIR show large improvements under modality removal and contrast degradation, including a 55.7% relative LAMR reduction on the best condition, and the framework also improves VFNet and SSD reimplemented as two-stream detectors.

Significance. If the central claim holds, the paper addresses a practical gap: most RGB-T detectors assume both modalities at test time. The proposed framework is simple, architecture-agnostic, and shows consistent gains across two datasets and several detector designs. The paper ships extensive experiments, including ablations, noise-robustness tests, feature visualizations, and an extended application to other detectors; these are strengths. The quality-aware interaction module is a reasonable mechanism. However, the load-bearing assumption that contrast/bias pseudo-degradation transfers to complete modality absence is not verified, and the headline 55% number is a single best cell, so the significance as stated is currently overstated.

major comments (3)

[Sec. 3.3 Eq. (6)-(7), Tables 1 and 6] The central robustness claim is tested under complete modality removal (the 'w/o RGB' and 'w/o TIR' columns in Table 1 and Table 6), but the training-time pseudo-degradation in Eq. (6) is b + c*x_m with c~U(0,0.7) and b~N(127.45,2440). For c=0 the degraded modality is a constant image equal to b, not a missing/zero input, and with the chosen b distribution near-black constants are essentially never sampled. In addition, the Table 6 test protocol ('contrast decreased by 0.5 and 0 times') does not specify whether the bias term is applied at test time; if c=0 is applied without bias, the test inputs are black images that were never seen in training. The interaction module in Eq. (5) therefore learns to downweight low-contrast, off-center images, while the test inputs lie off that manifold. The paper does not establish that pseudo-degradation transfers to complete modality absence; the observed gains could be due to the EMA/consistency framework or to degenerate responses to zero inputs rather than to the pseudo-degradation. Please add experiments that train with the actual test corruption (zeroing the modality, or at least c=0 with b=0) or otherwise justify the transfer.
[Sec. 4.4, Table 3] The ablation table and the prose are not consistent, which makes it hard to verify the component analysis. The rows of Table 3 are: baseline (no modules), interaction module only, pseudo-degradation only, interaction+pseudo-degradation, interaction+pseudo-degradation+auxiliary (the last row has all three). The text in 'Effect of pseudo degradation' refers to the 'second state' as the vanilla data augmentation, but the second row is the interaction-module-only row; the pseudo-degradation-only row shows R+T LAMR improving (16.93 vs 19.41 day) while R and T worsen (61.08 vs 53.54 and 37.68 vs 29.79 day), which contradicts the claim that vanilla augmentation improves imbalanced performance. Similarly, 'Effect of interaction modules' calls the 'third state' the base detector, but the third row is the pseudo-degradation-only row. Please renumber the rows and align each prose statement with the intended row, and re-evaluate the conclusions that depend on these comparisons.
[Abstract; Sec. 4.5, Table 6] The abstract's claim of 'decreasing the Missing Rate by 55%' is not representative of the results as a whole. In Table 6, the 55.7% relative improvement (70.72 to 31.35 LAMR) is the single best cell, corresponding to the base detector with the TIR modality removed at contrast 0. Other conditions show smaller relative improvements (e.g., w/o RGB at contrast 0: 28.96 to 16.63, about 42.6%; VFNet w/o TIR at contrast 0: 98.3 to 60.64, about 38.3%; SSD w/o TIR at contrast 0: 100 to 66.77, about 33%). The headline overstates the breadth of the evidence. Please report the average relative improvement over all imbalance conditions, or qualify the claim as applying to the most extreme single condition.

minor comments (5)

[Sec. 4.4, first paragraph] The statement that 'all models are trained with balanced dataset (i.e., the original training set)' is inaccurate for the pseudo-degradation rows, which deliberately train on degraded samples; please rephrase to clarify that the original paired data are used with or without pseudo-degradation.
[Sec. 3.1] The base detector is described as 'an unlearned network' despite being updated by EMA from the auxiliary detector; 'not directly optimized' would be more precise.
[Table 6 caption] The caption says 'The red and green records show the best improvement by 55%', but the submitted text does not display colors; please use explicit markers such as bold or asterisks to indicate the best improvement.
[Sec. 4.5] The abstract's phrase 'improving performance across various baseline detectors' is supported only by VFNet and SSD reimplemented as two-stream networks; please state in the abstract or conclusion that these are modified two-stream versions rather than the original released detectors.
[Sec. 4.3, Table 1] The percentages reported for CPFM ('605% increase' and '169% increase') are relative increases in LAMR; please clarify that they are relative rather than absolute changes to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 55% improvement is an empirical test-cell difference, the pseudo-degradation hyperparameters are hand-set rather than fitted, and the only self-citation (CPFM) is a comparison baseline, not a load-bearing premise.

full rationale

I walked the derivation chain from Eq. (1)-(8) and the experiment tables. The core claim is not a derivation: the network is trained with a detection loss plus a consistency loss (Eq. 8), and the reported robustness numbers (e.g., LAMR 31.35 in Table 1, and the 55% relative improvement in Table 6 from 70.72 to 31.35 in the w/o TIR contrast-0 column) are measured on the KAIST/FLIR test sets, not computed from the training inputs or from the hyperparameters. The pseudo-degradation of Eq. (6)-(7) is an augmentation rule with hand-set hyper-parameters (u=0.7, mu=127.45, sigma^2=2440, probability 0.3); it is not fitted to minimize the test metric, so none of the reported numbers is forced by construction. The consistency constraint of Eq. (4) is a training regularizer, not a prediction of the test-time result. The main scientific weakness, namely that training on contrast/bias corruption may not transfer to complete modality absence, is a real generalization assumption, and the paper itself acknowledges 'room for further improvement regarding robustness against modality information loss' (Sec. 5), but an unverified transfer assumption is a correctness/robustness concern, not circularity. The only self-citation that appears in a comparison role is CPFM [8], and although it overlaps with the present authors, it is used as a baseline to be outperformed, not as evidence for the method's validity. I find no step where an input is defined in terms of the output, no fitted parameter is relabeled as a prediction, and no load-bearing argument rests on a self-citation or imported uniqueness theorem.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the assumption that synthetic contrast/bias degradation simulates real modality imbalance, that the EMA teacher is a reliable target, and on several hand-selected hyperparameters. No invented entities are introduced.

free parameters (6)

degradation probability p = 0.3
Fraction of training samples subjected to pseudo-degradation; set by hand in Sec 4.2.
contrast multiplier upper bound u = 0.7
Upper bound of uniform distribution for contrast scaling c in Eq. 7; set by hand to control degradation strength.
bias Gaussian mean mu = 127.45
Mean of bias distribution for pseudo-degradation, chosen to map [-2.5,2.5] of standard normal to [0,255].
bias Gaussian variance sigma^2 = 2440
Variance of the bias Gaussian, ensuring 99.5% of biases fall in [0,255].
EMA replacement rate alpha = 0.001
Momentum for base detector update (Eq. 3); searched over [1e-4,1e-1] and set to 1e-3 in Sec 4.4.
backbone depth/width factors = 0.67 / 0.75
Scale factors for CSPDarknet53 backbone, chosen as YOLOv5-M medium level in Sec 4.2.

assumptions (3)

domain assumption Contrast decay and brightness bias are an adequate proxy for real-world modality degradation
Sec. 3.3 defines degradation only as Eq. 6 (linear contrast and bias). No evidence that this captures sensor failures, misalignment, or information loss that occur in practice.
domain assumption The EMA-updated base detector provides a stable and reliable target for consistency loss
Sec. 3.1 relies on [36] (YOLOX) for the claim that EMA-updated models have higher robustness; the base detector is treated as a teacher without validation of its own accuracy during training.
ad hoc to paper Pseudo-degradation does not disturb convergence because of the EMA teacher
Sec. 4.4 and Figure 5 show vanilla augmentation impairs convergence, while the proposed framework recovers it; this is an empirical result, not a derived property.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance." pith.science (2026). https://pith.science/paper/2RXLYRTD

@misc{pith2026250522154,
  author       = {Pith},
  title        = {Pith review of: Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2RXLYRTD}},
  note         = {Machine review of arXiv:2505.22154}
}

read the original abstract

RGB-Thermal (RGB-T) object detection utilizes thermal infrared (TIR) images to complement RGB data, improving robustness in challenging conditions. Traditional RGB-T detectors assume balanced training data, where both modalities contribute equally. However, in real-world scenarios, modality degradation-due to environmental factors or technical issues-can lead to extreme modality imbalance, causing out-of-distribution (OOD) issues during testing and disrupting model convergence during training. This paper addresses these challenges by proposing a novel base-and-auxiliary detector architecture. We introduce a modality interaction module to adaptively weigh modalities based on their quality and handle imbalanced samples effectively. Additionally, we leverage modality pseudo-degradation to simulate real-world imbalances in training data. The base detector, trained on high-quality pairs, provides a consistency constraint for the auxiliary detector, which receives degraded samples. This framework enhances model robustness, ensuring reliable performance even under severe modality degradation. Experimental results demonstrate the effectiveness of our method in handling extreme modality imbalances~(decreasing the Missing Rate by 55%) and improving performance across various baseline detectors.

Figures

Figures reproduced from arXiv: 2505.22154 by the authors.

**Figure 2.** Illustration of our proposed architecture. The green area is the base detector, where the detailed architecture of the backbone is presented. The proposed [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Qualitative comparison on KAIST and FLIR benchmark, including a [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: The µ and σ are solved by a linear scaling principles. The scaling maps the range of [-2.5, 2.5] to [0,255], where the integral probability is 0.995. -2.5 2.5 0 255 0.995 512 × 640 resolution, where the categories with few samples are eliminated, and three main categor…

**Figure 5.** Figure 5: Vanilla modal augmentation disturbs the convergence of the model. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: The visualized features of positive samples in the feature map. The yellow and blue points mean the positive samples from di [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 36 canonical work pages

[1]

Zhang, E

H. Zhang, E. Fromont, S. Lefevre, B. Avignon, Multispec- tral fusion for object detection with cyclic fuse-and-refine blocks, in: 2020 IEEE International conference on image processing (ICIP), IEEE, 2020, pp. 276–280

work page 2020
[2]

Hwang, J

S. Hwang, J. Park, N. Kim, Y . Choi, I. So Kweon, Mul- tispectral pedestrian detection: Benchmark dataset and baseline, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1037– 1045

work page 2015
[3]

Bijelic, T

M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, F. Heide, Seeing through fog without see- ing fog: Deep multimodal sensor fusion in unseen adverse weather, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11682–11692

work page 2020
[4]

G. Führ, C. R. Jung, Camera self-calibration based on nonlinear optimization and applications in surveillance systems, IEEE Transactions on Circuits and Systems for Video Technology 27 (2015) 1132–1142

work page 2015
[5]

Y . Liu, Q. Li, Y . Yuan, Q. Du, Q. Wang, Abnet: Adap- tive balanced network for multiscale object detection in remote sensing imagery, IEEE transactions on geoscience and remote sensing 60 (2021) 1–14

work page 2021
[6]

J. Ku, M. Mozifian, J. Lee, A. Harakeh, S. L. Waslander, Joint 3d proposal generation and object detection from view aggregation, in: 2018 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), IEEE, 2018, pp. 1–8

work page 2018
[7]

J. Liu, S. Zhang, S. Wang, D. N. Metaxas, Multispec- tral deep neural networks for pedestrian detection, arXiv preprint arXiv:1611.02644 (2016)

arXiv 2016
[8]

C. Tian, Z. Zhou, Y . Huang, G. Li, Z. He, Cross- modality proposal-guided feature mining for unregistered rgb-thermal pedestrian detection, IEEE Transactions on Multimedia (2024)

work page 2024

Show all 41 references

[9]

Zhang, X

L. Zhang, X. Zhu, X. Chen, X. Yang, Z. Lei, Z. Liu, Weakly aligned cross-modal learning for multispectral pedestrian detection, in: Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 5127–5137

2019
[10]

Zhang, E

H. Zhang, E. Fromont, S. Lefèvre, B. Avignon, Guided attentive feature fusion for multispectral pedestrian detec- tion, in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 72–80

2021
[11]

Q. Li, C. Zhang, Q. Hu, H. Fu, P. Zhu, Confidence- aware fusion using dempster-shafer theory for multispec- tral pedestrian detection, IEEE Transactions on Multime- dia (2022)

2022
[12]

Y .-T. Chen, J. Shi, Z. Ye, C. Mertz, D. Ramanan, S. Kong, Multimodal object detection via probabilistic ensembling, in: European Conference on Computer Vision, Springer, 2022, pp. 139–158

2022
[13]

S. Lee, J. Park, J. Park, Crossformer: Cross-guided atten- tion for multi-modal object detection, Pattern Recognition Letters 179 (2024) 144–150

2024
[14]

Y . Shao, Q. Huang, et al., Mod-yolo: Multispectral object detection based on transformer dual-stream yolo, Pattern Recognition Letters 183 (2024) 26–34

2024
[15]

J. U. Kim, S. Park, Y . M. Ro, Uncertainty-guided cross- modal learning for robust multispectral pedestrian detec- tion, IEEE Transactions on Circuits and Systems for Video Technology 32 (2021) 1510–1523

2021
[16]

K. Zhou, L. Chen, X. Cao, Improving multispectral pedes- trian detection by addressing modality imbalance prob- lems, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part XVIII 16, Springer, 2020, pp. 787–803

2020
[17]

Bochkovskiy, C.-Y

A. Bochkovskiy, C.-Y . Wang, H.-Y . M. Liao, Yolov4: Optimal speed and accuracy of object detection, arXiv preprint arXiv:2004.10934 (2020)

2020 arXiv
[18]

González, Z

A. González, Z. Fang, Y . Socarras, J. Serrat, D. Vázquez, J. Xu, A. M. López, Pedestrian detection at day/night time with visible and fir cameras: A comparison, Sensors 16 (2016) 820

2016
[19]

X. Jia, C. Zhu, M. Li, W. Tang, W. Zhou, Llvip: A visible- infrared paired dataset for low-light vision, in: Proceed- ings of the IEEE/CVF international conference on com- puter vision, 2021, pp. 3496–3504

2021
[20]

X. Guo, W. Zhou, T. Liu, Contrastive learning-based knowledge distillation for rgb-thermal urban scene se- mantic segmentation, Knowledge-Based Systems (2024) 111588

2024
[21]

Wagner, V

J. Wagner, V . Fischer, M. Herman, S. Behnke, et al., Mul- tispectral pedestrian detection using deep fusion convolu- tional neural networks., in: ESANN, volume 587, 2016, pp. 509–514

2016
[22]

Zhang, Z

L. Zhang, Z. Liu, S. Zhang, X. Yang, H. Qiao, K. Huang, A. Hussain, Cross-modality interactive attention network for multispectral pedestrian detection, Information Fusion 50 (2019) 20–29

2019
[23]

C. Li, D. Song, R. Tong, M. Tang, Illumination-aware faster r-cnn for robust multispectral pedestrian detection, Pattern Recognition 85 (2019) 161–171. 8

2019
[24]

Wanchaitanawong, M

N. Wanchaitanawong, M. Tanaka, T. Shibata, M. Oku- tomi, Multi-modal pedestrian detection with large mis- alignment based on modal-wise regression and multi- modal iou, in: 2021 17th International Conference on Machine Vision and Applications (MV A), IEEE, 2021, pp. 1–6

2021
[25]

J. Kim, H. Kim, T. Kim, N. Kim, Y . Choi, Mlpd: Multi- label pedestrian detector in multispectral domain, IEEE Robotics and Automation Letters 6 (2021) 7846–7853

2021
[26]

H. Ni, W. Wang, S. Yun, Z. Zhao, X. Zhang, Modality- independent regression and training for improving multi- spectral pedestrian detection, in: 2022 7th International Conference on Image, Vision and Computing (ICIVC), IEEE, 2022, pp. 75–80

2022
[27]

Y . Shi, B. Paige, P. Torr, N. Siddharth, Relating by con- trasting: A data-efficient framework for multimodal gen- erative models, in: International Conference on Learning Representations, 2020

2020
[28]

Y .-H. H. Tsai, P. P. Liang, A. Zadeh, L.-P. Morency, R. Salakhutdinov, Learning factorized multimodal rep- resentations, in: International Conference on Learning Representations, 2018

2018
[29]

M. Ma, J. Ren, L. Zhao, S. Tulyakov, C. Wu, X. Peng, Smil: Multimodal learning with severely missing modal- ity, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2021, pp. 2302–2310

2021
[30]

B. Yu, L. Zhou, L. Wang, J. Fripp, P. Bourgeat, 3d cgan based cross-modality mr image synthesis for brain tumor segmentation, in: 2018 IEEE 15th international sympo- sium on biomedical imaging (ISBI 2018), IEEE, 2018, pp. 626–630

2018
[31]

Sharma, G

A. Sharma, G. Hamarneh, Missing mri pulse sequence synthesis using multi-modal generative adversarial net- work, IEEE transactions on medical imaging 39 (2019) 1170–1183

2019
[32]

R. Azad, N. Khosravi, D. Merhof, Smu-net: Style matching u-net for brain tumor segmentation with miss- ing modalities, in: International Conference on Medical Imaging with Deep Learning, PMLR, 2022, pp. 48–62

2022
[33]

Zhang, H

Y . Zhang, H. Yu, Y . He, X. Wang, W. Yang, Illumination- guided rgbt object detection with inter-and intra-modality fusion, IEEE Transactions on Instrumentation and Mea- surement 72 (2023) 1–13

2023
[34]

Maheshwari, Y .-C

H. Maheshwari, Y .-C. Liu, Z. Kira, Missing modality robustness in semi-supervised multi-modal semantic seg- mentation, arXiv preprint arXiv:2304.10756 (2023)

2023 arXiv
[35]

Y .-J. Li, J. Park, M. O’Toole, K. Kitani, Modality-agnostic learning for radar-lidar fusion in vehicle detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 918–927

2022
[36]

Z. Ge, S. Liu, F. Wang, Z. Li, J. Sun, Yolox: Exceed- ing yolo series in 2021, arXiv preprint arXiv:2107.08430 (2021)

2021 arXiv
[37]

Qingyun, W

F. Qingyun, W. Zhaokui, Cross-modality attentive feature fusion for object detection in multispectral remote sensing imagery, Pattern Recognition 130 (2022) 108786

2022
[38]

K. Chen, J. Liu, H. Zhang, Igt: Illumination-guided rgb- t object detection with transformers, Knowledge-Based Systems 268 (2023) 110423

2023
[39]

Zhang, Y

H. Zhang, Y . Wang, F. Dayoub, N. Sunderhauf, Varifo- calnet: An iou-aware dense object detector, in: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8514–8523

2021
[40]

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.- Y . Fu, A. C. Berg, Ssd: Single shot multibox detector, in: European conference on computer vision, Springer, 2016, pp. 21–37

2016
[41]

Van der Maaten, G

L. Van der Maaten, G. Hinton, Visualizing data using t- sne., Journal of machine learning research 9 (2008). 9

2008

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Zhang, E

H. Zhang, E. Fromont, S. Lefevre, B. Avignon, Multispec- tral fusion for object detection with cyclic fuse-and-refine blocks, in: 2020 IEEE International conference on image processing (ICIP), IEEE, 2020, pp. 276–280

work page 2020

[2] [2]

Hwang, J

S. Hwang, J. Park, N. Kim, Y . Choi, I. So Kweon, Mul- tispectral pedestrian detection: Benchmark dataset and baseline, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1037– 1045

work page 2015

[3] [3]

Bijelic, T

M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, F. Heide, Seeing through fog without see- ing fog: Deep multimodal sensor fusion in unseen adverse weather, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11682–11692

work page 2020

[4] [4]

G. Führ, C. R. Jung, Camera self-calibration based on nonlinear optimization and applications in surveillance systems, IEEE Transactions on Circuits and Systems for Video Technology 27 (2015) 1132–1142

work page 2015

[5] [5]

Y . Liu, Q. Li, Y . Yuan, Q. Du, Q. Wang, Abnet: Adap- tive balanced network for multiscale object detection in remote sensing imagery, IEEE transactions on geoscience and remote sensing 60 (2021) 1–14

work page 2021

[6] [6]

J. Ku, M. Mozifian, J. Lee, A. Harakeh, S. L. Waslander, Joint 3d proposal generation and object detection from view aggregation, in: 2018 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), IEEE, 2018, pp. 1–8

work page 2018

[7] [7]

J. Liu, S. Zhang, S. Wang, D. N. Metaxas, Multispec- tral deep neural networks for pedestrian detection, arXiv preprint arXiv:1611.02644 (2016)

arXiv 2016

[8] [8]

C. Tian, Z. Zhou, Y . Huang, G. Li, Z. He, Cross- modality proposal-guided feature mining for unregistered rgb-thermal pedestrian detection, IEEE Transactions on Multimedia (2024)

work page 2024