Pith. sign in

REVIEW 4 major objections 5 minor 53 references

The key to visible-infrared pre-training is weighting patches by reliability, not aligning them all equally.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Reweighting visible-infrared pre-training patches by infrared structural reliability improves downstream segmentation, detection, and retrieval by small, mostly consistent margins.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection Good question, plausible method, but the evidence is muddier than the prose: the retrieval results contradict the paper's 'consistent improvement' claim, and the reproduced baseline is not a clean control. the 4 major comments →

arxiv 2607.20238 v1 pith:TM4JUTPD submitted 2026-07-22 cs.CV

Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training

Qiwei Ma , Bin Deng , Junjie Zhu , Qiangjuan Huang , Puhong Duan , Ke Yang , Xudong Kang , Shutao Li This is my paper
classification cs.CV
keywords visible-infrared alignmentcontrastive pre-trainingpatch importance samplinginfrared structural cuescurriculum learningsemantic segmentationobject detectioncross-modal retrieval
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that visible-infrared contrastive pre-training should not align all spatially paired patches with equal strength, because imaging-physics differences make many paired regions unreliable across modalities. It proposes Importance-Aware Sampling (IAS), which assigns each infrared patch an importance weight derived from infrared edge and gradient cues, optionally refined by a learned sampler and a curriculum that gradually admits harder patches. IAS reweights patch-level, correlation-based, and image-level contrastive objectives without changing the backbone. Across infrared segmentation, infrared detection, visible segmentation, and cross-modal retrieval, the authors report consistent gains over uniform baselines. If the claim holds, treating patch reliability as a first-class design choice is a simple and portable improvement for multimodal pre-training.

Core claim

The paper's claim is that uniformly aligning all spatially paired visible-infrared patches is suboptimal. It identifies four failure modes—textureless infrared regions, homogeneous weak-gradient areas, modality-specific artifacts, and residual misregistration—and shows empirically that infrared patches with low edge strength incur higher contrastive loss. IAS converts infrared structural cues into per-patch importance weights, optionally learned by a small network warm-started from those hand-crafted cues, and applies the weights to the contrastive objective through loss reweighting or importance pooling, with a curriculum that starts from confident patches and expands. On downstream infrare

What carries the argument

The load-bearing object is the per-patch importance weight, a scalar between zero and one that gates how much each visible-infrared patch pair contributes. It is instantiated three ways: a Sobel/HOG structural prior pooled per patch and normalized; a two-layer MLP sampler that predicts a soft importance mask from infrared patch embeddings, optionally warm-started by the prior; and a curriculum mask that keeps only the top fraction of patches early and linearly expands coverage. These weights enter three objective variants: they reweight patch-level contrastive losses, they weight correlations in a correlation-alignment loss, and they pool infrared tokens into a global representation for imag

Load-bearing premise

The decisive assumption is that the reference model used for comparison is an exact control, so the reported gains are caused by the sampling weights and not by implementation differences; the paper's own tables show reproduction gaps that are sometimes larger than the gains it attributes to sampling.

What would settle it

Re-run one downstream transfer benchmark with the exact same pre-training code and hyperparameters in two conditions—uniform patch weights and importance-aware weights—repeatedly enough to estimate run-to-run variance. The claim is falsified if the importance-weighted run does not beat the uniform run by more than that variance, or if the gap disappears once the baseline is re-run inside the same codebase.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Because IAS only changes patch selection and loss weighting, it can be dropped into image-level, patch-level, and correlation-based alignment objectives without retraining the backbone.
  • Soft importance weighting outperforms hard patch truncation, suggesting the benefit comes from down-weighting unreliable gradients rather than discarding patches.
  • A learned sampler warm-started by hand-crafted infrared cues beats either cue alone, indicating that the importance mask should adapt to data.
  • The same pre-trained representation improves infrared segmentation and detection while also transferring to visible-only segmentation, evidence that the weighting improves general alignment quality.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The observed relationship between edge strength and contrastive loss suggests a cheap, label-free diagnostic: monitoring the correlation between patch importance and patch loss during pre-training could flag when the weighting is mis-calibrated.
  • The same gating idea should transfer to other asymmetric modality pairs, such as visible and synthetic-aperture radar, which the paper lists as future work; one testable expectation is that gains scale with the fraction of unreliable correspondences.
  • If the gains survive at larger scales and with more recent backbones, importance-aware weighting could become a default component of multimodal pre-training rather than a task-specific trick.
  • The paper does not separate whether the improvement comes from suppressing noisy gradients or from a milder, implicit regularization effect; a controlled comparison against uniform weighting with increased loss sharpening could disentangle these.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that in visible-infrared (VIS-IR) pre-training, treating all spatially paired patches as equally trustworthy is suboptimal, because imaging-physics differences make some patches poorly alignable across modalities. It proposes Importance-Aware Sampling (IAS), a plug-and-play training strategy that (i) computes a hand-crafted structural prior (Sobel/HOG) from the infrared image to assign per-patch importance weights, (ii) optionally replaces or warm-starts a lightweight learned sampler that predicts soft importance masks from infrared patch embeddings, and (iii) applies a curriculum that gradually expands training from high-importance to low-importance patches. These weights are used to reweight image-level, patch-level, or correlation-based contrastive objectives. The method is evaluated on infrared semantic segmentation (MFNet, SODA, SCUTSEG, MSRS), infrared object detection (M3FD), visible semantic segmentation (ADE20K), and visible-infrared retrieval (M3FD subset). The central claim is that IAS consistently improves over strong baselines, including the official UNIV checkpoint and its own reproduction of UNIV.

Significance. If the central claim is fully supported, the paper makes a useful contribution: it identifies a rarely examined assumption in VIS-IR pre-training, proposes a simple and low-cost reweighting mechanism (0.107M additional parameters, according to the overhead analysis), and demonstrates plug-and-play behavior across three alignment paradigms. The formulation in Eqs. (9)-(13) is clear, and the ablation study covers importance sources, hard vs. soft weighting, and curriculum schedules. The code release would aid reproducibility. However, the evidence for the 'consistent improvements' claim is weakened by a large reproduction gap between the official UNIV* and the reproduced UNIV† baseline, by a direct contradiction in the retrieval results, by missing statistical reliability, and by an ablation that does not fully support the claim that the learned sampler is best. These issues are load-bearing because the method's benefit is measured relative to the reproduced baseline.

major comments (4)
  1. [§4.2, Table 3] The retrieval results contradict the paper's claim of 'consistent improvements over strong baselines.' On every metric in Table 3, IAS(UNIV-style) is below the official UNIV* checkpoint: e.g., V2I R@1 is 13.80 vs. 15.95, V2I R@10 is 44.52 vs. 48.69, and I2V R@10 is 29.64 vs. 39.52. The text acknowledges only gains over the reproduced UNIV† baseline, whose mRecall is 24.94 versus 30.65 for UNIV*. Moreover, the retrieval prose cites '56.71% R@1 for visible2infrared and 58.07% R@1 for infrared2visible,' but Table 3 reports IAS(Image-style) R@1 values of 82.14 and 83.33; the numbers 56.71/58.07 appear in no table. The authors should correct the prose, report official-baseline comparisons for all retrieval metrics, and either retract or substantiate the consistency claim for retrieval.
  2. [§4.2, Tables 2-5] The reproduced UNIV† baseline is not a neutral control for IAS. The gap between official UNIV* and reproduced UNIV† is sometimes larger than the reported IAS gain: on SODA, UNIV* is 70.22 mIoU vs. UNIV† 69.68, a difference of 0.54, while IAS(UNIV-style) exceeds UNIV* by only 0.37; on retrieval the difference is much larger (mRecall 30.65 vs. 24.94). The paper says the gap is due to 'implementation details' but provides no details. Since IAS modifies the training loss, any difference in optimizer settings, augmentation, checkpoint selection, or fine-tuning protocol could interact with the reweighting and produce the observed numbers. I request the exact reproduction protocol, same-seed comparisons, and ideally fine-tuning from the official UNIV* checkpoint so the effect of IAS is isolated.
  3. [§4.3, Table 7] Table 7 does not support the claim that the learned importance module 'performs best overall.' The learned-with-Sobel-warm-up source gives 52.16 on MFNet and 70.50 on SODA, which are only marginally better than the plain Sobel prior (52.14, 70.17), and on SCUTSEG it is worse than HOG (71.04 vs. 71.71). Additionally, it is unclear whether the curriculum is active in all rows of Table 7. Please clarify the experimental setup and state which variants are used in the final IAS results. As presented, the advantage of the learned sampler over the hand-crafted prior is not established with statistical confidence.
  4. [All experimental tables] The paper reports no error bars, no multiple seeds, and no significance tests for the main comparisons. Many of the headline gains are small in absolute terms (e.g., +0.37 mIoU on SODA over UNIV*, +0.89 mIoU on ADE20K over UNIV*, +0.74 mAP on M3FD over UNIV†). Given that the baseline reproduction gap is of similar size, single-run numbers are insufficient to support the central 'consistent improvement' claim. I request mean and standard deviation over at least three seeds for the key IAS-vs-baseline comparisons, and ideally for the main ablations.
minor comments (5)
  1. [§3.6, Eq. (19)] The retaining factor η is introduced but its value is never specified in the implementation details. Please report it.
  2. [§4.2] The detection paragraph says 'these results suggest that ISA improves infrared representation learning' — 'ISA' should be 'IAS.'
  3. [Figures] The segmentation text refers to 'Fig. 4 and Fig 6' when describing MFNet and M3FD; Fig. 6 is labeled as SCUTSEG, and detection results are in Fig. 5. Please correct the cross-references.
  4. [§4.3, Table 7] The overhead analysis mentions 'Sobel, HOG, and Random variants' but no Random row appears in any table. Either add the Random experiment or remove the mention.
  5. [§5 and §6] The visualizations are qualitative and rely on a small number of examples. It would strengthen the case study to report a quantitative comparison of low-edge regions, e.g., mean patch weights on such regions under Sobel vs. learned sampling.

Circularity Check

0 steps flagged

No circularity: IAS is an empirical training-time weighting scheme; its gains are evaluated on held-out downstream benchmarks, not derived from its own inputs.

full rationale

IAS does not present a derivation or a first-principles prediction that could reduce to its inputs. Patch importance weights are computed from infrared structural cues (Eq. 9-10) or from a learned module (Eq. 14-15), and then used to reweight contrastive objectives (Eq. 11-13). The learned sampler is optimized with the same contrastive objective it modulates plus a Sobel-guidance term (Eq. 16-17); this is standard self-supervised training, not circular, because the training objective is not the downstream evaluation target and no downstream labels are used to fit the weights. The central claims are assessed on external benchmarks (MFNet, SODA, SCUTSEG, MSRS, M3FD, ADE20K) with held-out splits, so the method is self-contained against external data. No load-bearing self-citation chain exists: the only self-reference in the bibliography [26] is not invoked to justify the method. The manuscript does contain a correctness/consistency issue that is not circularity: in Table 3, IAS(UNIV-style) is below official UNIV* on every retrieval metric (e.g., V2I R@1 13.80 vs 15.95), and the retrieval prose quotes 56.71/58.07 R@1 values that do not appear in any table. These issues undermine the 'consistent improvements' claim but do not constitute a circular derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical entities. Its free parameters are four hand-set hyperparameters of the sampling/curriculum module, none of which are individually ablated; the core modeling assumption is that infrared structure predicts cross-modal alignability.

free parameters (4)
  • Sampler temperature Ts = 0.5
    Controls sharpness of learned importance weights (Eq. 15); set by hand in Implementation Details; no ablation reported.
  • Guidance weight lambda_guide = decayed 1.0 -> 0.2 over first 5 epochs
    Strength of Sobel-guide loss during warm-up (Eq. 17); chosen without ablation.
  • Curriculum bounds rho_min, rho_max = 0.2, 1.0
    Coverage ratio bounds in Eq. (18); only linear vs cosine schedule is ablated, not sensitivity to these values.
  • Curriculum length T = not specified
    Epoch length over which coverage ramps (Eq. 18); value is never stated in Implementation Details.
axioms (4)
  • domain assumption Input VIS-IR pairs are pixel-aligned; residual misalignment is ignored.
    State explicitly in Sec. 1: 'under the assumption that pixel-level alignment has already been achieved'.
  • domain assumption Infrared edge/gradient strength is a reliable proxy for cross-modal patch alignability.
    Core heuristic behind Eq. (9)-(10); supported only by an illustrative Fig. 2 without numeric detail or a standalone validation of the proxy.
  • standard math Contrastive learning assumptions hold: co-located patches are positives, non-co-located are negatives, and similarity in embedding space is meaningful.
    Standard self-supervised assumption underlying Eqs. (5)-(8), inherited from prior work and not independently justified here.
  • domain assumption BCE-based correlation alignment with a pseudo-label matrix derived from visible attention (Eq. 8) is valid supervision.
    Inherited from UNIV [28]; used without independent justification in IAS.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training." pith.science (2026). https://pith.science/paper/TM4JUTPD

@misc{pith2026260720238,
  author       = {Pith},
  title        = {Pith review of: Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TM4JUTPD}},
  note         = {Machine review of arXiv:2607.20238}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Visible-infrared (VIS-IR) alignment is a key pre-training task for robust multi-sensor perception. Most existing methods use uniform patch-wise contrastive learning, but this can be unreliable in VIS-IR data because imaging-physics differences make some spatially paired regions inherently less comparable, and aligning them with equal strength hinders representation learning and downstream transfer. In this paper, we revisit VIS-IR pre-training from a sampling perspective and propose Importance-Aware Sampling (IAS), which adjusts training emphasis based on patch reliability. Specifically, IAS (i) derives patch weights from infrared structural cues and uses them to reweight the contrastive objective; (ii) learns a soft importance mask with a lightweight sampler, optionally warm-started from the hand-crafted prior; and (iii) employs a patch curriculum learning strategy that gradually expands from high-reliability regions to harder patches. It is worth noting that IAS is plug-and-play and works with both patch-/correlation-level alignment (e.g., UNIV-style) and image-level contrastive baselines (e.g., ImageBind-style). Extensive experiments on multiple VIS-IR benchmarks demonstrate consistent improvements over strong baselines, including for IR semantic segmentation, IR object detection and VIS semantic segmentation and cross-modal retrieval task. Code will be released on https://github.com/KlayMa527/IAS.

Figures

Figures reproduced from arXiv: 2607.20238 by Bin Deng, Junjie Zhu, Ke Yang, Puhong Duan, Qiangjuan Huang, Qiwei Ma, Shutao Li, Xudong Kang.

Figure 1
Figure 1. Figure 1: Factors that challenge VIS-IR alignment. We summa￾rize four common sources of unreliable patch correspondence. (a) Low-texture regions in the infrared image provide limited discriminative detail. (b) Weak-gradient regions that are ho￾mogeneous in both visible and infrared (e.g., sky/road/walls) offer little structural cue for alignment. (c) Modality-specific effects may appear only in one modality (e.g., r… view at source ↗
Figure 2
Figure 2. Figure 2: Patch-level contrastive loss decreases with increasing infrared edge strength. problems, under the assumption that pixel-level alignment has already been achieved. For such patches, enforcing equal-strength VIS-IR align￾ment is unnecessary and can even be harmful, since it introduces conflicting supervision and noisy gradients from regions that are only loosely matched across modalities. Em￾pirically, unif… view at source ↗
Figure 3
Figure 3. Figure 3: The overview of the proposal IAS framework. (A) Importance-aware sampling. (B) The Contrastive learning pretrain pipeline. (C) Alignment paradigm (C) The illustration of the downstream task. fusion [31, 42, 32]. Early approaches were mainly built upon hand-crafted descriptors with explicit matching [41]. Recently, cross-modal contrastive learning methods such as CLIP [34, 26] have advanced significantly. A… view at source ↗
Figure 4
Figure 4. Figure 4: The infrared semantic segmentation experiment result on the MFNet dataset. 4. Experiments 4.1. Experiment setting Dataset. For pre-training, we use the MVIP dataset from [28], which contains 98,992 aligned visible-infrared image pairs. The INF30 [23], MSIP [28], and INFMIX [45] contain 178,756, 305,241, and 859,375 single modality images, respectively. More details of pre-training datasets are de￾scribed i… view at source ↗
Figure 5
Figure 5. Figure 5: The infrared object detection experiment result on the M3FD dataset [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The infrared semantic segmentation experiment result on SCUTSEG dataset. downstream segmentation architecture. As shown in Ta￾ble 2, IAS consistently outperforms strong pretrained base￾lines, demonstrating that importance-aware sampling dur￾ing visible-infrared alignment can learn more transferable infrared representations. Specifically, IAS (UNIV-style) achieves the best mIoU on MFNet, SCUTSEG and MSRS, r… view at source ↗
Figure 7
Figure 7. Figure 7: The visible semantic segmentation experiment result on ADE20K dataset [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of feature on visible-infrared retrieval dataset. visible-infrared representations across different pre-training pipelines. Impact of Importance Sampling Sources [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: The visualization of layers of our IAS backbone [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 3 linked inside Pith

  1. [1]

    Atmospherictransmissionandthermalinertia induced blind road segmentation with a large-scale dataset tbrsd, in: Proc

    Chen,J.,Bai,X.,2023. Atmospherictransmissionandthermalinertia induced blind road segmentation with a large-scale dataset tbrsd, in: Proc. IEEE Int. Conf. Comput. Vis., pp. 1053–1063

    2023

  2. [2]

    Encoder-decoder with atrous separable convolution for semantic im- age segmentation, in: Proc

    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H., 2018. Encoder-decoder with atrous separable convolution for semantic im- age segmentation, in: Proc. Eur. Conf. Comput. Vis., pp. 801–818

    2018

  3. [3]

    Vision transformer adapter for dense predictions, in: Proc

    Chen, Z., Duan, Y., Wang, W., He, J., Lu, T., Dai, J., Qiao, Y., 2023. Vision transformer adapter for dense predictions, in: Proc. Int. Conf. Learn. Represent

    2023

  4. [4]

    Masked-attention mask transformer for universal image segmenta- tion,in:Proc.IEEEConf.Comput.Vis.PatternRecognit.,pp.1290– 1299

    Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., Girdhar, R., 2022. Masked-attention mask transformer for universal image segmenta- tion,in:Proc.IEEEConf.Comput.Vis.PatternRecognit.,pp.1290– 1299

    2022

  5. [5]

    ViM-VQ: Efficientpost-trainingvectorquantizationforvisualmamba,in:Proc

    Deng,J.,Li,S.,Wang,Z.,Xu,K.,Gu,H.,Huang,K.,2025. ViM-VQ: Efficientpost-trainingvectorquantizationforvisualmamba,in:Proc. IEEE Int. Conf. Comput. Vis., pp. 24518–24527

    2025

  6. [6]

    Lraf-net: Long-range attention fusion network for visible–infrared object detection

    Fu, H., Wang, S., Duan, P., Xiao, C., Dian, R., Li, S., Li, Z., 2023. Lraf-net: Long-range attention fusion network for visible–infrared object detection. IEEE Trans. Neural. Netw. Learn. Syst. 35, 13232– 13245

    2023

  7. [7]

    Cf- deformable detr: an end-to-end alignment-free model for weakly Qiwei Ma et al.:Preprint submitted to ElsevierPage 11 of 13 aligned visible-infrared object detection, in: Proc

    Fu, H., Yuan, J., Zhong, G., He, X., Lin, J., Li, Z., 2024. Cf- deformable detr: an end-to-end alignment-free model for weakly Qiwei Ma et al.:Preprint submitted to ElsevierPage 11 of 13 aligned visible-infrared object detection, in: Proc. Int. Joint Conf. Artif. Intell

    2024

  8. [8]

    MCMAE: Maskedconvolutionmeetsmaskedautoencoders,in:Proc.Adv.Neu- ral Inf

    Gao, P., Ma, T., Li, H., Lin, Z., Dai, J., Qiao, Y., 2022. MCMAE: Maskedconvolutionmeetsmaskedautoencoders,in:Proc.Adv.Neu- ral Inf. Process. Syst., pp. 35632–35644

    2022

  9. [9]

    Imagebind:Oneembeddingspacetobindthemall,in: Proc

    Girdhar,R.,El-Nouby,A.,Liu,Z.,Singh,M.,Alwala,K.V.,Joulin,A., Misra,I.,2023. Imagebind:Oneembeddingspacetobindthemall,in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 15180–15190

    2023

  10. [10]

    Grill, J.B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Ghesh- laghiAzar,M.,etal.,2020.Bootstrapyourownlatent-anewapproach to self-supervised learning. Proc. Adv. Neural Inf. Process. Syst. 33, 21271–21284

    2020

  11. [11]

    Guo,J.,Gao,C.,Liu,F.,Meng,D.,Gao,X.,2024.Damsdet:Dynamic adaptive multispectral detection transformer with competitive query selection and adaptive feature fusion, in: Proc. Eur. Conf. Comput. Vis., Springer. pp. 464–481

    2024

  12. [12]

    Mfnet: Towards real-time semantic segmentation for autonomous vehicleswithmulti-spectralscenes,in:Proc.Int.Conf.Intell.Robots Systems, pp

    Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., Harada, T., 2017. Mfnet: Towards real-time semantic segmentation for autonomous vehicleswithmulti-spectralscenes,in:Proc.Int.Conf.Intell.Robots Systems, pp. 5108–5115. doi:10.1109/IROS.2017.8206396

    arXiv 2017

  13. [13]

    Masked autoencoders are scalable vision learners, in: Proc

    He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R., 2022. Masked autoencoders are scalable vision learners, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 16000–16009

    2022

  14. [14]

    MaskR-CNN,in: Proc

    He,K.,Gkioxari,G.,Dollár,P.,Girshick,R.,2017. MaskR-CNN,in: Proc. IEEE Int. Conf. Comput. Vis., pp. 2961–2969

    2017

  15. [15]

    Global–local feature fusion networkforvisible–infraredvehicledetection

    Kang, X., Yin, H., Duan, P., 2024. Global–local feature fusion networkforvisible–infraredvehicledetection. IEEEGeosci.Remote Sens. Lett. 21, 1–5

    2024

  16. [16]

    Configuring data augmentations to reduce variance shift in positional embedding of vision transformers, in: Proc

    Kim, B.J., Kim, S.W., 2025. Configuring data augmentations to reduce variance shift in positional embedding of vision transformers, in: Proc. AAAI Conf. Artif. Intell., pp. 17840–17849

    2025

  17. [17]

    Segmentingobjects in day and night: Edge-conditioned cnn for thermal image semantic segmentation.IEEETrans.Neural.Netw.Learn.Syst.32,3069–3082

    Li,C.,Xia,W.,Yan,Y.,Luo,B.,Tang,J.,2020a. Segmentingobjects in day and night: Edge-conditioned cnn for thermal image semantic segmentation.IEEETrans.Neural.Netw.Learn.Syst.32,3069–3082

  18. [18]

    Segmentingobjects in day and night: Edge-conditioned cnn for thermal image semantic segmentation.IEEETrans.Neural.Netw.Learn.Syst.32,3069–3082

    Li,C.,Xia,W.,Yan,Y.,Luo,B.,Tang,J.,2020b. Segmentingobjects in day and night: Edge-conditioned cnn for thermal image semantic segmentation.IEEETrans.Neural.Netw.Learn.Syst.32,3069–3082

  19. [19]

    Li, F., Zhang, H., Xu, H., Liu, S., Zhang, L., Ni, L.M., Shum, H.Y.,

  20. [20]

    Explicit attention-enhanced fusion for rgb-thermal perception tasks

    Liang, M., Hu, J., Bao, C., Feng, H., Deng, F., Lam, T.L., 2023. Explicit attention-enhanced fusion for rgb-thermal perception tasks. IEEE Robot. Autom. Lett. 8, 4060–4067

    2023

  21. [21]

    COMO:Cross- mamba interaction and offset-guided fusion for multimodal object detection

    Liu,C.,Ma,X.,Yang,X.,Zhang,Y.,Dong,Y.,2026. COMO:Cross- mamba interaction and offset-guided fusion for multimodal object detection. Inf. Fusion 125, 103414. doi:https://doi.org/10.1016/ j.inffus.2025.103414

    arXiv 2026

  22. [22]

    Liu, F., Gao, C., Sun, Y., Zhao, Y., Yang, F., Qin, A., Meng, D.,

  23. [23]

    InfMAE: A foundation model in the infrared modality, in: Proc

    Liu, F., Gao, C., Zhang, Y., Guo, J., Wang, J., Meng, D., 2024. InfMAE: A foundation model in the infrared modality, in: Proc. Eur. Conf. Comput. Vis., Springer. pp. 420–437

    2024

  24. [24]

    Liu,J.,Fan,X.,Huang,Z.,Wu,G.,Liu,R.,Zhong,W.,Luo,Z.,2022. Target-aware dual adversarial learning and a multi-scenario multi- modality benchmark to fuse infrared and visible for object detection, in:Proc.IEEEConf.Comput.Vis.PatternRecognit.,pp.5802–5811

    2022

  25. [25]

    Bridging rgb-t image fusion and semantic segmentation via multi-task collaborative learning

    Lu, T., Wu, H., Fu, W., Fang, L., Li, S., 2026. Bridging rgb-t image fusion and semantic segmentation via multi-task collaborative learning. Inf. Fusion 129, 103988. doi:https://doi.org/10.1016/j. inffus.2025.103988

    arXiv 2026

  26. [26]

    Ma,Q.,Wang,Z.,Liu,W.,Lu,X.,Deng,B.,Duan,P.,Kang,X.,Li,S.,

  27. [27]

    Visualizing data using t-SNE

    Maaten, L.v.d., Hinton, G., 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605

    2008

  28. [28]

    UNIV: Unified foundation model for infrared and visible modalities

    Mao, F., Wang, S., Mei, J., Lu, S., Min, C., Liu, F., Feng, X., Wu, M., Hu, Y., 2025. UNIV: Unified foundation model for infrared and visible modalities. arXiv preprint arXiv:2509.15642

    Pith/arXiv arXiv 2025

  29. [29]

    Suppresscontentshift:Betterdiffusionfeaturesviaoff-the-shelfgen- eration techniques

    Meng, B., Xu, Q., Wang, Z., Yang, Z., Cao, X., Huang, Q., 2024. Suppresscontentshift:Betterdiffusionfeaturesviaoff-the-shelfgen- eration techniques. Proc. Adv. Neural Inf. Process. Syst. 37, 18910– 18939

    2024

  30. [30]

    Connecting joint-embedding predictive architecture with contrastive self-supervised learning

    Mo, S., Tong, S., 2024. Connecting joint-embedding predictive architecture with contrastive self-supervised learning. Proc. Adv. Neural Inf. Process. Syst. 37, 2348–2377

    2024

  31. [31]

    Attributefilterbased infrared and visible image fusion

    Mo,Y.,Kang,X.,Duan,P.,Sun,B.,Li,S.,2021. Attributefilterbased infrared and visible image fusion. Information Fusion 75, 41–54

    2021

  32. [32]

    Interactive visible and infrared image fusion and segmentation

    Nie, J., Lai, L., Wei, W., Zhang, L., Zhang, Y., 2026. Interactive visible and infrared image fusion and segmentation. Inf. Fusion 127, 103832. doi:https://doi.org/10.1016/j.inffus.2025.103832

    arXiv 2026

  33. [33]

    Learning by aligning: Visible-infrared person re-identification using cross-modal corre- spondences, in: Proc

    Park, H., Lee, S., Lee, J., Ham, B., 2021. Learning by aligning: Visible-infrared person re-identification using cross-modal corre- spondences, in: Proc. IEEE Int. Conf. Comput. Vis., pp. 12046– 12055

    2021

  34. [34]

    Learning transferable visual models from natural language supervision, in: Proc

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S.,Sastry,G.,Askell,A.,Mishkin,P.,Clark,J.,etal.,2021. Learning transferable visual models from natural language supervision, in: Proc. Int. Conf. Mach. Learn., PmLR. pp. 8748–8763

    2021

  35. [35]

    Drone-based rgb- infraredcross-modalityvehicledetectionviauncertainty-awarelearn- ing

    Sun, Y., Cao, B., Zhu, P., Hu, Q., 2022. Drone-based rgb- infraredcross-modalityvehicledetectionviauncertainty-awarelearn- ing. IEEE Trans. Circuits Syst. Video Technol. 32, 6700–6713

    2022

  36. [36]

    PIAFusion: A progressive infrared and visible image fusion network based on illumination aware

    Tang, L., Yuan, J., Zhang, H., Jiang, X., Ma, J., 2022. PIAFusion: A progressive infrared and visible image fusion network based on illumination aware. Information Fusion 83-84, 79–92. doi:https: //doi.org/10.1016/j.inffus.2022.03.007

    doi:10.1016/j.inffus.2022.03.007 2022

  37. [37]

    Yolov8: A novel object detection algorithm with enhanced performance and robustness, in: Proc

    Varghese, R., Sambath, M., 2024. Yolov8: A novel object detection algorithm with enhanced performance and robustness, in: Proc. Int. Conf. Adv. Data Eng. Intell. Comput. Syst., IEEE. pp. 1–6

    2024

  38. [38]

    Unifiedperceptual parsing for scene understanding, in: Proc

    Xiao,T.,Liu,Y.,Zhou,B.,Jiang,Y.,Sun,J.,2018. Unifiedperceptual parsing for scene understanding, in: Proc. Eur. Conf. Comput. Vis., pp. 418–434

    2018

  39. [39]

    SegFormer: Simple and efficient design for semantic segmentationwithtransformers

    Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P., 2021. SegFormer: Simple and efficient design for semantic segmentationwithtransformers. Proc.Adv.NeuralInf.Process.Syst. 34, 12077–12090

    2021

  40. [40]

    MCNet:Multi-levelcorrectionnet- work for thermal image semantic segmentation of nighttime driving scene

    Xiong,H.,Cai,W.,Liu,Q.,2021. MCNet:Multi-levelcorrectionnet- work for thermal image semantic segmentation of nighttime driving scene. Infrared Physics & Technology 113, 103628

    2021

  41. [41]

    Fast and robust matching for multimodal remote sensing image registration

    Ye, Y., Bruzzone, L., Shan, J., Bovolo, F., Zhu, Q., 2019. Fast and robust matching for multimodal remote sensing image registration. IEEETrans.Geosci.RemoteSens.57,9059–9070. doi:10.1109/TGRS. 2019.2924684

    arXiv 2019

  42. [42]

    SPGFusion: Semantic prior guided infrared and visible image fusion via pretrained vision models

    Zhang, H., Yao, S., Ma, J., Jiang, J., Zhang, Y., Zhou, H., 2026. SPGFusion: Semantic prior guided infrared and visible image fusion via pretrained vision models. Inf. Fusion 125, 103433. doi:https: //doi.org/10.1016/j.inffus.2025.103433

    arXiv 2026

  43. [43]

    CR2PQ:Continuousrelativerotarypositionalqueryfordense visual representation learning, in: Proc

    Zhang, S., Zhou, Q., Wu, S., Tan, H., Wang, Z., Huang, J., Yan, J., 2025a. CR2PQ:Continuousrelativerotarypositionalqueryfordense visual representation learning, in: Proc. Int. Conf. Learn. Represent

  44. [44]

    Zhang,T.,Ding,K.,Wen,J.,Xiong,Y.,Zhang,Z.,Xiang,S.,Pan,C.,

  45. [45]

    UNIP:Rethinkingpre-trainedattentionpatternsforinfraredsemantic segmentation, in: Proc

    Zhang, T., Wen, J., Chen, Z., Ding, K., Xiang, S., Pan, C., 2025b. UNIP:Rethinkingpre-trainedattentionpatternsforinfraredsemantic segmentation, in: Proc. Int. Conf. Learn. Represent

  46. [46]

    IEEE Conf

    Zhao,H.,Shi,J.,Qi,X.,Wang,X.,Jia,J.,2017.Pyramidsceneparsing network, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 2881–2890

    2017

  47. [47]

    Cddfuse:Correlation-drivendual-branchfeature decompositionformulti-modalityimagefusion,in:Proc.IEEEConf

    Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., Timofte, R., VanGool,L.,2023. Cddfuse:Correlation-drivendual-branchfeature decompositionformulti-modalityimagefusion,in:Proc.IEEEConf. Qiwei Ma et al.:Preprint submitted to ElsevierPage 12 of 13 Comput. Vis. Pattern Recognit., pp. 5906–5916

    2023

  48. [48]

    arXiv preprint arXiv:2312.08192

    PAD:self-supervisedpre-trainingwithpatchwise-scaleadapter for infrared images. arXiv preprint arXiv:2312.08192

    Pith/arXiv arXiv

  49. [49]

    Semantic understanding of scenes through the ade20k dataset

    Zhou, B., Zhao, H., Puig, X., Xiao, T., Fidler, S., Barriuso, A., Torralba, A., 2019. Semantic understanding of scenes through the ade20k dataset. Int J Comput Vis 127, 302–321. Qiwei Ma et al.:Preprint submitted to ElsevierPage 13 of 13

    2019

  50. [52]

    Equivariant multi-modality image fusion, in: Proc

    Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Zhang, K., Xu, S., Chen, D., Timofte, R., Van Gool, L., 2024. Equivariant multi-modality image fusion, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 25912–25921

    2024

  51. [2021]

    IEEE Trans

    Infraredandvisiblecross-modalimageretrievalthroughshared features. IEEE Trans. Circuits Syst. Video Technol. 31, 4485–4496

  52. [2023]

    IEEE Conf

    Mask dino: Towards a unified transformer-based framework for object detection and segmentation, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 3041–3050

  53. [2025]

    arXivpreprint arXiv:2510.22665

    SARVLM: A vision language foundation model for semantic understandingandtargetrecognitioninSARimagery. arXivpreprint arXiv:2510.22665

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.