Pith. sign in

REVIEW 3 major objections 5 minor 74 references

UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The hybrid attention pattern is the transferable currency: distilling it lets small models match large teachers on infrared segmentation.

desk verdict Solid, well-ablated empirical paper; the hybrid-attention mechanism has direct evidence (Tab. 19), and the main weaknesses are single-run results and a borrowed SOTA table. read the letter →

arxiv 2502.02257 v2 pith:RVZXUZ4F submitted 2025-02-04 cs.CV

Tao Zhang , Jinyong Wen , Zhen Chen , Kun Ding , Shiming Xiang , Chunhong Pan This is my paper · ORCID
classification cs.CV
keywords infraredsemanticsegmentationpre-trainingattentionpatternanalysisknowledgedistillationnormalizedmutualinformationmaskedimagemodelingcontrastivelearningvisiontransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that what makes an RGB pre-trained model transferable to infrared semantic segmentation is not its ImageNet accuracy but the layerwise pattern of its attention maps. Models whose middle layers attend to both nearby tokens and foreground objects, called the hybrid pattern, generalize best, while models dominated by purely local or purely global attention transfer worse. The paper proposes UNIP, a framework that distills that hybrid pattern from a large teacher into a small randomly initialized student, pre-trains on a mixed RGB/infrared dataset called InfMix, and fine-tunes with a last-layer feature pyramid (LL-FPN). On three infrared semantic segmentation benchmarks, UNIP raises average mIoU by up to 13.5 percentage points over standard pre-training, and the small student UNIP-S matches the large MAE-L teacher at roughly one tenth of the computational cost.

What carries the argument

The load-bearing device is NMI-HAD (NMI-guided Hybrid Attention pattern Distillation), built on the normalized mutual information between query and key tokens in the attention matrix. For head $m$ of layer $l$, the joint probability $p(q_i,k_j)=A^m_{i,j}/N$ yields $NMI^m(Q;K)=I^m(Q;K)/(H^m(Q)H^m(K))^{1/2}$, so local attention approaches 1 and global attention approaches 0. UNIP scans the teacher's latter half, selects the layer whose average NMI is closest to the empirical value $s=0.09$, and distills that layer's attention map into the student's last layer using KL divergence. The two supporting components are InfMix, a dataset of 859,375 images drawn from 23 infrared datasets plus ImageNet and COCO subsets with RGB versions converted to grayscale, and LL-FPN, which builds all decoder feature scales from the last layer only because that layer now carries the hybrid pattern.

What would settle it

Keep InfMix and LL-FPN fixed, and distill UNIP-S from MAE-L using a teacher layer chosen without the NMI rule, such as layer 24, where attention is local; if average fine-tuning mIoU stays near the 64.37% of the NMI-selected layer instead of dropping by roughly two points, then the NMI-guided layer choice is not the active ingredient.

Watch

Extended reading notes

Core claim

The central discovery is that pre-trained vision transformer attention maps fall into three layerwise patterns, local, hybrid, and global, and that the hybrid pattern is the one carrying semantic segmentation transfer. The paper quantifies patterns with the normalized mutual information between query and key tokens: identity-like local attention gives NMI close to 1, attention-collapsed global attention gives NMI close to 0, and hybrid layers sit between. It then uses NMI to find the teacher layer whose value is closest to $s=0.09$, and forces the student's last layer to imitate that layer's attention map under KL divergence. Combined with the InfMix dataset and the LL-FPN decoder, this recipe makes small students perform on par with or better than their large teachers, and it outperforms standard pre-training methods across all three infrared datasets.

Load-bearing premise

Everything rests on the claim that the teacher layer whose attention score lands nearest the empirical cutoff is the layer carrying the transferable hybrid pattern, and that imitating that map, rather than the larger dataset or the new decoder, produces the gains; the ablations show correlation but do not separate these factors cleanly.

Editorial extensions

If this is right

  • For choosing infrared backbones, target-domain linear probing is a better guide than ImageNet fine-tuning accuracy, since the paper reports Pearson correlations of 0.88 between infrared LP and FT versus 0.08 between ImageNet FT and infrared FT.
  • Small students can substitute large teachers: UNIP-S reaches 64.37% average mIoU against MAE-L's 64.35% at 1/10 of the compute, and UNIP-B exceeds MAE-L by 0.93% in fine-tuning and 5.06% in linear probing.
  • Every piece of InfMix matters: dropping the ImageNet subset, the COCO subset, or the grayscale conversion costs about 0.5–0.9 average mIoU, and increasing the pre-training dataset size improves performance monotonically.
  • LL-FPN is most effective when the distilled pattern is hybrid: moving the distillation target from MAE-L layer 18 (hybrid) to layer 24 (local) lowers average fine-tuning mIoU by about 2 points, and LL-FPN adds more in the hybrid case.
  • Supervised and contrastive pre-training transfer better to infrared than masked image modeling at small scales because they already exhibit hybrid layers; the gap narrows only when large MIM models develop hybrid patterns.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cheap experiment the paper does not run is testing whether $s=0.09$ stays optimal for other teacher families; Figure 7's flatness across MAE-L and DINO-B suggests it might be a universal prior, not a per-model fit.
  • Table 19's head-wise results imply that selecting only the hybrid heads inside a layer can outperform distilling the whole 16-head layer, so a future rule could operate at head granularity rather than layer granularity.
  • The layerwise probing evidence on ADE20K, MFNet-RGB, NYUDepthv2, and SUN-RGBD suggests the same distillation recipe would extend to RGB and depth segmentation, but full fine-tuning results on those modalities are not reported.
  • A direct way to isolate the mechanism is to distill an artificially constructed attention map whose NMI is close to 0.09; matching the teacher-layer result would prove the pattern itself, not the particular teacher layer, is the carrier.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper addresses the transfer of RGB pre-trained vision transformers to infrared semantic segmentation. The authors first benchmark six pre-training methods (DeiT, DeiT III, DINO, iBOT, MAE, CrossMAE) at four ViT sizes on three infrared datasets (SODA, MFNet-T, SCUT-Seg) under fine-tuning and linear probing, finding that ImageNet accuracy is not predictive of infrared fine-tuning performance, that supervised/contrastive methods beat MIM for small models, and that larger MIM models narrow the gap. To explain these differences, they introduce the NMI of the attention matrix as a quantitative descriptor and identify three attention patterns—local, hybrid, and global—claiming that the hybrid pattern (attending to nearby and foreground tokens simultaneously) is the most useful for segmentation, with layerwise linear probing peaking in layers where hybrid patterns appear. On this basis, they propose UNIP, a three-part recipe: NMI-HAD (distilling the attention map of the teacher layer whose NMI is closest to s=0.09, Eq. (2)), InfMix (a mixed RGB/grayscale/infrared dataset of 859,375 images), and LL-FPN (a last-layer feature pyramid used during fine-tuning). Reported results show gains of up to 13.57 points of average mIoU over same-size MAE baselines, with UNIP-S matching MAE-L in average mIoU at roughly 1/10 the inference FLOPs.

Significance. The paper is potentially a useful contribution on three counts. (1) The infrared pre-training benchmark is carefully controlled: layerwise decay rates are swept per method, training epochs are fixed at 100, all three infrared datasets share one protocol, and the paired MFNet-RGB/MFNet-T comparison in Tab. 2 is a clean natural experiment isolating the modality shift. (2) The NMI descriptor is simple and interpretable; the two extremal cases (identity attention gives NMI=1; uniform rows give NMI=0) are derived in Appendix D.1, the LLP peaks in Fig. 5 line up with the claimed pattern transitions, and the hybrid-pattern hypothesis is extended to RGB and depth in Tab. 10, making it a genuinely falsifiable empirical claim. (3) The method is thoroughly ablated: each of the three components (NMI-HAD target, InfMix composition, LL-FPN) has a dedicated ablation, and the paper ships released code, a data-scaling study (Tab. 15), and a continual-pretraining baseline (Tab. 9). The main shortfall is that the causal attribution to the hybrid pattern is not isolated from confounds, and the small-margin headline gains rest on single runs.

major comments (3)
  1. [§3.3, §4.2; Tabs. 6–7; App. E Tab. 19] The paper's signature mechanistic claim—that the fine-tuning gains of NMI-HAD are causally driven by the hybrid local-plus-foreground attention pattern—is underdetermined by the provided ablations. In Tab. 7, the hybrid-vs-local-vs-global comparison varies the teacher layer, and hence the target layer's average NMI, jointly with the pattern; in Tab. 6, the attention-vs-feature comparison changes the target type together with the loss function and the student head configuration; and the head-wise comparison in Tab. 19, while holding the layer fixed and approximately matching NMI, still varies both NMI (0.0985 vs 0.1049) and the identity of the six selected heads. No experiment holds the teacher, layer, NMI, data, and architecture fixed while destroying only the hybrid spatial structure. I recommend adding a control in which the same teacher layer's attention map is distilled after a spatial permutation of its rows or columns (which preserves NMI while destroying the local-plus-foreground geometry), or a synthetic non-hybrid map with matched NMI is used as the target; either experiment would directly test whether the hybrid structure, rather than any non-collapsed intermediate-layer map or the InfMix+LL-FPN combination, carries the gains. Because the design rationale of NMI-HAD and contribution (2) rest on this mechanism, the experiment is load-bearing rather than cosmetic.
  2. [§4.2; Tabs. 4 and 9] All headline results are single runs, and several of the paper's claims rest on margins that are small relative to the run-to-run variance of 100-epoch segmentation fine-tuning: UNIP-S vs DINO-S (+1.24 FT, +1.12 LP), UNIP-B vs iBOT-B (+1.27 FT), and the headline 'on par with MAE-L' statement (64.37 vs 64.35 in Tab. 4). Without multiple seeds, these claims are not statistically established, even though the large-margin results (e.g., +13.57 for UNIP-T from the MAE-L teacher) are robust. I ask for at least three seeds with mean plus/minus standard deviation for the entries in Tab. 4 where the reported improvement is below roughly 2 points, and a sentence reporting the observed variance.
  3. [Tab. 5 and abstract] The abstract claims that UNIP 'significantly surpasses state-of-the-art (SOTA) infrared or RGB segmentation methods,' but Tab. 5 reports only SODA and MFNet-T; no SCUT-Seg results are given for any of the compared methods, so the SOTA claim is not substantiated on the third benchmark dataset. In addition, all comparison numbers except PAD are borrowed from the TINN paper, so the protocols are not under the authors' control; the paper should state explicitly which claims are restricted to the two datasets, and ideally report its own runs for the main competitors on all three datasets.
minor comments (5)
  1. [§4.1, Eq. (2)] Equation (2) is under-specified: the domain of the arg max (l from L/2+1 to L) is stated only in the text; the symbol L denotes both the number of layers and the distillation loss; and the total pre-training objective that combines the KL term with any other loss is not given. Please rewrite the equation block so the objective is reconstructible.
  2. [§4.1, §4.2, Tab. 20] Typographical errors: 'hybird patterns' should be 'hybrid patterns' in §4.1; 'by as least 1.12%' should be 'by at least 1.12%' in §4.2; and in Tab. 20 the average width '1,00,1' for M3FD should presumably be '1,001'.
  3. [Fig. 6] In Fig. 6, the y-axis labels and legend are ambiguous: the curve labeled 'NMI' apparently plots delta-NMI (non-positive by definition), and the Pearson coefficient is computed between average FT and delta-NMI. Please relabel the axis and state in the caption how many images were used to compute NMI.
  4. [Abstract and §4.2] The '1/10 computational cost' claim is stated without qualification; it refers to inference FLOPs of ViT-S vs ViT-L, but the complete pipeline includes a 72.5-hour distillation stage (Tab. 9). Please specify inference vs training cost when making the efficiency claim.
  5. [App. C.1, Tab. 11] MAE-T and MAE-S are reproduced with modified decoder settings rather than taken from official checkpoints. The dagger mark in Tab. 11 is explained in the caption, but the main text should state that these are re-trained variants, so the benchmark comparison is not mistaken for evaluation of the released MAE checkpoints.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: UNIP's gains are held-out empirical results; NMI-HAD is an ablated design choice, not a fitted prediction.

full rationale

The paper's central claim is an empirical benchmark plus a method, not a derivation from fitted constants. The layer selection in NMI-HAD (Eq. 2) uses an empirically chosen s=0.09, but the selected layer is shown to be stable for s in [0.06, 0.12] (Fig. 7), and the downstream fine-tuning and linear probing numbers are measured on held-out SODA, MFNet-T, and SCUT-Seg datasets, which are distinct from the ImageNet data on which NMI is computed. The hybrid-pattern mechanism is supported by layerwise linear probing (Fig. 5), target-layer sweeps (Fig. 6), and ablations (Tabs. 6, 7, 19); although no same-NMI non-hybrid control fully isolates the mechanism, that is underdetermination rather than circularity. Self-citations such as PAD and MSIP (Zhang et al., 2023) appear only as baselines and dataset comparisons, not as load-bearing premises, and no uniqueness theorem or fitted quantity is renamed as a prediction. Therefore, no circular step meeting the evidentiary standard is present.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The central result rests on a small set of hand-chosen design decisions and domain assumptions rather than on derived physical constants. The most notable free parameter is the NMI threshold s=0.09 used to pick the distillation layer; it is stable under perturbation but is set from observed NMI values. The InfMix composition is also hand-designed. The causal premise that hybrid attention patterns drive segmentation transfer is supported by correlational evidence and ablations, but is not isolated from dataset scale and architecture changes.

free parameters (2)
  • NMI hybrid-pattern threshold s = 0.09
    Set after observing hybrid patterns with NMI in 0.06-0.12; used in Eq. (2) to select teacher distillation layer. Fig. 7 shows performance is stable over 0.06-0.12.
  • InfMix composition = InfPre 541,088; ImageNet subset 200,000; COCO 118,287
    Hand-selected mixing proportions in Sec. 4.1; Tab. 8 ablations show each component contributes, but the exact proportions are not optimized.
assumptions (4)
  • domain assumption Attention maps are a meaningful proxy for what a pre-trained ViT has learned and for downstream transfer quality.
    The entire analysis in Sec. 3 and the NMI-HAD target in Eq. (2) rest on this.
  • ad hoc to paper The three qualitative attention patterns (local, hybrid, global) are exhaustive and can be identified from NMI values with threshold s.
    Introduced in Sec. 3.1-3.2; hybrid range 0.06-0.12 is empirical.
  • domain assumption Distilling the attention map of one selected teacher layer into the student's last layer transfers the hybrid pattern and thereby improves segmentation.
    Core mechanism of NMI-HAD in Sec. 4.1; supported by Tab. 7 but not proven causally.
  • domain assumption Texture bias, as characterized by Park et al. (2023), is a major cause of infrared transfer degradation.
    Sec. 3.4 relies on this external classification and paired RGB-thermal results in Tab. 2.
invented entities (1)
  • Hybrid attention pattern as a load-bearing explanatory construct independent evidence
    purpose: Explains why supervised/CL pre-training transfers better than MIM and motivates NMI-HAD distillation target.
    Supported within the paper by attention visualizations (Fig. 3), NMI curves (Fig. 4), layerwise probe peaks (Fig. 5), and head-wise distillation ablations (Tab. 19), but the causal role is inferred, not isolated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation." pith.science (2026). https://pith.science/paper/RVZXUZ4F

@misc{pith2026250202257,
  author       = {Pith},
  title        = {Pith review of: UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RVZXUZ4F}},
  note         = {Machine review of arXiv:2502.02257}
}
read the original abstract

Pre-training techniques significantly enhance the performance of semantic segmentation tasks with limited training data. However, the efficacy under a large domain gap between pre-training (e.g. RGB) and fine-tuning (e.g. infrared) remains underexplored. In this study, we first benchmark the infrared semantic segmentation performance of various pre-training methods and reveal several phenomena distinct from the RGB domain. Next, our layerwise analysis of pre-trained attention maps uncovers that: (1) There are three typical attention patterns (local, hybrid, and global); (2) Pre-training tasks notably influence the pattern distribution across layers; (3) The hybrid pattern is crucial for semantic segmentation as it attends to both nearby and foreground elements; (4) The texture bias impedes model generalization in infrared tasks. Building on these insights, we propose UNIP, a UNified Infrared Pre-training framework, to enhance the pre-trained model performance. This framework uses the hybrid-attention distillation NMI-HAD as the pre-training target, a large-scale mixed dataset InfMix for pre-training, and a last-layer feature pyramid network LL-FPN for fine-tuning. Experimental results show that UNIP outperforms various pre-training methods by up to 13.5\% in average mIoU on three infrared segmentation tasks, evaluated using fine-tuning and linear probing metrics. UNIP-S achieves performance on par with MAE-L while requiring only 1/10 of the computational cost. Furthermore, UNIP significantly surpasses state-of-the-art (SOTA) infrared or RGB segmentation methods and demonstrates broad potential for application in other modalities, such as RGB and depth. Our code is available at https://github.com/casiatao/UNIP.

Figures

Figures reproduced from arXiv: 2502.02257 by the authors.

Figure 1
Figure 1. The Chain-of-Thought (CoT) of our work. Step1 (Sec. 2): We benchmark the infrared segmentation performance of various pre-trained models and derive several insights. Step2 (Sec. 3): We explore the reasons for the varying behaviors of these models by analyzing the pre-trained atten￾tion maps. Step3 (Sec. 4): Based on these findings, we propose UNIP, a unified framework aimed to enhance the performance of small pre-tr… view at source ↗
Figure 2
Figure 2. The performance of pre-trained models across various methods and sizes. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Attention maps for different query tokens in three representative layers. Each query token’s [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: NMI on ImageNet. between the query and key tokens, while attention entropy im￾plies the concentration of the attention distribution. However, both metrics depict the relationship between one query and mul￾tiple key tokens and are unable to reflect differences in the at…
Figure 5
Figure 5. Figure 5: The layerwise linear probing performance of different methods on SODA ( [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The average FT and NMI of each target layer. Each model is distilled for 20 epochs. Impact of Distillation Target Layers [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: The average FT when employing dif￾ferent s in Eq. (2). Impact of the Hyperparameter s. The parameter s in Eq. (2) deter￾mines the layer chosen for distillation. As presented in [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Illustrations of different transfer architectures for semantic segmentation tasks. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: CKA representation analysis of different models. UNIP-S aligns well with DINO-S in the [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Visualizations of attention maps in supervised and CL models. The attention maps [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: Visualizations of layerwise attention maps in MAE and UNIP-S distilled from MAE-L. [PITH_FULL_IMAGE:figures/full_fig_p025_11.png]
Figure 12
Figure 12. Figure 12: Visualizations of attention maps in MAE and UNIP distilled from MAE-L. Attention [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: Attention maps of RGB image inputs for different query tokens in three representative [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 50 canonical work pages

  1. [1]

    URL http://adas.cvc.uab.es/elektra/enigma-portfolio/cvc-14-visible-fir-day-night-pedestrian-sequen\ -dataset/

    CVC -14 pedestrian dataset, 2016 a . URL http://adas.cvc.uab.es/elektra/enigma-portfolio/cvc-14-visible-fir-day-night-pedestrian-sequen\ -dataset/

    work page 2016

  2. [2]

    URL http://adas.cvc.uab.es/elektra/enigma-portfolio/item-1/

    CVC -9 pedestrian dataset, 2016 b . URL http://adas.cvc.uab.es/elektra/enigma-portfolio/item-1/

    work page 2016

  3. [3]

    Iris thermal/visible face database

    Besma Abidi. Iris thermal/visible face database. URL http://vcipl-okstate.org/pbvs/bench/

    work page

  4. [4]

    Bahnsen and Thomas B

    Chris H. Bahnsen and Thomas B. Moeslund. Rain Removal in Traffic Surveillance : Does it matter? IEEE TITS, 2019

    work page 2019

  5. [5]

    Yuille, Yuyin Zhou, and Cihang Xie

    Yutong Bai, Zeyu Wang, Junfei Xiao, Chen Wei, Huiyu Wang, Alan L. Yuille, Yuyin Zhou, and Cihang Xie. Masked autoencoders enable efficient knowledge distillers. In CVPR, 2023

    work page 2023

  6. [6]

    BeiT : BERT pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BeiT : BERT pre-training of image transformers. In ICLR, 2022

    work page 2022

  7. [7]

    BIRDSAI : A dataset for detection and tracking in aerial thermal infrared videos

    Elizabeth Bondi, Raghav Jain, Palash Aggrawal, Saket Anand, Robert Hannaford, Ashish Kapoor, Jim Piavis, Shital Shah, Lucas Joppa, Bistra Dilkina, and Milind Tambe. BIRDSAI : A dataset for detection and tracking in aerial thermal infrared videos. In WACV, 2020

    work page 2020

  8. [8]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020

    2020

Show all 74 references
  1. [9]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv\'e J\'egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021

    2021

  2. [10]

    Atmospheric transmission and thermal inertia induced blind road segmentation with a large-scale dataset tbrsd

    Junzhang Chen and Xiangzhi Bai. Atmospheric transmission and thermal inertia induced blind road segmentation with a large-scale dataset tbrsd. In ICCV, 2023

    2023

  3. [11]

    Infrared city database, 2021 a

    Kai Chen, Chenglong Zhou, and Shuigen Wang. Infrared city database, 2021 a . URL http://openai.raytrontek.com/apply/E_Universal_video.html/

    2021

  4. [12]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV, 2018

    2018

  5. [13]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In ICCV, 2021 b

    2021

  6. [14]

    Vision transformer adapter for dense predictions

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. In ICLR, 2023

    2023

  7. [15]

    Schwing, Alexander Kirillov, and Rohit Girdhar

    Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In CVPR, 2022

    2022

  8. [16]

    MMSegmentation : Openmmlab semantic segmentation toolbox and benchmark, 2020

    MMSegmentation Contributors. MMSegmentation : Openmmlab semantic segmentation toolbox and benchmark, 2020. URL https://github.com/open-mmlab/mmsegmentation

    2020

  9. [17]

    ImageNet : A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet : A large-scale hierarchical image database. In CVPR, 2009

    2009

  10. [18]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  11. [19]

    EVA : Exploring the limits of masked visual representation learning at scale

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. EVA : Exploring the limits of masked visual representation learning at scale. In CVPR, 2023

    2023

  12. [20]

    Rethinking patch dependence for masked autoencoders

    Letian Fu, Long Lian, Renhao Wang, Baifeng Shi, Xudong Wang, Adam Yala, Trevor Darrell, Alexei A Efros, and Ken Goldberg. Rethinking patch dependence for masked autoencoders. arXiv preprint arXiv:2401.14391, 2024

    2024 arXiv

  13. [21]

    MFNet : Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes

    Qishen Ha, Kohei Watanabe, Takumi Karasawa, Yoshitaka Ushiku, and Tatsuya Harada. MFNet : Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes. In IROS, 2017

    2017

  14. [22]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016

    2016

  15. [23]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll\'ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022

    2022

  16. [24]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  17. [25]

    Milan: Masked image pretraining on language assisted representation

    Zejiang Hou, Fei Sun, Yen-Kuang Chen, Yuan Xie, and Sun-Yuan Kung. Milan: Masked image pretraining on language assisted representation. arXiv preprint arXiv:2208.06049, 2022

    2022 arXiv

  18. [26]

    Multispectral pedestrian detection: Benchmark dataset and baselines

    Soonmin Hwang, Jaesik Park, Namil Kim, Yukyung Choi, and In So Kweon. Multispectral pedestrian detection: Benchmark dataset and baselines. In CVPR, 2015

    2015

  19. [27]

    LLVIP : A visible-infrared paired dataset for low-light vision

    Xinyu Jia, Chuang Zhu, Minzhen Li, Wenqi Tang, and Wenli Zhou. LLVIP : A visible-infrared paired dataset for low-light vision. In ICCVW, 2021

    2021

  20. [28]

    Billion-scale similarity search with gpus

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. TBD, 2021

    2021

  21. [29]

    Similarity of neural network representations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In ICML, 2019

    2019

  22. [30]

    RGB-T object tracking: Benchmark and baseline

    Chenglong Li, Xinyan Liang, Yijuan Lu, Nan Zhao, and Jin Tang. RGB-T object tracking: Benchmark and baseline. PR, 2019

    2019

  23. [31]

    Segmenting objects in day and night: Edge-conditioned cnn for thermal image semantic segmentation

    Chenglong Li, Wei Xia, Yan Yan, Bin Luo, and Jin Tang. Segmenting objects in day and night: Edge-conditioned cnn for thermal image semantic segmentation. TNNLS, 2021 a

    2021

  24. [32]

    LasHeR : A large-scale high-diversity benchmark for rgbt tracking

    Chenglong Li, Wanlin Xue, Yaqing Jia, Zhichen Qu, Bin Luo, Jin Tang, and Dengdi Sun. LasHeR : A large-scale high-diversity benchmark for rgbt tracking. IEEE TIP, 2022 a

    2022

  25. [33]

    Infrared ship database, 2021

    ChunLiu Li and Shuigen Wang. Infrared ship database, 2021. URL http://openai.raytrontek.com/apply/E_Sea_shipping.html

    2021

  26. [34]

    Ni, and Heung-Yeung Shum

    Feng Li, Hao Zhang, Huaizhe Xu, Shilong Liu, Lei Zhang, Lionel M. Ni, and Heung-Yeung Shum. Mask DINO : Towards a unified transformer-based framework for object detection and segmentation. In CVPR, 2023

    2023

  27. [35]

    On-vehicle visible and infrared object detection database, 2021 b

    Gangqiang Li, Jiansheng Wang, and Shuigen Wang. On-vehicle visible and infrared object detection database, 2021 b . URL http://openai.raytrontek.com/apply/E_Double_light_vehicle.html/

    2021

  28. [36]

    Exploring plain vision transformer backbones for object detection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In ECCV, 2022 b

    2022

  29. [37]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C. Lawrence Zitnick. Microsoft COCO : Common objects in context. In ECCV, 2014

    2014

  30. [38]

    InfMAE : A foundation model in infrared modality

    Fangcen Liu, Chenqiang Gao, Yaming Zhang, Junjie Guo, Jinhao Wang, and Deyu Meng. InfMAE : A foundation model in infrared modality. arXiv preprint arXiv:2402.00407, 2024 a

    2024 arXiv

  31. [39]

    Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection

    Jinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu, Risheng Liu, Wei Zhong, and Zhongxuan Luo. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. In CVPR, 2022

    2022

  32. [40]

    Cross-modal collaborative representation learning and a large-scale rgbt benchmark for crowd counting

    Lingbo Liu, Jiaqi Chen, Hefeng Wu, Guanbin Li, Chenglong Li, and Liang Lin. Cross-modal collaborative representation learning and a large-scale rgbt benchmark for crowd counting. In CVPR, 2021 a

    2021

  33. [41]

    LSOTB-TIR : A large-scale high-diversity thermal infrared object tracking benchmark

    Qiao Liu, Xin Li, Zhenyu He, Chenglong Li, Jun Li, Zikun Zhou, Di Yuan, Jing Li, Kai Yang, Nana Fan, and Feng Zheng. LSOTB-TIR : A large-scale high-diversity thermal infrared object tracking benchmark. In ACM MM, 2020

    2020

  34. [42]

    General-purpose dual-sensor (infrared/visible) video database, 2021 b

    Qing Liu, Zhaofei Xu, Ronglu Jin, and Shuigen Wang. General-purpose dual-sensor (infrared/visible) video database, 2021 b . URL http://openai.raytrontek.com/apply/E_Infrared_security.html/

    2021

  35. [43]

    Infrared aerial photography database, 2021 c

    Qing Liu, Zhaofei Xu, and Shuigen Wang. Infrared aerial photography database, 2021 c . URL http://openai.raytrontek.com/apply/E_Aerial_mancar.html

    2021

  36. [44]

    Exploring target representations for masked autoencoders

    Xingbin Liu, Jinghao Zhou, Tao Kong, Xianming Lin, and Rongrong Ji. Exploring target representations for masked autoencoders. In ICLR, 2024 b

    2024

  37. [45]

    Fully convolutional networks for semantic segmentation

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015

    2015

  38. [46]

    Seasons in Drift : A long term thermal imaging dataset for studying concept drift

    Ivan Adriyanov Nikolov, Mark Philip Philipsen, Jinsong Liu, Jacob Velling Dueholm, Anders Skaarup Johansen, Kamal Nasrollahi, and Thomas B Moeslund. Seasons in Drift : A long term thermal imaging dataset for studying concept drift. In NeurIPS, 2021

    2021

  39. [47]

    Maxime Oquab, Timoth \'e e Darcet, Th \'e o Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Micha...

    2024

  40. [48]

    Multi-modal rgb--depth--thermal human body segmentation

    Cristina Palmero, Albert Clap \'e s, Chris Bahnsen, Andreas M gelmose, Thomas B Moeslund, and Sergio Escalera. Multi-modal rgb--depth--thermal human body segmentation. IJCV, 2016

    2016

  41. [49]

    What do self-supervised vision transformers learn? In ICLR, 2023

    Namuk Park, Wonjae Kim, Byeongho Heo, Taekyung Kim, and Sangdoo Yun. What do self-supervised vision transformers learn? In ICLR, 2023

    2023

  42. [50]

    PyTorch : An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  43. [51]

    Mathematical contributions to the theory of evolution

    Karl Pearson. Mathematical contributions to the theory of evolution. iii. regression, heredity, and panmixia. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 1896

  44. [52]

    TinyMIM : An empirical study of distilling mim pre-trained models

    Sucheng Ren, Fangyun Wei, Zheng Zhang, and Han Hu. TinyMIM : An empirical study of distilling mim pre-trained models. In CVPR, 2023

    2023

  45. [53]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012

    2012

  46. [54]

    Lichtenberg, and Jianxiong Xiao

    Shuran Song, Samuel P. Lichtenberg, and Jianxiong Xiao. SUN RGB-D : A rgb-d scene understanding benchmark suite. In CVPR, 2015

    2015

  47. [55]

    Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning

    Yiming Sun, Bing Cao, Pengfei Zhu, and Qinghua Hu. Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning. IEEE TCSVT, 2022

    2022

  48. [56]

    Multispectral object detection for autonomous vehicles

    Karasawa Takumi, Kohei Watanabe, Qishen Ha, Antonio Tejero-De-Pablos, Yoshitaka Ushiku, and Tatsuya Harada. Multispectral object detection for autonomous vehicles. In ACM MM, 2017

    2017

  49. [57]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers & distillation through attention. In ICML, 2021

    2021

  50. [58]

    DeiT III : Revenge of the vit

    Hugo Touvron, Matthieu Cord, and Herv \'e J \'e gou. DeiT III : Revenge of the vit. In ECCV, 2022

    2022

  51. [59]

    RGBT Salient Object Detection : A large-scale dataset and benchmark

    Zhengzheng Tu, Yan Ma, Zhun Li, Chenglong Li, Jieming Xu, and Yongtao Liu. RGBT Salient Object Detection : A large-scale dataset and benchmark. IEEE TMM, 2023

    2023

  52. [60]

    A closer look at self-supervised lightweight vision transformers

    Shaoru Wang, Jin Gao, Zeming Li, Xiaoqin Zhang, and Weiming Hu. A closer look at self-supervised lightweight vision transformers. In ICML, 2023

    2023

  53. [61]

    Unified perceptual parsing for scene understanding

    Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. Unified perceptual parsing for scene understanding. In ECCV, 2018

    2018

  54. [62]

    Alvarez, and Ping Luo

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, and Ping Luo. SegFormer : Simple and efficient design for semantic segmentation with transformers. In NeurIPS, 2021

    2021

  55. [63]

    MCNet : Multi-level correction network for thermal image semantic segmentation of nighttime driving scene

    Haitao Xiong, Wenjie Cai, and Qiong Liu. MCNet : Multi-level correction network for thermal image semantic segmentation of nighttime driving scene. Infrared Physics & Technology, 2021

    2021

  56. [64]

    EfficientSAM : Leveraged masked image pretraining for efficient segment anything

    Yunyang Xiong, Bala Varadarajan, Lemeng Wu, Xiaoyu Xiang, Fanyi Xiao, Chenchen Zhu, Xiaoliang Dai, Dilin Wang, Fei Sun, Forrest Iandola, Raghuraman Krishnamoorthi, and Vikas Chandra. EfficientSAM : Leveraged masked image pretraining for efficient segment anything. In CVPR, 2024

    2024

  57. [65]

    ROMA : Cross-domain region similarity matching for unpaired nighttime infrared to daytime visible video translation

    Zhenjie Yu, Kai Chen, Shuang Li, Bingfeng Han, Chi Harold Liu, and Shuigen Wang. ROMA : Cross-domain region similarity matching for unpaired nighttime infrared to daytime visible video translation. In ACM MM, 2022

    2022

  58. [66]

    Visible-Thermal UAV Tracking : A large-scale benchmark and new baseline

    Pengyu Zhang, Jie Zhao, Dong Wang, Huchuan Lu, and Xiang Ruan. Visible-Thermal UAV Tracking : A large-scale benchmark and new baseline. In CVPR, 2022

    2022

  59. [67]

    PAD : Self-supervised pre-training with patchwise-scale adapter for infrared images

    Tao Zhang, Kun Ding, Jinyong Wen, Yu Xiong, Zeyu Zhang, Shiming Xiang, and Chunhong Pan. PAD : Self-supervised pre-training with patchwise-scale adapter for infrared images. arXiv preprint arXiv:2312.08192, 2023

    2023 arXiv

  60. [68]

    Pyramid scene parsing network

    Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, 2017

    2017

  61. [69]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, 2017

    2017

  62. [70]

    Image BERT pre-training with online tokenizer

    Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. Image BERT pre-training with online tokenizer. In ICLR, 2022

    2022

  63. [71]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  64. [72]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  65. [73]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  66. [74]

    Therefore, we focus on enhancing small pre-trained models by introducing a comprehensive framework, UNIP, and validating its effectiveness through extensive experiments

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.