Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Adaptive Perception for Unified Visual Multi-modal Object Tracking

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A unified multi-modal tracker with equal modality treatment beats separate trackers on RGB-T, RGB-E, and RGB-D.

desk verdict A plausible unified multi-modal tracker with solid component ablations, but the headline claim over task-specific trackers is undercut by a training-data confound. read the letter →

arxiv 2502.06583 v1 pith:6EEI7VUL submitted 2025-02-10 cs.CV

Xiantao Hu , Bineng Zhong , Qihua Liang , Zhiyi Mo , Liangtao Shi , Ying Tai , Jian Yang This is my paper · ORCID
classification cs.CV
keywords visualobjecttrackingmulti-modalRGB-TRGB-DRGB-eventunifiedmodeladaptiveperceptionlearnabletokens
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

APTrack is a single multi-modal object tracker that gives RGB, thermal, event, and depth inputs the same modeling treatment instead of making RGB the dominant modality. The paper's central claim is that this equal-modality strategy, together with an adaptive modality interaction module built on learnable tokens, lets one set of network weights reach state-of-the-art results on RGB-T, RGB-E, and RGB-D tracking without task-specific fine-tuning. If the claim holds, a single deployed model could replace separate trackers for each sensor pairing and would keep working when the informative modality changes across scenes. The evidence comes from five benchmarks, RGBT234, LasHeR, VisEvent, DepthTrack, and VOT-RGBD2022, where APTrack is reported to outperform both previous unified trackers and trackers specialized for a single multi-modal task.

What carries the argument

The central object is the adaptive modality interaction (AMI) module, inserted after some transformer blocks, alongside an equal modeling backbone that shares one patch-embedding layer and one transformer block across modalities so there is no built-in bias toward RGB. AMI has three stages: token learning, which uses softmax-weighted spatial attention to distill each modality into a small number of learnable tokens; a global modal perceptor, which runs two reciprocal attention operations (Q-Attention merges queries from both modalities, and KV-Attention merges keys and values) so each modality can query the other's information; and token embedding, which returns the exchanged information to the full token sequence through a learned spatial weight matrix. This machinery transfers the currently useful modality's advantages while keeping computation low, because only a few tokens participate in the expensive interaction.

What would settle it

Retrain APTrack on only one task's training set, such as LasHeR alone, and compare it with task-specific RGB-T trackers trained on the same LasHeR split; if the lead over those trackers shrinks to zero or reverses, then the unified-training data, not equal modeling, was driving the reported advantage.

Watch

Extended reading notes

Core claim

APTrack's discovery is that treating every modality equally at the architectural level is enough to let the network itself decide which modality to trust: the same shared patch embedding, shared transformer blocks, and an adaptive modality interaction (AMI) module process RGB and the X modality symmetrically. AMI compresses each modality's features into a few learned tokens, has the two modalities exchange information through query-guided and key-value-guided attention, and embeds the exchanged information back into both feature streams. With one set of parameters trained jointly on LasHeR, VisEvent, and DepthTrack, the model reports a 58.9 success score on LasHeR, 61.8 success on VisEvent, 62.1 F-score on DepthTrack, and 77.4 EAO on VOT-RGBD2022, exceeding the listed unified and task-specific competitors. The paper interprets these results as showing that adaptive perception, not a fixed dominant-auxiliary design, is the better route to a general RGB-X tracker.

Load-bearing premise

The performance comparison assumes that beating single-task trackers shows the architecture is better, even though APTrack was trained on three datasets at once while the single-task rivals were trained on one, so the larger and more varied training set could be the real source of the gap.

Editorial extensions

If this is right

  • A single APTrack model can be deployed across RGB-T, RGB-E, and RGB-D tracking without per-task fine-tuning, which simplifies both training and deployment pipelines.
  • Trackers that adopt equal modality modeling should degrade gracefully when one modality becomes uninformative, because the attention mechanism can shift to the other modality frame by frame.
  • The AMI module's use of a small number of learned tokens shows that cross-modal fusion does not require full-sequence interaction, pointing to cheaper multi-modal adaptation for real-time use.
  • The reported 50.5 FPS runtime on a single RTX 4090 indicates that the unified design does not trade away speed for generality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If equal modeling is the active ingredient, the same recipe should transfer to other RGB-X pairings, such as RGB with polarization, RGB with night-vision, or RGB with natural-language prompts, as long as paired training data exist.
  • The strongest comparison list includes task-specific trackers trained on a single dataset, so the headline advantage may partly reflect the larger, more diverse joint training set; a matched-data comparison would be needed to isolate the architectural contribution.
  • The learned-token count ablation (0, 16, 32, 64 tokens) suggests there is an optimal token budget, so a natural extension is to make the token count or interaction depth adaptive to input difficulty rather than fixed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes APTrack, a unified multi-modal visual object tracker that treats RGB and a secondary modality (TIR, event, or depth) symmetrically via an equal-modeling strategy, and introduces an adaptive modality interaction (AMI) module based on learnable tokens and a global modal perceptor. The model is trained jointly on LasHeR, VisEvent, and DepthTrack, and evaluated on five benchmarks including RGBT234 and VOT-RGBD2022. The central claim is that a single set of parameters outperforms both previous unified multi-modal trackers and task-specific trackers designed for a single multi-modal task.

Significance. If the central claim is valid, APTrack would be a valuable step towards a truly unified multi-modal tracker, since the equal-modeling design and the token-based AMI are simple and computationally plausible. The paper provides ablations (Table III) showing that the adaptive-perception structure and the two AMI components each contribute on LasHeR, which supports the architectural reasoning. However, the headline comparison against task-specific trackers is currently confounded by the training-data volume difference (Section IV-A), and the reported margins are small and lack statistical backing, so the significance is conditional on those issues being resolved.

major comments (3)
  1. [Section IV-A, Table I] The comparison against task-specific trackers is confounded by training data. APTrack is trained jointly on LasHeR, VisEvent, and DepthTrack (Section IV-A), while the task-specific methods in Table I (AINet, BAT, GMMT, TBSI, etc.) are trained on their single corresponding dataset. On LasHeR the margin over AINet is only 0.7 success points (58.9 vs. 58.2). The observed advantage could therefore reflect the larger, more diverse training set rather than the equal-modeling architecture, which is the paper's central claim. To establish the claim, please provide a controlled comparison: either train APTrack on a single dataset, train the task-specific baselines on the same three-dataset union, or add a training-data-matched variant of APTrack.
  2. [Table I and Table IV] All reported benchmark numbers are single-run point estimates without error bars, and the number of learnable tokens N_t is chosen by evaluating on the LasHeR test set (Table IV). With margins as small as 0.7 points on LasHeR and 0.7 F-score on DepthTrack, it is unclear whether the improvements are statistically meaningful. Please report results over multiple seeds (mean and standard deviation) and either avoid test-set-based hyperparameter tuning or clearly flag that the reported LasHeR numbers include test-set selection.
  3. [Section IV-C] The component ablation is performed only on LasHeR. The claim that AMI benefits all three tasks (RGB-T, RGB-E, RGB-D) rests on the final model's performance on VisEvent and DepthTrack, but no ablation on those datasets is given. Please add ablations on at least one RGB-E and one RGB-D dataset to demonstrate that the contributions generalize beyond RGB-T.
minor comments (6)
  1. [Section II-A and Table I] OneTracker is described as a unified multi-modal tracker in Section II-A, but in Table I it appears under the 'Specific' category; please clarify the categorization and justify the placement.
  2. [Table I] ViPT appears twice with different numbers (unified: 49.0 success; specific: 52.5 success); this should be explained, as it is confusing without a footnote.
  3. [Section IV-C] The first sentence says 'we refer to Un-Track [8] to perform ablation experiments', but the next sentence says 'We used the dual-template ViPT [7] as our baseline'; please make this consistent.
  4. [Figure 1 caption] The caption contains a typo ('🤖Inetraction').
  5. [Section III-C, Eq. (8)] The notation for Q-Attention and KV-Attention is under-specified; please define Q, K, V, ˆQ, ˆK, ˆV and the Attention function used in Eq. (8).
  6. [Section V] The limitations paragraph is appreciated, but it does not mention the training-data confound or the test-set-based hyperparameter choice; please discuss these in the limitations.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity; only a mild test-set hyperparameter selection on LasHeR.

  1. fitted input called prediction [Section IV-C, 'Number of learnable Tokens Analysis' and Table IV; effect on Table I.]
    "we will perform ablation experiments and explorations on the LasHeR [28] test set to validate the model components. ... ABLATION STUDIES ABOUT NUMBER OF LEARNABLE TOKENS. WE USE GRAY COLOR TO DENOTE OUR FINAL TRACKERS SETTING. Number Precision Success 0 72.9 58.7 16 73.2 58.3 32 74.1 58.9 64 73.9 58.6"

    The final number of learnable tokens (Nt=32) is chosen by comparing Success rates on the LasHeR test set, and 32 is the row with the highest Success (58.9). The same LasHeR test set is then used in Table I to report APTrack's headline SOTA comparison (58.9 Success vs. AINet's 58.2). Thus the reported LasHeR advantage is not a fully out-of-sample prediction: the test set has already influenced the selection of this hyperparameter. This is a mild 'fitted input called prediction' effect, because the number being claimed as a result is partly selected on the same data used to measure it. It does not, however, reduce the core architectural derivation, which is supported by ablations and by other benchmarks not used for this selection.

full rationale

APTrack is an empirical architecture paper: the 'adaptive perception' claim is supported by benchmark measurements, and the AMI module is described by equations that are not derived from the benchmark numbers. I found no derivation step in which a predicted quantity equals a fitted input by construction. The one mild statistical issue is test-set hyperparameter selection: Section IV-C states ablations are run on the LasHeR test set, and Table IV selects 32 learnable tokens as the argmax Success on that set; the same LasHeR numbers then appear in the Table I SOTA comparison. This makes the LasHeR advantage slightly selected rather than purely predicted, but it is a narrow effect and does not make the central architecture claim circular. The self-citations ([10], [49], [54]) appear only as related-work context and are not load-bearing. The unequal training data for task-specific baselines (APTrack trained on three datasets vs. specific trackers on one) is a correctness/fairness concern, not circularity, and is not scored here.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard benchmark evaluation and on a few hand-set hyperparameters. The most notable free parameter is the number of learnable tokens, which is tuned on the test set. No new physical or conceptual entities are introduced.

free parameters (3)
  • Number of learnable tokens N_t = 32
    Selected via ablation on the LasHeR test set (Table IV); the paper reports 32 as the best setting.
  • Dynamic template update interval and threshold = 5 frames; score > 0.65
    Set by hand in Section IV-A without ablation or justification.
  • Loss weights lambda_1, lambda_2 = 5 and 2
    Fixed in Equation (11) from prior tracking literature; not fitted in this paper.
assumptions (3)
  • domain assumption Transformer blocks with shared weights preserve sufficient modality-specific information.
    The equal modeling strategy assumes the same backbone can encode RGB and X modalities without modality-specific adapters.
  • domain assumption Benchmark datasets and evaluation protocols are correctly used.
    Tables I and II rely on standard benchmarks; any protocol mismatch would change the numbers.
  • domain assumption Event data can be represented as image-like frames.
    VisEvent is treated as an RGB-E task with patch embedding on event frames, but the exact event representation is not specified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Perception for Unified Visual Multi-modal Object Tracking." pith.science (2026). https://pith.science/paper/6EEI7VUL

@misc{pith2026250206583,
  author       = {Pith},
  title        = {Pith review of: Adaptive Perception for Unified Visual Multi-modal Object Tracking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6EEI7VUL}},
  note         = {Machine review of arXiv:2502.06583}
}
read the original abstract

Recently, many multi-modal trackers prioritize RGB as the dominant modality, treating other modalities as auxiliary, and fine-tuning separately various multi-modal tasks. This imbalance in modality dependence limits the ability of methods to dynamically utilize complementary information from each modality in complex scenarios, making it challenging to fully perceive the advantages of multi-modal. As a result, a unified parameter model often underperforms in various multi-modal tracking tasks. To address this issue, we propose APTrack, a novel unified tracker designed for multi-modal adaptive perception. Unlike previous methods, APTrack explores a unified representation through an equal modeling strategy. This strategy allows the model to dynamically adapt to various modalities and tasks without requiring additional fine-tuning between different tasks. Moreover, our tracker integrates an adaptive modality interaction (AMI) module that efficiently bridges cross-modality interactions by generating learnable tokens. Experiments conducted on five diverse multi-modal datasets (RGBT234, LasHeR, VisEvent, DepthTrack, and VOT-RGBD2022) demonstrate that APTrack not only surpasses existing state-of-the-art unified multi-modal trackers but also outperforms trackers designed for specific multi-modal tasks.

Figures

Figures reproduced from arXiv: 2502.06583 by the authors.

Figure 1
Figure 1. APTrack is a unified multi-modal tracker that can be applied to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall structure of APTrack. APTrack is composed of shared embedding, shared transformer block, AMI and Head. The method of modal [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The detailed architecture of Global Modal Perceptor. These perceptors [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Visual result of RGB-T. The three sequences from top to bottom represent the following scenarios: both modalities exhibit low effectiveness, the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: More MPR/MSR comparisons on RGBT234. VisEvent. VisEvent [32] contains 500 training video se￾quences and 320 testing video sequences, which is currently the largest RGB-E dataset. In VisEvent [32], we compared with recent high-performance multi-modal trackers, including…
Figure 6
Figure 6. Figure 6: Visualization comparison of our method with other unified multi-modal trackers on RGB-E and RGB-D tasks. Four challenging scenarios including [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Visualization of the attention maps for a representative pair sequence. The modal dominance of these two sequences is not consistent, specifically [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Success rate of different attributes on the LasHeR [ [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Explicit Context Reasoning with Supervision for Visual Tracking

    cs.CV 2025-07 conditional novelty 6.0 of 10

    RSTrack supervises a Mamba-based state reasoning module with true target states, improving visual tracking accuracy on six benchmarks.

Reference graph

Works this paper leans on

60 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [1]

    Backbone is all your need: A simplified architecture for visual object tracking,

    B. Chen, P. Li, L. Bai, L. Qiao, Q. Shen, B. Li, W. Gan, W. Wu, and W. Ouyang, “Backbone is all your need: A simplified architecture for visual object tracking,” in ECCV (22), ser. Lecture Notes in Computer Science, vol. 13682. Springer, 2022, pp. 375–392

    work page 2022

  2. [2]

    Mixformer: End-to-end tracking with iterative mixed attention,

    Y . Cui, C. Jiang, L. Wang, and G. Wu, “Mixformer: End-to-end tracking with iterative mixed attention,” in CVPR. IEEE, 2022, pp. 13 598– 13 608

    work page 2022

  3. [3]

    Joint feature learning and relation modeling for tracking: A one-stream framework,

    B. Ye, H. Chang, B. Ma, S. Shan, and X. Chen, “Joint feature learning and relation modeling for tracking: A one-stream framework,” in ECCV (22), ser. Lecture Notes in Computer Science, vol. 13682. Springer, 2022, pp. 341–357

    work page 2022

  4. [4]

    Seqtrack: Sequence to sequence learning for visual object tracking,

    X. Chen, H. Peng, D. Wang, H. Lu, and H. Hu, “Seqtrack: Sequence to sequence learning for visual object tracking,” in CVPR. IEEE, 2023, pp. 14 572–14 581

    work page 2023

  5. [5]

    Swintrack: A simple and strong baseline for transformer tracking,

    L. Lin, H. Fan, Z. Zhang, Y . Xu, and H. Ling, “Swintrack: A simple and strong baseline for transformer tracking,” in NeurIPS, 2022

    work page 2022

  6. [6]

    Autoregressive visual tracking,

    X. Wei, Y . Bai, Y . Zheng, D. Shi, and Y . Gong, “Autoregressive visual tracking,” in CVPR. IEEE, 2023, pp. 9697–9706

    work page 2023

  7. [7]

    Visual prompt multi- modal tracking,

    J. Zhu, S. Lai, X. Chen, D. Wang, and H. Lu, “Visual prompt multi- modal tracking,” in CVPR. IEEE, 2023, pp. 9516–9526

    work page 2023

  8. [8]

    Single-Model and Any-Modality for Video Object Tracking

    Z. Wu, J. Zheng, X. Ren, F.-A. Vasluianu, C. Ma, D. P. Paudel, L. Van Gool, and R. Timofte, “Single-model and any-modality for video object tracking,” arXiv preprint arXiv:2311.15851 , 2023

    work page Pith review arXiv 2023

Show all 60 references
  1. [9]

    Robust tracking via mamba-based context-aware token learning,

    J. Xie, B. Zhong, Q. Liang, N. Li, Z. Mo, and S. Song, “Robust tracking via mamba-based context-aware token learning,” arXiv preprint arXiv:2412.13611, 2024

    2024 arXiv

  2. [10]

    Exploiting multimodal spatial-temporal patterns for video object tracking,

    X. Hu, Y . Tai, X. Zhao, C. Zhao, Z. Zhang, J. Li, B. Zhong, and J. Yang, “Exploiting multimodal spatial-temporal patterns for video object tracking,” arXiv preprint arXiv:2412.15691 , 2024

    2024 arXiv

  3. [11]

    Curricular contrastive regularization for physics-aware single image dehazing,

    Y . Zheng, J. Zhan, S. He, J. Dong, and Y . Du, “Curricular contrastive regularization for physics-aware single image dehazing,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 5785–5794

    2023

  4. [12]

    Dynamic group difference coding based on thermal infrared face image for fever screening,

    M. Yan, J. Qian, R. Wang, S. Gao, and J. Yang, “Dynamic group difference coding based on thermal infrared face image for fever screening,” IEEE Transactions on Instrumentation and Measurement , vol. 72, pp. 1–13, 2023

    2023

  5. [13]

    Few-shot learning with long- tailed labels,

    H. Zhang, S. Chen, L. Luo, and J. Yang, “Few-shot learning with long- tailed labels,” Pattern Recognition, vol. 156, p. 110806, 2024

    2024

  6. [14]

    SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation,

    X. An, L. Zhao, C. Gong, N. Wang, D. Wang, and J. Yang, “SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation,” Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 2, pp. 691–699, Mar. 2024

    2024

  7. [15]

    An emotion recognition method based on eye movement and audiovisual features in mooc learning environment,

    J. Bao, X. Tao, and Y . Zhou, “An emotion recognition method based on eye movement and audiovisual features in mooc learning environment,” IEEE Transactions on Computational Social Systems , vol. 11, no. 1, pp. 171–183, 2022

    2022

  8. [16]

    3d- guided multi-feature semantic enhancement network for person re-id,

    E. Ning, W. Li, J. Fang, J. Yuan, Q. Duan, and G. Wang, “3d- guided multi-feature semantic enhancement network for person re-id,” Information Fusion, vol. 117, p. 102863, 2025

    2025

  9. [17]

    Multi-branch enhanced discriminative network for vehicle re-identification,

    J. Lian, D.-H. Wang, Y . Wu, and S. Zhu, “Multi-branch enhanced discriminative network for vehicle re-identification,” IEEE Transactions on Intelligent Transportation Systems , 2023

    2023

  10. [18]

    Guided real image dehazing using ycbcr color space,

    W. Fang, J. Fan, Y . Zheng, J. Weng, Y . Tai, and J. Li, “Guided real image dehazing using ycbcr color space,” arXiv preprint arXiv:2412.17496 , 2024

    2024 arXiv

  11. [19]

    Onetracker: Unifying visual object tracking with foundation models and efficient tuning,

    L. Hong, S. Yan, R. Zhang, W. Li, X. Zhou, P. Guo, K. Jiang, Y . Chen, J. Li, Z. Chen et al. , “Onetracker: Unifying visual object tracking with foundation models and efficient tuning,” arXiv preprint arXiv:2403.09634, 2024

    2024 arXiv

  12. [20]

    Prompting for multi-modal tracking,

    J. Yang, Z. Li, F. Zheng, A. Leonardis, and J. Song, “Prompting for multi-modal tracking,” in ACM Multimedia . ACM, 2022, pp. 3492– 3500

    2022

  13. [21]

    Sdstrack: Self-distillation symmetric adapter learning for multi-modal visual object tracking,

    X. Hou, J. Xing, Y . Qian, Y . Guo, S. Xin, J. Chen, K. Tang, M. Wang, Z. Jiang, L. Liu et al., “Sdstrack: Self-distillation symmetric adapter learning for multi-modal visual object tracking,” arXiv preprint arXiv:2403.16002, 2024

    2024 arXiv

  14. [22]

    Bridging search region interaction with template for RGB-T tracking,

    T. Hui, Z. Xun, F. Peng, J. Huang, X. Wei, X. Wei, J. Dai, J. Han, and S. Liu, “Bridging search region interaction with template for RGB-T tracking,” in CVPR. IEEE, 2023, pp. 13 630–13 639

    2023

  15. [23]

    Bi-directional adapter for multi- modal tracking,

    B. Cao, J. Guo, P. Zhu, and Q. Hu, “Bi-directional adapter for multi- modal tracking,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 2, 2024, pp. 927–935

    2024

  16. [24]

    Spiking transformers for event-based single object tracking,

    J. Zhang, B. Dong, H. Zhang, J. Ding, F. Heide, B. Yin, and X. Yang, “Spiking transformers for event-based single object tracking,” in CVPR. IEEE, 2022, pp. 8791–8800

    2022

  17. [25]

    Lasot: A high-quality benchmark for large-scale single object tracking,

    H. Fan, L. Lin, F. Yang, P. Chu, G. Deng, S. Yu, H. Bai, Y . Xu, C. Liao, and H. Ling, “Lasot: A high-quality benchmark for large-scale single object tracking,” in CVPR. Computer Vision Foundation / IEEE, 2019, pp. 5374–5383

    2019

  18. [26]

    Got-10k: A large high-diversity benchmark for generic object tracking in the wild,

    L. Huang, X. Zhao, and K. Huang, “Got-10k: A large high-diversity benchmark for generic object tracking in the wild,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 43, no. 5, pp. 1562–1577, 2021

    2021

  19. [27]

    Trackingnet: A large-scale dataset and benchmark for object tracking in the wild,

    M. M ¨uller, A. Bibi, S. Giancola, S. Al-Subaihi, and B. Ghanem, “Trackingnet: A large-scale dataset and benchmark for object tracking in the wild,” in ECCV (1), ser. Lecture Notes in Computer Science, vol. 11205. Springer, 2018, pp. 310–327

    2018

  20. [28]

    Lasher: A large-scale high-diversity benchmark for RGBT tracking,

    C. Li, W. Xue, Y . Jia, Z. Qu, B. Luo, J. Tang, and D. Sun, “Lasher: A large-scale high-diversity benchmark for RGBT tracking,” IEEE Trans. Image Process., vol. 31, pp. 392–404, 2022

    2022

  21. [29]

    RGB-T object tracking: Benchmark and baseline,

    C. Li, X. Liang, Y . Lu, N. Zhao, and J. Tang, “RGB-T object tracking: Benchmark and baseline,” Pattern Recognit., vol. 96, 2019

    2019

  22. [30]

    Depthtrack: Unveiling the power of rgbd tracking,

    S. Yan, J. Yang, J. K ¨apyl¨a, F. Zheng, A. Leonardis, and J.-K. K¨am¨ar¨ainen, “Depthtrack: Unveiling the power of rgbd tracking,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10 725–10 733

    2021

  23. [31]

    The visual object tracking vot2015 challenge results,

    M. Kristan, J. Matas, A. Leonardis, M. Felsberg, L. Cehovin, G. Fer- nandez, T. V ojir, G. Hager, G. Nebehay, and R. Pflugfelder, “The visual object tracking vot2015 challenge results,” in Proceedings of the IEEE international conference on computer vision workshops, 2015, pp. 1–23

    2015

  24. [32]

    Visevent: Reliable object tracking via collaboration of frame and event flows,

    X. Wang, J. Li, L. Zhu, Z. Zhang, Z. Chen, X. Li, Y . Wang, Y . Tian, and F. Wu, “Visevent: Reliable object tracking via collaboration of frame and event flows,” IEEE Transactions on Cybernetics , 2023

    2023

  25. [33]

    Unified-io: A unified model for vision, language, and multi-modal tasks,

    J. Lu, C. Clark, R. Zellers, R. Mottaghi, and A. Kembhavi, “Unified-io: A unified model for vision, language, and multi-modal tasks,” in The Eleventh International Conference on Learning Representations , 2022

    2022

  26. [34]

    Imagebind: One embedding space to bind them all,

    R. Girdhar, A. El-Nouby, Z. Liu, M. Singh, K. V . Alwala, A. Joulin, and I. Misra, “Imagebind: One embedding space to bind them all,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 15 180–15 190. 10

    2023

  27. [35]

    Mutex: Learning uni- fied policies from multimodal task specifications,

    R. Shah, R. Mart ´ın-Mart´ın, and Y . Zhu, “Mutex: Learning uni- fied policies from multimodal task specifications,” arXiv preprint arXiv:2309.14320, 2023

    2023 arXiv

  28. [36]

    Siamese vision transformers are scalable audio-visual learners,

    Y .-B. Lin and G. Bertasius, “Siamese vision transformers are scalable audio-visual learners,” arXiv preprint arXiv:2403.19638 , 2024

    2024 arXiv

  29. [37]

    A unified audio-visual learning framework for localization, separation, and recognition,

    S. Mo and P. Morgado, “A unified audio-visual learning framework for localization, separation, and recognition,” in International Conference on Machine Learning . PMLR, 2023, pp. 25 006–25 017

    2023

  30. [38]

    Learning visual representation from modality-shared contrastive language-image pre-training,

    H. You, L. Zhou, B. Xiao, N. Codella, Y . Cheng, R. Xu, S.-F. Chang, and L. Yuan, “Learning visual representation from modality-shared contrastive language-image pre-training,” in European Conference on Computer Vision. Springer, 2022, pp. 69–87

    2022

  31. [39]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2018

    2018

  32. [40]

    Weighted sparse representa- tion regularized graph learning for rgb-t object tracking,

    C. Li, N. Zhao, Y . Lu, C. Zhu, and J. Tang, “Weighted sparse representa- tion regularized graph learning for rgb-t object tracking,” in Proceedings of the 25th ACM international conference on Multimedia , 2017, pp. 1856–1864

    2017

  33. [41]

    Generative-based fusion mechanism for multi-modal tracking,

    Z. Tang, T. Xu, X. Wu, X.-F. Zhu, and J. Kittler, “Generative-based fusion mechanism for multi-modal tracking,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 6, 2024, pp. 5189– 5197

    2024

  34. [42]

    Transformer tracking,

    X. Chen, B. Yan, J. Zhu, D. Wang, X. Yang, and H. Lu, “Transformer tracking,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 8126–8135

    2021

  35. [43]

    Learning spatio-temporal transformer for visual tracking,

    B. Yan, H. Peng, J. Fu, D. Wang, and H. Lu, “Learning spatio-temporal transformer for visual tracking,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 10 448–10 457

    2021

  36. [44]

    Aiatrack: Attention in attention for transformer visual tracking,

    S. Gao, C. Zhou, C. Ma, X. Wang, and J. Yuan, “Aiatrack: Attention in attention for transformer visual tracking,” in European Conference on Computer Vision. Springer, 2022, pp. 146–164

    2022

  37. [45]

    Rgbd1k: A large-scale dataset and benchmark for rgb-d object tracking,

    X.-F. Zhu, T. Xu, Z. Tang, Z. Wu, H. Liu, X. Yang, X.-J. Wu, and J. Kittler, “Rgbd1k: A large-scale dataset and benchmark for rgb-d object tracking,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 3, 2023, pp. 3870–3878

    2023

  38. [46]

    Focal loss for dense object detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2980–2988

    2017

  39. [47]

    Generalized intersection over union: A metric and a loss for bounding box regression,

    H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. D. Reid, and S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” in CVPR. Computer Vision Foundation / IEEE, 2019, pp. 658–666

    2019

  40. [48]

    Depthtrack: Unveiling the power of RGBD tracking,

    S. Yan, J. Yang, J. K ¨apyl¨a, F. Zheng, A. Leonardis, and J. K ¨am¨ar¨ainen, “Depthtrack: Unveiling the power of RGBD tracking,” in ICCV. IEEE, 2021, pp. 10 705–10 713

    2021

  41. [49]

    Transformer tracking via frequency fusion,

    X. Hu, B. Zhong, Q. Liang, S. Zhang, N. Li, X. Li, and R. Ji, “Transformer tracking via frequency fusion,” IEEE Trans. Circuits Syst. Video Technol., vol. 34, no. 2, pp. 1020–1031, 2024

    2024

  42. [50]

    Multiple source domain adaptation for multiple object tracking in satellite video,

    X. Zheng, H. Cui, and X. Lu, “Multiple source domain adaptation for multiple object tracking in satellite video,” IEEE Transactions on Geoscience and Remote Sensing , vol. 61, pp. 1–11, 2023

    2023

  43. [51]

    Explicit visual prompts for visual object tracking,

    L. Shi, B. Zhong, Q. Liang, N. Li, S. Zhang, and X. Li, “Explicit visual prompts for visual object tracking,” in AAAI. AAAI Press, 2024, pp. 4838–4846

    2024

  44. [52]

    Autoregressive queries for adaptive tracking with spatio-temporal trans- formers,

    J. Xie, B. Zhong, Z. Mo, S. Zhang, L. Shi, S. Song, and R. Ji, “Autoregressive queries for adaptive tracking with spatio-temporal trans- formers,” in CVPR, 2024, pp. 19 300–19 309

    2024

  45. [53]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. Kim, C. Hallacy, A. Ramesh, G. Goh, Agarwal, and et al., “Learning transferable visual models from natural language supervision,” Cornell University - arXiv,Cornell University - arXiv , Feb 2021

    2021

  46. [54]

    Towards modalities correlation for rgb-t tracking,

    X. Hu, B. Zhong, Q. Liang, S. Zhang, N. Li, and X. Li, “Towards modalities correlation for rgb-t tracking,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

    2024

  47. [55]

    RGBD1K: A large-scale dataset and benchmark for RGB-D object tracking,

    X. Zhu, T. Xu, Z. Tang, Z. Wu, H. Liu, X. Yang, X. Wu, and J. Kittler, “RGBD1K: A large-scale dataset and benchmark for RGB-D object tracking,” in AAAI. AAAI Press, 2023, pp. 3870–3878

    2023

  48. [56]

    Siamban: Target-aware tracking with siamese box adaptive network,

    Z. Chen, B. Zhong, G. Li, S. Zhang, R. Ji, Z. Tang, and X. Li, “Siamban: Target-aware tracking with siamese box adaptive network,” IEEE Trans- actions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 5158–5173, 2022

    2022

  49. [57]

    Reliable object tracking by multimodal hybrid feature extrac- tion and transformer-based fusion,

    H. Sun, R. Liu, W. Cai, J. Wang, Y . Wang, H. Tang, Y . Cui, D. Yao, and D. Guo, “Reliable object tracking by multimodal hybrid feature extrac- tion and transformer-based fusion,” arXiv preprint arXiv:2405.17903 , 2024

    2024 arXiv

  50. [58]

    Depthrefiner: Adapting rgb trackers to rgbd scenes via depth-fused refinement,

    S. Lai, D. Wang, and H. Lu, “Depthrefiner: Adapting rgb trackers to rgbd scenes via depth-fused refinement,” in 2024 IEEE International Conference on Multimedia and Expo (ICME) . IEEE, 2024, pp. 1–6

    2024

  51. [59]

    Cross- modulated attention transformer for rgbt tracking,

    Y . Xiao, J. Zhao, A. Lu, C. Li, Y . Lin, B. Yin, and C. Liu, “Cross- modulated attention transformer for rgbt tracking,” arXiv preprint arXiv:2408.02222, 2024

    2024 arXiv

  52. [60]

    Rgbt tracking via all-layer multimodal interactions with progressive fusion mamba,

    A. Lu, W. Wang, C. Li, J. Tang, and B. Luo, “Rgbt tracking via all-layer multimodal interactions with progressive fusion mamba,” arXiv preprint arXiv:2408.08827, 2024

    2024 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.