Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

What You Have is What You Track: Adaptive and Robust Multimodal Tracking

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One tracker stays state-of-the-art even when video sensors drop frames.

desk verdict A promising heterogeneous-MoE tracker whose central equations don't type-check and whose headline gains aren't in the tables—send to reviewers but require major fixes. read the letter →

arxiv 2507.05899 v1 pith:YYJ7UUV6 submitted 2025-07-08 cs.CV

Yuedong Tan , Jiawei Shao , Eduard Zamfir , Ruanjun Li , Zhaochong An , Chao Ma , Danda Paudel , Luc Van Gool
show 2 more authors
Radu Timofte Zongwei Wu
This is my paper
classification cs.CV
keywords multimodalobjecttrackingtemporallymissingmodalitiesHeterogeneousMixture-of-Expertsvideo-levelmaskingRGB-thermalRGB-depthRGB-eventadaptiveinferencecomplexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a single multimodal visual tracker can stay accurate when sensor data goes missing temporarily, not just when every frame is complete. Existing trackers lose a lot of accuracy under such conditions because they assume every frame supplies both modalities, so the authors propose FlexTrack, a unified framework that treats missing data as a cue and dynamically adjusts how much computation it spends. Its two key ingredients are a heterogeneous mixture-of-experts fusion module, which routes whole video clips to experts of different sizes, and a video-level masking strategy that keeps at least one modality per frame during training, preserving temporal continuity and spatial completeness. On nine benchmarks, including new missing-modality variants of four datasets, FlexTrack reports the best results, with a larger margin over prior work under missing data than under complete data. The paper's core claim is that one tracker, with no extra branch or parameters for missing inputs, can handle both scenarios.

What carries the argument

The load-bearing machinery is the Heterogeneous Mixture-of-Experts Fusion (HMoE-Fuse) module combined with a training-time video-level masking strategy. HMoE-Fuse routes whole video clips, rather than individual tokens, through a gating function that activates the top-$K$ experts ($K=2$) among experts with different hidden dimensions ($2d$ for $d \in \{2,\dots,D-1\}$), so the model can spend more capacity when whole modalities are absent and less when data is plentiful. The masking strategy, described in Algorithm 1, samples a search-region mask from five fixed patterns and, with probability $\alpha$, applies per-clip masks that keep at least one modality in every frame, enforcing temporal consistency while preserving spatial completeness within available frames. Switch-Transformer-style balance and importance auxiliary losses keep the heterogeneous experts from collapsing onto a single route.

What would settle it

Run FlexTrack and the strongest baselines on sequences recorded with genuine hardware sync failures, such as sporadic depth or thermal frame drops logged by the sensors, and compare precision and AUC with the numbers in Table 5. If FlexTrack's margin over STTrack and SUTrack shrinks toward its 2.6% complete-data margin, then the missing-modality advantage is tied to the synthetic mask distribution rather than to real dropout behavior.

Watch

Extended reading notes

Core claim

The paper claims that a single unified tracker, FlexTrack, can be trained once on RGB-thermal, RGB-depth, and RGB-event data and then serve both fully aligned videos and videos with temporally missing modalities, without adding parameters or branches for the missing case. On the complete benchmarks it reports state-of-the-art results, for example 77.3 precision on LasHeR, 81.4 precision on VisEvent, and an F-score of 67.0 on DepthTrack, and it reports that the lead over prior methods widens under missing-modality variants, reaching a 10.2% margin compared with 2.6% on complete data. The authors attribute the missing-data gains to the interaction of the heterogeneous mixture-of-experts fusion (HMoE-Fuse) module and the video-level masking strategy, which their ablations show contributes 3.8 and 6.5 precision points on the RGB-thermal and RGB-event missing benchmarks.

Load-bearing premise

The central assumption is that the synthetic masks used for training and evaluation, which drop one modality from some frames or clips while keeping at least one, represent how real sensors actually fail, so that the large reported margins under missing data would transfer to real synchronization failures.

Editorial extensions

If this is right

  • If the reported results hold, a single model can replace separate complete-data and missing-data trackers, with no additional prompt parameters or modality-specific branches.
  • Missing-modality robustness becomes a trainable property: the paper reports a 10.2% margin over prior state of the art under missing modalities, larger than the 2.6% margin on complete data.
  • Adaptive expert selection implies a test-time compute trade-off: easy full-modality frames can be processed by smaller experts, while hard or incomplete frames get larger experts.
  • The video-level masking recipe can act as a training-time augmentation for other multimodal trackers, improving missing-modality precision by 3.8 to 6.5 points in the paper's ablations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the correlation between missing rate and expert size suggests a natural controller that sets expert budget from sensor-health metadata, turning robustness into an explicit compute-allocation problem.
  • Editorial inference: the same video-level masking could apply to other video tasks that consume multimodal streams, such as action recognition or multi-object tracking, where intermittent sensor loss also breaks temporal continuity.
  • Editorial inference: the missing-modality benchmarks are synthetic masks, so whether the large margins transfer to real hardware dropout patterns is not directly measured in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes FlexTrack, a multimodal visual tracker designed for scenarios where modalities (RGB plus depth, thermal, or event) are temporally incomplete. The two claimed contributions are a Heterogeneous Mixture-of-Experts fusion module (HMoE-Fuse) that dynamically adjusts computational complexity according to missing-data rates and scene difficulty, and a video-level masking training strategy that preserves temporal consistency and spatial completeness. The authors construct missing-modality variants of existing RGB-thermal, RGB-event, and RGB-depth datasets, report state-of-the-art results on both complete and missing settings, and claim gains of 2.6% on complete benchmarks and 10.2% under missing modalities. Code and benchmark are promised but not currently available.

Significance. Temporally incomplete multimodal data is a realistic and understudied problem in visual tracking, and the paper's motivation is strong. If the proposed architecture and training strategy perform as claimed, the unified treatment of complete and missing-modality tracking would be a useful step beyond prompt-based or modality-specific solutions. The paper also contains extensive comparisons across nine benchmarks and several ablations, which is a valuable empirical contribution. However, the significance is currently limited by two load-bearing issues: the central HMoE-Fuse equations are not dimensionally well-defined and cannot be implemented from the text, and the missing-modality evaluation protocol is author-generated, unreleased, and potentially aligned with the training masking distribution. These issues must be resolved before the reported state-of-the-art numbers can be attributed to the proposed mechanism.

major comments (3)
  1. [§3.2, Eqs. (1)–(5)] The central fusion mechanism is not well-defined as written. The dimensions of Tv, W1, W2, W3, W4, and the expert outputs En(Tv) are never specified; because the experts are said to have hidden dimension 2d with d varying over {2,...,D-1}, the sum in Eq. (1) is undefined unless every expert projects to a common output space. Moreover, with Tv ∈ R^{L×D} and W1, W2 ∈ R^{D×D}, Eq. (2) evaluates to (Tv W1)(Tv W2)^T ∈ R^{L×L}, an attention matrix rather than a token transformation; Eq. (3) then multiplies a token matrix by the transpose of that L×L matrix, which is undefined unless D=L. Since the promised code is not available, the HMoE-Fuse module cannot be implemented or checked from the paper as submitted, and this affects all results attributed to HMoE-Fuse, including Tables 5, 7, and 8.
  2. [§4.3 and Algorithm 1] The missing-modality evaluation is vulnerable to train/eval distribution matching. The missing-modality benchmarks are constructed by the authors and are not released, and the evaluation protocol is not specified: the paper does not state the missing-rate distribution, the random seed, or the exact mask-generation procedure used at test time. Since the training masks in Algorithm 1 (random patterns from Ms and Mvi, always keeping at least one modality per frame) closely resemble the author-generated missing patterns used in evaluation, the large missing-modality margins in Table 5 may partly reflect alignment between training and evaluation masking rather than transferable robustness. The 10.2% margin claim therefore needs a public benchmark with a fixed evaluation harness, a sensitivity analysis under mismatched training/evaluation missing statistics, or evaluation on real sensor dropout sequences.
  3. [§4.4 and Figure 4] The adaptive-complexity claim is supported only by qualitative visualization. The paper asserts that the model selects larger experts when the missing rate increases or the scene is more difficult, but it reports no quantitative analysis of expert selection, such as average gating weights, selected expert capacity as a function of missing rate, or routing statistics across videos. Without such measurements, the central claim that the tracker "dynamically activates computational units based on missing data rates" is not established; the gating behavior in Fig. 4 could reflect scene appearance or dataset-specific cues rather than an interpretable missing-modality-driven policy. The ablation in Table 8 varies the fixed expert size and does not test routing behavior, so it does not fill this gap.
minor comments (6)
  1. [Abstract and §1] The abstract and GitHub link in the paper are inconsistent: the abstract says the code and benchmark will be available "at here", while the full text gives a GitHub URL, and the repository is not currently accessible; since the method equations are incomplete, the promised code release is essential and should be made available before publication.
  2. [§4.2] The text says "despite training exclusively on the VisEvent [53], LasHeR [33], and VisEvent [53] datasets", which duplicates VisEvent and omits DepthTrack; this appears to be a typo that should be corrected to "VisEvent, LasHeR, and DepthTrack".
  3. [§4.2, Tables 2–4] Several table cross-references are mismatched: the text referring to Table 2 discusses LasHeR and RGBT234, but those results appear in Table 3, and the caption of Table 2 says "RGB-Event" while the table shows VisEvent; all table references and captions should be carefully checked.
  4. [§4.3, Table 5] The missing-modality benchmark names (DepthTrackmiss, LasHeRmiss, RGBT234miss, VisEventmiss) are introduced without a precise definition of how missing rates are sampled, and IPT [38] is not evaluated on DepthTrackmiss or VisEventmiss, which makes the comparison across datasets uneven; adding the missing protocol details and completing the baseline matrix would strengthen the evaluation.
  5. [Abstract and §1] The claim "first comprehensive study on tracker performance with temporally incomplete multimodal data" should be softened or supported with a more detailed comparison to IPT [38], which also addresses missing-modality RGBT tracking; the current wording overstates the novelty relative to the cited prior work.
  6. [§3.4] The loss weights are described as hyperparameters but there is no sensitivity analysis or justification for the chosen values; a brief sentence on how λ1...λ4 were selected would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's SOTA claims are evaluated on external benchmarks and its ablations are independent of the claimed conclusions.

full rationale

I walked the claimed derivation chain: FlexTrack's two main contributions, the Heterogeneous Mixture-of-Experts Fusion (HMoE-Fuse) and the video-level masking strategy, are defined independently of the reported results. The complete-modality SOTA numbers are measured against external benchmarks (VisEvent, LasHeR, RGBT234, DepthTrack, VOT-RGBD22), and the missing-modality numbers are compared against existing trackers on author-constructed variants. The missing-modality evaluation does share the 'at least one modality per frame' constraint with Algorithm 1's training masks, but the paper explicitly states this exclusion, making it a scoped benchmark-design choice rather than a derivation step that reduces to its own input. The adaptive-complexity discussion in Fig. 4 is a post-hoc interpretation of learned gating behavior, not a prediction forced by definition. No load-bearing self-citation or imported uniqueness theorem is used; citations to prior work by the same authors are standard baselines and do not carry the argument. The dimensional inconsistencies in Eqs. (2)-(5) and the absence of released code are correctness/reproducibility risks, not circularity, and thus do not raise the circularity score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on hand-chosen hyperparameters (K, alpha, expert sizes, loss weights, mask patterns) and on the assumption that the synthetic missing patterns used in training and evaluation capture real sensor failures. No new physical entities are introduced. The free parameters are not fitted to a benchmark objective in a classical sense, but they are necessary design choices that shape the reported gains and are not fully specified in the text.

free parameters (5)
  • Top-K expert count K = 2
    Set to 2 in the gating function and used in Eq. (1); no ablation over K is reported, so it is a hand-chosen architectural constant.
  • Video-level masking threshold alpha = not specified
    Algorithm 1 applies clip masking when p < alpha, but alpha is never given a value, so the training augmentation is not fully specified.
  • Expert hidden dimension set (2d, d in {2,...,D-1}) = not specified (D and M not given)
    The heterogeneity of experts is central to the adaptive-complexity claim, yet the actual hidden sizes and number of experts are not stated; Table 8 gives only 512, 64, and 4 as comparison points.
  • Training mask pattern sets (M_s and M_vi) = five search-region patterns and three clip patterns
    The predefined patterns in Algorithm 1 define which modality-missing combinations the model learns; these patterns also resemble the missing patterns used for evaluation, so their choice directly shapes the reported gains.
  • Loss weights lambda_1..lambda_4 = 1, 5, 2, 1
    Weights for auxiliary, classification, L1, and GIoU losses in Eq. (7); chosen by hand and not ablated.
assumptions (5)
  • ad hoc to paper The matrix operations in Eqs. (1)-(5) are well-defined and implement a valid attention-plus-MoE layer.
    The method section presents these equations as the fusion core, but no tensor dimensions are provided; for the claim to hold, the operations must be implementable as written or with unstated projections.
  • domain assumption Real sensor synchronization failures are well approximated by the synthetic missing patterns used in training and evaluation.
    The paper motivates the problem with sensor desynchronization but validates only on author-created missing-modality variants, not on real dropout data.
  • domain assumption Auxiliary modalities depth, thermal, and event can be treated as a single unified X modality in a joint training pipeline.
    The unified model is trained on LasHeR (thermal), VisEvent (event), and DepthTrack (depth) together, assuming one shared X encoder works for all three.
  • standard math Switch Transformer load-balancing losses prevent expert collapse in the heterogeneous, video-level routing setting.
    The auxiliary loss in Eq. (6) is taken from the Switch Transformer paper; the authors assume it behaves as intended with heterogeneous experts and clip-level routing.
  • ad hoc to paper The gating function learns to route on missing-modality informativeness rather than only on scene appearance.
    The adaptive-complexity story in Figure 4 claims experts are selected by missing rate and scene complexity, but no auxiliary missing-rate signal is provided to the gating network; it must infer this from the token sequence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What You Have is What You Track: Adaptive and Robust Multimodal Tracking." pith.science (2026). https://pith.science/paper/YYJ7UUV6

@misc{pith2026250705899,
  author       = {Pith},
  title        = {Pith review of: What You Have is What You Track: Adaptive and Robust Multimodal Tracking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YYJ7UUV6}},
  note         = {Machine review of arXiv:2507.05899}
}
read the original abstract

Multimodal data is known to be helpful for visual tracking by improving robustness to appearance variations. However, sensor synchronization challenges often compromise data availability, particularly in video settings where shortages can be temporal. Despite its importance, this area remains underexplored. In this paper, we present the first comprehensive study on tracker performance with temporally incomplete multimodal data. Unsurprisingly, under such a circumstance, existing trackers exhibit significant performance degradation, as their rigid architectures lack the adaptability needed to effectively handle missing modalities. To address these limitations, we propose a flexible framework for robust multimodal tracking. We venture that a tracker should dynamically activate computational units based on missing data rates. This is achieved through a novel Heterogeneous Mixture-of-Experts fusion mechanism with adaptive complexity, coupled with a video-level masking strategy that ensures both temporal consistency and spatial completeness which is critical for effective video tracking. Surprisingly, our model not only adapts to varying missing rates but also adjusts to scene complexity. Extensive experiments show that our model achieves SOTA performance across 9 benchmarks, excelling in both conventional complete and missing modality settings. The code and benchmark will be publicly available at https://github.com/supertyd/FlexTrack/tree/main.

Figures

Figures reproduced from arXiv: 2507.05899 by the authors.

Figure 1
Figure 1. Synchronization has long been a challenge in multisen [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our framework: Conventional trackers operate under the strong assumption that all frames are always available, disregarding practical scenarios with missing modalities. To address this limitation, we introduce heterogeneous experts, which dynamically switch between different experts to adapt test-time complexity. To further strengthen the robust learning, we employ a video-level masking strategy during training, enc… view at source ↗
Figure 3
Figure 3. Comparison of different masking strategies for multimodal settings. (a) Conventional random or MAE-like masking [ [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: We are interested in the following question: With a higher missing rate, would a model favor a lighter expert or a more complex expert? To investigate this, we analyze the impact of the missing modality rate on the same video clip, as shown in (a) and (b). When the mis…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Models as Tools: An Agentic Coordination Framework for Unified Multimodal Visual Tracking

    cs.CV 2026-08 conditional novelty 6.0 of 10

    ACTrack coordinates a tracker, a segmentation model, and a VLM through event-triggered conflicts, reporting state-of-the-art RGB and multimodal tracking with 30% trainable parameters.

Reference graph

Works this paper leans on

75 extracted references · 67 canonical work pages · cited by 1 Pith paper

  1. [1]

    Multimodal machine learning: A survey and tax- onomy

    Tadas Baltru ˇsaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and tax- onomy. TPAMI, 41(2):423–443, 2018. 2

    work page 2018

  2. [2]

    Learning discriminative model prediction for track- ing

    Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte. Learning discriminative model prediction for track- ing. In ICCV, 2019. 6

    work page 2019

  3. [3]

    Robust object modeling for visual tracking

    Yidong Cai, Jie Liu, Jie Tang, and Gangshan Wu. Robust object modeling for visual tracking. In ICCV, 2023. 1

    work page 2023

  4. [4]

    Transformer tracking

    Xin Chen, Bin Yan, Jiawen Zhu, Dong Wang, Xiaoyun Yang, and Huchuan Lu. Transformer tracking. In CVPR, 2021. 5

    work page 2021

  5. [5]

    Seqtrack: Sequence to sequence learning for visual ob- ject tracking

    Xin Chen, Houwen Peng, Dong Wang, Huchuan Lu, and Han Hu. Seqtrack: Sequence to sequence learning for visual ob- ject tracking. In CVPR, 2023. 1, 2, 7

    work page 2023

  6. [6]

    Sutrack: Towards simple and unified single object tracking

    Xin Chen, Ben Kang, Wanting Geng, Jiawen Zhu, Yi Liu, Dong Wang, and Huchuan Lu. Sutrack: Towards simple and unified single object tracking. arXiv preprint arXiv:2412.19138, 2024. 2, 5, 6, 7

    arXiv 2024

  7. [7]

    Siamban: Target-aware tracking with siamese box adaptive network

    Zedu Chen, Bineng Zhong, Guorong Li, Shengping Zhang, Rongrong Ji, Zhenjun Tang, and Xianxian Li. Siamban: Target-aware tracking with siamese box adaptive network. TPAMI, 45(4):5158–5173, 2022. 5

    work page 2022

  8. [8]

    Diffusion models in vision: A survey

    Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. TPAMI, 45(9):10850–10869, 2023. 2

    work page 2023

Show all 75 references
  1. [9]

    Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models

    Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y Wu, et al. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models. arXiv preprint arXiv:2401.06066, 2024. 2, 3, 7

    2024 arXiv

  2. [10]

    High-performance long- term tracking with meta-updater

    Kenan Dai, Yunhua Zhang, Dong Wang, Jianhua Li, Huchuan Lu, and Xiaoyun Yang. High-performance long- term tracking with meta-updater. In CVPR, 2020. 5, 6

    2020

  3. [11]

    Atom: Accurate tracking by overlap max- imization

    Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. Atom: Accurate tracking by overlap max- imization. In CVPR, 2019. 5, 6

    2019

  4. [12]

    Proba- bilistic regression for visual tracking

    Martin Danelljan, Luc Van Gool, and Radu Timofte. Proba- bilistic regression for visual tracking. In CVPR, 2020. 5

    2020

  5. [13]

    Lasot: A high-quality benchmark for large-scale single ob- ject tracking

    Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. Lasot: A high-quality benchmark for large-scale single ob- ject tracking. In CVPR, 2019. 5

    2019

  6. [14]

    Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity. JMLR, 23(120):1–39, 2022. 5

    2022

  7. [15]

    Sparsett: Visual tracking with sparse transform- ers

    Zhihong Fu, Zehua Fu, Qingjie Liu, Wenrui Cai, and Yun- hong Wang. Sparsett: Visual tracking with sparse transform- ers. In IJCAI, 2022. 1

    2022

  8. [16]

    Aiatrack: Attention in attention for trans- former visual tracking

    Shenyuan Gao, Chunluan Zhou, Chao Ma, Xinggang Wang, and Junsong Yuan. Aiatrack: Attention in attention for trans- former visual tracking. In ECCV. Springer, 2022. 2

    2022

  9. [17]

    Deep adaptive fusion network for high perfor- mance RGBT tracking

    Yuan Gao, Chenglong Li, Yabin Zhu, Jin Tang, Tao He, and Futian Wang. Deep adaptive fusion network for high perfor- mance RGBT tracking. In ICCVW, 2019. 6

    2019

  10. [18]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 2

    2020

  11. [19]

    Siamcar: Siamese fully convolutional classification and regression for visual tracking

    Dongyan Guo, Jun Wang, Ying Cui, Zhenhua Wang, and Shengyong Chen. Siamcar: Siamese fully convolutional classification and regression for visual tracking. In CVPR,

  12. [20]

    Fusemoe: Mixture-of-experts transformers for flexi- modal fusion

    Xing Han, Huy Nguyen, Carl Harris, Nhat Ho, and Suchi Saria. Fusemoe: Mixture-of-experts transformers for flexi- modal fusion. NeurIPS, 37:67850–67900, 2025. 2, 3

    2025

  13. [21]

    Onetracker: Unifying visual object tracking with foundation models and efficient tuning

    Lingyi Hong, Shilin Yan, Renrui Zhang, Wanyun Li, Xinyu Zhou, Pinxue Guo, Kaixun Jiang, Yiting Chen, Jinglun Li, Zhaoyu Chen, et al. Onetracker: Unifying visual object tracking with foundation models and efficient tuning. In CVPR, 2024. 2, 5, 6

    2024

  14. [22]

    Sdstrack: Self-distillation symmetric adapter learning for multi-modal visual object tracking

    Xiaojun Hou, Jiazheng Xing, Yijie Qian, Yaowei Guo, Shuo Xin, Junhao Chen, Kai Tang, Mengmeng Wang, Zhengkai Jiang, Liang Liu, et al. Sdstrack: Self-distillation symmetric adapter learning for multi-modal visual object tracking. In CVPR, 2024. 2, 5, 6, 7

    2024

  15. [23]

    Exploiting multimodal spatial-temporal patterns for video object tracking

    Xiantao Hu, Ying Tai, Xu Zhao, Chen Zhao, Zhenyu Zhang, Jun Li, Bineng Zhong, and Jian Yang. Exploiting multimodal spatial-temporal patterns for video object tracking. arXiv preprint arXiv:2412.15691, 2024. 2, 5, 6, 7

    2024 arXiv

  16. [24]

    Got-10k: A large high-diversity benchmark for generic object tracking in the wild

    Lianghua Huang, Xin Zhao, and Kaiqi Huang. Got-10k: A large high-diversity benchmark for generic object tracking in the wild. TPAMI, 43(5):1562–1577, 2019. 5

    2019

  17. [25]

    Cola-diff: Conditional latent diffusion model for multi- modal mri synthesis

    Lan Jiang, Ye Mao, Xiangfeng Wang, Xi Chen, and Chao Li. Cola-diff: Conditional latent diffusion model for multi- modal mri synthesis. In MICCAI, pages 398–408. Springer,

  18. [26]

    Exploring enhanced contextual infor- mation for video-level object tracking

    Ben Kang, Xin Chen, Simiao Lai, Yang Liu, Yi Liu, and Dong Wang. Exploring enhanced contextual infor- mation for video-level object tracking. arXiv preprint arXiv:2412.11023, 2024. 5, 6, 7

    2024 arXiv

  19. [27]

    The seventh visual object tracking VOT2019 challenge results

    Matej Kristan, Jiri Matas, Ales Leonardis, Michael Fels- berg, Roman Pflugfelder, Joni-Kristian Kamarainen, Luka Cehovin Zajc, Ondrej Drbohlav, Alan Lukezic, Amanda Berg, et al. The seventh visual object tracking VOT2019 challenge results. In ICCVW, 2019. 6

    2019

  20. [28]

    The eighth visual object tracking VOT2020 challenge results

    Matej Kristan, Ale ˇs Leonardis, Ji ˇr´ı Matas, Michael Fels- berg, Roman Pflugfelder, Joni-Kristian K ¨am¨ar¨ainen, Martin Danelljan, Luka ˇCehovin Zajc, Alan Luke ˇziˇc, Ondrej Dr- bohlav, et al. The eighth visual object tracking VOT2020 challenge results. In ECCVW, 2020. 6

    2020

  21. [29]

    The tenth visual object tracking vot2022 challenge re- sults

    Matej Kristan, Ale ˇs Leonardis, Jiˇr´ı Matas, Michael Felsberg, Roman Pflugfelder, Joni-Kristian K ¨am¨ar¨ainen, Hyung Jin Chang, Martin Danelljan, Luka ˇCehovin Zajc, Alan Lukeˇziˇc, et al. The tenth visual object tracking vot2022 challenge re- sults. In ECCVW, 2023. 6

    2023

  22. [30]

    Weighted sparse representation regularized graph learning for RGB-T object tracking

    Chenglong Li, Nan Zhao, Yijuan Lu, Chengli Zhu, and Jin Tang. Weighted sparse representation regularized graph learning for RGB-T object tracking. In ACMMM, 2017. 6

    2017

  23. [31]

    Rgb-t object tracking: Benchmark and baseline

    Chenglong Li, Xinyan Liang, Yijuan Lu, Nan Zhao, and Jin Tang. Rgb-t object tracking: Benchmark and baseline. PR, 96:106977, 2019. 1, 2, 5, 6, 7

    2019

  24. [32]

    Challenge-aware RGBT tracking

    Chenglong Li, Lei Liu, Andong Lu, Qing Ji, and Jin Tang. Challenge-aware RGBT tracking. In ECCV, 2020. 6

    2020

  25. [33]

    Lasher: A large-scale high- diversity benchmark for rgbt tracking

    Chenglong Li, Wanlin Xue, Yaqing Jia, Zhichen Qu, Bin Luo, Jin Tang, and Dengdi Sun. Lasher: A large-scale high- diversity benchmark for rgbt tracking. TIP, 31:392–404,

  26. [34]

    Siambag: Band attention grouping-based siamese object tracking net- work for hyperspectral videos

    Wei Li, Zengfu Hou, Jun Zhou, and Ran Tao. Siambag: Band attention grouping-based siamese object tracking net- work for hyperspectral videos. TGRS, 61:1–12, 2023. 2

    2023

  27. [35]

    Swintrack: A simple and strong baseline for trans- former tracking

    Liting Lin, Heng Fan, Zhipeng Zhang, Yong Xu, and Haibin Ling. Swintrack: A simple and strong baseline for trans- former tracking. NeurIPS, 35:16743–16754, 2022. 1, 2

    2022

  28. [36]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 5

    2014

  29. [37]

    Context-aware three-dimensional mean- shift with occlusion handling for robust object tracking in RGB-D videos

    Ye Liu, Xiao-Yuan Jing, Jianhui Nie, Hao Gao, Jun Liu, and Guo-Ping Jiang. Context-aware three-dimensional mean- shift with occlusion handling for robust object tracking in RGB-D videos. TMM, pages 664–677, 2018. 6

    2018

  30. [38]

    Modality-missing rgbt tracking: Invertible prompt learning and high-quality benchmarks

    Andong Lu, Chenglong Li, Jiacong Zhao, Jin Tang, and Bin Luo. Modality-missing rgbt tracking: Invertible prompt learning and high-quality benchmarks. IJCV, pages 1–21,

  31. [39]

    Smil: Multimodal learning with severely missing modality

    Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng. Smil: Multimodal learning with severely missing modality. InAAAI, pages 2302–2310, 2021. 1, 2

    2021

  32. [40]

    Are multimodal transformers robust to missing modality? In CVPR, pages 18177–18186, 2022

    Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng. Are multimodal transformers robust to missing modality? In CVPR, pages 18177–18186, 2022. 1, 2

    2022

  33. [41]

    Learning target candidate association to keep track of what not to track

    Christoph Mayer, Martin Danelljan, Danda Pani Paudel, and Luc Van Gool. Learning target candidate association to keep track of what not to track. In ICCV, 2021. 1

    2021

  34. [42]

    Trackingnet: A large-scale dataset and benchmark for object tracking in the wild

    Matthias Muller, Adel Bibi, Silvio Giancola, Salman Al- subaihi, and Bernard Ghanem. Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. In ECCV, 2018. 5

    2018

  35. [43]

    Learning multi-domain convolutional neural networks for visual tracking

    Hyeonseob Nam and Bohyung Han. Learning multi-domain convolutional neural networks for visual tracking. In CVPR,

  36. [44]

    Dal: A deep depth- aware long-term tracker

    Yanlin Qian, Song Yan, Alan Luke ˇziˇc, Matej Kristan, Joni- Kristian K ¨am¨ar¨ainen, and Ji ˇr´ı Matas. Dal: A deep depth- aware long-term tracker. In ICPR, 2021. 6

    2021

  37. [45]

    Vital: Visual tracking via adversarial learning

    Yibing Song, Chao Ma, Xiaohe Wu, Lijun Gong, Linchao Bao, Wangmeng Zuo, Chunhua Shen, Rynson WH Lau, and Ming-Hsuan Yang. Vital: Visual tracking via adversarial learning. In CVPR, 2018. 5

    2018

  38. [46]

    Fast-itpn: Integrally pre- trained transformer pyramid network with token migration

    Yunjie Tian, Lingxi Xie, Jihao Qiu, Jianbin Jiao, Yaowei Wang, Qi Tian, and Qixiang Ye. Fast-itpn: Integrally pre- trained transformer pyramid network with token migration. TPAMI, 2024. 5

    2024

  39. [47]

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. NeurIPS, 35:10078– 10093, 2022. 4, 7

    2022

  40. [48]

    Paul V oigtlaender, Jonathon Luiten, Philip H. S. Torr, and Bastian Leibe. Siam R-CNN: Visual tracking by re- detection. In CVPR, 2020. 5

    2020

  41. [49]

    Cross-modal pattern- propagation for rgb-t tracking

    Chaoqun Wang, Chunyan Xu, Zhen Cui, Ling Zhou, Tong Zhang, Xiaoya Zhang, and Jian Yang. Cross-modal pattern- propagation for rgb-t tracking. In CVPR, 2020. 6

    2020

  42. [50]

    Learnable cross- modal knowledge distillation for multi-modal learning with missing modality

    Hu Wang, Congbo Ma, Jianpeng Zhang, Yuan Zhang, Jodie Avery, Louise Hull, and Gustavo Carneiro. Learnable cross- modal knowledge distillation for multi-modal learning with missing modality. In MICCAI, pages 216–226. Springer,

  43. [51]

    Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, and Philip H. S. Torr. Fast online object tracking and segmenta- tion: A unifying approach. In CVPR, 2019. 5

    2019

  44. [52]

    Visev- ent: Reliable object tracking via collaboration of frame and event flows

    Xiao Wang, Jianing Li, Lin Zhu, Zhipeng Zhang, Zhe Chen, Xin Li, Yaowei Wang, Yonghong Tian, and Feng Wu. Visev- ent: Reliable object tracking via collaboration of frame and event flows. IEEE Transactions on Cybernetics, 2023. 1

    2023

  45. [53]

    Visev- ent: Reliable object tracking via collaboration of frame and event flows

    Xiao Wang, Jianing Li, Lin Zhu, Zhipeng Zhang, Zhe Chen, Xin Li, Yaowei Wang, Yonghong Tian, and Feng Wu. Visev- ent: Reliable object tracking via collaboration of frame and event flows. TCYB, pages 1–14, 2023. 2, 5, 6, 7

    2023

  46. [54]

    Event stream-based visual object tracking: A high-resolution benchmark dataset and a novel baseline

    Xiao Wang, Shiao Wang, Chuanming Tang, Lin Zhu, Bo Jiang, Yonghong Tian, and Jin Tang. Event stream-based visual object tracking: A high-resolution benchmark dataset and a novel baseline. In CVPR, 2024. 1

    2024

  47. [55]

    Autoregressive visual tracking

    Xing Wei, Yifan Bai, Yongchao Zheng, Dahu Shi, and Yi- hong Gong. Autoregressive visual tracking. In CVPR, 2023. 1, 2

    2023

  48. [56]

    Towards good practices for missing modality robust action recognition

    Sangmin Woo, Sumin Lee, Yeonju Park, Muhammad Adi Nugroho, and Changick Kim. Towards good practices for missing modality robust action recognition. In AAAI, pages 2776–2784, 2023. 2

    2023

  49. [57]

    Dropmae: Masked autoen- coders with spatial-attention dropout for tracking tasks

    Qiangqiang Wu, Tianyu Yang, Ziquan Liu, Baoyuan Wu, Ying Shan, and Antoni B Chan. Dropmae: Masked autoen- coders with spatial-attention dropout for tracking tasks. In CVPR, 2023. 4

    2023

  50. [58]

    Single-model and any-modality for video ob- ject tracking

    Zongwei Wu, Jilai Zheng, Xiangxuan Ren, Florin-Alexandru Vasluianu, Chao Ma, Danda Pani Paudel, Luc Van Gool, and Radu Timofte. Single-model and any-modality for video ob- ject tracking. In CVPR, 2024. 2, 5, 6

    2024

  51. [59]

    Attribute-based progressive fusion network for rgbt tracking

    Yun Xiao, Mengmeng Yang, Chenglong Li, Lei Liu, and Jin Tang. Attribute-based progressive fusion network for rgbt tracking. In AAAI, 2022. 6

    2022

  52. [60]

    Correlation-aware deep tracking

    Fei Xie, Chunyu Wang, Guangting Wang, Yue Cao, Wankou Yang, and Wenjun Zeng. Correlation-aware deep tracking. In CVPR, 2022. 6

    2022

  53. [61]

    Learning spatio-temporal transformer for vi- sual tracking

    Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu. Learning spatio-temporal transformer for vi- sual tracking. In ICCV, 2021. 5

    2021

  54. [62]

    Depthtrack: Un- veiling the power of rgbd tracking

    Song Yan, Jinyu Yang, Jani K ¨apyl¨a, Feng Zheng, Ale ˇs Leonardis, and Joni-Kristian K ¨am¨ar¨ainen. Depthtrack: Un- veiling the power of rgbd tracking. In ICCV, 2021. 1, 5, 6, 7

    2021

  55. [63]

    Prompting for multi-modal tracking

    Jinyu Yang, Zhe Li, Feng Zheng, Ales Leonardis, and Jingkuan Song. Prompting for multi-modal tracking. In ACMMM, 2022. 5, 6

    2022

  56. [64]

    Joint feature learning and relation modeling for tracking: A one-stream framework

    Botao Ye, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Joint feature learning and relation modeling for tracking: A one-stream framework. In ECCV, 2022. 2, 5, 6

    2022

  57. [65]

    Flex-moe: Modeling arbitrary modality combination via the flexible mixture-of-experts

    Sukwon Yun, Inyoung Choi, Jie Peng, Yangfan Wu, Jingx- uan Bao, Qiyiwen Zhang, Jiayi Xin, Qi Long, and Tianlong Chen. Flex-moe: Modeling arbitrary modality combination via the flexible mixture-of-experts. In NeurIPS. 2

  58. [66]

    Object tracking in RGB-T videos using modal-aware attention net- work and competitive learning

    Hui Zhang, Lei Zhang, Li Zhuo, and Jing Zhang. Object tracking in RGB-T videos using modal-aware attention net- work and competitive learning. Sensors, page 393, 2020. 6

    2020

  59. [67]

    Multi-modal fusion for end-to-end rgb-t tracking

    Lichao Zhang, Martin Danelljan, Abel Gonzalez-Garcia, Joost Van De Weijer, and Fahad Shahbaz Khan. Multi-modal fusion for end-to-end rgb-t tracking. In ICCVW, 2019. 6

    2019

  60. [68]

    Jointly modeling motion and appearance cues for robust RGB-T tracking

    Pengyu Zhang, Jie Zhao, Chunjuan Bo, Dong Wang, Huchuan Lu, and Xiaoyun Yang. Jointly modeling motion and appearance cues for robust RGB-T tracking. TIP, pages 3335–3347, 2021. 6

    2021

  61. [69]

    Visible-thermal uav tracking: A large-scale benchmark and new baseline

    Pengyu Zhang, Jie Zhao, Dong Wang, Huchuan Lu, and Xiang Ruan. Visible-thermal uav tracking: A large-scale benchmark and new baseline. In CVPR, 2022. 6

    2022

  62. [70]

    Missing modality imagination network for emotion recognition with uncertain missing modalities

    Jinming Zhao, Ruichen Li, and Qin Jin. Missing modality imagination network for emotion recognition with uncertain missing modalities. In ACL, pages 2608–2618, 2021. 1, 2

    2021

  63. [71]

    Visual prompt multi-modal tracking

    Jiawen Zhu, Simiao Lai, Xin Chen, Dong Wang, and Huchuan Lu. Visual prompt multi-modal tracking. InCVPR,

  64. [72]

    Llama-moe: Building mixture-of-experts from llama with continual pre-training

    Tong Zhu, Xiaoye Qu, Daize Dong, Jiacheng Ruan, Jingqi Tong, Conghui He, and Yu Cheng. Llama-moe: Building mixture-of-experts from llama with continual pre-training. arXiv preprint arXiv:2406.16554, 2024. 2

    2024 arXiv

  65. [73]

    Rgbd1k: A large-scale dataset and benchmark for rgb-d object tracking

    Xue-Feng Zhu, Tianyang Xu, Zhangyong Tang, Zucheng Wu, Haodong Liu, Xiao Yang, Xiao-Jun Wu, and Josef Kit- tler. Rgbd1k: A large-scale dataset and benchmark for rgb-d object tracking. In AAAI, 2023. 2, 6

    2023

  66. [74]

    Dense feature aggregation and pruning for RGBT tracking

    Yabin Zhu, Chenglong Li, Bin Luo, Jin Tang, and Xiao Wang. Dense feature aggregation and pruning for RGBT tracking. In ACMMM, 2019. 6

    2019

  67. [75]

    Quality- aware feature aggregation network for robust RGBT track- ing

    Yabin Zhu, Chenglong Li, Jin Tang, and Bin Luo. Quality- aware feature aggregation network for robust RGBT track- ing. TIV, pages 121–130, 2020. 6

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.