Pith. sign in

REVIEW 3 major objections 4 minor 70 references

The paper claims that state-of-the-art tracking comes not from a larger monolithic tracker but from coordinating a matching tracker, a segmentation model, and a vision-language model as tools, with the VLM called only under persistent confl

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 00:11 UTC pith:KPFREK3B

load-bearing objection Genuinely novel coordination framework, but the empirical claims are conditional on code release and validation-based hyperparameters; the VLM arbiter is an unmeasured load-bearing component. the 3 major comments →

arxiv 2608.00847 v1 pith:KPFREK3B submitted 2026-08-01 cs.CV

Models as Tools: An Agentic Coordination Framework for Unified Multimodal Visual Tracking

Wenrui Cai , Yuzhe Li , Qingjie Liu , Yunhong Wang This is my paper
classification cs.CV
keywords visual object trackingagentic tool coordinationfoundation modelsSAM3vision-language repromptingmultimodal trackingparameter-efficient adaptationevent-triggered arbitration
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

ACTrack is a claim about how to use foundation models for visual tracking: instead of adapting one model into a complete tracker, the paper treats a matching-based tracker, SAM3, and a vision-language model as separate tools and coordinates them through a shared target state. A SAM3 Motion Tool re-anchors the tracker's search region with mask-derived motion, a SAM3 Perception Tool detects when the tracker and the motion mask have locked onto different physical objects, and a VLM Reprompt Tool is invoked only when the same conflict persists for five frames. On RGB benchmarks the paper reports that this coordination pushes LaSOT AUC past 80 for the first time and beats the strongest and largest published trackers; on multimodal benchmarks the same policy, with only the matching tool retrained, outperforms prior unified trackers on RGB-T, RGB-E, RGB-D, and RGB-Language data with about 30% of parameters trainable. The underlying message is that failure modes matter more than scale: different models fail in complementary ways, and detecting and arbitrating those disagreements can substitute for further model growth.

Core claim

The central discovery claimed is that an event-triggered tool-coordination policy, rather than any single tracker architecture, is what pushes performance past current state of the art. ACTrack maintains a shared target state and invokes tools in a fixed order: SAM3 propagates a mask and converts it to a motion box; this box re-anchors the search region of the matching tracker; SAM3's perception branch then assigns both the tracker's box and the motion box to detected instances, producing a conflict flag when they belong to different objects. Only after the same conflict persists for K frames does ACTrack query the VLM, which returns Matching, Motion, or Uncertain; a Matching verdict injects

What carries the argument

The load-bearing mechanism is the shared target state with an event-triggered conflict signal. Each frame, the Motion Tool (SAM3 with Kalman-filtered mask selection) outputs a mask-derived motion box; the Perception Tool (SAM3's detection branch) assigns both that box and the Instance Matching Tool's box to detected instance identities and sets a conflict flag when the two tools lock onto different objects; a sliding buffer triggers the VLM Reprompt Tool only when the same conflict persists for K consecutive frames. The VLM's three-way verdict decides whether to accept the tracker's box, keep the motion box, or leave the default decision, and a Matching verdict refreshes SAM3's prompt. This

Load-bearing premise

The whole error-correction path assumes the VLM's three-way verdict is correct, but the paper reports no accuracy, prompt template, temperature, or reliability check for it; a wrong 'Matching' verdict would inject a drifted box into SAM3's memory and turn a recoverable conflict into long-term corruption.

What would settle it

On the LaSOT conflict frames, replace the VLM with (a) a random verdict and (b) an oracle that always picks the tool matching ground truth. If random verdicts preserve most of the reported AUC gain, or if the oracle does not improve over the VLM, the claim that semantic arbitration drives performance is refuted. A cheaper check: measure the VLM's agreement with ground-truth identity on a sample of triggered frames; if agreement is near chance, the Reprompt Tool cannot be the cause of the gains.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the claim is right, state-of-the-art RGB tracking no longer requires a single giant tracker: ACTrack-B224, assembled from frozen off-the-shelf components, is reported as the first method to exceed 80 AUC on LaSOT, and ACTrack-L384 raises it further.
  • Event-triggered VLM invocation keeps semantic reasoning cheap: the VLM is called only a few times per sequence on average, with each call around 2.1 seconds, so persistent-conflict arbitration adds little to per-frame cost.
  • The same coordination policy transfers across modalities by swapping only the Instance Matching Tool; ACTrack-EM reports state-of-the-art results on RGB-T, RGB-E, RGB-D, and RGB-Language benchmarks with 173.3M trainable parameters.
  • The framework is tracker-agnostic: replacing the Instance Matching Tool with different underlying trackers consistently improves them, so the coordination policy, not the specific matcher, is claimed to be the source of the gains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment the paper leaves implicit is measuring the VLM's arbitration accuracy against ground truth on triggered frames; the framework's ceiling may be set by that accuracy.
  • If VLM reliability varies across domains, a confidence-gated fallback or a human-in-the-loop review for the Uncertain verdict would be a direct extension of the same event-triggered design.
  • Beyond tracking, the pattern of a fast matcher, a precise segmenter, and a sparse semantic judge arbitrating persistent disagreements could transfer to other perception tasks with similar complementary failure modes.
  • Because the VLM is stochastic and closed, the reported gains may vary across runs; testing the pipeline across sampling temperatures and repeated seeds would show how much of the result depends on the particular verdicts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. ACTrack proposes an agentic coordination framework for visual tracking, replacing monolithic trackers with four tools: an off-the-shelf matching tracker (MCITrack) as Instance Matching Tool, a SAM3-based Motion Tool providing mask-derived motion priors and search-region re-anchoring, a SAM3 Perception Tool detecting instance conflicts, and a Seed 2.0 Pro VLM Reprompt Tool invoked after K consecutive conflicts to arbitrate between tracker and motion boxes. The paper reports state-of-the-art results on eight RGB benchmarks and strong results on RGB-T/D/E/L benchmarks, with a parameter-efficient multimodal variant ACTrack-EM that uses about 31% trainable parameters. The core claimed contribution is the event-triggered tool-coordination policy rather than a new tracker architecture.

Significance. The conceptual direction is well motivated: decomposing tracking into complementary tools and using a VLM only for sparse identity arbitration is a sensible departure from monolithic foundation-model trackers. The ablation in Table 9 shows that each added tool contributes, and the ACTrack-EM design usefully demonstrates that multimodal adaptation can be isolated inside the matching tool. If the headline numbers are out-of-sample and reproducible, the paper would be an important step for the tracking community. At present those conditions are not met: the VLM tool is an unspecified closed model, key hyperparameters are selected on the test benchmarks, and no code or repeated-run statistics are provided. The significance is therefore conditional on the evidence being completed.

major comments (3)
  1. [§3.5, Eq. (4), Algorithm 2 (lines 19–27)] The VLM Reprompt Tool is load-bearing but unverified. Eq. (4) defines the verdict y_t in {Matching, Motion, Uncertain} from Seed 2.0 Pro, but the paper gives no prompt template, image resolution, sampling temperature, or measured verdict accuracy. A 'Matching' verdict triggers S_t ← Reprompt(S_{t-1}, b^r_t), writing the tracker box into SAM3 memory; if the tracker has drifted to a distractor, this can contaminate all subsequent mask propagation. Table 11 reports only the trigger interval K and call counts, not arbitration accuracy. To support the claim that ACTrack provides long-term error correction, the authors need to specify the protocol, validate VLM verdicts against ground truth on conflict frames, and analyze cases where a wrong 'Matching' verdict occurs. Without this, the +0.5 AUC gain attributed to Reprompt in Table 9 is not trustworthy.
  2. [§4.1 and §4.4 (Tables 10, 11, 13)] The main configuration is selected on the same test splits that are later reported as achievements. Table 10 varies rho on the LaSOT, TNL2K, VisEvent, LasHeR, and DepthTrack test sets and then fixes rho=2.5/1.5; Table 11 chooses K=5 on the same test sets; Table 13 chooses W_m and P_obj on the LaSOT test split. No validation split or nested selection procedure is described. Since several of the differences are small (e.g., rho: 80.1 vs 79.6; K: 80.1 vs 79.5), this test-set selection can be sufficient to determine the reported optimum. Please either add a validation-based selection protocol or report the full ranges as evidence of insensitivity, and state explicitly which numbers are out-of-sample.
  3. [§4.1 and Tables 2–7] No code, checkpoints, or repeated-run statistics are provided for the SOTA numbers. ACTrack-B/L are described as assembled from off-the-shelf checkpoints, and the VLM pipeline is stochastic, yet no random-seed variations or error bars are reported. Claims such as 'first to push AUC beyond 80' on LaSOT and a >10 AUC gain on VisEvent require either deterministic inference or mean±std over multiple runs. The authors should release the code and evaluation scripts, and report stability of the VLM-dependent pipeline.
minor comments (4)
  1. [§2.4] Typo: 'ubstantially underperforms' should be 'substantially underperforms'.
  2. [Figure 4] The caption and axis text contain garbled '/uni000...' tokens that render as unreadable text; the figure needs to be regenerated.
  3. [Table 13] The row 'w/KF (W_m=0.15, P_obj=0.269)' introduces an unexplained objectness threshold value. The implementation details in §4.1 fix P_obj=0.5; please explain how the 0.269 threshold was obtained and why it is included.
  4. [§4.2, Table 3] The note that GOT-10k 'may no longer strictly conform to its one-shot protocol' is buried in the text. Since Table 3 is used as a comparison table, the training data used by each baseline and by ACTrack should be clearly disclosed so readers can judge comparability.

Circularity Check

1 steps flagged

Benchmark 'predictions' partly fitted: the flagship LaSOT 80.1 is the argmax of a test-set hyperparameter grid (rho, K, W_m, P_obj), not an out-of-sample result; no derivation-chain circularity found otherwise.

specific steps
  1. fitted input called prediction [Sec. 4.1 'Tool coordination configuration'; Sec. 4.4 'The Impact of Search Region Cropping Factor...' and 'Triggering Interval...' (Tables 10, 11, 13); Sec. 4.2 LaSOT results]
    "On the RGB dataset, a cropping factor of 2.5 yields better performance, whereas on datasets of other modalities, a cropping factor of 1.5 yields better performance. ... The results indicate that triggering on five consecutive conflicting frames achieves the best accuracy ... we adopt a crop expansion ratio of rho=2.5 for RGB and RGB-Language datasets ... a motion-prior weight of W_m=0.15 ... for 5 consecutive frames, the VLM Reprompt Tool is invoked"

    These values are not chosen from a held-out validation set or from a stated principle; they are the argmax of the same test splits later reported as achievements. Table 10 on LaSOT test gives 80.1 for rho=2.5 vs 79.8/79.6/79.6 for rho=1.5/2.0/4.0; Table 11 gives 80.1 for K=5 vs 79.5/79.5 for K=10/20; Table 13 selects W_m=0.15, P_obj=0.5 as the best 'SAM3 + Instance Matching Tool' AUC (78.9) on LaSOT test. Sec. 4.2 then reports 'ACTrack-B224 attains an AUC of 80.1' as a headline SOTA. The reported number is therefore the maximum of a test-set grid search, not an independent out-of-sample prediction of a fixed method: the 'prediction' is fitted by construction.

full rationale

The framework itself is not derived from the benchmarks: the coordination policy, Algorithm 2, and the tool interfaces are specified independently of any dataset, and the RGB variants use off-the-shelf MCITrack checkpoints. No self-citation chain is load-bearing: SPMTrack/HIPTrack/Uni-MDTrack are cited as related work or comparison, not as the justification for ACTrack's design. The VLM arbitration being an unmeasured black box is a robustness/correctness concern but not a circularity, since the VLM verdict is an external input rather than an output of the derivation. The one genuine circular element is the evaluation practice: the configuration (rho, K, W_m, P_obj) is selected by maximizing the LaSOT test AUC in Tables 10/11/13, and that same selected value is then presented as the headline prediction in Sec. 4.2. This makes the flagship 'first AUC beyond 80' claim fitted, while the broader claim of coordination gains retains independent content; hence a moderate score of 5 rather than a higher one.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The framework is an empirical composition of existing models. Its main 'free' inputs are the coordination hyperparameters, all chosen on the test benchmarks, and two strong domain assumptions: SAM3 instance detections are reliable for conflict detection, and the closed VLM arbitrates correctly enough to correct SAM3 memory. No new physical entities are introduced.

free parameters (6)
  • Search region expansion rate rho = 2.5 for RGB/RGB-L, 1.5 for RGB-D/T/E
    Chosen from test-set ablations in Table 10 to maximize AUC/F-score on LaSOT, TNL2K, VisEvent, LasHeR, DepthTrack.
  • VLM trigger interval K = 5 consecutive conflict frames
    Chosen from Table 11 on test splits to maximize AUC; higher K reduces VLM calls but hurts LaSOT/DepthTrack.
  • Motion-prior weight W_m = 0.15
    Selected via standalone and in-framework ablations on LaSOT test (Table 13); the paper notes the best standalone SAM3 weight is not the best in-framework weight.
  • Objectness threshold P_obj = 0.5 (sigmoid) with warm-up 15 frames at IoU>0.3
    Set in Sec 4.1 and ablated in Table 13 (P_obj=0.269 also tested).
  • Auxiliary template update threshold = center confidence > 0.9, interval floor(n/5)
    Selected via ablation on LaSOT test (Table 15).
  • LoRA rank r and scale alpha = r=64, alpha=64
    Hyperparameters for ACTrack-EM; not ablated in the paper.
axioms (4)
  • standard math Constant-velocity Kalman filter with box state x in R^8 adequately models target motion
    Used in Sec 3.3, Eq. (2), following DeepSORT [47]; assumes roughly constant box center, aspect, height, and velocities between frames.
  • domain assumption SAM3's object detections are reliable enough that assigning box centers to detection masks via Algorithm 1 yields correct instance identities
    Invoked in Sec 3.3 and Algorithm 1; if detections miss instances or produce false merges, the conflict signal C_t is wrong and the default policy can pick the wrong tool.
  • ad hoc to paper The VLM returns a correct three-way identity verdict from the four-panel image
    Sec 3.5, Eq. (4): the entire error-correction path depends on y_t in {Matching, Motion, Uncertain}; no calibration, prompt details, or error rate are reported for Seed 2.0 Pro.
  • ad hoc to paper Persistent conflict over K frames is a necessary and sufficient trigger for semantic arbitration
    Sec 3.5 and Algorithm 2 assume that K identical conflict flags indicate genuine identity disagreement and that short (<=K-1) conflicts should be resolved by the default policy alone.

pith-pipeline@v1.3.0-alltime-deepseek · 30600 in / 14040 out tokens · 138540 ms · 2026-08-05T00:11:06.257470+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Models as Tools: An Agentic Coordination Framework for Unified Multimodal Visual Tracking." pith.science (2026). https://pith.science/paper/KPFREK3B

@misc{pith2026260800847,
  author       = {Pith},
  title        = {Pith review of: Models as Tools: An Agentic Coordination Framework for Unified Multimodal Visual Tracking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KPFREK3B}},
  note         = {Machine review of arXiv:2608.00847}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Most current visual trackers adopt a matching-based architecture trained exclusively on tracking datasets, whose performance gains depend heavily on the length of the input context, and have now reached a bottleneck. While high-performance tracking increasingly relies on foundation models, existing methods use them monolithically, adapting a foundation model into a tracker or modify a segmentation foundation model into a tracking pipeline, which fails to exploit complementary strengths. Matching-based trackers excel at instance-level correspondence but lack semantic discrimination and fine-grained foreground perception, whereas segmentation foundation models produce precise masks yet struggle with instance discrimination and multimodal extension. Both paradigms also lack error-correction capabilities for long-term tracking. To address these issues, we propose ACTrack, an agentic coordination framework that treats heterogeneous models as invocable tools under an event-triggered mechanism. ACTrack coordinates a Tracker-based Instance Matching Tool for target discrimination, a SAM3 Motion Tool for mask-derived motion priors, a SAM3 Perception Tool for detecting distractors and instance-conflict cues, and a VLM Reprompt Tool activated only under persistent conflict to mitigate error accumulation. We design a complete tool-invocation trigger mechanism and an inter-tool coordination mechanism, enabling the complementary strengths of different model tools to be fully integrated. Experiments show that ACTrack substantially surpasses the strongest and the largest trackers on eight RGB benchmarks. Furthermore, a parameter-efficient adaptation strategy enables parameter sharing and reuse across tools, achieving unified multimodal tracking with only 30\% trainable parameters while substantially outperforming prior methods on multimodal benchmarks such as LasHeR, VisEvent, TNL2K, and DepthTrack.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

70 extracted references · 61 canonical work pages · 1 internal anchor

  1. [1]

    In: European Conference on Computer Vision, pp

    Ye, B., Chang, H., Ma, B., Shan, S., Chen, X.: Joint feature learning and relation model- ing for tracking: A one-stream framework. In: European Conference on Computer Vision, pp. 341–357 (2022). Springer

    work page 2022

  2. [2]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision 21 and Pattern Recognition (CVPR), pp

    Cui, Y., Jiang, C., Wang, L., Wu, G.: Mixformer: End-to-end tracking with itera- tive mixed attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision 21 and Pattern Recognition (CVPR), pp. 13608– 13618 (2022)

    work page 2022

  3. [3]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Chen, X., Peng, H., Wang, D., Lu, H., Hu, H.: Seqtrack: Sequence to sequence learning for visual object tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14572– 14581 (2023)

    work page 2023

  4. [4]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Wei, X., Bai, Y., Zheng, Y., Shi, D., Gong, Y.: Autoregressive visual tracking. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9697–9706 (2023)

    work page 2023

  5. [5]

    Proceedings of the AAAI Conference on Arti- ficial Intelligence38(7), 7588–7596 (2024) ht tps://doi.org/10.1609/aaai.v38i7.28591

    Zheng, Y., Zhong, B., Liang, Q., Mo, Z., Zhang, S., Li, X.: Odtrack: Online dense temporal token learning for visual tracking. Proceedings of the AAAI Conference on Arti- ficial Intelligence38(7), 7588–7596 (2024) ht tps://doi.org/10.1609/aaai.v38i7.28591

    doi:10.1609/aaai.v38i7.28591 2024

  6. [6]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Bai, Y., Zhao, Z., Gong, Y., Wei, X.: Artrackv2: Prompting autoregressive tracker where to look and how to describe. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19048–19057 (2024)

    work page 2024

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pp

    Xie, J., Zhong, B., Mo, Z., Zhang, S., Shi, L., Song, S., Ji, R.: Autoregressive queries for adaptive tracking with spatio-temporal trans- formers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pp. 19300–19309 (2024)

    work page 2024

  8. [8]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Cai, W., Liu, Q., Wang, Y.: Hiptrack: Visual tracking with historical prompts. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19258–19267 (2024)

    work page 2024

  9. [9]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Kang, B., Chen, X., Lai, S., Liu, Y., Liu, Y., Wang, D.: Exploring enhanced contextual information for video-level object tracking. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pp. 4194–4202 (2025)

    work page 2025

  10. [10]

    In: Pro- ceedings of the AAAI Conference on Artificial Intelligence, vol

    Li, X., Zhong, B., Liang, Q., Li, G., Mo, Z., Song, S.: Mambalct: Boosting tracking via long-term context state space model. In: Pro- ceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pp. 4986–4994 (2025)

    work page 2025

  11. [11]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Wu, W., Liang, Q., Zhong, B., Xia, H., Mo, Z., Song, S.: An efficient token compression framework for visual object tracking. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6857–6867 (2026)

    work page 2026

  12. [12]

    Transactions on Machine Learning Research Journal, 1–31 (2024)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features with- out supervision. Transactions on Machine Learning Research Journal, 1–31 (2024)

    work page 2024

  13. [13]

    In: Leonardis, A., Ricci, E., Roth, S., Rus- sakovsky, O., Sattler, T., Varol, G

    Lin, L., Fan, H., Zhang, Z., Wang, Y., Xu, Y., Ling, H.: Tracking meets lora: Faster training, larger model, stronger performance. In: Leonardis, A., Ricci, E., Roth, S., Rus- sakovsky, O., Sattler, T., Varol, G. (eds.) Computer Vision – ECCV 2024, pp. 300–318. Springer, Cham (2025)

    work page 2024

  14. [14]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2025)

    Lin, L., Fan, H., Zhang, Z., Huang, Y., Wang, Y., Xu, Y., Ling, H.: Loratv2: Enabling low-cost temporal modeling in one-stream trackers. In: Advances in Neural Information Processing Systems (NeurIPS) (2025)

    work page 2025

  15. [15]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Cai, W., Liu, Q., Wang, Y.: Spmtrack: Spatio-temporal parameter-efficient fine- tuning with mixture of experts for scalable visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16871–16881 (2025)

    work page 2025

  16. [16]

    In: The Thirteenth International Confer- ence on Learning Representations (2025)

    Ravi, N., Gabeur, V., Hu, Y.-T., Hu, R., Ryali, C., Ma, T., Khedr, H., R¨ adle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K.V., Carion, N., Wu, C.-Y., Gir- shick, R., Dollar, P., Feichtenhofer, C.: Sam 2: Segment anything in images and videos. In: The Thirteenth International Confer- ence on Learning Representations (2025). https://open...

    work page 2025

  17. [17]

    https: //arxiv.org/abs/2511.16719

    Carion, N., Gustafson, L., Hu, Y.-T., Deb- nath, S., Hu, R., Suris, D., Ryali, C., Alwala, K.V., Khedr, H., Huang, A., Lei, J., Ma, T., Guo, B., Kalla, A., Marks, M., Greer, J., Wang, M., Sun, P., R¨ adle, R., Afouras, T., Mavroudi, E., Xu, K., Wu, T.-H., Zhou, Y., Momeni, L., Hazra, R., Ding, S., Vaze, S., Porcher, F., Li, F., Li, S., Kamath, A., Cheng, ...

    Pith/arXiv arXiv 2026

  18. [18]

    IEEE Transactions on Image Processing (2026)

    Yang, C.-Y., Huang, H.-W., Jiang, Z., Chai, W., Hwang, J.-N.: Samurai: Motion-aware memory for training-free visual object track- ing with sam 2. IEEE Transactions on Image Processing (2026)

    work page 2026

  19. [19]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Videnovic, J., Lukezic, A., Kristan, M.: A distractor-aware memory for visual object tracking with sam2. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24255– 24264 (2025)

    work page 2025

  20. [20]

    http s://arxiv.org/abs/2511.21631

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, ...

    Pith/arXiv arXiv 2025

  21. [21]

    In: Advances in Neural Information Processing Systems (2024)

    Sun, Y., Yu, F., Chen, S., Zhang, Y., Huang, J., Li, Y., Li, C., Wang, C.: Chattracker: Enhancing visual tracking performance via chatting with multimodal large language model. In: Advances in Neural Information Processing Systems (2024)

    work page 2024

  22. [22]

    In: European Con- ference on Computer Vision, pp

    Yu, E., Zhao, L., Wei, Y., Yang, J., Wu, D., Kong, L., Wei, H., Wang, T., Ge, Z., Zhang, X.,et al.: Merlin: Empowering multimodal llms with foresight minds. In: European Con- ference on Computer Vision, pp. 425–443 (2024). Springer

    work page 2024

  23. [23]

    https://arxiv.org/abs/2512.22799

    Wang, J., Zhou, K., Wu, Z., Ji, K., Huang, D., Zheng, Y.: VPTracker: Global Vision- Language Tracking via Visual Prompt (2026). https://arxiv.org/abs/2512.22799

    Pith/arXiv arXiv 2026

  24. [24]

    https://arxiv.org/abs/2506.172 52

    Huang, Z., Ban, Y., Fu, L., Li, X., Dai, Z., Li, J., Wang, D.: Adaptive Batch-Wise Sample Scheduling for Direct Preference Optimiza- tion (2026). https://arxiv.org/abs/2506.172 52

    work page 2026

  25. [25]

    https: //arxiv.org/abs/2602.08354

    Huang, Z., Xia, X., Ren, Y., Zheng, J., Wang, X., Zhang, Z., Xie, H., Liang, S., Chen, Z., Xiao, X., Zhuang, F., Li, J., Ban, Y., Wang, D.: Does Your Reasoning Model Implicitly Know When to Stop Thinking? (2026). https: //arxiv.org/abs/2602.08354

    Pith/arXiv arXiv 2026

  26. [26]

    https://arxiv.org/abs/2601.226 64

    Huang, Z., Xia, X., Ren, Y., Zheng, J., Xiao, X., Xie, H., Huaqiu, L., Liang, S., Dai, Z., Zhuang, F., Li, J., Ban, Y., Wang, D.: Real- Time Aligned Reward Model beyond Seman- tics (2026). https://arxiv.org/abs/2601.226 64

    work page 2026

  27. [27]

    arXiv preprint arXiv:2304.14394 (2023)

    Chen, X., Kang, B., Zhu, J., Wang, D., Peng, H., Lu, H.: Unified sequence- to-sequence learning for single-and multi- modal visual object tracking. arXiv preprint arXiv:2304.14394 (2023)

    Pith/arXiv arXiv 2023

  28. [28]

    In: AAAI (2025)

    Chen, X., Kang, B., Geng, W., Zhu, J., Liu, Y., Wang, D., Lu, H.: Sutrack: Towards sim- ple and unified single object tracking. In: AAAI (2025)

    work page 2025

  29. [29]

    https://arxiv.org/abs/2603 .14452

    Cai, W., Lu, Z., Li, Y., Feng, Y., Zhang, J., Liu, Q., Wang, Y.: Uni-MDTrack: Learn- ing Decoupled Memory and Dynamic States for Parameter-Efficient Visual Tracking in All Modality (2026). https://arxiv.org/abs/2603 .14452

    work page 2026

  30. [30]

    In: ECCV, pp

    Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.: Fully-convolutional siamese networks for object tracking. In: ECCV, pp. 850–865 (2016) 23

    work page 2016

  31. [31]

    In: CVPR, pp

    Li, B., Yan, J., Wu, W., Zhu, Z., Hu, X.: High performance visual tracking with siamese region proposal network. In: CVPR, pp. 8971–8980 (2018)

    work page 2018

  32. [32]

    In: CVPR, pp

    Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., Lu, H.: Transformer tracking. In: CVPR, pp. 8126–8135 (2021)

    work page 2021

  33. [33]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp

    Yan, B., Peng, H., Fu, J., Wang, D., Lu, H.: Learning spatio-temporal trans- former for visual tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10448–10457 (2021)

    work page 2021

  34. [34]

    IEEE Transactions on Image Processing31, 392– 404 (2022) https://doi.org/10.1109/TIP.20 21.3130533

    Li, C., Xue, W., Jia, Y., Qu, Z., Luo, B., Tang, J., Sun, D.: Lasher: A large-scale high- diversity benchmark for rgbt tracking. IEEE Transactions on Image Processing31, 392– 404 (2022) https://doi.org/10.1109/TIP.20 21.3130533

    work page doi:10.1109/tip.20 2022

  35. [35]

    In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV), pp

    Yan, S., Yang, J., K¨ apyl¨ a, J., Zheng, F., Leonardis, A., K¨ am¨ ar¨ ainen, J.-K.: Depth- track: Unveiling the power of rgbd tracking. In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV), pp. 10725–10733 (2021)

    work page 2021

  36. [36]

    IEEE Transactions on Cybernetics54(3), 1997–2010 (2024) https: //doi.org/10.1109/TCYB.2023.3318601

    Wang, X., Li, J., Zhu, L., Zhang, Z., Chen, Z., Li, X., Wang, Y., Tian, Y., Wu, F.: Visevent: Reliable object tracking via collaboration of frame and event flows. IEEE Transactions on Cybernetics54(3), 1997–2010 (2024) https: //doi.org/10.1109/TCYB.2023.3318601

    work page arXiv 1997

  37. [37]

    In: Proceedings of the 30th ACM International Conference on Multimedia, pp

    Yang, J., Li, Z., Zheng, F., Leonardis, A., Song, J.: Prompting for multi-modal track- ing. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 3492–3500 (2022)

    work page 2022

  38. [38]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Zhu, J., Lai, S., Chen, X., Wang, D., Lu, H.: Visual prompt multi-modal tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9516–9526 (2023)

    work page 2023

  39. [39]

    arXiv preprint arXiv:2406.20024 (2024)

    Chen, Y., Wang, L.: emoe-tracker: Envi- ronmental moe-based transformer for robust event-guided object tracking. arXiv preprint arXiv:2406.20024 (2024)

    Pith/arXiv arXiv 2024

  40. [40]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Hou, X., Xing, J., Qian, Y., Guo, Y., Xin, S., Chen, J., Tang, K., Wang, M., Jiang, Z., Liu, L., Liu, Y.: Sdstrack: Self-distillation symmetric adapter learning for multi-modal visual object tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26551– 26561 (2024)

    work page 2024

  41. [41]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Hong, L., Yan, S., Zhang, R., Li, W., Zhou, X., Guo, P., Jiang, K., Chen, Y., Li, J., Chen, Z., Zhang, W.: Onetracker: Unifying visual object tracking with foundation mod- els and efficient tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19079– 19091 (2024)

    work page 2024

  42. [42]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Wu, Z., Zheng, J., Ren, X., Vasluianu, F.- A., Ma, C., Paudel, D.P., Van Gool, L., Timofte, R.: Single-model and any-modality for video object tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19156–19166 (2024)

    work page 2024

  43. [43]

    Elysium: Exploring Object-level Perception in Videos via MLLM

    Wang, H., Ye, Y., Wang, Y., Nie, Y., Huang, C.: Elysium: Exploring Object-level Percep- tion in Videos via MLLM (2024). https://ar xiv.org/abs/2403.16558

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  44. [44]

    In: CVPR, pp

    Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., Ling, H.: Lasot: A high-quality benchmark for large- scale single object tracking. In: CVPR, pp. 5374–5383 (2019)

    work page 2019

  45. [45]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pp

    Wang, X., Shu, X., Zhang, Z., Jiang, B., Wang, Y., Tian, Y., Wu, F.: Towards more flexible and accurate object tracking with natural language: Algorithms and bench- mark. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pp. 13763–13773 (2021)

    work page 2021

  46. [46]

    In: CVPR, pp

    Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., Yan, J.: Siamrpn++: Evolution of siamese visual tracking with very deep networks. In: CVPR, pp. 4282–4291 (2019)

    work page 2019

  47. [47]

    In: IEEE International Conference on Image Processing (ICIP), pp

    Wojke, N., Bewley, A., Paulus, D.: Simple 24 online and realtime tracking with a deep association metric. In: IEEE International Conference on Image Processing (ICIP), pp. 3645–3649 (2017). IEEE

    work page 2017

  48. [48]

    0 model card: Towards intelligence frontier for real-world complexity

    Seed, B.: Seed2. 0 model card: Towards intelligence frontier for real-world complexity. Available at ByteDance Seed Model Cards (2026)

    work page 2026

  49. [49]

    https://doi.org/10.4 8550/arXiv.2507.21732

    Xu, Q., Zhu, L., Liu, C., Lin, G., Long, C., Li, Z., Zhao, R.: SAMITE: Position Prompted SAM2 with Calibrated Memory for Visual Object Tracking (2025). https://doi.org/10.4 8550/arXiv.2507.21732 . https://arxiv.org/ abs/2507.21732

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.21732 2025

  50. [50]

    https://arxiv.org/abs/2605.073 79

    Chen, X., Sun, C., Xu, J., Peng, H., Wang, D., Lu, H., Ma, K.: RELO: Reinforcement Learning to Localize for Visual Object Track- ing (2026). https://arxiv.org/abs/2605.073 79

    work page 2026

  51. [51]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Liang, S., Bai, Y., Gong, Y., Wei, X.: Autore- gressive sequential pretraining for visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7254–7264 (2025)

    work page 2025

  52. [52]

    TPAMI (2019)

    Huang, L., Zhao, X., Huang, K.: Got-10k: A large high-diversity benchmark for generic object tracking in the wild. TPAMI (2019)

    work page 2019

  53. [53]

    In: ECCV, pp

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ ar, P., Zit- nick, C.L.: Microsoft coco: Common objects in context. In: ECCV, pp. 740–755 (2014)

    work page 2014

  54. [54]

    In: ECCV, pp

    Muller, M., Bibi, A., Giancola, S., Alsubaihi, S., Ghanem, B.: Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. In: ECCV, pp. 300–317 (2018)

    work page 2018

  55. [55]

    In: Glober- son, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C

    Peng, L., Gao, J., Liu, X., Li, W., Dong, S., Zhang, Z., Fan, H., Zhang, L.: Vasttrack: Vast category visual object tracking. In: Glober- son, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C. (eds.) Advances in Neural Information Processing Systems, vol. 37, pp. 130797–130818 (2024). https://proceedings.neurips.cc/paper files/p ap...

    work page 2024

  56. [56]

    In: 7th International Conference on Learning Representations, ICLR (2019)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: 7th International Conference on Learning Representations, ICLR (2019)

    work page 2019

  57. [57]

    In: CVPR, pp

    Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., Savarese, S.: Gen- eralized intersection over union: A metric and a loss for bounding box regression. In: CVPR, pp. 658–666 (2019)

    work page 2019

  58. [58]

    Ma, Y., Tang, Y., Yang, W., Zhang, T., Zhang, J., Kang, M.: Unifying Visual and Vision-Language Tracking via Contrastive Learning (2024)

    work page 2024

  59. [59]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Huang, Y., Li, X., Zhou, Z., Wang, Y., He, Z., Yang, M.-H.: Rtracker: Recoverable track- ing via pn tree structured memory. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19038–19047 (2024)

    work page 2024

  60. [60]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Zhou, L., Zhou, Z., Mao, K., He, Z.: Joint visual grounding and tracking with natural language specification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 23151– 23160 (2023)

    work page 2023

  61. [61]

    TPAMI37(9), 1834–1848 (2015) https://doi.org/10.1109/TPAMI.2014.23882 26

    Wu, Y., Lim, J., Yang, M.: Object tracking benchmark. TPAMI37(9), 1834–1848 (2015) https://doi.org/10.1109/TPAMI.2014.23882 26

    work page arXiv 2015

  62. [62]

    In: ECCV, pp

    Mueller, M., Smith, N., Ghanem, B.: A benchmark and simulator for uav tracking. In: ECCV, pp. 445–461 (2016)

    work page 2016

  63. [63]

    In: Proceedings of the IEEE Inter- national Conference on Computer Vision (ICCV) (2017)

    Kiani Galoogahi, H., Fagg, A., Huang, C., Ramanan, D., Lucey, S.: Need for speed: A benchmark for higher frame rate object tracking. In: Proceedings of the IEEE Inter- national Conference on Computer Vision (ICCV) (2017)

    work page 2017

  64. [64]

    In: Computer 25 Vision–ECCV 2022: 17th European Confer- ence, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII, pp

    Gao, S., Zhou, C., Ma, C., Wang, X., Yuan, J.: Aiatrack: Attention in attention for transformer visual tracking. In: Computer 25 Vision–ECCV 2022: 17th European Confer- ence, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII, pp. 146–164 (2022). Springer

    work page 2022

  65. [65]

    In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp

    Mayer, C., Danelljan, M., Paudel, D.P., Van Gool, L.: Learning target candidate asso- ciation to keep track of what not to track. In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 13444–13454 (2021)

    work page 2021

  66. [66]

    In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp

    Cai, Y., Liu, J., Tang, J., Wu, G.: Robust object modeling for visual tracking. In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9589–9600 (2023)

    work page 2023

  67. [67]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pp

    Wu, Q., Yang, T., Liu, Z., Wu, B., Shan, Y., Chan, A.B.: Dropmae: Masked autoencoders with spatial-attention dropout for tracking tasks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pp. 14561–14571 (2023)

    work page 2023

  68. [68]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Gao, S., Zhou, C., Zhang, J.: Generalized relation modeling for transformer tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18686–18695 (2023)

    work page 2023

  69. [69]

    arXiv preprint arXiv:2507.05899 (2025)

    Tan, Y., Shao, J., Zamfir, E., Li, R., An, Z., Ma, C., Paudel, D., Van Gool, L., Timofte, R., Wu, Z.: What you have is what you track: Adaptive and robust multimodal tracking. arXiv preprint arXiv:2507.05899 (2025)

    Pith/arXiv arXiv 2025

  70. [70]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Hu, X., Tai, Y., Zhao, X., Zhao, C., Zhang, Z., Li, J., Zhong, B., Yang, J.: Exploiting mul- timodal spatial-temporal patterns for video object tracking. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pp. 3581–3589 (2025) 26

    work page 2025