REVIEW 3 major objections 5 minor 1 cited by
MambaLCT: Boosting Tracking via Long-term Context State Space Model
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Visual tracking usually forgets the target's past; MambaLCT argues that a state space model can remember every frame from the first to the present, and that this long-term memory is what pushes performance to new highs on six benchmarks.
desk verdict A well-ablated Mamba-based tracker whose long-term memory claim is undercut by a 2-frame training clip; the gains are real but the mechanism isn't proven. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Context Mamba module is a unidirectional state space model that scans frame features in temporal order. Each frame's features $f_i$ update the hidden state $H_i^t = A H_i^{t-1} + B f_i^t$, and an empty token $T_i$ carries the aggregated history between frames via $H_i^T = A H_i^L + B T_i$, producing $Y_i^T = C H_i^T$ that updates the context token $c_p$. The context token has length 1, so the entire history is compressed into one vector that is injected into the attention of the ucaEncoder, a hierarchical ViT that jointly models template, search frame, and context. This separation of labor — transformer for spatial appearance, Mamba for temporal memory — is what lets the context length grow to the whole sequence at linear cost.
What would settle it
Run inference while resetting the Context Mamba hidden state and context token every $K$ frames (for example every 50 frames), and compare AUC on LaSOT with the full-history version. If performance does not drop, the benchmark gains come from module capacity rather than from long-term memory; alternatively, if a fixed two-frame context window matches full-history performance, the long-term mechanism is not load-bearing.
Extended reading notes
Core claim
The central claim is that long-term context — not just the initial template or a short window — is what a tracker needs to stay accurate through deformation, full occlusion, and fast motion. MambaLCT proposes to build that context by feeding the autoregressive appearance features of each search frame, along with a learned context token, into a Context Mamba module; the selective scanning mechanism accumulates target-related information in a hidden state that carries across frames. The updated context token is then concatenated into the next encoder step, so the template-search relationship is modeled with knowledge of the target's entire history. Experiments on six benchmarks, with ablations on LaSOT, are used to support the claim that this long-term context is responsible for the gains, not the extra parameters or the video sampling alone.
Load-bearing premise
The hidden state of the Context Mamba, trained only on two-frame clips, is assumed to keep accumulating useful target cues when unrolled over the full test sequence, even though the paper states that training and testing phases cannot be unified.
Editorial extensions
If this is right
- MambaLCT-384 reaches 73.6 AUC on LaSOT, 53.3 AUC on LaSOText, 85.2 AUC on TrackingNet, and 58.5 AUC on TNL2K, ahead of the short-context trackers it is compared with.
- The same model runs at 45.3 fps on a Tesla V100, so long-term context can be added without sacrificing real-time speed.
- Attribute-level results on LaSOT show the largest gains in motion blur, full occlusion, and deformation, the situations where remembering the target's history should matter most.
- Ablations indicate that sequence sampling by itself does not help; the improvement appears only when the sampled video is fed through the Context Mamba module, locating the effect in the long-term context mechanism.
- A single context token of length 1 outperforms longer context token sequences, consistent with Mamba's autoregressive accumulation of redundant information.
Reading between the lines
- If the mechanism is doing what the paper claims, then training on longer clips or with a curriculum that gradually extends sequence length should close the train-test gap and push all six benchmarks higher still.
- The unidirectional scan means the context token only knows the past; a bidirectional or chunked scan might help after a target disappears and reappears, since the tracker could then also use evidence from frames after the occlusion.
- Because the context token is a single compact vector, the method should compose naturally with online template update or memory replay: store the token and update it cheaply, without re-encoding the full history.
- The same context-injection idea could transfer to other single-object tracking frameworks or to video object segmentation, where target appearance also drifts over long clips.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MambaLCT, a single-object tracker that augments a HiViT-based encoder with a unidirectional Context Mamba module intended to aggregate target-related cues from the first frame to the current frame. The context tokens are updated by a recurrent state-space update (Eqs. 7-8) and injected into the encoder's attention. Training uses clips of length 2 sampled from GOT-10K, LaSOT, COCO, and TrackingNet, while inference unrolls the recurrence over entire test sequences. The authors report results on LaSOT, LaSOText, GOT-10K, TrackingNet, TNL2K and UAV123, plus ablations on LaSOT, and claim new SOTA on six benchmarks.
Significance. If the long-term context mechanism were validated, the paper would provide a practically interesting combination of Mamba's linear-time recurrence with transformer appearance modeling for visual tracking, and the reported real-time speed (45.3 fps at 384 resolution) and public code and models are useful assets. The empirical coverage is broad, and the ablation in Table 4(a) isolates the Mamba addition from the HiViT backbone. However, as detailed below, the paper's central attribution of gains to long-term memory is not directly supported by the training setup, and the SOTA claim is contradicted by its own GOT-10K numbers.
major comments (3)
- [Training Details; Inference Details; Limitation; Eqs. (7)-(8); Table 4(a)] Training Details state that the video clip sampling length is 2, and the Limitation section concedes that 'the training and testing phases cannot be unified.' The Context Mamba recurrence in Eqs. (7)-(8) is therefore trained only through two frame-to-frame transitions, while at inference it is unrolled over sequences of hundreds of frames. Under this mismatch, the gain of the Mamba module in Table 4(a) (71.8 vs 70.1 for #4 vs #3) cannot be attributed specifically to long-term context; it could come from the added Vim-Small capacity or from two-frame context alone. Please provide evidence that the hidden-state dynamics are stable for lengths far beyond the training clip length (e.g., train with clip lengths 4 or 8, or ablate with variable context at inference), or soften the long-term-context claim accordingly.
- [Abstract; Table 2] The abstract claims 'new SOTA performance on six benchmarks', but Table 2 shows that on GOT-10K, ODTrack achieves AO 77.0, SR0.5 87.9 and SR0.75 75.1, all higher than MambaLCT-384's 76.2, 86.7 and 74.3. Since GOT-10K is one of the six benchmarks listed, the SOTA claim is not supported by the reported numbers. Please correct the claim (e.g., state the benchmarks and resolutions on which the method is actually SOTA).
- [Training Details; Table 2 footnote] The training data are stated as GOT-10K, LaSOT, COCO and TrackingNet, while Table 2's footnote says the star on GOT-10K denotes trackers trained only on GOT-10K. If the GOT-10K protocol requires training only on GOT-10K, the MambaLCT GOT-10K result is not protocol-compliant; if it does not, the footnote is misleading. Please clarify this discrepancy and, if necessary, report a model trained only on GOT-10K for that column.
minor comments (5)
- [Table 4(b)] The header says 'Sampling Length' while the surrounding text discusses the length of the context token cp; please make the label consistent with the variable being varied.
- [Eqs. (7)-(8)] The notation uses t for the token index within a frame and T for the cross-frame transition token, which is easy to confuse; please define both superscripts explicitly.
- [Table 1] The header contains the typo 'Spees' for 'Speed'.
- [Figure 6] The qualitative attention maps are suggestive but not quantitative; consider reporting an average attention-target overlap metric over a larger set of sequences to substantiate the claim that attention becomes more focused with sequence length.
- [Related Work] The statement that ODTrack and AQATrack construct fixed-length context of length 4 should be verified against the original papers, since ODTrack's temporal token is also described as propagating across the whole sequence.
Circularity Check
No circularity: benchmark claims are empirical comparisons, not derived from fitted parameters or self-citation chains.
full rationale
MambaLCT's central claims—that long-term context aggregation improves robustness and that it achieves new SOTA on six benchmarks—are empirical measurements against published baselines. The Context Mamba recurrence (Eqs. 6–8) is a network architecture trained with the loss in Eq. (10); no predicted quantity is defined in terms of the benchmark target, and no parameter is fitted to a benchmark and then renamed as a prediction. The comparisons to ODTrack and AQATrack are against published numbers from prior work, including same-group papers, but those numbers are external constraints, not inputs to the derivation, so this is self-citation without load-bearing circularity. The stated Limitation—'the training and testing phases cannot be unified'—is a genuine generalization and validation risk for the long-term-context mechanism, but it concerns distribution shift between 2-frame training clips and full-sequence inference, not definitional circularity. Similarly, the observation that ODTrack's GOT-10K AO (77.0) exceeds MambaLCT-384 (76.2) in Tab. 2 is a correctness inconsistency with the 'new SOTA' claim, not a circularity. The paper is self-contained against external benchmarks for its empirical claims, so no circular step is present.
Assumptions & free parameters
free parameters (9)
- loss weight lambda1 =
5
- loss weight lambda2 =
2
- context token length Nc =
1
- Mamba layer insertion indices =
(3, 6, 9)
- training clip sampling length =
2
- sampling quantity =
30000
- backbone learning rate =
2e-4
- other learning rate =
2e-3
- batch size =
16
assumptions (4)
- domain assumption Pretrained HiViT-Base (MAE) and Vim-Small checkpoints provide useful initialization.
- domain assumption Mamba's selective scan compresses target-relevant features into a fixed-size hidden state without losing critical cues.
- domain assumption The Transformer encoder can effectively combine context, template, and search tokens through attention.
- domain assumption Standard classification, L1, and GIoU losses are sufficient to train the tracking framework.
Cite this review
Pith. "Pith review of MambaLCT: Boosting Tracking via Long-term Context State Space Model." pith.science (2026). https://pith.science/paper/43ERK33L
@misc{pith2026241213615,
author = {Pith},
title = {Pith review of: MambaLCT: Boosting Tracking via Long-term Context State Space Model},
year = {2026},
howpublished = {\url{https://pith.science/paper/43ERK33L}},
note = {Machine review of arXiv:2412.13615}
}
read the original abstract
Effectively constructing context information with long-term dependencies from video sequences is crucial for object tracking. However, the context length constructed by existing work is limited, only considering object information from adjacent frames or video clips, leading to insufficient utilization of contextual information. To address this issue, we propose MambaLCT, which constructs and utilizes target variation cues from the first frame to the current frame for robust tracking. First, a novel unidirectional Context Mamba module is designed to scan frame features along the temporal dimension, gathering target change cues throughout the entire sequence. Specifically, target-related information in frame features is compressed into a hidden state space through selective scanning mechanism. The target information across the entire video is continuously aggregated into target variation cues. Next, we inject the target change cues into the attention mechanism, providing temporal information for modeling the relationship between the template and search frames. The advantage of MambaLCT is its ability to continuously extend the length of the context, capturing complete target change cues, which enhances the stability and robustness of the tracker. Extensive experiments show that long-term context information enhances the model's ability to perceive targets in complex scenarios. MambaLCT achieves new SOTA performance on six benchmarks while maintaining real-time running speeds.
Figures
Forward citations
Cited by 1 Pith paper
-
Explicit Context Reasoning with Supervision for Visual Tracking
RSTrack supervises a Mamba-based state reasoning module with true target states, improving visual tracking accuracy on six benchmarks.
Reference graph
Works this paper leans on
-
[1]
Bertinetto, L.; Valmadre, J.; Henriques, J. F.; Vedaldi, A.; and Torr, P. H. 2016. Fully-convolutional siamese networks for object tracking. In Computer Vision--ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II 14, 850--865. Springer
work page 2016
-
[2]
Cai, Y.; Liu, J.; Tang, J.; and Wu, G. 2023. Robust object modeling for visual tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 9589--9600
work page 2023
-
[3]
Chen, X.; Peng, H.; Wang, D.; Lu, H.; and Hu, H. 2023. Seqtrack: Sequence to sequence learning for visual object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14572--14581
work page 2023
-
[4]
Chen, X.; Yan, B.; Zhu, J.; Wang, D.; Yang, X.; and Lu, H. 2021. Transformer tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8126--8135
work page 2021
-
[5]
Chen, Z.; Zhong, B.; Li, G.; Zhang, S.; Ji, R.; Tang, Z.; and Li, X. 2022. SiamBAN: Target-aware tracking with Siamese box adaptive network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4): 5158--5173
2022
-
[6]
Cui, Y.; Jiang, C.; Wang, L.; and Wu, G. 2022. Mixformer: End-to-end tracking with iterative mixed attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 13608--13618
work page 2022
-
[7]
Danelljan, M.; Bhat, G.; Shahbaz Khan, F.; and Felsberg, M. 2017. Eco: Efficient convolution operators for tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition, 6638--6646
work page 2017
-
[8]
Fan, H.; Lin, L.; Yang, F.; Chu, P.; Deng, G.; Yu, S.; Bai, H.; Xu, Y.; Liao, C.; and Ling, H. 2019. Lasot: A high-quality benchmark for large-scale single object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5374--5383
work page 2019
Show all 35 references
-
[9]
Gao, S.; Zhou, C.; Ma, C.; Wang, X.; and Yuan, J. 2022. Aiatrack: Attention in attention for transformer visual tracking. In European Conference on Computer Vision, 146--164. Springer
2022
-
[10]
Gao, S.; Zhou, C.; and Zhang, J. 2023. Generalized relation modeling for transformer tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18686--18695
2023
-
[11]
He, K.; Chen, X.; Xie, S.; Li, Y.; Doll \'a r, P.; and Girshick, R. 2022. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16000--16009
2022
-
[12]
Hu, X.; Zhong, B.; Liang, Q.; Zhang, S.; Li, N.; Li, X.; and Ji, R. 2023. Transformer Tracking via Frequency Fusion. IEEE Transactions on Circuits and Systems for Video Technology
2023
-
[13]
Huang, L.; Zhao, X.; and Huang, K. 2019. Got-10k: A large high-diversity benchmark for generic object tracking in the wild. IEEE transactions on pattern analysis and machine intelligence, 43(5): 1562--1577
2019
-
[14]
Huang, T.; Pei, X.; You, S.; Wang, F.; Qian, C.; and Xu, C. 2024. Localmamba: Visual state space model with windowed selective scan. arXiv preprint arXiv:2403.09338
2024 arXiv
-
[15]
Li, B.; Wu, W.; Wang, Q.; Zhang, F.; Xing, J.; and Yan, J. 2019. Siamrpn++: Evolution of siamese visual tracking with very deep networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4282--4291
2019
-
[16]
Li, K.; Li, X.; Wang, Y.; He, Y.; Wang, Y.; Wang, L.; and Qiao, Y. 2024. Videomamba: State space model for efficient video understanding. arXiv preprint arXiv:2403.06977
2024 arXiv
-
[17]
Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 1...
2014
-
[18]
I.; Jiang, C.; Liu, Z.; and Wang, H
Liu, J.; Han, J.; Liu, L.; Aviles-Rivero, A. I.; Jiang, C.; Liu, Z.; and Wang, H. 2024. MAMBA4D: Efficient Long-Sequence Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Models. arXiv preprint arXiv:2405.14338
2024 arXiv
-
[19]
Loshchilov, I.; and Hutter, F. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101
2017 arXiv
-
[20]
P.; and Van Gool, L
Mayer, C.; Danelljan, M.; Paudel, D. P.; and Van Gool, L. 2021. Learning target candidate association to keep track of what not to track. In Proceedings of the IEEE/CVF international conference on computer vision, 13444--13454
2021
-
[21]
Muller, M.; Bibi, A.; Giancola, S.; Alsubaihi, S.; and Ghanem, B. 2018. Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. In Proceedings of the European conference on computer vision (ECCV), 300--317
2018
-
[22]
Shi, L.; Zhong, B.; Liang, Q.; Li, N.; Zhang, S.; and Li, X. 2024. Explicit Visual Prompts for Visual Object Tracking. arXiv preprint arXiv:2401.03142
2024 arXiv
-
[23]
Wei, X.; Bai, Y.; Zheng, Y.; Shi, D.; and Gong, Y. 2023. Autoregressive visual tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9697--9706
2023
-
[24]
Xie, F.; Chu, L.; Li, J.; Lu, Y.; and Ma, C. 2023. Videotrack: Learning to track objects via video transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 22826--22835
2023
-
[25]
Xie, J.; Zhong, B.; Mo, Z.; Zhang, S.; Shi, L.; Song, S.; and Ji, R. 2024. Autoregressive Queries for Adaptive Tracking with Spatio-Temporal Transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19300--19309
2024
-
[26]
Yan, B.; Peng, H.; Fu, J.; Wang, D.; and Lu, H. 2021. Learning spatio-temporal transformer for visual tracking. In Proceedings of the IEEE/CVF international conference on computer vision, 10448--10457
2021
-
[27]
Yang, D.; He, J.; Ma, Y.; Yu, Q.; and Zhang, T. 2023. Foreground-background distribution modeling transformer for visual object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10117--10127
2023
-
[28]
Ye, B.; Chang, H.; Ma, B.; Shan, S.; and Chen, X. 2022. Joint feature learning and relation modeling for tracking: A one-stream framework. In European conference on computer vision, 341--357. Springer
2022
-
[29]
Ye, J.; Zhong, B.; Liang, Q.; Zhang, S.; Li, X.; and Ji, R. 2023. Positive-Sample-Free Object Tracking via a Soft Constraint. IEEE Transactions on Circuits and Systems for Video Technology
2023
-
[30]
Zhang, X.; Tian, Y.; Xie, L.; Huang, W.; Dai, Q.; Ye, Q.; and Tian, Q. 2023. Hivit: A simpler and more efficient design of hierarchical vision transformer. In The Eleventh International Conference on Learning Representations
2023
-
[31]
Zhang, Z.; Peng, H.; Fu, J.; Li, B.; and Hu, W. 2020. Ocean: Object-aware anchor-free tracking. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXI 16, 771--787. Springer
2020
-
[32]
Zheng, Y.; Zhong, B.; Liang, Q.; Mo, Z.; Zhang, S.; and Li, X. 2024. ODTrack: Online Dense Temporal Token Learning for Visual Tracking. arXiv preprint arXiv:2401.01686
2024 arXiv
-
[33]
Zhu, L.; Liao, B.; Zhang, Q.; Wang, X.; Liu, W.; and Wang, X. 2024. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417
2024 arXiv
-
[34]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
-
[35]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.