{"id":"037a4378-e0a7-4170-9e53-516e11e176dd","arxiv_id":"2412.13615","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"MambaLCT combines a unidirectional Mamba scan of all past search frames with a Transformer encoder to build long-term context for single-object tracking.","lead":"MambaLCT is a visual object tracker that uses a Mamba state-space module to accumulate target appearance history across the entire video and inject it into a Transformer encoder. The authors report state-of-the-art results on several tracking benchmarks and release code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The long-term context claim is not validated: the Context Mamba is trained on 2-frame clips but unrolled over full test sequences, so hidden-state dynamics at inference are out-of-distribution; benchmark gains may reflect added capacity rather than long-term memory.","rationale":"The paper is well-engineered and the ablations are informative: the Context Mamba module does improve AUC over the +sequence-sampling baseline (70.1 -> 71.8, Tab. 4a). However, the mechanism is described as long-term context from the first frame to the current frame, and the reported training protocol uses clips of length 2. This is exactly the weakest point: the recurrent state in Eqs. 7-8 is trained only on two-step rollouts, and the paper's own Limitation admits the train/test mismatch. The benchmark gains may therefore reflect extra capacity or short-range context rather than validated long-term memory. The reader's weakest_assumption captures this accurately. I would keep the CONDITIONAL verdict: the paper is acceptable conditional on demonstrating that the length-2-trained model actually benefits from full-sequence state propagation, e.g., by varying training clip length or by comparing stateful vs reset inference. The SOTA claim should also be corrected to per-resolution/per-benchmark statements, since Tab. 2 shows ODTrack better on GOT-10K.","tokens_in":12710,"tokens_out":4171,"duration_ms":38292,"concrete_test":"Retrain MambaLCT-256 under identical settings with clip sampling lengths 2, 4, 8, and 16, and evaluate on LaSOT test. If the length-2 model already matches the length-16 model, or if increasing training clip length does not improve AUC on the longest LaSOT sequences (e.g., >1000 frames), the long-term context mechanism is not validated. As a complementary check, run inference on LaSOT with the hidden state reset every 2 frames; if full-sequence stateful inference does not beat the reset variant, the gains are not from long-term memory.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that aggregating target cues from the first frame to the current frame via the Context Mamba improves robustness. This requires the recurrent hidden state H^T_i (Eq. 8) to remain stable and informative when unrolled over hundreds of frames. But Training Details state 'we set the video clip sampling length to 2', and the Limitation concedes 'the training and testing phases cannot be unified'. During training, gradients only propagate through two frame-to-frame transitions, so the Mamba parameters are optimized for a state that has integrated at most two frames of history. At inference the same recurrence is applied for the full sequence; hidden states accumulate information never seen in training, so the model can silently drift or saturate. The reported gains over the non-temporal baseline (#3 vs #4 in Tab. 4) therefore cannot be attributed specifically to long-term memory; they could come from the extra Vim-Small module's capacity or from the two-frame context alone. The paper's attribute-based analysis and Fig. 6 show attention sharpening with sequence length, but these are qualitative and do not control for the training/inference mismatch. This is a correctness risk for the mechanism, not just a scope limitation. A secondary issue: Tab. 2 itself contradicts 'new SOTA on six benchmarks', since ODTrack's GOT-10K AO (77.0) exceeds MambaLCT-384 (76.2).","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MambaLCT, a single-object tracker that augments a HiViT-based encoder with a unidirectional Context Mamba module intended to aggregate target-related cues from the first frame to the current frame. The context tokens are updated by a recurrent state-space update (Eqs. 7-8) and injected into the encoder's attention. Training uses clips of length 2 sampled from GOT-10K, LaSOT, COCO, and TrackingNet, while inference unrolls the recurrence over entire test sequences. The authors report results on LaSOT, LaSOText, GOT-10K, TrackingNet, TNL2K and UAV123, plus ablations on LaSOT, and claim new SOTA on six benchmarks.","tokens_in":13010,"tokens_out":6650,"duration_ms":53023,"significance":"If the long-term context mechanism were validated, the paper would provide a practically interesting combination of Mamba's linear-time recurrence with transformer appearance modeling for visual tracking, and the reported real-time speed (45.3 fps at 384 resolution) and public code and models are useful assets. The empirical coverage is broad, and the ablation in Table 4(a) isolates the Mamba addition from the HiViT backbone. However, as detailed below, the paper's central attribution of gains to long-term memory is not directly supported by the training setup, and the SOTA claim is contradicted by its own GOT-10K numbers.","major_comments":[{"comment":"Training Details state that the video clip sampling length is 2, and the Limitation section concedes that 'the training and testing phases cannot be unified.' The Context Mamba recurrence in Eqs. (7)-(8) is therefore trained only through two frame-to-frame transitions, while at inference it is unrolled over sequences of hundreds of frames. Under this mismatch, the gain of the Mamba module in Table 4(a) (71.8 vs 70.1 for #4 vs #3) cannot be attributed specifically to long-term context; it could come from the added Vim-Small capacity or from two-frame context alone. Please provide evidence that the hidden-state dynamics are stable for lengths far beyond the training clip length (e.g., train with clip lengths 4 or 8, or ablate with variable context at inference), or soften the long-term-context claim accordingly.","section":"Training Details; Inference Details; Limitation; Eqs. (7)-(8); Table 4(a)"},{"comment":"The abstract claims 'new SOTA performance on six benchmarks', but Table 2 shows that on GOT-10K, ODTrack achieves AO 77.0, SR0.5 87.9 and SR0.75 75.1, all higher than MambaLCT-384's 76.2, 86.7 and 74.3. Since GOT-10K is one of the six benchmarks listed, the SOTA claim is not supported by the reported numbers. Please correct the claim (e.g., state the benchmarks and resolutions on which the method is actually SOTA).","section":"Abstract; Table 2"},{"comment":"The training data are stated as GOT-10K, LaSOT, COCO and TrackingNet, while Table 2's footnote says the star on GOT-10K denotes trackers trained only on GOT-10K. If the GOT-10K protocol requires training only on GOT-10K, the MambaLCT GOT-10K result is not protocol-compliant; if it does not, the footnote is misleading. Please clarify this discrepancy and, if necessary, report a model trained only on GOT-10K for that column.","section":"Training Details; Table 2 footnote"}],"minor_comments":[{"comment":"The header says 'Sampling Length' while the surrounding text discusses the length of the context token cp; please make the label consistent with the variable being varied.","section":"Table 4(b)"},{"comment":"The notation uses t for the token index within a frame and T for the cross-frame transition token, which is easy to confuse; please define both superscripts explicitly.","section":"Eqs. (7)-(8)"},{"comment":"The header contains the typo 'Spees' for 'Speed'.","section":"Table 1"},{"comment":"The qualitative attention maps are suggestive but not quantitative; consider reporting an average attention-target overlap metric over a larger set of sequences to substantiate the claim that attention becomes more focused with sequence length.","section":"Figure 6"},{"comment":"The statement that ODTrack and AQATrack construct fixed-length context of length 4 should be verified against the original papers, since ODTrack's temporal token is also described as propagating across the whole sequence.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":"The main technical concern (training/testing context-length mismatch) is acknowledged by the authors in their Limitation section, which is honest but does not remove the need for a direct validation. If the authors cannot run a longer-clip training experiment, they should reframe the contribution as short-context temporal modeling and retract the SOTA-on-six-benchmarks claim. The GOT-10K protocol issue should also be checked against the benchmark's official rules before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: MambaLCT is a clearly written, well-ablated tracker that adds a unidirectional Mamba module to accumulate target cues across frames. The gains over its HiViT baseline are real (about +1.3 AUC on LaSOT), and the paper is honest about its main weakness: training on 2-frame clips while testing on full sequences. That weakness is significant enough that the \"long-term context\" story is not directly proven.\n\nThe genuinely new piece is using a state-space model for video-level context in single-object tracking, as opposed to the fixed-window Transformers of ODTrack and AQATrack. The ablation table is clean: sequence sampling alone doesn't help, adding Mamba does. The speed numbers are competitive. That's a solid contribution to a mature field.\n\nThe soft spots: (1) The abstract says \"new SOTA on six benchmarks,\" but Table 2 shows ODTrack beats MambaLCT-384 on GOT-10K (77.0 vs 76.2 AO). That's an overclaim. (2) The train/test mismatch is not a minor detail. With clip length 2, the Mamba parameters are only ever trained to integrate two frames; at inference it unrolls over hundreds. The ablation shows the module adds capacity, but it doesn't show that capacity is being used for long-term memory. The visualization in Fig. 6 is suggestive but not controlled. (3) No variance or multiple-seed reporting, which is common but would help given the small margins.\n\nThe limitations paragraph acknowledges the train/test gap and calls it future work. That's honest, but it also means the central claim of the paper—that long-term context is what helps—remains unverified. The +1.3 AUC could come from the extra Vim parameters or from two-frame information. Still, the paper is worth engaging: the design is sensible, the experiments are thorough enough to be informative, and the train/test mismatch is a testable hypothesis for follow-up. I'd send it to review. The referee should ask for a version with longer training clips (even 4 or 8) and per-resolution SOTA claims.","headline":"A well-ablated Mamba-based tracker whose long-term memory claim is undercut by a 2-frame training clip; the gains are real but the mechanism isn't proven.","tokens_in":13552,"tokens_out":2613,"would_cite":true,"duration_ms":22466,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Visual tracking usually forgets the target's past; MambaLCT argues that a state space model can remember every frame from the first to the present, and that this long-term memory is what pushes performance to new highs on six benchmarks.","keywords":["visual object tracking","long-term context","state space model","Mamba","selective scanning","transformer encoder","context token","benchmark evaluation"],"falsifier":"Run inference while resetting the Context Mamba hidden state and context token every $K$ frames (for example every 50 frames), and compare AUC on LaSOT with the full-history version. If performance does not drop, the benchmark gains come from module capacity rather than from long-term memory; alternatively, if a fixed two-frame context window matches full-history performance, the long-term mechanism is not load-bearing.","tokens_in":12505,"feed_emoji":"🎯","tokens_out":6191,"duration_ms":51783,"temperature":0.7,"pith_summary":"Existing trackers build target context from adjacent frames or short video clips, so they lose sight of how the target has changed over a long sequence. MambaLCT instead scans the features of every search frame from the first frame to the current frame with a unidirectional Mamba state space model, compressing target-related cues into a single evolving context token. That token is injected into the attention of a transformer encoder that jointly models template, search frame, and context, so appearance learning and long-term memory are unified. The paper reports that this design reaches new state-of-the-art scores on LaSOT, LaSOText, GOT-10K, TrackingNet, TNL2K and UAV123 while running in real time, and that the context token's attention indeed focuses on the target as the sequence grows.","feed_headline":"Tracker's frame-by-frame memory tops six benchmarks in real time","feed_subtitle":"A Context Mamba module rolls target cues from frame one to the present, and the gains show up on long and hard sequences.","key_machinery":"The Context Mamba module is a unidirectional state space model that scans frame features in temporal order. Each frame's features $f_i$ update the hidden state $H_i^t = A H_i^{t-1} + B f_i^t$, and an empty token $T_i$ carries the aggregated history between frames via $H_i^T = A H_i^L + B T_i$, producing $Y_i^T = C H_i^T$ that updates the context token $c_p$. The context token has length 1, so the entire history is compressed into one vector that is injected into the attention of the ucaEncoder, a hierarchical ViT that jointly models template, search frame, and context. This separation of labor — transformer for spatial appearance, Mamba for temporal memory — is what lets the context length grow to the whole sequence at linear cost.","core_discovery":"The central claim is that long-term context — not just the initial template or a short window — is what a tracker needs to stay accurate through deformation, full occlusion, and fast motion. MambaLCT proposes to build that context by feeding the autoregressive appearance features of each search frame, along with a learned context token, into a Context Mamba module; the selective scanning mechanism accumulates target-related information in a hidden state that carries across frames. The updated context token is then concatenated into the next encoder step, so the template-search relationship is modeled with knowledge of the target's entire history. Experiments on six benchmarks, with ablations on LaSOT, are used to support the claim that this long-term context is responsible for the gains, not the extra parameters or the video sampling alone.","pith_inferences":["If the mechanism is doing what the paper claims, then training on longer clips or with a curriculum that gradually extends sequence length should close the train-test gap and push all six benchmarks higher still.","The unidirectional scan means the context token only knows the past; a bidirectional or chunked scan might help after a target disappears and reappears, since the tracker could then also use evidence from frames after the occlusion.","Because the context token is a single compact vector, the method should compose naturally with online template update or memory replay: store the token and update it cheaply, without re-encoding the full history.","The same context-injection idea could transfer to other single-object tracking frameworks or to video object segmentation, where target appearance also drifts over long clips."],"forward_implications":["MambaLCT-384 reaches 73.6 AUC on LaSOT, 53.3 AUC on LaSOText, 85.2 AUC on TrackingNet, and 58.5 AUC on TNL2K, ahead of the short-context trackers it is compared with.","The same model runs at 45.3 fps on a Tesla V100, so long-term context can be added without sacrificing real-time speed.","Attribute-level results on LaSOT show the largest gains in motion blur, full occlusion, and deformation, the situations where remembering the target's history should matter most.","Ablations indicate that sequence sampling by itself does not help; the improvement appears only when the sampled video is fed through the Context Mamba module, locating the effect in the long-term context mechanism.","A single context token of length 1 outperforms longer context token sequences, consistent with Mamba's autoregressive accumulation of redundant information."],"supporting_citations":[{"why":"Supplies the Vim-Small Mamba backbone that MambaLCT modifies to a unidirectional scanning mode for temporal context construction.","marker":"(Zhu et al. 2024)"},{"why":"Provides a fixed-length short-term context baseline (ODTrack) whose token-propagation method MambaLCT extends to full-sequence context.","marker":"(Zheng et al. 2024)"},{"why":"Provides the AQATrack baseline that builds spatio-temporal context within a window; MambaLCT compares against it on four benchmarks.","marker":"(Xie et al. 2024)"},{"why":"Serves as the OSTrack one-stream framework that MambaLCT builds on, replacing ViT with HiViT and injecting context into the attention.","marker":"(Ye et al. 2022)"},{"why":"The LaSOT benchmark, whose long sequences and attribute labels (motion blur, full occlusion, deformation) motivate and evaluate long-term context.","marker":"(Fan et al. 2019)"},{"why":"The GOT-10K dataset used both for training and for reporting average overlap and success-rate results.","marker":"(Huang, Zhao, and Huang 2019)"},{"why":"Supplies the HiViT hierarchical backbone that replaces the vanilla ViT in the ucaEncoder.","marker":"(Zhang et al. 2023)"},{"why":"Provides the MAE pretraining that initializes the backbone parameters before tracking training.","marker":"(He et al. 2022)"}],"fun_headline_variants":["Memory-based tracker from frame one to now beats six benchmarks","Full-history context state model lifts tracker to SOTA on six benchmarks","Tracker's long-term context leads to new bests on six benchmarks in real time","Context Mamba accumulates target cues across frames for real-time SOTA","Frame-to-frame memory elevates tracker to top scores on six tests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The hidden state of the Context Mamba, trained only on two-frame clips, is assumed to keep accumulating useful target cues when unrolled over the full test sequence, even though the paper states that training and testing phases cannot be unified.","fun_headline_variants_meta":{"raw":{"variants":["Memory-based tracker from frame one to now beats six benchmarks","Full-history context state model lifts tracker to SOTA on six benchmarks","Tracker's long-term context leads to new bests on six benchmarks in real time","Context Mamba accumulates target cues across frames for real-time SOTA","Frame-to-frame memory elevates tracker to top scores on six tests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000186,"raw_usage":{"total_tokens":1312,"prompt_tokens":922,"completion_tokens":390,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":298}},"tokens_in":538,"tokens_out":390,"duration_ms":21502,"temperature":1.0,"reasoning_tokens":298,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:57:02.814542+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run inference while resetting the Context Mamba hidden state and context token every $K$ frames (for example every 50 frames), and compare AUC on LaSOT with the full-history version. If performance does not drop, the benchmark gains come from module capacity rather than from long-term memory; alternatively, if a fixed two-frame context window matches full-history performance, the long-term mechanism is not load-bearing.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The LaSOT benchmark, whose long sequences and attribute labels (motion blur, full occlusion, deformation) motivate and evaluate long-term context."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The GOT-10K dataset used both for training and for reporting average overlap and success-rate results."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the HiViT hierarchical backbone that replaces the vanilla ViT in the ucaEncoder."}],"review_version":1}