{"id":"fd897b26-0eac-4bb1-893c-4305958f3328","arxiv_id":"2412.11023","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"MCITrack uses Mamba hidden states and per-block cross-attention to pass video-level context through a transformer tracker, reaching 76.6% AUC on LaSOT and 80.0% AO on GOT-10k.","lead":"This paper introduces MCITrack, a visual object tracker that carries information from past video frames in the hidden states of a Mamba model and merges that context into the current frame's features. It reports consistent benchmark gains across eight tracking datasets, including 76.6% AUC on LaSOT and 80.0% AO on GOT-10k.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Inference-time hidden-state gating (unnamed threshold, interval T, memory bank) is unspecified and mismatched with training; if tuned on LaSOT test, the 76.6% AUC claim may overstate the method.","rationale":"After reading the full manuscript, I find no contradiction in the equations or the architecture description; the CIF module equations are internally consistent, and the ablations are directionally plausible. The most fragile point is the inference-time update protocol. The paper states that hidden states are updated only when the classification score exceeds a threshold and at an update interval T, but neither value is reported. Because this protocol is not used in training, it is a hand-designed test-time heuristic. The headline LaSOT number depends on it, and the paper does not demonstrate that the gain comes from Mamba hidden states rather than from this heuristic. The reader's weakest_assumption identified the same underspecification and test-set tuning risk; I partially agree, adding that Table 6 lacks evidence that the competing propagation methods were evaluated under the identical gating rule. This concern does not warrant rejection: the method is clearly described at the architectural level, code is promised, and the improvements are consistent across benchmarks. However, without the missing values and a sensitivity analysis, the SOTA claim cannot be fully verified. Therefore the conditional verdict stands.","tokens_in":15007,"tokens_out":4583,"duration_ms":40701,"concrete_test":"Run MCITrack-L384 on LaSOT with a swept threshold grid (0.0, 0.2, 0.4, 0.6, 0.8) and T in {1, 5, 10, 20}, using the released code; report AUC for every combination and the exact values used for the 76.6% claim. Additionally, rerun Table 6 with all propagation methods forced to use the same inference gating (same threshold and T) and with gating disabled (threshold=0, T=1). If the 76.6% result is not robust to a ±0.2 threshold change, or if the Mamba-vs-Extra-Token gap shrinks below 0.5% under identical gating, the load-bearing claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Mamba hidden states transmit richer context than extra tokens rests on the LaSOT results, but the inference protocol in 'Implementation Details - Inference' is incompletely specified and appears to deviate from training. During training, hidden states are updated on every frame of the two-step unroll, with no confidence gating described. At inference, the authors maintain a memory bank and update the video clip only at an unstated interval T, and update hidden states only when the classification score exceeds an unnamed threshold. These are free parameters whose values are never given. If the threshold and T were selected on LaSOT test, the reported 76.6% AUC is an upper envelope rather than a fixed method, and the comparison to ODTrack et al. is not a fair apples-to-apples evaluation. Furthermore, the ablation table (Table 6) that supports the 'richer context' claim ('Baseline' vs 'Extra Token' vs 'LSTM') does not state whether all propagation methods used the identical gating rule and memory-bank protocol. If the Mamba baseline benefits from a tuned threshold while the alternatives use a fixed or naive update, the +1.9% over 'Extra Token' should be attributed to the inference heuristic, not to the hidden-state representation. No formal verification or variance analysis is provided, so the reported differences are not shown to be significant.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MCITrack, a video-level object tracker that propagates contextual information through Mamba hidden states instead of the few extra tokens used by prior video-level trackers. The core Contextual Information Fusion (CIF) module pairs each backbone block with a mamba layer and two cross-attention layers: the mamba layer stores historical context in hidden states, the in-attention layer injects that context into backbone features, and the out-attention layer extracts current-frame information back into the CIF block. The model is trained with 5-frame clips and a two-step unroll, then evaluated on eight tracking benchmarks. The central empirical claim is state-of-the-art performance, including 76.6% AUC on LaSOT and 80.0% AO on GOT-10k, with ablations on LaSOT supporting the CIF components, the clip length, the hidden-state size, and the choice of Mamba over extra tokens, LSTM, or previous-frame features.","tokens_in":15263,"tokens_out":4903,"duration_ms":43498,"significance":"If the reported results hold under a fixed and fully specified inference protocol, the paper makes a useful contribution to video-level tracking: the architecture is clearly described, the ablations cover the main design choices, the model family spans a reasonable efficiency-accuracy trade-off, and code and models are promised. The strength of the empirical case is reduced by two issues: the inference-time gating protocol has unspecified parameters that directly affect the reported numbers, and several key design choices were selected on the same LaSOT benchmark used for the headline state-of-the-art claim. These issues are addressable and do not invalidate the architectural idea, but they need to be resolved before the central claim can be fully trusted.","major_comments":[{"comment":"The inference protocol is incomplete: the paper says hidden states are updated only when the classification score exceeds a threshold and that the video clip is refreshed at an update interval T using frames from a memory bank, but it never gives numerical values for the threshold, T, or the memory-bank update rule. These parameters determine which frames propagate into the hidden state and therefore directly affect the reported AUC/AO numbers. Please specify the threshold, T, the memory-bank capacity and update rule, and state whether these values were chosen before or after seeing test-set results; without that information the method cannot be reproduced and comparisons with ODTrack and ARTrackV2 may reflect differences in inference heuristics rather than in the CIF architecture.","section":"Implementation Details, Inference"},{"comment":"Training updates hidden states on every step with no confidence gating, while inference gates updates by the classification score and periodically refreshes the clip at interval T. This training-inference mismatch is not discussed. If the threshold and T were tuned on LaSOT test, the headline 76.6% AUC would be an upper envelope rather than the performance of a fixed method. Please clarify whether gating was applied during training, and if it was not, report the sensitivity of the main results to the threshold and T on a validation split, and confirm that all compared methods and all ablation variants use exactly the same inference protocol.","section":"Implementation Details, Training vs. Inference"},{"comment":"The hidden-state size (16), the number of CIF blocks (4), the video clip length (5), and the propagation method are selected by ablations on LaSOT, and LaSOT is also the benchmark used for the headline 'unprecedented AUC' claim. Selecting the final configuration on the same test set used for the state-of-the-art comparison can inflate the reported gains over the 'Extra Token' and 'LSTM' alternatives. Please report ablations on a held-out split or on additional benchmarks, or otherwise justify that the final configuration was frozen before test evaluation.","section":"Tables 4-6 and State-of-the-Art Comparisons"},{"comment":"The ablation table does not state whether the Baseline, Extra Token, LSTM, and Previous Features rows all used the same hidden-state update threshold, interval T, and memory-bank protocol. If the Mamba baseline benefited from a tuned gating rule while the alternatives used a naive or fixed update, the +1.9% AUC over Extra Token would be attributable to the inference heuristic rather than to the hidden-state representation. Please specify the protocol used in each row and, ideally, include repeated-run variance or results on another benchmark, because the reported differences (0.9-1.9 AUC) are small and no significance information is provided.","section":"Table 6, Contextual Information Propagation Methods"}],"minor_comments":[{"comment":"The text contains a typo: 'set a thresholda for updating' should read 'set a threshold for updating'.","section":"Implementation Details, Inference"},{"comment":"The column header 'P N orm P' appears with broken spacing in several tables and should be rendered as 'P Norm P'.","section":"Tables 1, 3, 4, 5, 6"},{"comment":"The symbol N is used both for the number of backbone/CIF blocks and for the SSM state size in Eq. (2); renaming one of them would avoid ambiguity.","section":"Model Architecture, Backbone and State Space Model"},{"comment":"The training paragraph says a 5-frame video clip and two search regions are used, but the overview describes inputs as one video clip and one search region; please clarify how the two search regions are sampled and paired with the clip.","section":"Implementation Details, Training"},{"comment":"The Limitation paragraph addresses training speed and clip overhead but does not mention the sensitivity of the results to the inference threshold and interval T; adding a sentence on the chosen values and their effect would strengthen reproducibility.","section":"Limitation"}],"recommendation":"major_revision","confidential_remarks":"The architectural idea is solid and the experimental coverage is broad, but the unspecified inference gating and the selection of design choices on the same benchmark used for the headline claim are load-bearing concerns. I recommend asking the authors for full disclosure of the threshold and interval values, a validation-based selection protocol, and a clear statement that all comparisons share the same inference heuristics before reconsidering the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nYou should know about MCITrack because it is a real, if incremental, step in video-level tracking: instead of packing context into a few extra tokens, it carries context in Mamba hidden states fused into every backbone block via cross-attention. That design choice is new in this niche, and the paper does a good job of supporting it. The gains are consistent across eight benchmarks, and the ablations show Mamba hidden states beating LSTM and extra-token baselines. Training details are thorough, and code and models are promised, which matters for a paper like this.\n\nThe soft spots are real but not fatal. The inference protocol is underspecified: hidden states update only when the classification score exceeds an unnamed threshold, and the memory-bank refresh interval T is not given. During training, hidden states update on every frame, so there is a train/inference mismatch that is not discussed. If the threshold and T were tuned on LaSOT test—the same benchmark that anchors the 76.6% AUC claim—the headline number is an upper envelope rather than a fixed method. Also, the ablation comparing propagation methods does not state whether all variants used the same gating rule, so the +1.9% over Extra Token could be partly due to the inference heuristic rather than the hidden-state representation. That said, the underlying architecture is clearly specified, and the code is available, so a referee can verify.\n\nI would send this to peer review. The central idea is sound and the experiments are extensive. The authors need to report the threshold/T, specify which split the ablation used, and either align training and inference or explain the mismatch. If the code checks out, this is a solid publishable tracking paper. No variance or significance analysis is provided, but that is typical for this community and not a dealbreaker.\n\nBest","headline":"A capable, well-executed tracking paper that earns its SOTA claims on the evidence shown, but the unnamed threshold and interval T in the inference protocol plus ablation-on-LaSOT leave reproducibility and potential overfitting questions that a referee should press.","tokens_in":15827,"tokens_out":3898,"would_cite":true,"duration_ms":30581,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that video-level context carried in Mamba hidden states, fused into each backbone block by cross-attention, lets MCITrack surpass token-based video trackers and reach state-of-the-art accuracy.","keywords":["video object tracking","Mamba","state space model","hidden states","contextual information","cross-attention","LaSOT","GOT-10k"],"falsifier":"Run MCITrack-L384 on LaSOT with the hidden-state update threshold fixed to a constant (for example 0.5) and the update interval set to every frame, then compare the AUC to the reported 76.6%; a large drop would indicate the numbers depend on undisclosed update tuning rather than on hidden-state capacity.","tokens_in":14761,"feed_emoji":"🎯","tokens_out":4047,"duration_ms":32242,"temperature":0.7,"pith_summary":"The paper introduces MCITrack, a video-level object tracker that replaces the few extra tokens used by prior video-level trackers with Mamba hidden states that continuously record historical context. The central claim is that these hidden states transmit richer contextual information than a handful of tokens, and the numbers offered are state-of-the-art accuracy of 76.6% AUC on LaSOT and 80.0% average overlap on GOT-10k. A sympathetic reader would care because, if true, the bottleneck in video tracking shifts from how much context can be packed into extra tokens to how well a recurrent state can compress the video history.","feed_headline":"Mamba hidden states lift video tracking to 76.6% AUC","feed_subtitle":"MCITrack stores video history in a state instead of extra tokens, posting top scores on LaSOT and GOT-10k.","key_machinery":"The Contextual Information Fusion (CIF) module is the load-bearing component, and its core is the Mamba layer, a selective state space model whose hidden state $h_t = \\bar{A} h_{t-1} + \\bar{B} x_t$ carries the video history. The hidden state is updated only when the tracker's classification score beats an unspecified threshold, at an update interval $T$, using a memory bank of reliable frames to avoid injecting erroneous context. Four CIF blocks are paired with four backbone blocks, so context is injected at multiple feature levels rather than once at the end. This machinery is what the paper claims transfers more contextual information than the few extra tokens of prior methods.","core_discovery":"The paper claims that carrying video-level context in Mamba hidden states, fused into each backbone block through cross-attention, transmits more information than the additional tokens used by ODTrack and similar trackers. Its Contextual Information Fusion (CIF) module combines a Mamba layer that stores and updates a fixed-size hidden state with in-attention and out-attention layers that inject this context before each backbone block and extract current-frame features back into the state. On standard benchmarks, the largest variant MCITrack-L384 reports 76.6% AUC on LaSOT and 80.0% average overlap on GOT-10k, which the paper presents as the best published numbers on both.","pith_inferences":["The hidden-state approach might transfer to other video-sequence tasks such as action recognition or multi-object tracking, where a compact stateful summary of history could replace token queues.","The undisclosed update threshold could likely be replaced by a learned confidence gate, removing a hyperparameter while retaining robustness to unreliable frames.","A stress test on very long videos (thousands of frames) would reveal whether the hidden state saturates or forgets early context, since the reported benchmarks are mostly on minute-scale sequences."],"forward_implications":["Video-level trackers no longer need to grow token counts to carry context; a fixed-size recurrent state can in principle serve the same role with less overhead.","Multi-level injection means context can help shallow high-resolution features and deep semantic features simultaneously, which may reduce drift in long-term tracking.","The reported efficiency (MCITrack-B224 beats ODTrack-B384 with lower FLOPs and a lower input resolution) suggests hidden-state context transmission is cheaper than token-based propagation.","The CIF module is a generic wrapper, so the same architectural recipe could be applied to other transformer backbones beyond the Fast-iTPN used here."],"supporting_citations":[{"why":"Supplies the selective state space model whose hidden states store and transmit contextual information.","marker":"(Gu and Dao 2023)"},{"why":"The ODTrack baseline using extra tokens; it is the main comparison the paper claims to beat with hidden states.","marker":"(Zheng et al. 2024)"},{"why":"Fast-iTPN backbone that MCITrack wraps with CIF blocks.","marker":"(Tian et al. 2024)"},{"why":"Provides the prediction head and loss function used by MCITrack.","marker":"(Ye et al. 2022)"},{"why":"LaSOT, the headline benchmark where 76.6% AUC is reported.","marker":"(Fan et al. 2019)"},{"why":"GOT-10k, where 80.0% average overlap is reported.","marker":"(Huang, Zhao, and Huang 2019)"}],"fun_headline_variants":["MCITrack: Mamba state carries video context to SOTA tracking","Hidden state stores video history for better object tracking","Mamba memory tracks objects across video frames with top scores","Video tracker uses Mamba state to beat token-based context methods","Storing context in Mamba state lifts video tracking to new highs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmark gains assume that the hidden-state update rule (the classification-score threshold and the update interval $T$, whose values are never stated) was not tuned on the same LaSOT test set used for the headline result, and that this rule transfers to long videos without re-tuning.","fun_headline_variants_meta":{"raw":{"variants":["MCITrack: Mamba state carries video context to SOTA tracking","Hidden state stores video history for better object tracking","Mamba memory tracks objects across video frames with top scores","Video tracker uses Mamba state to beat token-based context methods","Storing context in Mamba state lifts video tracking to new highs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000482,"raw_usage":{"total_tokens":2361,"prompt_tokens":902,"completion_tokens":1459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1374}},"tokens_in":518,"tokens_out":1459,"duration_ms":9801,"temperature":1.0,"reasoning_tokens":1374,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:22:48.195018+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MCITrack-L384 on LaSOT with the hidden-state update threshold fixed to a constant (for example 0.5) and the update interval set to every frame, then compare the AUC to the reported 76.6%; a large drop would indicate the numbers depend on undisclosed update tuning rather than on hidden-state capacity.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The ODTrack baseline using extra tokens; it is the main comparison the paper claims to beat with hidden states."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Fast-iTPN backbone that MCITrack wraps with CIF blocks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LaSOT, the headline benchmark where 76.6% AUC is reported."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GOT-10k, where 80.0% average overlap is reported."}],"review_version":1}