{"id":"f57759a4-fcff-418e-8931-182ddc76434d","arxiv_id":"2507.10302","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DisCo assigns each visual token to a unique concept from the caption and aligns its attention across frames, improving video MLLM accuracy and token efficiency.","lead":"DisCo is a new visual connector for video AI models that turns video frames into a small set of tokens with distinct meanings and consistent focus across frames. It lifts video question-answering scores on several benchmarks and lets the model use far fewer tokens without losing accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TFC's Eq. 5 attention-proxy assumption is the load-bearing risk: FFA could reinforce diffuse/background focus rather than true instance tracking, and the paper offers only one qualitative 'pot' example as validation.","rationale":"The reader's weakest_assumption already points at TFC, and I agree. I considered alternatives: the uncontrolled SOTA table, the missing error bars, and the 16-vs-64 token efficiency plot are all real, but they affect how strongly 'remarkably outperforms' can be stated, not whether the proposed mechanism produces coherent tokens. The SFT-controlled ablations (Tab. 4) are a genuine strength: they show the components add accuracy over the same data. However, accuracy gains are an indirect witness to the mechanism. For the paper's central definition 'temporal coherence of each visual token to its instance across every frame', the only direct support is Eq. 5 plus the FFA loss. If the assumption behind Eq. 5 is wrong, the loss can be minimized by aligning irrelevant frame-level statistics, and the qualitative pot example would be an exception rather than the rule. The proposed check is feasible with existing video segmentation data and would settle the issue. Because the failure mode is unproven rather than demonstrated, I keep the reader's CONDITIONAL verdict rather than moving to REJECT.","tokens_in":19620,"tokens_out":5014,"duration_ms":64829,"concrete_test":"Use a per-frame instance-mask benchmark (e.g., DAVIS 2017 or YTVIS validation) and, for each video, obtain DisCo's semantic concepts from the caption and match each visual-token group to its concept via the paper's Hungarian matching. Compute the mean over frames of the normalized cross-attention map C_t_k's IoU with the ground-truth mask of the matched instance for InternVideo2-HD+DisCo and for the SFT baseline from Tab. 4. If DisCo's target-instance IoU is not clearly higher, with a reduced fraction of low-attention-mass frames, then the Eq. 5 proxy assumption fails and the temporal-coherence claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two pillars: VCD gives distinct semantics, TFC gives temporal coherence. TFC's only mechanism is the FFA loss (Eq. 6), which treats the frame-level attention feature a_t_i in Eq. 5 as a faithful summary of 'which instance token i is looking at'. That proxy is unvalidated. If a token's cross-attention maps are diffuse or dominated by background, or if multiple instances share appearance, FFA can pull frame features together while never tracking a real object. The paper's evidence for the proxy is one qualitative 'pot' example (Fig. 6b); Tab. 6 shows accuracy gains, but downstream accuracy does not establish that attention maps consistently track instances. Since temporal coherence is asserted as a defining attribute (Sec. 3.5), a failure of this proxy would directly undermine the central claim, even if VCD's semantic-distinctness story survives. This is a correctness risk, not a style complaint.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DisCo, a plug-and-play visual encapsulation module for video MLLMs, with two components: a Visual Concept Discriminator (VCD), which aligns groups of visual tokens with distinct semantic concepts extracted from captions via GPT-4 and trained with contrastive/matching losses, and a Temporal Focus Calibrator (TFC), which uses a frame-level focus alignment (FFA) loss on cross-attention features to encourage each visual token to keep tracking its instance across frames. DisCo is applied to ST-LLM and InternVideo2 (including the HD variant), and experiments report consistent gains on MVBench, STAR, PerceptionTest, EgoSchema, MLVU, VideoMME and VideoChatGPT-Bench, along with token-efficiency and ablation studies.","tokens_in":19790,"tokens_out":5216,"duration_ms":61675,"significance":"If the reported effects hold, DisCo is a useful and unusually well-scoped contribution: it is plug-and-play, works across two base models, and its ablation table compares against an SFT baseline trained on the same corpus, which is stronger than the common practice of comparing only to the published baseline. The token-efficiency result, where a 16-token DisCo outperforms a 64-token resampler, is practically relevant. The manuscript also ships code and gives detailed hyperparameters. The main caveats are that the temporal-coherence mechanism is validated only qualitatively and that the headline state-of-the-art claims rest on uncontrolled comparisons.","major_comments":[{"comment":"The TFC module's only training signal is the FFA loss, which treats the pooled cross-attention feature a_t_i as a faithful summary of the instance that token i is tracking. The paper does not provide quantitative evidence for this proxy. Fig. 6(b) shows one qualitative 'pot' example, and downstream accuracy (Tab. 6) is an indirect and insensitive check, because a model could gain from the auxiliary loss even if the attention maps do not actually track instances. If the attention maps are diffuse or background-dominated, FFA can reinforce a wrong focus, and the claimed temporal-coherence attribute is not established. Please add a quantitative validation of the proxy, such as measuring attention-map overlap with ground-truth instance tracks or masks on a subset, or an intervention that isolates the FFA mechanism, and report the distribution rather than a single example.","section":"Sec. 3.3, Eqs. (5)-(6)"},{"comment":"The 'state-of-the-art' comparisons are not controlled: different rows use different visual encoders, LLMs, training corpora, token counts, and input resolutions, so the reported margins over prior systems cannot be attributed to the proposed module. The paper's own SFT baseline in Table 4 is the right control, but it is reported for only one run per condition and without standard deviations or significance tests. Several numeric differences in Tables 2 and 12 (e.g., MVBench 68.2 vs. 68.1, VideoMME 47.4 vs. 46.3) are within plausible run-to-run noise. Please add multiple seeds or a statistical statement, and present 'outperforms previous state-of-the-art' claims only with a controlled comparison where the sole change is the proposed method.","section":"Sec. 4.1, Tables 2-3"},{"comment":"The token-efficiency claim needs clarification and stronger controls. Fig. 5 varies 'local visual tokens' but does not state how the number of global tokens and the VCD group count Ng change when the total drops from 64 to 16, and the x-axis label 'Step time (s/step)' measures training time, while the abstract's '75% fewer tokens' refers to input-token count. Table 12 compares DisCo (64 tokens) with TokenPacker and DeCo using different underlying models and training procedures. Please make the comparison setup explicit (same base model, same data, same token-count convention) or restrict the efficiency claim to a single controlled setting.","section":"Sec. 4.2, Fig. 5 and Table 12"}],"minor_comments":[{"comment":"The text and equations refer to 'i-th visual token' while the actual matching unit is a group of N/Ng tokens; please clarify how group-level features are pooled and whether each token within a group receives the same concept label.","section":"Sec. 3.2, Eq. (2)"},{"comment":"The contrastive temperature tau is introduced but its value is never reported in the implementation details; please specify it.","section":"Eq. (3)"},{"comment":"EgoSchema is evaluated on a 'subset' and MLVU on a 'subset'; please state the subset sizes and cite the convention used by prior work.","section":"Table 2"},{"comment":"There are several typos: 'basd' should be 'based', Table 5 uses 'Lvtc/Lvtm' where the text uses 'Lvsc/Lvsm', and the Table 11 caption uses 'λ_fsc' where Eq. (7) defines 'λ_ffa'.","section":"Appendix A and Tables 5, 11"},{"comment":"The x-axis is labeled 'Step time (s/step)' while the caption says the figure reports 'Performance and efficiency under different number of visual tokens'; please clarify which quantity is on each axis and report inference-time variation as well as training time.","section":"Fig. 5"},{"comment":"Stage 1 mixes 900K video dense captions with 23K image captions; please state how image-only data is used, for example whether it is batch-level mixing or a separate auxiliary objective, and whether the image data also receives the VCD/TFC losses.","section":"Sec. 3.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer-vision venue, and I saw no problematic citation patterns or novelty-disclosure issues. The central risk is overclaiming on the SOTA comparisons and on the TFC mechanism; both can be addressed with additional experiments, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DisCo is a clean, plausible trainable connector for video MLLMs, and the authors do the reader the courtesy of a controlled ablation: both the VCD and TFC components add a nontrivial margin over an SFT baseline trained on the same corpus. That is the strongest part of the paper. The headline 'state-of-the-art' comparisons in Table 2 are mostly cross-model comparisons with different training data and sizes; the real evidence is the paired gains on ST-LLM and InternVideo2. The token-efficiency experiment (16 tokens beating 64) is suggestive but underreported—a line plot with no numbers or error bars.\n\nThe novelty is the specific combination, not any single component. Pairwise token-concept contrastive alignment has clear ancestors in BLIP-style losses, and cross-attention regularization is not new; but tying a Hungarian assignment over GPT-4-extracted concepts to a frame-level attention-calibration loss is a distinct recipe. It is a useful plug-in for Q-Former-style connectors rather than a new paradigm.\n\nThe soft spots are real but not fatal. First, no standard deviations or multiple seeds appear anywhere; on MVBench the component gains are 0.6–1.7 points, which could be noise. Second, hyperparameters (Ng, loss weights) are selected by looking at the same benchmarks where the final numbers are reported. Third, the TFC mechanism rests on an unvalidated proxy: treating a token's averaged cross-attention map as its 'focused instance' (Eq. 5). The FFA loss could just be pulling frame attention to whatever is already dominant, including background. One qualitative 'pot' example is not enough. That said, the ablation shows TFC improves accuracy, so the worst case is that the paper's stated interpretation of why it works is too strong, not that the method fails. I would ask the authors for an attention-tracking metric—pointing accuracy against ground truth or a simple objectness mask—before believing the temporal-coherence claim.\n\nThis paper deserves a serious referee. The right outcome is probably major revision asking for seeds or error bars, code release, and a quantitative attention-focus check, not rejection.","headline":"A plausible plug-and-play video connector whose controlled ablations are the real contribution, but the SOTA and temporal-coherence claims need more evidence before they're taken at face value.","tokens_in":20335,"tokens_out":3015,"would_cite":false,"duration_ms":33664,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A plug-and-play visual connector that pairs each video token with a distinct concept and tracks that concept across every frame yields consistent accuracy gains on video understanding benchmarks while using about 75% fewer tokens.","keywords":["video multimodal large language models","visual encapsulation","resampler","semantic distinctness","temporal coherence","contrastive learning","token efficiency","video question answering"],"falsifier":"Measure attention faithfulness directly: on a video dataset with instance masks (e.g., DAVIS or a video instance-segmentation benchmark), locate the token assigned to a known object by the VCD matching, and compute the overlap of that token's per-frame cross-attention maps with the ground-truth object mask across all frames. If attention frequently falls on background or loses the object when it is occluded or absent, DisCo's temporal-coherence claim is measurably false.","tokens_in":19398,"feed_emoji":"🎬","tokens_out":4858,"duration_ms":48607,"temperature":0.7,"pith_summary":"Video multimodal LLMs convert a video into a small set of visual tokens for the language model, and the quality of that conversion limits video understanding. This paper argues that existing connectors, whether linear projectors or resamplers, produce tokens that are semantically redundant—several tokens attend the same object while others are ignored—and temporally unstable, losing track of an object in some frames. DisCo instead supervises each token to represent one distinct concept, extracted from the caption as a noun phrase, and to keep attending to that concept in every frame. On ST-LLM and InternVideo2, DisCo consistently improves accuracy on MVBench, STAR, PerceptionTest, EgoSchema, MLVU, VideoMME, and video conversation benchmarks, and a 16-token DisCo outperforms a 64-token resampler. If correct, this points to a new design principle for visual connectors: token quality, not token count, is what carries video understanding.","feed_headline":"DisCo cuts video tokens 75% and beats SOTA","feed_subtitle":"Plug-in module assigns each token a distinct concept and tracks it across frames, lifting video QA accuracy.","key_machinery":"The machinery is the resampler's cross-attention mechanism treated as a readable, steerable signal. DisCo writes into it with two auxiliary losses: the Visual Concept Discriminator (VCD) uses bipartite matching between visual-token groups and caption-derived noun phrases, then applies contrastive and matching losses to give each token group distinct semantics; the Temporal Focus Calibrator (TFC) extracts frame-level attended features from the cross-attention maps and aligns each token's per-frame features to their centroid with the Frame-level Focus Alignment (FFA) loss. The same cross-attention maps that produce the tokens are thus used as the training signal to make tokens distinct and temporally consistent.","core_discovery":"DisCo claims that the deficiencies of resampler-based video tokens—semantic indistinctness, where multiple tokens pile onto the same instance and other instances get no token, and temporal incoherence, where a token attends its instance in some frames but drops it in others—can be directly supervised away. The Visual Concept Discriminator extracts non-repetitive noun phrases from the video caption with GPT-4, partitions visual tokens into groups, matches groups to phrases by Hungarian bipartite matching, and trains with pairwise contrastive and matching losses so each token group locks onto its phrase. The Temporal Focus Calibrator takes the resampler's per-frame cross-attention maps, builds frame-wise attended features for each token, and applies a Frame-level Focus Alignment loss that pulls every frame's feature toward the token's temporal centroid and repels other tokens' features, so each token keeps its instance in view throughout the video. A set of global tokens preserves information the phrases miss. The reported result is that these two modules, added as a plug-in to existing resamplers, raise accuracy across a wide spectrum of video QA and conversation benchmarks while allowing token counts to drop by about 75% without losing performance.","pith_inferences":["The frame-level attention-alignment idea could generalize to image MLLMs as an object-centric token regularizer, encouraging each token to track an entity across spatial regions rather than across time.","If the 75% token reduction holds at scale, inference cost of video MLLMs could shrink substantially, and model capacity could be redirected from redundant tokens to more concepts (more token groups) rather than more tokens per group.","The method's ceiling is set by caption quality: since concepts come from captions via GPT-4, objects never mentioned in captions will not receive dedicated tokens; testing on open-vocabulary video QA with unseen objects would stress this boundary.","The similarity-matrix and attention visualizations suggest a cheap diagnostic: token-to-concept similarity entropy could be used as a monitoring metric during training to detect when semantic distinctness collapses, without needing full benchmark evaluation."],"forward_implications":["Plug-in gains: DisCo raises accuracy on six video QA benchmarks and the VideoChatGPT conversation benchmark when added to ST-LLM and InternVideo2, with differences beyond the SFT control that uses the same data.","Token efficiency: a 16-token DisCo still outperforms a standard 64-token resampler on MVBench, implying an effective 75% reduction in visual tokens at similar or better accuracy.","Generality across captions: gains persist with both long dense captions (ShareGPT4o) and short captions (WebVid2M), though the token configuration should be adapted to the caption's instance count.","Both losses matter: ablations show VCD and TFC each contribute, and dropping either the visual-semantic contrastive loss or the matching loss reduces MVBench accuracy."],"supporting_citations":[{"why":"Supplies the Q-Former resampler architecture that DisCo builds on and modifies.","marker":"[36]"},{"why":"Source of the pairwise contrastive and matching losses that the Visual Concept Discriminator adopts.","marker":"[37]"},{"why":"Provides the Hungarian bipartite matching used to assign visual-token groups to semantic concepts.","marker":"[6]"},{"why":"Vision-language alignment technique behind the VSC loss formulation.","marker":"[57]"},{"why":"One of the two video MLLM baselines (InternVideo2) on which DisCo is plugged in and evaluated.","marker":"[63]"},{"why":"The other baseline (ST-LLM) used to demonstrate DisCo's plug-and-play generality.","marker":"[47]"},{"why":"TokenPacker, a token-compression method DisCo is compared against in token efficiency experiments.","marker":"[41]"},{"why":"DeCo, another token-compression method compared in the same efficiency table.","marker":"[70]"},{"why":"GPT-4 is used to extract non-repetitive semantic instances from video captions for the VCD module.","marker":"[1]"}],"fun_headline_variants":["DisCo gives video tokens distinct focus, 75% savings","DisCo: 75% fewer tokens, better video QA","Plug-in DisCo sharpens video token semantics","DisCo boosts video MLLMs with distinct tokens","DisCo: coherent tokens, SOTA video understanding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The Temporal Focus Calibrator assumes that a visual token's cross-attention map is a faithful measure of which instance the token represents; if attention is diffuse or glued to background, the frame-alignment loss will reinforce the wrong focus instead of tracking the intended object.","fun_headline_variants_meta":{"raw":{"variants":["DisCo gives video tokens distinct focus, 75% savings","DisCo: 75% fewer tokens, better video QA","Plug-in DisCo sharpens video token semantics","DisCo boosts video MLLMs with distinct tokens","DisCo: coherent tokens, SOTA video understanding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000583,"raw_usage":{"total_tokens":2775,"prompt_tokens":1010,"completion_tokens":1765,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":1686}},"tokens_in":626,"tokens_out":1765,"duration_ms":16164,"temperature":1.0,"reasoning_tokens":1686,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:33:41.547730+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure attention faithfulness directly: on a video dataset with instance masks (e.g., DAVIS or a video instance-segmentation benchmark), locate the token assigned to a known object by the VCD matching, and compute the overlap of that token's per-frame cross-attention maps with the ground-truth object mask across all frames. If attention frequently falls on background or loses the object when it is occluded or absent, DisCo's temporal-coherence claim is measurably false.","supporting_citations":[{"cited_title":"In: ICML (2023) 1, 2, 3","cited_arxiv_id":null,"evidence_quote":"Supplies the Q-Former resampler architecture that DisCo builds on and modifies."},{"cited_title":"In: ICML (2022) 4, 6","cited_arxiv_id":null,"evidence_quote":"Source of the pairwise contrastive and matching losses that the Visual Concept Discriminator adopts."},{"cited_title":"In: ICML (2021) 4","cited_arxiv_id":null,"evidence_quote":"Vision-language alignment technique behind the VSC loss formulation."},{"cited_title":"In: ECCV (2024) 1, 2, 5, 6, 9","cited_arxiv_id":null,"evidence_quote":"One of the two video MLLM baselines (InternVideo2) on which DisCo is plugged in and evaluated."},{"cited_title":"In: ECCV (2024) 5, 6, 9","cited_arxiv_id":null,"evidence_quote":"The other baseline (ST-LLM) used to demonstrate DisCo's plug-and-play generality."}],"review_version":1}