{"id":"8738905f-808e-42ce-a7f4-b94aeada0441","arxiv_id":"2506.03990","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DynTok dynamically merges similar adjacent visual tokens into groups, reducing video token counts to 44.4% with comparable or better video understanding accuracy.","lead":"A new video token compression method, DynTok, merges visually similar neighboring image tokens row by row, cutting the token count to about 44% while keeping accuracy on video QA benchmarks. The method is simple, parameter-free at inference, and could make long-video understanding in large vision-language models cheaper and more scalable.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No control against fixed/random grouping at the same token budget means the 2.2x-no-loss claim does not yet establish that cosine-similarity merging is the operative mechanism.","rationale":"The paper is a straightforward, useful compression method, and the zero-shot result is a genuinely strong piece of evidence that high cosine similarity frequently indicates mergeable content. However, the central mechanistic claim goes one step further: it attributes the preserved accuracy to the adaptive similarity threshold. The submitted evidence never separates that attribution from the generic effect of local averaging at a 2.2x token budget. Since the authors themselves report that zero-shot compression hurts OCR and action recognition (Table 2), the similarity proxy is known to be imperfect; training recovers these tasks, but that recovery is a property of fine-tuning on averaged tokens and would likely occur for any fixed grouping. Therefore, the most load-bearing gap is not whether the method achieves the reported numbers (though held-out threshold/frame selection and code release remain important), but whether the reported numbers support the paper's proposed mechanism rather than an equally simple fixed-grouping baseline. The proposed control test would settle this directly. The reader's weakest assumption about cosine similarity as a redundancy proxy is closely related, but the missing control is the sharper, falsifiable form of it. I therefore keep the reader's CONDITIONAL verdict: the engineering result may hold, but the method-specific claim requires this ablation and clearer token accounting.","tokens_in":12343,"tokens_out":14170,"duration_ms":144638,"concrete_test":"Train a control model with the identical data, architecture, and DynTok training recipe, but replace similarity-based grouping with a fixed row-wise grouping that produces the same total token count at the Sth=0.6 operating point (for 14-patch rows, use e.g. five groups per row plus one grid marker, about a 43% token ratio, and six groups per row, about 51%, to bracket 44.4%). Add grid markers exactly as in DynTok, and evaluate both zero-shot (on the baseline model) and after training on Video-MME and MLVU, with multiple seeds if compute allows. If the fixed-grouping control matches DynTok within benchmark noise on all tasks, the cosine-similarity threshold is not the load-bearing component; if DynTok is consistently better by more than 1 point on the long-video benchmarks, the mechanism is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that row-wise cosine-similarity threshold merging preserves accuracy at 44.4% of the tokens. The evidence for the similarity mechanism is a comparison against the uncompressed baseline; the paper never trains or evaluates a control that removes only the similarity signal. If a fixed grouping (e.g., merging equal blocks of patches per row plus grid markers) at the same 2.2x token budget gives the same zero-shot and trained accuracy, then the threshold and the cosine signal are not the source of the result, and the 'adaptive, information-preserving' mechanism is unsupported. The paper's zero-shot Table 2 already shows the proxy is weakest on fine-grained OCR and action; after training, recovery could come from the model tolerating averaged tokens, not from the grouping being semantically correct. This is testable and should be a required ablation before the method-specific claims are accepted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DynTok, a dynamic visual-token compression method for video LLMs. For each row of image-patch tokens, DynTok computes the cosine similarity between adjacent SigLIP/CLIP features; whenever the similarity exceeds a threshold Sth, the token is merged into the current group, and each group is averaged into one token, with a grid marker appended at each row end. The method is parameter-free at inference and can be used zero-shot or combined with fine-tuning. Experiments on MVBench, PerceptionTest, NextQA, LongVideoBench, MLVU, and Video-MME show that a DynTok-trained model at Sth=0.6 uses 44.4% of the baseline tokens while matching or slightly exceeding baseline accuracy (e.g., 64.0 vs. 63.6 on Video-MME), and that using more input frames yields 65.3% on Video-MME and 72.5% on MLVU. The paper also reports a per-task breakdown showing that zero-shot DynTok loses most on OCR and action tasks, while training with DynTok recovers much of this gap.","tokens_in":12449,"tokens_out":5077,"duration_ms":53516,"significance":"If the central claim is validated, DynTok is a practically attractive method: it is simple, training-compatible, and offers a 2.2x token reduction with no aggregate accuracy loss, while also enabling more frames under a fixed token budget. The zero-shot variant and the per-task analysis are useful contributions, and the method is clearly distinct from fixed-pooling and learned-compressor baselines. The main significance hinges on whether the adaptive cosine-similarity grouping is actually the operative mechanism, and whether the long-video gains survive a proper evaluation protocol.","major_comments":[{"comment":"The central 2.2x-reduction-without-loss claim is established only against the uncompressed baseline. There is no control that removes the similarity signal while keeping the same token budget, the same group-averaging operation, and the same row grid markers. A random grouping per row, a fixed block-merging scheme, or a fixed-stride pooling at matched token counts should be evaluated in both the zero-shot and trained settings. If such a control matches DynTok's accuracy, then the adaptive similarity threshold is not the source of the result, and the claim that DynTok 'preserves essential content' via similarity is unsupported. This is a required ablation before the mechanism-specific claims can be accepted.","section":"Section 4.2, Table 1 and Figure 3"},{"comment":"The 'DynTok-moreframes' results (65.3% on Video-MME and 72.5% on MLVU) appear to be obtained after sweeping Sth and the number of frames on the test benchmarks. The paper does not state whether these configurations were selected on a validation split or pre-registered before evaluation. If the reported numbers are the best over the test-set sweeps in Figures 3 and 4, they are optimistically biased and should not be compared directly with baseline numbers from a fixed protocol. Please specify the selection procedure or report a fixed configuration (for example, Sth=0.6 and a fixed frame count) for all benchmarks.","section":"Section 4.2, Figures 3-4, Table 1 last row"},{"comment":"Similarity is computed on the CLIP/SigLIP representation X, while the averaging is applied to the post-MLP tokens H. Because the MLP connector is not guaranteed to preserve cosine neighborhoods, the paper should justify this choice or provide a small ablation that computes similarity directly on H, or otherwise show that cosine similarity in X correlates with redundancy in H. This is not a fatal issue, but it is load-bearing for the claimed information-preservation mechanism, especially given the zero-shot drops on fine-grained tasks in Table 2.","section":"Section 3.2, Eq. (1)"}],"minor_comments":[{"comment":"The text reads 'adaptively splitting and fussing the similar adjacent tokens'; this should be 'fusing'.","section":"Section 1, Contributions"},{"comment":"There is a typo 'trianing' in the description of the single-image stage, and 'avaliable' in Section 4.2; both should be corrected.","section":"Section 4.1"},{"comment":"The notation H' ∈ R^{l,demb} should clarify whether the grid markers are included in the count l, since the markers are part of the concatenated token sequence.","section":"Section 3.2"},{"comment":"The claim that 160 frames with DynTok produce 'roughly equivalent' token consumption to 64 baseline frames would benefit from an explicit token-count calculation, including grid-marker tokens, because the current numbers (15k vs. 12.5k) are not exactly matched.","section":"Section 4.2"},{"comment":"The paper first says the baseline is trained on '10 million samples' and later states the video dataset contains 1.79M training samples; please clarify whether the 10M figure includes the single-image stage and whether all video samples are used for the video stage.","section":"Section 4.1 and Table 1"},{"comment":"The rightmost subplot plots accuracy against token count, but the legend labels only frame counts; please make the mapping between frame counts and token counts explicit, since it is central to the token-budget comparison.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a straightforward empirical contribution from an industry group. The method is simple and the experiments are extensive, but the lack of a random/fixed-grouping control and the apparent test-set selection for the headline long-video numbers are real concerns that should be addressed before publication. I would also encourage the authors to release code and the promised training data, since reproducibility is otherwise limited by the proprietary nature of the training mixture."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clean, modest token-compression paper for video LLMs, and the zero-shot evidence is the strongest part. The mechanism attribution to cosine similarity is not yet supported, because there is no control against fixed grouping at the same token budget. Still, the paper deserves serious review.\n\nWhat is actually new: the row-local greedy grouping with per-row markers is a small but real twist on ToMe-style merging, and it is parameter-free and simple to integrate. The best evidence is the zero-shot run: at 44.4% of the tokens, DynTok-zero loses only about one point on Video-MME and stays at parity on most other benchmarks. That is a solid demonstration that the compression preserves enough information without training. The more-frames-for-same-token-budget result in Figure 4 is the most original takeaway: because DynTok cuts per-frame cost, you can sample more frames and the model keeps improving when the baseline plateaus. That tradeoff is genuinely useful.\n\nSoft spots, in order. First, the missing control. The paper attributes the result to similarity-based merging but never compares against a fixed grouping (e.g., merging every k-th patch per row) at identical token counts. If static grouping gives the same accuracy, the adaptive cosine signal is not doing the work. The stress-test note lands here, and the experiment is cheap. Second, the long-video headline numbers (65.3 Video-MME, 72.5 MLVU) come from choosing Sth and frame count after seeing the test curves; there is no held-out split, no error bars, single run. The qualitative trend in Figure 4 is consistent, so I trust the direction more than the exact points. Third, no code or data released, which makes the missing ablation worse. Minor: Table 2 is honest — zero-shot OCR and action drop most, which supports the idea that the proxy is content-dependent.\n\nThe paper is coherent, cites the relevant prior work fairly, and the limitations section is honest about scope. I recommend sending it to peer review. The core 2.2x-reduction-with-parity claim holds as an empirical finding; the mechanism claim needs the fixed-grouping control before it is established.","headline":"Modest but useful token compression; zero-shot evidence is solid, mechanism attribution needs a fixed-grouping control.","tokens_in":13025,"tokens_out":3834,"would_cite":false,"duration_ms":34008,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DynTok compresses a video's visual tokens to 44.4% of their original count while keeping accuracy, and spends the savings on more frames to lift long-video scores.","keywords":["video understanding","visual token compression","token merging","cosine similarity","long video","vision-language model","DynTok","model efficiency"],"falsifier":"On a video benchmark dominated by dense text or small moving objects, run DynTok zero-shot at $S_{\\mathrm{th}}=0.6$ and compare per-task accuracy with the uncompressed baseline; if OCR or action accuracy collapses while the token savings stay near 2.2x, the similarity-is-redundancy assumption fails exactly where fine-grained local detail matters.","tokens_in":12106,"feed_emoji":"🎞️","tokens_out":7205,"duration_ms":62102,"temperature":0.7,"pith_summary":"DynTok is a compression method that needs no learned parameters, and it cuts the visual tokens that a video language model sends to the large language model (LLM) backbone. It examines each row of image patches, measures the cosine similarity between neighboring patch embeddings from the vision encoder, and merges consecutive patches whose similarity exceeds a threshold into a single averaged token. The paper reports that this reduces the token count to 44.4% of the baseline, a 2.2x reduction, while keeping accuracy essentially unchanged. Because each frame uses fewer tokens, the model can afford to take in more frames per video, reaching 65.3% on Video-MME and 72.5% on MLVU. The practical claim is that much of a video's visual signal is redundant at the patch level, and exploiting that redundancy locally preserves spatial layout while costing only linear similarity computation.","feed_headline":"Video tokens cut to 44 percent with no accuracy loss","feed_subtitle":"Row-wise merging of similar patches lets video models use more frames at lower compute and lift long-video scores.","key_machinery":"The central object is the row-wise dynamic token group. For each horizontal line of image patches, DynTok computes the cosine similarity $s = \\frac{x_{k-1} \\cdot x_k}{\\|x_{k-1}\\| \\|x_k\\|}$ between the CLIP/SigLIP representations of neighboring patches and compares it with threshold $S_{\\mathrm{th}}$. Consecutive patches above threshold are fused by averaging into a single visual token; a patch below threshold starts a new group, so high-detail regions naturally keep more tokens. The row-local scope preserves spatial order, and the appended grid marker maintains row boundary information for the LLM. Because only left-neighbor similarities are needed, the compression cost is linear in the number of tokens.","core_discovery":"The paper's central claim is that a purely local, threshold-driven merging of adjacent visual tokens can remove more than half of a video's visual tokens without hurting, and in some settings improving, video understanding. DynTok operates on each row of the patch grid produced by a vision encoder such as SigLIP: a patch starts a new group whenever its cosine similarity to the patch on its left falls below a threshold $S_{\\mathrm{th}}$; otherwise it joins its predecessor's group, and every group is averaged into one token. A grid marker appended to each row tells the LLM where rows end. The authors show this works zero-shot on an already trained model, keeping at least 96% of baseline accuracy on MVBench and above 98% on most other benchmarks while using under half the tokens, and that training with the compressed tokens closes most of the remaining gap. The same token budget can then be spent on more frames, which is why the method's long-video numbers, 65.3% on Video-MME and 72.5% on MLVU, exceed the baseline that uses fewer frames.","pith_inferences":["Editorial inference: the same row-wise merging should transfer to still images with the same threshold, giving image models a content-adaptive token budget; the paper explicitly leaves image understanding for future work.","Editorial inference: since the mechanism compresses before the LLM and the paper's own numbers show almost no zero-shot loss on temporal tasks, pairing DynTok with per-layer token dropping inside the LLM, which the paper lists as unexplored, should compound the savings without much additional accuracy cost.","Editorial inference: the zero-shot drops on OCR and action tasks suggest a cheap extension: make $S_{\\mathrm{th}}$ task- or layer-dependent rather than fixed, which could recover those losses while keeping compression on smooth regions.","Editorial inference: the reported scaling behavior implies a token-budget allocation rule, spend saved tokens on more frames when the video is long and temporal reasoning dominates, and raise $S_{\\mathrm{th}}$ when spatial detail dominates, which the paper sketches but does not state as a policy."],"forward_implications":["A 2.2x token reduction means the LLM's attention cost over the visual prefix drops by roughly 4.8x, since attention scales quadratically with sequence length.","Within a fixed token budget, DynTok can process roughly twice as many frames as the baseline, which is exactly how it converts compression into better long-video accuracy.","The threshold $S_{\\mathrm{th}}$ is a continuous dial: $S_{\\mathrm{th}}=0.4$ gives 5x compression with near-baseline accuracy, $S_{\\mathrm{th}}=0.5$ gives 3x with above-baseline accuracy, and $S_{\\mathrm{th}}=0.6$ gives 2.2x.","Because the compression is parameter-free, any existing video LLM can adopt it without retraining for an immediate token saving, and retraining on merged tokens recovers most of the zero-shot losses."],"supporting_citations":[{"why":"Supplies the LLaVA-OneVision architecture, training procedure, and single-image stage that DynTok builds on and compares against.","marker":"Li et al. (2025)"},{"why":"Provides the SigLIP vision encoder whose contrastively trained representations DynTok uses for cosine similarity and patch encoding.","marker":"Zhai et al. (2023)"},{"why":"Provides the LLaVA-Video training data, evaluation configuration, and baseline to which DynTok is compared.","marker":"Zhang et al. (2024b)"},{"why":"Dynamic-VLM is the pooling-based token compression baseline whose fixed pooling DynTok contrasts with adaptive grouping.","marker":"Wang et al. (2024a)"},{"why":"DyCoke is the prior similarity-based token fusion method for video LLMs that DynTok improves on by keeping fusion local.","marker":"Tao et al. (2024)"},{"why":"Token Merging supplies the underlying idea of fusing similar tokens, which DynTok adapts to row-wise video compression.","marker":"Bolya et al. (2023)"},{"why":"Video-MME is the long-video benchmark where DynTok reports 65.3% accuracy and the per-task breakdown.","marker":"Fu et al. (2024a)"},{"why":"MLVU is the second long-video benchmark where DynTok reports 72.5% accuracy with more frames.","marker":"Zhou et al. (2024)"}],"fun_headline_variants":["Dynamic token merging halve video tokens, keep accuracy","Video tokens cut to 44% with no quality dip","Adaptive row merging trims 56% of video tokens","Token compression lets video models use more frames","Slim video tokens, beef up long-video understanding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire compression rests on cosine similarity between neighboring patches being a faithful measure of redundancy: the paper assumes that when two adjacent patches look alike to the vision encoder, averaging them loses nothing the task needs.","fun_headline_variants_meta":{"raw":{"variants":["Dynamic token merging halve video tokens, keep accuracy","Video tokens cut to 44% with no quality dip","Adaptive row merging trims 56% of video tokens","Token compression lets video models use more frames","Slim video tokens, beef up long-video understanding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000193,"raw_usage":{"total_tokens":1357,"prompt_tokens":961,"completion_tokens":396,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":318}},"tokens_in":577,"tokens_out":396,"duration_ms":4706,"temperature":1.0,"reasoning_tokens":318,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:49:14.571183+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a video benchmark dominated by dense text or small moving objects, run DynTok zero-shot at $S_{\\mathrm{th}}=0.6$ and compare per-task accuracy with the uncompressed baseline; if OCR or action accuracy collapses while the token savings stay near 2.2x, the similarity-is-redundancy assumption fails exactly where fine-grained local detail matters.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Token Merging supplies the underlying idea of fusing similar tokens, which DynTok adapts to row-wise video compression."}],"review_version":1}