{"id":"f2d91d50-204f-4894-b05c-770bb8c4f2ab","arxiv_id":"2506.21184","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Video-X2L uses bi-level KV compression with task-aware selective reloading to improve long-video QA accuracy and reduce decode-time memory versus uniform KV compression.","lead":"A new training-free method, Video-X2L, compresses long-video KV caches into fine and coarse levels during pre-filling, then reloads fine-grained caches only for the chunks a relevance model deems most important for the question. On LVU benchmarks it improves on the uniform-compression baseline Video-XL at similar or lower memory cost, though gains are modest and tuned per benchmark.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline advantage rests on per-benchmark top-k tuning on the test benchmarks; a fixed-k re-evaluation is needed before the claim is trusted.","rationale":"The reader's weakest assumption focused on the relevance oracle's accuracy, but the paper's own experiments (Table 3) show modest sensitivity to oracle choice and the limitation is explicitly conceded. The more consequential concern is that the claimed advantage is selected over top-k on the test benchmarks, which is a methodological artifact rather than a model-capability limitation. If a fixed k removes the margins, the central 'huge advantage' claim fails; if the margins persist, the method is genuinely robust. This does not overturn the CONDITIONAL verdict, but it sharpens the condition: the authors must provide out-of-sample or fixed-top-k results, which is a concrete and feasible check. Therefore the verdict remains CONDITIONAL, and my read does not change the reader's verdict.","tokens_in":14031,"tokens_out":2082,"duration_ms":25733,"concrete_test":"Re-run the main experiments in Table 1 with a single fixed top-k value (e.g., k=3, matching the ablations in Tables 2 and 3) across all benchmarks, and report MLVU Dev, LongVideoBench, and VNBench. If any margin versus Video-XL: 2x becomes non-positive, or the average improvement over Video-XL: 2x drops below roughly 1 point, then the per-benchmark top-k tuning is load-bearing. Also report the standard deviation across at least three repeated runs for the key configurations.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Video-X2L outperforms existing KV-compression methods by a large margin (Table 1). Section 4.1 states that Video-X2L is optimized by tuning the top-k parameter of L-KV selection for each task across all benchmarks, with k ranging from 1 to 5. This means the reported numbers are likely maxima over a small hyperparameter grid evaluated on the same test sets used for comparison. Appendix A.2 shows accuracy varies nonmonotonically with k and the optimal k differs across task types, so per-benchmark test-set tuning can inflate every reported advantage. The relevance-oracle limitation is honestly ablated in Table 3, where several oracles perform similarly, and in Section 4.4 it is explicitly acknowledged; the top-k tuning is not similarly controlled. Without a fixed-k result or validation-based selection, the reported improvements over Video-XL: 2x and 32x are not clearly out-of-sample and the 'huge advantage' is not yet established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Video-X^2L, a training-free extension of KV-compressible video MLLMs. During pre-filling, the video is encoded twice into low-compression KVs (L-KVs) and high-compression KVs (H-KVs). During decoding, a relevance oracle (LanguageBind in the main experiments) scores each video chunk against the task, and L-KVs are reloaded for the top-k chunks while H-KVs are used for the rest. Experiments on MLVU, VideoMME, LongVideoBench, VNBench, and a needle-in-a-haystack setting report that Video-X^2L outperforms Video-XL at both 2x and 32x compression while cutting decode-time cache by about 86% and speeding up decoding by about 4.5x. Ablations examine the role of the relevance oracle, the compression-ratio pair, the top-k value, and the two KV types.","tokens_in":14161,"tokens_out":2907,"duration_ms":35867,"significance":"If the reported gains are out-of-sample, the paper would make a useful practical contribution: a training-free, plug-in mechanism that preserves fine-grained detail exactly where a given question needs it, at a fraction of the decode-time cost of uniform low compression. The paper deserves credit for being explicit about the mechanism, for releasing code, and for ablating the relevance oracle and the two KV types (Tables 3 and 6). The central empirical claim, however, is currently weakened by test-set hyperparameter tuning, so the significance of the headline numbers is not yet established.","major_comments":[{"comment":"The main results are produced after tuning top-k separately for each benchmark on the same test sets, with k ranging from 1 to 5 (Section 4.1). Appendix A.2 shows that accuracy is nonmonotonic in k and that the optimal k differs across task types. The numbers in Table 1 are therefore best-case selections over a test-set grid, and the reported '+1.7' or '+2.5' improvements over Video-XL are not clearly out-of-sample. Please report results with a fixed k (for example, k=3, which is used in most ablations), or select k on a validation split, and show that the headline conclusions are unchanged.","section":"Section 4.1, Table 1, Appendix A.2"},{"comment":"The default compression-ratio pair (2x low, 32x high) is the best-performing row in Table 2 on MLVU. This is another selection made after looking at test performance. The paper should either state that the pair was fixed before the main comparison, or provide a robustness analysis showing that the conclusions in Table 1 hold for a range of pairs (e.g., 2x16x, 2x32x, 2x72x) rather than only for the empirically best one.","section":"Section 4.4, Table 2"},{"comment":"Table 3 shows that on MLVU the Random baseline (63.7) and the Uniform heuristic (64.0) are close to the LanguageBind oracle (65.4), and on LongVideoBench Random (50.3) and Uniform (50.6) are close to LanguageBind (51.7). This weakens the claim that task-aware scoring is what drives the improvement: much of the gain could come from simply having some L-KVs in the context. Please add a discussion of this gap, report variance or significance across oracle choices, and temper the 'task-aware' causal claim accordingly.","section":"Section 4.6, Table 3"}],"minor_comments":[{"comment":"There are several typos and duplicated phrases, e.g., 'geneartes' in Section 1, 'oeprations' in the contribution list, and the repeated sentence 'This is because the two key innovative mechanism...' in Section 4.5. A careful proofread is needed.","section":"Throughout"},{"comment":"The caption says 'Video-XL: m×n× uses a low compression ratio of m and a high compression ratio of n', but the settings are configurations of Video-X^2L, not Video-XL. This should be corrected to avoid confusion.","section":"Table 2 caption"},{"comment":"The text refers to 'Recall@topk' but the metric is not defined, and the figure is not included in the provided text. Please define the metric and ensure the figure is actually present.","section":"Appendix A.2, Figure 3"},{"comment":"The abbreviation 'M-avg' is used without a definition, and 'LongVideo.' is an unclear truncation of LongVideoBench. Please expand these terms in the table caption or footnote.","section":"Table 1"},{"comment":"The notation 'K(V:i)' and 'V(V:i)' is ambiguous; it should be clarified whether these denote the KVs of all previous visual summary tokens before chunk i, and the indexing should be made precise.","section":"Section 3.3, Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The central idea is interesting and the paper is generally readable, but the main comparative claim rests on per-benchmark test-set tuning of top-k and on selecting the best compression-ratio pair from Table 2. A fixed-k or validation-based evaluation is essential before this can be accepted. The relevance-oracle ablation also deserves a more measured interpretation, since random and uniform selection capture much of the observed gain. I would encourage the editor to request the additional experiments rather than reject, because the mechanism is simple, training-free, and potentially useful."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. The core idea is clean: run the Video-XL compression twice, once at low and once at high ratio, then use an external relevance oracle to decide which chunks get the fine-grained L-KVs at decode time. The assembly is new even though each part has precedents, and it is training-free, which makes it immediately practical if it works. The ablations are more honest than most: Table 6 shows both KV types earn their keep, Table 3 shows the oracle choice is not knife-edge, and the 72x compression drop is explicitly blamed on the oracle's limits. I appreciate that they admit timestamps break their oracle in the limitation section.\n\nThe soft spot is real and it is exactly what the stress-test note says: Section 4.1 says top-k was tuned per benchmark with k from 1 to 5, and the results in Table 1 and Table 2 come after that tuning. Since the same test sets are used for comparison, the reported margins over Video-XL are maxima over a small grid, not out-of-sample numbers. The paper shows the metric is nonmonotonic in k, so this is not a harmless knob. Missing error bars and the placeholder repo also matter. The efficiency claims (86.1% cache reduction, 4.5x speedup) depend on the chosen 2x/32x ratio, which is the best row in Table 2; that is fine as a demonstration, but it is not a free lunch. LongVU is cited but not run as a main-table baseline, which weakens the \"huge advantage\" framing.\n\nNone of this kills the mechanism. The design is plausible, the oracle limitation is honestly shown, and the VLNIAH result at 2048 frames suggests the idea has legs. The fix is straightforward: rerun with a single fixed k across all benchmarks, or select k on a validation split, report variance across a few runs, and release working code. If the advantage survives that, this is a broadly useful inference-time trick for long-video QA.\n\nI would send it to peer review, but with a clear request for a fixed-k or validation-selected re-evaluation before the headline numbers are accepted. The reader's conditional verdict is about right; my own score would be conditional in the same way.","headline":"A genuinely useful, training-free KV-compression idea that is well ablated but whose headline numbers are tuned per benchmark and need a fixed-k out-of-sample rerun before they can be believed.","tokens_in":14809,"tokens_out":764,"would_cite":true,"duration_ms":10600,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By keeping fine detail only for task-critical chunks, Video-X2L beats uniform KV compression on long-video QA while cutting cache size by 86 percent.","keywords":["long-video understanding","KV cache compression","multimodal large language models","task-aware selection","selective KV re-loading","relevance oracle","inference efficiency","compression ratio"],"falsifier":"Pick a long video and a question whose answer depends on a detail inside a chunk that the relevance oracle ranks below the top-k, run Video-X2L with and without L-KV reloading, and compare accuracies on such items. If accuracy drops to the no-L-KV baseline whenever the oracle ranks the critical chunk low, then the oracle, not the reloading mechanism, is the binding constraint; if accuracy stays high even under a deliberately bad oracle, then the H-KVs alone retain enough detail to support the central claim.","tokens_in":13744,"feed_emoji":"🎬","tokens_out":10298,"duration_ms":108559,"temperature":0.7,"pith_summary":"Long-video question answering is expensive because every frame's visual tokens become key-value (KV) cache entries the model must attend to during generation. Uniform KV compression shrinks the whole video at one fixed rate, so at high ratios it loses exactly the fine details detailed questions need. Video-X2L instead compresses each video chunk twice: once into fine-grained low-compression KVs (L-KVs) and once into abstract high-compression KVs (H-KVs). For a given question, a relevance oracle scores every chunk, and only the top-k chunks are decoded with L-KVs while the rest stay at H-KV compression. The paper reports that this task-aware redistribution outperforms uniform low compression on several long-video benchmarks while reducing the KV cache by 86.1% and speeding decoding by 4.5x.","feed_headline":"Video-X2L cuts video-AI memory 86% while beating fixed compression","feed_subtitle":"Stores fine detail only for frames a question needs, beating uniform compression on four long-video benchmarks.","key_machinery":"The central mechanism is bi-level KV compression followed by selective KV re-loading. A KV cache stores the key and value vectors that a transformer's attention layers compute for each token; in a KV-compressible multimodal large language model, visual summary tokens inserted into each video chunk absorb the chunk's information, and their KV states become a compressed stand-in for the chunk. Video-X2L runs this compression twice per chunk to produce L-KVs (small compression ratio, fine detail) and H-KVs (large compression ratio, abstract summary), keeps both in CPU memory after the first pass, and at decode time uses a relevance oracle to rank chunks for the given question. Only the top-k chunks' L-KVs are moved back to GPU memory, merged with the H-KVs and re-ordered by temporal position, so the final context has hybrid compression ratios. This is what carries the argument: the model sees most of the video at high compression but decodes the few chunks that matter for the question at near-full detail.","core_discovery":"Video-X2L's central claim is that the information lost by high KV compression is not equally costly across a video: most long-video tasks rely on a small number of critical chunks, so the compression budget should be allocated per task instead of globally. In the pre-filling stage, the model produces both L-KVs and H-KVs for every chunk (for example, 2× and 32× compression), and during decoding it re-loads L-KVs only for the top-k chunks ranked by a relevance oracle such as LanguageBind, using H-KVs elsewhere. The paper reports that on MLVU, LongVideoBench, and VNBench this hybrid cache beats uniform 2× compression while cutting KV cache size by 86.1% and achieving a 4.5x decoding speedup at the default 2×/32× setting. The approach is training-free, plug-compatible with any KV-compressible MLLM, and its advantage grows at higher compression ratios and longer videos.","pith_inferences":["A natural stress test the paper does not run is to make the relevance oracle adversarially wrong: if a question-relevant chunk is always ranked below the top-k, Video-X2L should behave like the H-KV-only baseline, which would confirm the oracle as the bottleneck and define the boundary of the method's promise.","The same two-level cache idea transfers to other long multimodal inputs, such as multi-page documents or high-resolution image grids, where most queries depend on localized regions and a cheap retriever could pick which pages or patches get fine-grained KVs.","Because the paper tunes top-k per task, replacing the fixed k with a confidence threshold or learned policy over the oracle's score distribution could remove a per-benchmark hyperparameter and make the method fully adaptive.","The paper's savings are measured at decode time; the pre-filling stage still runs two compression passes. Deriving H-KVs from the already-computed L-KVs, or making the second pass cheaper, is a direct extension that would push the efficiency gain to the whole pipeline."],"forward_implications":["Any existing KV-compressible MLLM can adopt Video-X2L without retraining; the paper shows gains with Video-XL-7B as the backbone and with several relevance oracles including LanguageBind, InternVideo2, SigLIP, and attention scores.","At the default 2×/32× setting, Video-X2L reduces KV cache by 86.1%, shortens time-to-first-token from 139.7 ms to 52.1 ms on MLVU, and still beats uniform 2× compression on MLVU, LongVideoBench, and VNBench.","On detail-oriented tasks, including single- and multi-detail MLVU categories and VNBench needle tasks, the selective reloading is where the gains concentrate; holistic tasks stay comparable because the first and last chunks are always kept at low compression.","In the extra-long Needle-In-A-Haystack setting, Video-X2L supports up to 2,048 frames on one A100 GPU and keeps 100% needle accuracy below 1,000 frames, degrading more slowly than Video-XL beyond that length.","A light post-training on 4K video QA samples widens the gap over the backbone on MLVU and VideoMME, indicating the mechanism is compatible with further adaptation."],"supporting_citations":[{"why":"Supplies the Video-XL backbone and chunked visual-summary-token compression workflow that Video-X2L runs twice to produce L-KVs and H-KVs.","marker":"[29]"},{"why":"Serves as the default relevance oracle that scores each chunk's importance to a given task.","marker":"[47]"},{"why":"Provides the MLVU benchmark whose holistic, single-detail, and multi-detail splits drive the task-type analysis.","marker":"[46]"},{"why":"Provides VideoMME, whose timestamped questions expose the relevance oracle limitation and where gains are smallest.","marker":"[8]"},{"why":"Provides LongVideoBench, one of the benchmarks where the method gains most over uniform compression.","marker":"[37]"},{"why":"Provides VNBench needle tasks that test fine detail perception and where Video-X2L reports leading performance.","marker":"[45]"},{"why":"Supplies the LongVA baseline and the Needle-In-A-Haystack evaluation construction for extra-long video comparisons.","marker":"[42]"},{"why":"Supplies VideoRope QA pairs used for the extended V-NIAH evaluation.","marker":"[36]"}],"fun_headline_variants":["Video-X2L caches only key frames: 86% less memory, 4.5x faster","Task-aware KV loading beats uniform compression on long-video tasks","Training-free Video-X2L reloads detail only for critical chunks","Video-X2L outperforms uniform KV compression with selective cache"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the relevance oracle correctly ranks the genuinely critical chunks at or near the top; when it fails, as the paper notes for timestamped questions, those chunks are decoded from heavily compressed H-KVs and the fine detail is lost.","fun_headline_variants_meta":{"raw":{"variants":["Video-X2L caches only key frames: 86% less memory, 4.5x faster","Task-aware KV loading beats uniform compression on long-video tasks","Training-free Video-X2L reloads detail only for critical chunks","Video-X2L outperforms uniform KV compression with selective cache"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000394,"raw_usage":{"total_tokens":2114,"prompt_tokens":1038,"completion_tokens":1076,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":994}},"tokens_in":654,"tokens_out":1076,"duration_ms":10600,"temperature":1.0,"reasoning_tokens":994,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:32:12.262799+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pick a long video and a question whose answer depends on a detail inside a chunk that the relevance oracle ranks below the top-k, run Video-X2L with and without L-KV reloading, and compare accuracies on such items. If accuracy drops to the no-L-KV baseline whenever the oracle ranks the critical chunk low, then the oracle, not the reloading mechanism, is the binding constraint; if accuracy stays high even under a deliberately bad oracle, then the H-KVs alone retain enough detail to support the central claim.","supporting_citations":[],"review_version":1}