{"id":"171601ad-4704-4663-a63b-e5279665f5a3","arxiv_id":"2506.19225","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Video-XL-2 cuts long-video inference cost with chunked pre-filling and query-gated dense-or-sparse KV reloading, reporting half the FLOPs and a third less decoding memory at roughly equal benchmark scores.","lead":"Video-XL-2 is a 7B video-understanding model that watches very long videos by splitting the input into chunks and keeping detailed memory only for the parts a question is about. It reports processing 10,000 frames on a single A100 GPU while staying competitive on long-video benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline SOTA claim is undermined by Table 2's grouping: Seed1.5-VL-8B and Eagle2.5-8B are open-weight 8B models classified as closed-source, and both outperform Video-XL-2 on MLVU dev and VideoMME.","rationale":"I read the paper in good faith: the efficiency story is coherent, the ablation in Table 3 is clean, and the 10,000-frame memory claim is plausible if the chunked KV cache behaves as described. The reader's identified weakest assumption, the unspecified relevance oracle in Section 3.4, is a legitimate reproducibility and correctness concern because a misranking oracle could drop exactly the chunks needed for a fine-grained answer. However, I see a more directly checkable problem with the central claim itself: Table 2 groups Seed1.5-VL-8B and Eagle2.5-8B under closed-source models, yet both are open-weight 8B models on public release channels. If this grouping is incorrect, the abstract's 'outperforming existing open-source lightweight models' is false on the paper's own numbers, regardless of how well the KV-sparsification mechanisms work. This is an external factual check rather than an internal inconsistency, but it is the most load-bearing issue because it targets the headline contribution. The reader's conditional verdict already includes fixing baseline classification, so my recommendation does not move the verdict; it sharpens the condition and supplies a concrete verification step.","tokens_in":12156,"tokens_out":6007,"duration_ms":59926,"concrete_test":"Check the official release pages, model cards, and licenses for Seed1.5-VL-8B and Eagle2.5-8B. If either is open-weight, move it into the 'Open-source Light Models' block of Table 2 and recompute the comparisons across MLVU dev, VideoMME, LongVideoBench, LVBench, and V-STaR. If Video-XL-2 remains ahead on all reported columns, the SOTA claim survives this specific objection; if not, the Abstract and Section 1 must be revised to state the benchmarks on which the claim actually holds and the claim should be downgraded accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is not internal to the KV-sparsification mechanism but to the comparative claim it supports. The Abstract and Section 1 (contribution 1) state that Video-XL-2 achieves state-of-the-art performance, outperforming all existing open-source lightweight (7B/8B) MLLMs. Table 2, however, places Seed1.5-VL-8B and Eagle2.5-8B under 'Closed-source Models' even though both are publicly released open-weight 8B models. Under the paper's own definition, these baselines belong in the open-source lightweight comparison set. On MLVU dev, Video-XL-2 scores 74.8 while Seed1.5-VL-8B scores 82.1 and Eagle2.5-8B scores 77.6; on VideoMME, Video-XL-2 scores 66.6 while Eagle2.5-8B scores 72.4. The preprint gives no justification for the closed-source categorization, and the cited reports document open releases. If the classification is wrong, the paper's headline superiority claim is contradicted by its own table. This is load-bearing because the abstract and contributions lead with this SOTA claim, independent of whether the chunk-based prefilling and bi-level KV decoding mechanisms are effective.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Video-XL-2, a 7B-parameter video MLLM built on Qwen2.5-7B, with a vision encoder, dynamic token synthesis (DTS), MLP projector, and an LLM. Its efficiency contributions are chunk-based pre-filling (full attention within chunks, sparse attention across chunks with historical timestamp tokens) and bi-level KV decoding (dense KVs for query-relevant chunks, downsampled sparse KVs for the rest, selected via a relevance oracle). The authors claim state-of-the-art results on long-video understanding benchmarks among open-source lightweight models, processing over 10,000 frames on an 80GB A100, and report substantial FLOPs and KV-cache reductions. The paper includes training stages, main benchmark results in Table 2, efficiency ablations in Table 3, and a needle-in-haystack evaluation.","tokens_in":12447,"tokens_out":3752,"duration_ms":38911,"significance":"The efficiency mechanisms, especially chunk-based pre-filling with timestamp tokens and task-aware KV selection, are practically motivated and, if correctly evaluated, would be a useful contribution to long-video MLLM inference. The authors have publicly released the model and provide concrete training and efficiency numbers, which supports reproducibility of the method's architecture. However, the paper's central comparative claim is currently undercut by its own Table 2, and the unspecified relevance oracle makes the main efficiency-performance tradeoff difficult to verify. As a result, the headline significance claims are not yet established in the submitted version.","major_comments":[{"comment":"The claim that Video-XL-2 'outperforms all existing open-source lightweight (7B or 8B) MLLMs' is contradicted by the paper's own table. Seed1.5-VL-8B and Eagle2.5-8B are listed under 'Closed-source Models' even though both are publicly released open-weight 8B models. On MLVU Dev, Seed1.5-VL-8B scores 82.1 and Eagle2.5-8B scores 77.6, while Video-XL-2 scores 74.8; on VideoMME, Eagle2.5-8B scores 72.4 versus Video-XL-2's 66.6. Under the paper's own classification criterion, these baselines belong in the open-source lightweight comparison set, so the abstract and contribution 1 overstate the result. This must be corrected by regrouping the baselines or by revising the superiority claims.","section":"Table 2 and Section 1 (Contribution 1)"},{"comment":"The relevance oracle that selects dense versus sparse KV chunks is never specified for the main results. The text says the oracle 'can be implemented via various methods such as an MLLM embedder, attention score mechanism or a more complex, custom-designed relevance computation pipeline,' but no concrete implementation, hyperparameters, or dense-chunk selection ratio are given for the experiments in Table 2 and Table 3. Without this information, the reported 38.8% average KV-cache reduction and the performance preservation in Table 3 cannot be reproduced or audited. Please disclose the exact oracle used, its parameters, and an ablation showing sensitivity to oracle quality.","section":"Section 3.4 (Bi-level KVs Decoding)"},{"comment":"The efficiency comparison uses 'Visual Input Length (k)' and 'FLOPs (G)' computed from 'the average of the maximum frame counts utilized across all long video understanding benchmarks.' This definition is ambiguous: it is unclear whether FLOPs include vision encoder and DTS computations, how the maximum frame count is determined for each model, and how different sampling strategies affect the comparison. Since the paper's efficiency advantage is a central contribution, please provide a precise methodology for both metrics and, ideally, report the underlying frame counts per benchmark.","section":"Section 4.1 and Table 2 (Efficiency metrics)"},{"comment":"MLVU [49] is authored by members of the same research group, and the paper highlights MLVU Dev as the headline result where Video-XL-2 surpasses GPT-4o. To rule out evaluation-protocol bias or accidental training-data overlap, please disclose whether any MLVU data was used in training and provide a detailed evaluation protocol, including how the subset of MLVU Dev used for Table 2 was selected. Independent confirmation on benchmarks not authored by the group would substantially strengthen the claim.","section":"Section 4.2 and MLVU benchmark"}],"minor_comments":[{"comment":"There are several typos in the related work, such as 'LLaV A' instead of 'LLaVA,' and 'MLP projetor' instead of 'MLP projector' in Section 3.3.","section":"Section 2"},{"comment":"The column headers 'M-avg' and 'M-avg w/o sub' are not defined; please spell out the metrics and specify which benchmarks contribute to each average.","section":"Table 2"},{"comment":"The frame sampling strategy depends on a 'pre-defined frame upper bound' and a 'pre-defined maximum sampling rate,' but neither value is stated; please report the actual values used in the experiments.","section":"Section 3.2"},{"comment":"The needle-in-haystack evaluation is described only qualitatively; please provide the number of questions, the scoring rule, and numeric results for Video-XL-2 and Video-XL so the claimed ability to process 10,000 frames can be assessed.","section":"Section 4.4 and Figure 5"},{"comment":"The rows 'Video-XL2 100% 100%' and '+Bi-level KVs Decoding 48.2% 61.2%' are inconsistent with the reported average FLOPs and KV-cache values: please clarify whether the first row is the dense baseline and how the percentages are normalized.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper's technical core is interesting, but the current version has a serious internal inconsistency in Table 2 that directly undermines the abstract and contribution 1. The unspecified relevance oracle is a second load-bearing gap. I recommend major revision rather than rejection because both issues appear fixable via regrouping/claim softening and by disclosing the oracle and evaluation details. I would also ask the editor to pay attention to the overlap between the authors and the MLVU benchmark; a transparency statement and independent evaluation would help."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked for a read on Video-XL-2. The short version: the efficiency work is real and worth attention, but the paper's headline \"outperforming all open-source lightweight MLLMs\" is false on its own numbers, and the key decoding oracle is undisclosed. I'd send it to referees, but with a clear expectation of major revision.\n\nWhat's genuinely new: chunk-based pre-filling (full attention within chunks, sparse attention across chunks via historical timestamp tokens and a sliding window) and bi-level KV decoding (dense vs. pooled-sparse KVs per chunk, selected by query relevance). The internal ablation (Table 3) is clean: chunk-based pre-filling cuts FLOPs to 48.8% with less than 0.5 average performance drop, and bi-level KV decoding cuts decoding KV cache to 61.2% while recovering and slightly improving scores. The measured 10,000-frame processing on an 80GB A100 and the linear pre-filling scaling are concrete, practical results. The authors also engage with their own prior Video-XL line and with ReTaKe; the bi-level dense/sparse selection is a real, sensible idea.\n\nThe soft spots are load-bearing. First, the abstract and contribution 1 claim SOTA among open-source lightweight (7B/8B) models, but Table 2 classifies Seed1.5-VL-8B and Eagle2.5-8B as closed-source. Both are publicly released open-weight models. Under the paper's own definition, they belong in the open lightweight set. On MLVU Dev, Seed1.5-VL-8B scores 82.1 vs. Video-XL-2's 74.8; Eagle2.5-8B scores 77.6. On VideoMME, Eagle2.5-8B scores 72.4 vs. 66.6. The headline claim is contradicted by the paper's own table. That is not a minor presentational slip.\n\nSecond, the relevance oracle is unspecified. Section 3.4 lists options (MLLM embedder, attention scores, custom pipeline) but never says which one produced the main results. The entire decoding-stage memory savings depend on the oracle correctly ranking chunks: if it misranks, the fine-grained KVs for the part of the video the query needs are dropped. Without disclosure, the results aren't reproducible and the \"task-aware\" claim can't be checked. The references to the authors' own retrieval systems suggest a custom pipeline, but the reader is left guessing.\n\nMinor issues: the repository link is a placeholder, there are no error bars on the ablations, and MLVU Dev is authored by the same group. Those matter but are secondary.\n\nWho is this for: practitioners and researchers working on long-video deployment and KV-cache compression for multimodal LLMs. The efficiency mechanism is plausible enough to deserve a serious referee. I'd expect reviewers to demand a corrected baseline table, full disclosure of the oracle and dense-chunk selection policy, actual code, and ideally error bars. As written, it's a conditional accept at best.","headline":"Solid efficiency engineering, but the headline SOTA claim is contradicted by the paper's own table and the core oracle is a black box.","tokens_in":13079,"tokens_out":4860,"would_cite":false,"duration_ms":47419,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Video-XL-2 claims to beat every open-source lightweight MLLM on long-video benchmarks while cutting prefill FLOPs roughly in half and running 10,000 frames on a single A100 GPU.","keywords":["long video understanding","multimodal large language model","KV cache sparsification","chunk-based pre-filling","bi-level KV decoding","temporal grounding","inference efficiency","video question answering"],"falsifier":"Pick a benchmark question whose answer depends on a detail in one brief moment, then deliberately make the relevance scorer rank that moment lowest; if the model still answers, the dense-versus-sparse distinction is not what preserves accuracy, and if it fails, the published results hinge entirely on scorer quality. A companion check is to replace the scorer with a random number generator and rerun MLVU, VideoMME, and LongVideoBench; if accuracies stay near the reported 74.8, 66.6, and 61.0, the query-aware selection is not doing the work claimed.","tokens_in":11896,"feed_emoji":"🎬","tokens_out":8935,"duration_ms":85247,"temperature":0.7,"pith_summary":"The paper tries to settle a practical trade-off: long-video understanding models are either accurate but too expensive, or fast but lossy. Video-XL-2's wager is that most of a long video is irrelevant to any particular question, so most of the video should be stored and read at coarse grain, with fine detail reserved for the few chunks a query actually touches. To that end it introduces two mechanisms, chunk-based pre-filling and bi-level KV decoding, that together claim to halve prefill compute, cut decoding KV cache by nearly forty percent, and still improve benchmark scores. If the claims hold, hour-scale video understanding becomes feasible on a single 80GB GPU, and open 7B/8B models can compete with much larger and closed models.","feed_headline":"Long-video AI tops lightweight rivals at 10,000 frames on one GPU","feed_subtitle":"Chunked pre-filling and query-aware KV reloads let a 7B model answer hour-long videos fast.","key_machinery":"The load-bearing object is task-aware KV sparsification, realized as two mechanisms. Chunk-based pre-filling splits the visual token sequence into equal chunks; each chunk attends fully to itself and sparsely to earlier chunks through historical timestamp tokens and a sliding window of overlap, so prefill cost scales with chunk length and roughly linearly with frame count instead of quadratically. Bi-level KV decoding keeps, for every video chunk, both the original dense key-value pairs and a downsampled sparse version; at query time a relevance oracle scores each chunk and the decoder loads dense KVs for high-scoring chunks and sparse KVs for the rest, producing a mixed KV set that is lighter and, the paper argues, more focused. The oracle is the component that decides where fine detail is preserved.","core_discovery":"Video-XL-2 argues that the reason long-video MLLMs are expensive is not that they need all their visual tokens at full resolution everywhere; it is that they keep dense key-value state for the entire input even when a specific question only needs fine detail in a few places. The paper's claim is that by running pre-filling in chunks, with full attention inside each chunk and sparse attention across chunks through timestamp tokens and a sliding window, and by maintaining two resolutions of KV cache per chunk, dense and downsampled, then loading dense KVs only for chunks a relevance oracle ranks as important to the query, a 7B model can match or beat all open-source lightweight rivals on long-video benchmarks while using 48.8% of the original prefill FLOPs and 61.2% of the original decoding KV cache. State-of-the-art results are reported on MLVU dev/test, VideoMME without subtitles, LVBench, and VideoEval-Pro among lightweight open models, with competitive temporal grounding on Charades-STA and V-STaR.","pith_inferences":["The headline efficiency numbers obscure one degree of freedom: Table 3 compares the full oracle-based pipeline against no sparsification, so the contribution of the relevance ranking itself is not isolated. A random-selection ablation would separate downsampling from task-awareness.","The near-linear prefill scaling suggests the 10,000-frame ceiling is a hardware boundary rather than a method boundary; multi-GPU or stronger pooling could plausibly reach feature-length video, but correctness at that scale is untested.","The historical timestamp tokens function as a compressed summary of past chunks, so the relevance oracle could be replaced by an actual video-to-text retriever, connecting this decoding scheme with retrieval-augmented long-video systems."],"forward_implications":["Long-video question answering on hour-scale footage becomes feasible on a single 80GB GPU: the paper reports processing up to 10,000 frames on one A100 and thousands of frames in seconds.","Prefill cost grows nearly linearly rather than quadratically with frame count, so the same recipe should keep working on still longer inputs, limited mainly by hardware rather than attention complexity.","Keeping dense detail only where the query points and coarse detail elsewhere gives other multimodal models a new design point: efficiency and accuracy need not be traded off, as Table 3 shows FLOPs falling to 48.8% with small benchmark gains.","Temporal grounding in long video should improve because the timestamp tokens that carry cross-chunk history give the model explicit time anchors; the paper reports competitive results on Charades-STA and V-STaR."],"supporting_citations":[{"why":"Supplies the Dynamic Token Synthesize (DTS) compression module and the reconstruction-based pre-training that the architecture inherits.","marker":"[16]"},{"why":"The earlier chunked Video-XL model that Video-XL-2 builds on; its 2048-frame limit in the needle-in-haystack test is the baseline the new model exceeds.","marker":"[40]"},{"why":"ReTaKe, a chunk-level KV-dropping baseline; it motivates the query-adaptive design by showing the limits of fixed-ratio drops.","marker":"[41]"},{"why":"VideoChat-Flash, the strongest lightweight efficiency baseline; its FLOPs and benchmark numbers are the direct comparison for the reported gains.","marker":"[6]"},{"why":"SnapKV, cited as evidence that query-relevant KV sparsity exists before generation, grounding the bi-level decoding choice.","marker":"[18]"},{"why":"SigLIP, the vision encoder that produces the dense per-frame visual features.","marker":"[42]"},{"why":"Qwen2.5-7B, the LLM backbone whose KV cache the two efficiency mechanisms must compress.","marker":"[43]"},{"why":"MLVU, the long-video benchmark whose dev/test scores anchor the state-of-the-art claim.","marker":"[49]"},{"why":"VideoMME, the benchmark for the no-subtitle accuracy used as the main general long-video comparison.","marker":"[50]"},{"why":"LongVideoBench, the retrieval-focused long-video benchmark used in the efficiency ablation.","marker":"[51]"}],"fun_headline_variants":["Video-XL-2: Sparse KV cache slashes long-video AI cost","Chunked pre-filling + task-aware KV cuts long-video memory","7B model beats lightweight rivals on 10k-frame videos","Long-video MLLM uses sparse KV cache to run 10k frames on one GPU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's accuracy rests on an assumption about the scoring step: it must rarely rank the one chunk that contains the answer as irrelevant, because chunks marked irrelevant are stored and read only at coarse resolution.","fun_headline_variants_meta":{"raw":{"variants":["Video-XL-2: Sparse KV cache slashes long-video AI cost","Chunked pre-filling + task-aware KV cuts long-video memory","7B model beats lightweight rivals on 10k-frame videos","Long-video MLLM uses sparse KV cache to run 10k frames on one GPU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000842,"raw_usage":{"total_tokens":3700,"prompt_tokens":1008,"completion_tokens":2692,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":2606}},"tokens_in":624,"tokens_out":2692,"duration_ms":19030,"temperature":1.0,"reasoning_tokens":2606,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:07:45.383929+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pick a benchmark question whose answer depends on a detail in one brief moment, then deliberately make the relevance scorer rank that moment lowest; if the model still answers, the dense-versus-sparse distinction is not what preserves accuracy, and if it fails, the published results hinge entirely on scorer quality. A companion check is to replace the scorer with a random number generator and rerun MLVU, VideoMME, and LongVideoBench; if accuracies stay near the reported 74.8, 66.6, and 61.0, the query-aware selection is not doing the work claimed.","supporting_citations":[{"cited_title":"Qwen2.5 technical report","cited_arxiv_id":null,"evidence_quote":"Qwen2.5-7B, the LLM backbone whose KV cache the two efficiency mechanisms must compress."}],"review_version":1}