{"id":"f4ee81a5-a234-4f7a-b45b-a645842156ef","arxiv_id":"2607.23193","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A modality-decoupled, query-anchored token compression method preserves audio and video cues independently and outperforms unidirectional compression baselines on omnimodal QA at 25-45% token retention.","lead":"OmniScope is a training-free system that compresses audio and video tokens for multimodal AI by scoring each modality separately against the user query, instead of letting one modality decide what the other keeps. On four audio-video QA benchmarks it reports the best average accuracy among compression methods, with faster prefill and lower memory at high compression.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 7's proportional budget allocation is undefined for negative cosine scores, which CLIP and audio-cosine scores produce; the paper never specifies a non-negative transform.","rationale":"The reader identifies query-score proxy validity as the weakest assumption. I agree that is a relevant concern, but a more fundamental, reproduction-blocking issue is that the budget-allocation formula in Eq. 7 is mathematically undefined when scores are negative. The paper's own scoring definitions (Eq. 3 and Eq. 5) produce cosine similarities that are not bounded below by zero, and a long video will contain many frames/seconds that are semantically unrelated to the query. Without a non-negativity transformation, Eq. 7 would allocate negative token counts, which is impossible. Since the released code is the only place that could resolve this, the empirical claims cannot currently be independently verified from the text alone. This is an internally load-bearing omission: the central comparison to OmniZip and the claimed 'best average accuracy' rest on the allocation step working. If the code applies a hidden normalization, the paper is incomplete rather than wrong, and the reader's CONDITIONAL verdict remains appropriate. I therefore recommend UNCHANGED (CONDITIONAL), but for a different reason than the reader's. My concern is not about whether CLIP similarity is an ideal proxy, but about whether the allocation formula can be executed as written at all.","tokens_in":17252,"tokens_out":5965,"duration_ms":92766,"concrete_test":"Inspect the released code (or reimplement from the paper) and record the raw scores s_i for all frames/seconds before Eq. 7 on a small subset of WorldSense/DailyOmni. If any s_i < 0 and no explicit non-negative transform is applied, the allocation is invalid. If the code does transform scores, verify whether Eq. 7 as written matches the executed operation; if not, the paper must be corrected to state the transform. A useful additional check: replace all scores with |s_i| or with random positive scores; if the accuracy gap vs. OmniZip disappears, the specific allocation formula is load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on query-conditioned scores driving token allocation. In Sec. 3.3.3, Eq. 7 sets n_keep(i) = floor( s_i / sum_j s_j * N_budget + 0.5 ). But s_i is a raw cosine similarity (Eq. 3, CLIP) or a weighted average of cosine similarities (Eq. 5), both of which can be negative. With negative s_i, the formula yields negative or zero retention counts at low-salience positions, which is not a valid token budget. The paper never specifies a shift, ReLU, min-max normalization, or softmax to make scores non-negative. Without this, the allocation step is mathematically ill-defined; with an undocumented transform, the method is not reproducible from the paper. Since the budget allocation is the mechanism that implements modality-decoupled, query-aware compression, the entire empirical comparison rests on an unstated assumption. Appendix D's use of CLIP because internal vision-text alignment is weak makes the visual scores especially likely to be negative for many frames in a long video.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OmniScope, a training-free token compression framework for omnimodal LLMs, instantiated on Qwen2.5-Omni at 7B and 3B scales. Its central idea is that query-relevant moments differ between audio and video, so compressing one modality using the other modality's salience (as in OmniZip) is fragile. OmniScope therefore estimates relevance separately: CLIP score per frame for video, a gated cosine similarity between audio-encoder outputs and query embeddings for audio, then allocates per-position token budgets proportionally. Visual tokens are pruned with an anchor/delta spatio-temporal scheme, and audio tokens are merged per second via bipartite soft matching. Experiments cover WorldSense, DailyOmni, OmniVideoBench, and Video-MME at 45% and 25% overall retention, reporting the best average accuracy among compression baselines in all four settings, with up to 3.53x prefill speedup and over 15% GPU memory reduction at 25% retention.","tokens_in":17536,"tokens_out":7415,"duration_ms":79179,"significance":"If the results are reliable, OmniScope is a useful, simple, and training-free contribution: it introduces a clean design principle — share the query across modalities but decouple the salience estimates — and it consistently beats the unidirectional OmniZip baseline on aggregate accuracy. The paper's strengths include a clear motivational analysis (Appendix F uses CLIP/CLAP as an independent probe), consistent ablations of the main components, external comparisons, and a promise of released code. The central claim is empirically defensible, but it rests on a formally underspecified budget-allocation step and on single-run accuracy numbers whose margins are sometimes small. These issues are fixable and do not invalidate the empirical direction, but they need to be addressed before the paper is fully reproducible and before the 'best average accuracy' claim can be taken at face value.","major_comments":[{"comment":"The budget allocation is undefined for the scores actually produced by Eqs. (3) and (5). Both are cosine similarities or weighted averages of cosine similarities and can be negative. With negative s_i, Eq. (7) can produce negative or zero retention counts at low-salience positions, and if the denominator sum is non-positive the ratio is undefined. No shift, ReLU, min-max normalization, or softmax is specified. Additionally, the formula does not clamp n_keep(i) to the number of tokens available in that frame/second, and the floor rounding means the total allocated count need not equal N_budget. Because this equation is the mechanism that implements modality-decoupled, query-aware compression, the method as described is not reproducible from the text alone. Please specify a non-negative transform and a rounding/renormalization rule, or give the exact implementation.","section":"Sec. 3.3.3, Eq. (7)"},{"comment":"All accuracy numbers are single runs with no error bars or significance tests, yet the central claim is a ranking: 'best average accuracy across all compression settings.' Several margins are small — for example, Qwen2.5-Omni-3B at 45% retention gives 50.18 (OmniScope) vs. 49.70 (OmniZip), and 7B at 25% gives 51.00 vs. 50.53 (FastV). Without repeated runs or a significance test, the reader cannot tell whether these differences are stable. Please report means and standard deviations over at least three runs, or explicitly state the single-run limitation and provide seed sensitivity.","section":"Sec. 4.2, Table 1"},{"comment":"The method depends on CLIP cosine similarity for visual tokens and internal gated cosine similarity for audio tokens as proxies for what the LLM actually needs to answer a query. The paper shows that the model's internal vision-text alignment is too weak and therefore chooses an external CLIP scorer, but it does not validate that CLIP's relevance judgments agree with the OmniLLM's answer-critical frames. The empirical results are consistent with the proxy working, but an analysis of per-question agreement or of failure cases where CLIP scores misdirect the budget would make the central mechanism load-bearing rather than assumed. This is a correctness-risk concern, not a circularity claim.","section":"Sec. 3.3.1-3.3.2 and Appendix D"}],"minor_comments":[{"comment":"When no second s' satisfies 0<|s-s'|≤R, the maximum in the boost factor is over an empty set. Define b_s = 1 in that case.","section":"Eq. (6)"},{"comment":"After the floor operation, the sum of n_keep may not equal N_budget, and n_keep(i) should be clamped to the capacity of position i. A specific rounding/renormalization procedure should be stated.","section":"Eq. (7)"},{"comment":"The phrase 'averaging the top-k token scores within each second' is ambiguous: specify whether k is selected over token positions within the second and whether k=5 is the value listed in Table 7.","section":"Sec. 3.3.2"},{"comment":"The axes of Figure 5 appear unlabeled/placeholder. Provide properly labeled axes and describe the measurement setup.","section":"Appendix C, Fig. 5"},{"comment":"The caption says 'same for Tables 5 and 4' but then references Table 4; reorder for clarity.","section":"Table 3 caption"},{"comment":"The diagnostic uses CLAP for audio scores while the actual OmniScope pipeline uses the LLM audio encoder. State explicitly whether the observed cross-modal mismatch is expected to transfer to the internal audio scoring used in the method.","section":"Appendix F"},{"comment":"Sensitivity analysis is only reported for τ_r. Since the audio-scoring hyperparameters (τ, top-k, β, γ, R, P) directly control budget allocation, a sensitivity study for at least the most sensitive of those would strengthen the 'fixed hyperparameters' claim.","section":"Table 7 / Appendix E"}],"recommendation":"major_revision","confidential_remarks":"To the editor: this is a solid systems paper with a clear and timely message, and the negative-score issue in Eq. (7) is fixable. My recommendation of major revision is driven by reproducibility of the allocation step and by the lack of any uncertainty quantification for the small accuracy margins. I do not see a reason to reject: the empirical trend is consistent across settings and the ablations support the design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is real. Decoupling per-modality query relevance instead of letting one modality guide the other is a clean design principle for omnimodal compression, and the paper supports it with an explicit ablation (Table 3) showing that decoupled scoring beats cross-guided variants. The AD-STC anchor-delta strategy and per-second bipartite audio merging are sensible engineering choices, and the ablations (Tables 4, 5) back them up. All hyperparameters are fixed across benchmarks and both model scales, and the code is released. That is solid, reproducible systems work.\n\nThe soft spots are real but not fatal. First, Eq. 7 is formally undefined as written: raw cosine similarities can be negative, and the proportional allocation floor(s_i / sum_j s_j * N_budget + 0.5) then produces negative or zero retention counts. The paper never specifies a non-negative shift, ReLU, or softmax. The stress-test note is correct. This matters because the budget allocation is the mechanism that implements the decoupling. The code likely resolves it, but the paper itself is not reproducible on this point. Second, the margins over OmniZip are small in several settings (3B/25% is 0.37 points), and there are no error bars or significance tests. The abstract's \"only a 0.35-point drop\" is the 7B/25% number; at 3B/25% the drop is 1.50. That is selective reporting. Third, the query-similarity proxy itself — CLIP for visual, gated cosine on the audio encoder for audio — is reasonable but not independently validated. The paper honestly shows the internal vision-text alignment is too weak, which is why CLIP is used, but it doesn't test whether CLIP's notion of relevance matches what the LLM actually needs. That is a legitimate assumption to make, but it remains an assumption.\n\nWho gets value from this: anyone working on efficient multimodal inference, especially on Qwen2.5-Omni-style models. It is an engineering contribution with a novel design principle, not a theoretical breakthrough. The paper deserves a serious referee. The main fixes before publication: clarify the non-negative transformation in Eq. 7, report variance or additional runs, and present the 3B and 7B results symmetrically in the abstract. I would accept it for peer review.","headline":"A genuinely new and useful idea for omni-token compression, with consistent ablations, but the paper has an ambiguous budget-allocation formula and an abstract that cherry-picks the most favorable numbers.","tokens_in":18028,"tokens_out":1883,"would_cite":true,"duration_ms":23854,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-04T03:29:35.829204+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":2}