{"id":"13bb8088-efb2-4efa-9ce6-9c36253d8627","arxiv_id":"2607.25125","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":5,"one_line_summary":"Adaptive allocation of a limited frame budget between CLIP-prompted spatial zoom-ins and graph-based temporal hyperframes beats prior keyframe samplers on long-video MLLM benchmarks.","lead":"LENS is a training-free keyframe sampler that splits a fixed frame budget between query-guided spatial zoom-ins and multi-frame temporal zoom-outs. It lifts long-video QA accuracy under tight context limits, e.g. Video-MME 53.3%→60.7% with Qwen2.5-VL at 8 frames.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The \"matched frame budget\" claim rests on slot accounting: each temporal hyperframe concatenates τ=4 source frames into one slot, so LENS at \"8 frames\" actually shows the MLLM content from up to ~20 distinct frames, and Table 1's own numbers (uniform@32 = 61.1 > LENS@8 = 60.7; BOLT@32 = 64.1 > LENS@","rationale":"The reader correctly identified CLIP-as-relevance-oracle as a weak assumption, and I agree it is one — but it is acknowledged in the limitations, ablated (CLIP vs. BLIP in Table 3), and genre-standard; it does not by itself threaten the empirical claim. The more load-bearing issue is the budget accounting underneath the strongest claim. Because hyperframes pack τ=4 source frames per slot, \"matched frame budget\" comparisons systematically give LENS more temporal coverage than baselines at the same nominal budget. The paper's own Table 1 shows baselines at higher budgets matching or beating LENS at lower budgets at roughly comparable effective coverage, which means the \"consistent outperformance under matched budgets\" is sensitive to how the budget is counted. This does not make the paper wrong — the method is genuinely training-free, the token budget is matched, multi-resolution/budget-reallocation tricks are accepted in this literature (Q-Frame does similar), and the ablations (Table 3, Table A1) support that the components each contribute. But the abstract's quantitative claim (53.3→60.7 at 8 frames) is implicitly a slot-accounted claim, and the strongest version of the reader's strongest_claim does not survive distinct-frame accounting without qualification. Hence CONDITIONAL rather than ACCEPT: the paper should either add the matched-effective-frame comparison or explicitly scope the claim to token/slot budgets. The suspicious identical AKS rows at 16/32 frames additionally warrant a verification pass on the re-implemented baseline numbers before full acceptance.","tokens_in":18728,"tokens_out":3281,"duration_ms":20918,"concrete_test":"Using the released code, log the per-query allocation ratio r and compute each video's distinct source-frame count D = B_spa + τ·B_tem. Then re-run uniform, BOLT, and Q-Frame on Video-MME at frame budgets matched to the average D (≈20 for B=8, ≈40 for B=16 with r≈0.5, τ=4). If any baseline at matched distinct-frame coverage reaches within ~1 point of LENS's 60.7/63.1, the headline claim holds only under slot accounting and the abstract/Table 1 framing needs revision. Separately, re-run AKS at 16 and 32 frames to verify the identical rows are not a table copy error.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that LENS beats prior samplers \"under matched frame budgets.\" The budget B is defined (§3.4) as the number of frame slots given to the MLLM. But the temporal zoom-out branch (§3.3, \"Hyperframe Aggregation\") concatenates τ−1=3 neighboring frames onto each selected frame before downsampling. With the default τ=4 and the controller's r≈0.5 for mixed queries, an 8-frame budget yields B_spa=4 spatial frames plus 4 hyperframes covering 16 distinct source frames — roughly 20 distinct frames of visual content, versus 8 for uniform/BOLT/AKS/Q-Frame at the same nominal budget. The paper is transparent that the design \"trades spatial detail for broader temporal coverage\" while \"keeping the token budget fixed,\" so this is not hidden; but the claim as stated is about frame budgets, and the informative comparison is not run.\n\nTable 1 already contains evidence that this accounting is load-bearing. LENS@8 scores 60.7 overall on Video-MME, while uniform sampling @32 scores 61.1 and Q-Frame@16 scores 60.7 — i.e., baselines with comparable or smaller distinct-frame coverage than LENS@8's effective ~20 frames match or exceed it. LENS@16 (63.1, effective ~40 frames with r≈0.5) is beaten by BOLT@32 (64.1) and roughly matched by Q-Frame@32 (62.3) and uniform@32 (61.1) at smaller effective coverage. Only LENS@32 (67.1, effective ~80 frames) lacks a baseline at comparable coverage, so its margin is untested. The \"consistent outperformance\" pattern is therefore partially an artifact of comparing at equal slot counts rather than equal visual evidence. A secondary red flag: the AKS rows for 16 and 32 frames report identical Video-MME numbers (66.4/56.9/48.6/57.3) across both budgets, which is implausible for a genuine re-run and suggests a copy error in the re-implemented baseline table — worth checking since baseline numbers underpin the superiority claim.","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper presents LENS, a training-free keyframe sampling framework for long-form video QA with MLLMs. A fixed frame budget B is split by an MLLM-predicted ratio r between (i) a spatial zoom-in branch that selects query-relevant frames via CLIP/BLIP similarity plus watershed segmentation and overlays CLIP-attention-derived saliency masks, and (ii) a temporal zoom-out branch that propagates similarity scores over a pairwise-similarity video graph (Zhou et al.-style label propagation) and packs each selected frame with τ−1 neighbors into a downsampled \"hyperframe.\" On Video-MME, LongVideoBench, and MLVU with Qwen2.5-VL at 8/16/32 frame budgets, LENS reports consistent gains over uniform sampling and over BOLT, AKS, and Q-Frame (e.g., 53.3→60.7 overall on Video-MME at 8 frames), with transfer results on LLaVA-OneVision and GPT-5-Mini, component ablations (Table 3), hyperparameter sweeps (Table A1), and released code.","tokens_in":19297,"tokens_out":3763,"duration_ms":185721,"significance":"If the central comparison holds up, this is a useful, practical contribution: a training-free, plug-and-play sampler with multi-benchmark, multi-budget, multi-backbone evaluation, re-implemented baselines (a genuine plus for comparability), component-level ablations isolating each branch and the allocator, appendix hyperparameter sweeps showing stability, the allocation prompt and implementation details in the appendix, and public code. The efficiency accounting (Table 2 and the per-query breakdown) is also unusually transparent. The main caveat on significance is that the headline claim — consistent outperformance \"under matched frame budgets\" — is currently entangled with a slot-accounting issue (Major Comment 1): the temporal branch's hyperframes show the MLLM content from substantially more distinct source frames than the nominal budget suggests, and the paper's own Table 1 shows coverage-matched baselines meeting or beating LENS at lower nominal budgets. The τ=1 ablation (59.7 overall at 8 slots) indicates the method likely retains an edge without any extra coverage, so the contribution is probably real but its magnitude is uncalibrated.","major_comments":[{"comment":"The central claim of 'consistent outperformance under matched frame budgets' rests on slot accounting that is not matched in terms of distinct visual content. With default τ=4 (§4.1), each temporal-branch slot concatenates 4 source frames; at r≈0.5 an 8-slot budget shows the MLLM content from up to ~4 + 4×4 = 20 distinct frames, versus 8 for uniform/BOLT/AKS/Q-Frame. Table 1's own cross-budget rows show this is load-bearing: uniform@32 (61.1) exceeds LENS@8 (60.7), Q-Frame@16 (60.7) ties it, and BOLT@32 (64.1) exceeds LENS@16 (63.1) — i.e., baselines at comparable or smaller effective distinct-frame coverage match or beat LENS at lower nominal budgets. Only LENS@32 (67.1) lacks a coverage-matched baseline. I do not think this invalidates the method — the τ=1 ablation in Table A1 (59.7 at 8 slots, still above Q-Frame@8's 58.7) shows much of the gain survives without aggregation — but the","section":"§3.3–3.4, Table 1"},{"comment":"The claim that hyperframes keep 'the token budget fixed' is asserted but never substantiated with numbers. The hyperframe is 'downsampled to a lower spatial resolution' — the target resolution is not specified anywhere, and for Qwen2.5-VL, which uses dynamic-resolution ViT encoding, a 4-wide concatenated composite downsampled to an unspecified size may well consume more visual tokens than a single native frame. Please report (i) the exact downsampling resolution and (ii) measured visual-token counts per input for LENS vs. uniform at each nominal budget and backbone. This is a prerequisite for either resolution of Major Comment 1.","section":"§3.3 (Hyperframe Aggregation), §B.1"},{"comment":"The adaptivity claim is under-supported relative to its prominence in the title and abstract. Adaptive allocation beats the fixed 4/4 split by only +0.5 overall (60.7 vs 60.2, Table 3), and no statistics on the controller's actual behavior are given: the prompt (§B.2) anchors r to discrete bands, but the distribution of predicted r across benchmarks, its correlation with query type, and per-query-type accuracy vs. the fixed split are all unreported. Without these, 'adaptively allocates frame budgets based on the text query' is a design description rather than an evidenced finding. A histogram of r plus a breakdown of gains on detail- vs. event-type questions (Video-MME has category labels) would close this gap cheaply.","section":"§3.4, §4.3 (Budget Allocation)"}],"minor_comments":[{"comment":"The citation supporting 'the final similarity is primarily governed by the deepest MSA layers' is given as [19,45], but [19] is the visual instruction tuning paper (LLaVA), which does not establish this; presumably a CLIP-decomposition reference (e.g., [11] or related interpretability work) was intended. Also, V_n^(l,h) = W_v z_n is defined but never used; s(n,l,h) should be defined explicitly in terms of it.","section":"§3.2, Eq. (2)–(3)"},{"comment":"Typographical/grammatical: 'reaching 60.6% on LVB and 66.7 on MLVU' (missing %); 'as more budgets becomes available'; 'fullvideosintothemodel' and similar missing-space artifacts in the introduction (likely PDF extraction, but worth checking the source).","section":"§4.2"},{"comment":"The runtime breakdown (15.8s/query vs 11.0s baseline) is welcome. Please clarify whether the 2.1s budget-allocation call uses the QA backbone itself (an extra MLLM forward pass per query) and how this cost is handled for the GPT-5-Mini API setting, where the controller call and QA call are both paid API requests.","section":"Table 2 / §4.3"},{"comment":"Decoding parameters are said to be 'default'; please state the temperature explicitly and, if greedy decoding is used, note that results are single-run deterministic. Otherwise some estimate of run-to-run variance (or at least per-benchmark question counts enabling significance reasoning) would strengthen the 2–3 point margins in Table 1.","section":"§4.1 / §B.1"},{"comment":"The Anomaly/Count category (AC) jumps from 19.0 to 40.9 at 8 frames — by far the largest category gain. A sentence explaining which component drives this (presumably temporal zoom-out coverage) would be informative, especially in light of Major Comment 1.","section":"Table A3 (MLVU categories)"},{"comment":"Please mark which displayed frames are hyperframes (with their τ window) versus spatially prompted frames; the green circle/hexagon legend is hard to read at print size and the downsampling of hyperframes is not visually evident.","section":"Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is competently executed and the empirical package (re-implemented baselines, ablations, code) is above average for this subfield. My decision hinges on the budget-accounting issue in Major Comment 1: the abstract's '53.3% to 60.7%' framing and the 'matched frame budget' comparisons are, as they stand, partially an artifact of hyperframes carrying ~4× the distinct-frame content per slot. The τ=1 ablation suggests a real contribution survives a fair accounting, so I view this as fixable within the manuscript's scope rather than disqualifying — but the authors should not be allowed to keep the current framing without either the matched-coverage experiment or an explicit token-budget redefinition. The margin over Q-Frame, the closest prior method, is also modest (~2 points) once accounting is clarified, so the novelty positioning relative to Q-Frame's multi-resolution idea deserves a careful sentence in revision."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: LENS is a clean, usable dual-branch sampler—CLIP attention prompting for spatial zoom-in, graph-smoothed scores plus hyperframes for temporal zoom-out, and a cheap LLM split of the slot budget. Under equal input slots it beats uniform and the usual training-free baselines on Video-MME / LVB / MLVU, with transfer to LLaVA-OneVision and GPT-5-Mini and a linked codebase. That is real systems progress for long-video MLLM inference, not a foundational leap.\n\nWhat is new is the package, not any single piece. CLIP relevance, diversity/watershed, multi-res allocation, and object-cue search already exist (BOLT, AKS, Q-Frame, T*). Combining query-conditioned visual prompting with graph message-passing and explicit spatial-vs-temporal budget control is a sensible composition, and the ablations (scoring alone, each branch, fixed 4/4, adaptive) actually isolate the pieces. Hyperparameter sweeps in the appendix are stable. Circularity is low; this is external benchmark accuracy.\n\nSoft spots, in proportion. First, the stress-test on budget accounting is partly right and partly overstated. Hyperframes (τ=4) pack multiple source frames into one downsampled slot, so LENS@8 can expose ~20 distinct frames of content while baselines@8 show eight full-res frames. The paper is open that it trades spatial detail for temporal coverage at fixed token/slot count—that is the design, and for MLLMs the slot budget is the binding constraint. Still, the abstract’s “matched frame budgets / consistently outperforms” language invites the wrong comparison; Table 1 already shows uniform@32 ≈ LENS@8 and BOLT@32 beating LENS@16 on overall Video-MME. They should report an equal-source-frame or equal-pixel control, or state the claim strictly as equal MLLM inputs. Second, AKS rows for 16 and 32 frames are identical across short/medium/long/overall—that is almost certainly a copy error in the re-implementation table and undercuts trust in the baseline column until fixed. Third, no error bars; CLIP as relevance oracle is a known limit they acknowledge. Novelty is compositional.\n\nWho it is for: people shipping long-video QA under tight context who want a plug-in sampler. Math is standard (label propagation, attention rollout); citations are on-genre. I would send it to peer review, ask for the equal-evidence control and a corrected baseline table, and still engage—the core idea is sound and the gains under fixed slots are the right metric for practice.","headline":"Solid training-free systems paper with real gains under fixed MLLM slots; the hyperframe packing makes “matched frame budget” a bit slippery, and the AKS table rows look broken.","tokens_in":20536,"tokens_out":651,"would_cite":true,"duration_ms":23977,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Long-video models reason better when a fixed frame budget is split, by query, between spatial zoom-ins and temporal zoom-outs.","keywords":["long-form video understanding","keyframe sampling","multi-modal large language models","spatio-temporal reasoning","visual prompting","video graph","training-free"],"falsifier":"On the same Video-MME / LongVideoBench / MLVU splits and identical frame budgets, replace LENS’s adaptive dual-zoom set with the best prior sampler (or with fixed 50/50 or all-spatial / all-temporal ablations) and check whether the reported overall accuracy gaps (e.g., 60.7% vs 58.7% vs 53.3% at 8 frames on Qwen2.5-VL) disappear or reverse.","tokens_in":20131,"feed_emoji":"🎬","tokens_out":1047,"duration_ms":30134,"temperature":0.7,"pith_summary":"Long multi-modal language models cannot feed whole videos into their context windows, so they must pick a small set of frames. Prior keyframe pickers treat every frame as a single, fixed-granularity unit and therefore force a hard trade-off between fine spatial detail and broad temporal coverage. LENS claims that this trade-off can be made query-dependent and training-free: a lightweight controller reads the question and divides the same frame budget between spatial zoom-ins (CLIP-attention masks that highlight query-relevant regions inside a frame) and temporal zoom-outs (graph-propagated relevance plus multi-frame “hyperframes” that widen the temporal field of view). On standard long-video QA benchmarks the resulting multi-granular evidence set consistently beats uniform sampling and prior training-free samplers under matched budgets, lifting Video-MME accuracy from 53.3% to 60.7% with an 8-frame Qwen2.5-VL backbone and producing similar gains on other models and benchmarks. A sympathetic reader cares because the method is plug-and-play, adds only modest runtime, and shows that how evidence is allocated across scale matters as much as which frames are chosen.","feed_headline":"Split the frame budget: zoom in or zoom out by query","feed_subtitle":"Training-free LENS lifts long-video QA from 53.3% to 60.7% by allocating frames across spatial detail and temporal context.","key_machinery":"LENS’s dual zoom plus adaptive allocator: spatial zoom-in (CLIP patch-attention visual prompting on watershed-selected frames) and temporal zoom-out (pairwise visual-similarity graph message-passing followed by hyperframe aggregation), with an MLLM controller that outputs the budget split ratio r from the text query alone.","core_discovery":"Under a fixed frame budget, adaptively allocating some frames to query-conditioned spatial zoom-ins and the rest to graph-based temporal zoom-outs supplies complementary multi-granular evidence that lets off-the-shelf video MLLMs answer long-form questions more accurately than uniform sampling or prior training-free keyframe methods.","pith_inferences":["If the allocator’s decision correlates with question type, future work could cache or distill the split ratio so the controller call itself becomes free at inference time.","The same spatial-vs-temporal budget idea may transfer to other token-scarce settings (long documents with figures, multi-camera streams) where “zoom in on a region” and “summarize a window” are natural dual operations.","Failure modes on abstract or domain-specific queries noted in the limitations suggest pairing LENS with a lightweight domain adapter or retrieval step rather than replacing the dual-zoom core."],"forward_implications":["Keyframe sampling for long video should treat spatial resolution and temporal coverage as a query-conditioned budget allocation problem, not only a frame-ranking problem.","Training-free dual-scale selection can be dropped in front of open-source and proprietary video MLLMs without weight updates and still raise accuracy under tight context limits.","Detail-oriented questions should receive more spatial zoom-ins; event-level or sequence questions should receive more temporal zoom-outs, and an LLM can decide the split from text alone.","Modest extra compute (graph message-passing and attention prompting) is enough to outperform relevance-only and multi-resolution baselines while remaining far cheaper than agentic or RAG long-video pipelines."],"fun_headline_variants":["LENS splits frame budget between spatial zoom-ins and temporal zoom-outs","Query-guided zoom in or out under fixed frames lifts long-video QA","Adaptive spatio-temporal zooming beats uniform keyframe sampling","Training-free LENS allocates frames for detail and long-range context","Fixed budget, multi-granular evidence: zoom-ins plus temporal zoom-outs"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Off-the-shelf image–text similarity scores, CLIP attention maps, and pairwise visual-similarity graphs are faithful enough proxies for “what the question needs to see” that watershed selection and hyperframe packing will surface the right cues for arbitrary long-video questions.","fun_headline_variants_meta":{"raw":{"variants":["LENS splits frame budget between spatial zoom-ins and temporal zoom-outs","Query-guided zoom in or out under fixed frames lifts long-video QA","Adaptive spatio-temporal zooming beats uniform keyframe sampling","Training-free LENS allocates frames for detail and long-range context","Fixed budget, multi-granular evidence: zoom-ins plus temporal zoom-outs"]},"model":"grok-4.5","effort":"low","cost_usd":0.003844,"raw_usage":{"total_tokens":1207,"prompt_tokens":795,"num_sources_used":0,"completion_tokens":101,"cost_in_usd_ticks":38444000,"prompt_tokens_details":{"text_tokens":795,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":311,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":795,"tokens_out":101,"duration_ms":5744,"temperature":1.0,"reasoning_tokens":311,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T00:36:57.372060+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same Video-MME / LongVideoBench / MLVU splits and identical frame budgets, replace LENS’s adaptive dual-zoom set with the best prior sampler (or with fixed 50/50 or all-spatial / all-temporal ablations) and check whether the reported overall accuracy gaps (e.g., 60.7% vs 58.7% vs 53.3% at 8 frames on Qwen2.5-VL) disappear or reverse.","supporting_citations":[],"review_version":1}