{"id":"e189fd33-4620-4e09-a97e-851f45c27fea","arxiv_id":"2608.05707","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"MEC constructs one query-conditioned, nested ranking of video frames whose prefixes work for any frame budget, improving accuracy over uniform sampling by 3.77 points and cutting selection latency by about half.","lead":"A new training-free frame selection method, MEC, builds a single priority ranking of video frames that can be cut at any budget to feed a large multimodal model, so the same ranking serves tight and loose context limits without rerunning. It reports accuracy gains over uniform sampling and latency savings versus budget-specific selectors across four long-video benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sparse-discovery recall is the load-bearing risk: if decisive frames never enter C(q), no budget can recover them; Figure 7's all-budget failure shows this is real, and a candidate-pool recall check would settle how often it binds.","rationale":"The reader's weakest_assumption names the same premise: candidate-pool completeness before ranking. My independent reading of Section 3.3, Section D, and Figure 7 confirms that all budgets share one pool, so an omitted decisive frame cannot be repaired by truncating R at a larger K. The ranking objective in Eq. (3) and the greedy construction are internally coherent; the empirical average gain is not invalidated by a single counterexample. The missing piece is a quantitative recall estimate: how often does the sparse discovery stage omit answer-critical evidence? Without it, the 'any budget' wording overstates what is established. This is a condition for acceptance, not a rejection. The reader's CONDITIONAL verdict remains appropriate, so I set verdict_should_be to UNCHANGED. Other weaknesses (lack of error bars, only two baselines, no release) are addressable but less load-bearing than the recall premise.","tokens_in":21120,"tokens_out":5776,"duration_ms":53038,"concrete_test":"On a random sample of 200 Video-MME Long questions, run MEC to obtain C(q) and R. Build an augmented pool C_dense(q) = C(q) union Uniform(K=1024) (or one frame per second, whichever is smaller), rerun the same greedy ranking with the same M and weights on C_dense(q), and compare accuracy at K = 8, 16, 32, 64, 128, and 256. If the dense-pool variant gains materially (e.g., more than 1 pp at K = 64) or if inspection shows more than 5% of questions have decisive frames outside C(q), sparse-discovery recall is the binding constraint and the paper should report candidate-pool recall as an explicit metric. If no gain appears, Figure 7 is an outlier and the central ranking claim is safe.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that one truncated ranking serves any budget rests on the candidate pool C(q) containing answer-critical evidence before ranking begins. MEC deliberately limits cost: it uses only P = min(N, floor(P0 + min(T/tau_p, 1) P1)) sparse probes (Eq. 4), activates only the top eta = 0.25 segments, and zooms around a single anchor per active segment (Eq. 8). Any decisive frame that falls between probes, or in an inactive segment with diluted segment-average scores, never enters C(q). Since every prefix R:K is drawn from C(q), later coverage and diversity cannot recover the omission at any larger budget. The paper itself documents this exact failure in Figure 7: the final planter state is absent from the pool, so MEC answers incorrectly at all six budgets while Uniform is correct from K = 32 onward. Section D calls this a 'sparse-discovery blind spot,' and the failure does not contradict the average 3.77 pp gain. However, the paper provides no recall statistic for C(q), so the frequency of such omissions is unquantified. If the frequency is material, the phrase 'any budget' is conditional on sparse discovery having already found the evidence, which is exactly the load-bearing assumption that needs explicit support.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MEC (Matryoshka Evidence-to-Context) Frame Selection, a training-free method for long-video frame selection that constructs a single budget-independent ranked list of frames. The ranking is built by (i) creating a reusable sparse index of low-resolution appearance, local visual change, and observability; (ii) discovering query-conditioned evidence via duration-adaptive sparse probing, segment activation, and anchor-centered local zooming; and (iii) greedily selecting frames with a position-adaptive score that emphasizes evidence in early ranks and temporal coverage plus visual diversity in later ranks. Any budget K is served by truncating the ranking to its K-th prefix and restoring chronological order. Experiments on four long-video benchmarks (Video-MME, Video-MME-v2, LongVideoBench, MLVU) with three LMMs and six frame budgets report an average +3.77 pp accuracy gain over uniform sampling, competitiveness with AKS and WFS-SB, and lower cached end-to-end selection latency. The paper includes ablations on the ranking schedule and components, an analysis of the ranking horizon M, and an explicitly documented failure case in which sparse discovery misses a decisive frame.","tokens_in":21384,"tokens_out":7571,"duration_ms":61473,"significance":"The problem is practically important and the proposed formulation is clean: serving every budget from one truncated ranking is attractive for deployment, and the paper makes a falsifiable claim that can be independently checked. Strengths include a shared configuration across benchmarks, LMMs, and budgets; three downstream LMMs; six to eleven budget points; component and schedule ablations; and an unusually honest limitations section that includes a concrete failure case (Figure 7). The method is training-free and does not require accessing the LMM's internal tokens. However, the central 'any budget' claim is only as strong as the candidate-generation stage, and the main accuracy and latency claims lack statistical and amortization detail. These gaps are fixable and should be addressed before publication.","major_comments":[{"comment":"The 'any budget' claim is load-bearing on sparse-discovery recall. Because every prefix FK is drawn from the candidate pool C(q), a decisive frame that never enters C(q) cannot be recovered by coverage or diversity at any larger budget. The paper itself documents this in Figure 7: for the planter comparison question, MEC fails at all six budgets while uniform sampling succeeds from K=32 onward. Section D acknowledges this 'sparse-discovery blind spot,' but the manuscript reports no statistic on how frequently such omissions occur. I request a candidate-pool recall analysis: for example, measure the fraction of questions for which the pool contains the frames selected by a dense oracle (or the answer-critical frames identified by a stronger reference), and report per-benchmark recall. Without this, the central claim that one ranking serves 'any budget' is conditional on an unvalidated premise.","section":"Section D / Figure 7 (Appendix E.3)"},{"comment":"The main accuracy claim lacks statistical support. The reported differences are often small (e.g., Video-MME Overall at K=8: MEC 63.85 vs WFS-SB 63.52; at K=128: 73.74 vs 73.37), and the 3.77 pp average improvement over uniform is a point estimate with no confidence intervals or significance tests. Across 24 benchmark-by-budget cells, many gains may be within sampling noise. Please report bootstrap confidence intervals or paired statistical tests over benchmark questions/videos for the headline comparisons, at least for the full MEC vs. uniform and MEC vs. strongest baseline.","section":"Table 1 / Section 4.2"},{"comment":"The end-to-end latency reduction of 47.37–51.19% is measured with 'reusable caches available,' but the amortization model is unspecified. It is unclear whether the 5.48 s of MEC preprocessing per query includes building the sparse index once per video (amortized over the three questions in Video-MME) or rebuilding it per query, and whether the same cache reuse applies to AKS and WFS-SB. Since the latency claim is in the abstract, the paper should state the cache-reuse assumption explicitly and, ideally, report both cold and amortized numbers.","section":"Table 4 / Section 4.3"}],"minor_comments":[{"comment":"There are several typographical and ligature artifacts, e.g., 'Efficiently', 'official', 'difficult', and 'Y et scaling' in the Introduction; these should be cleaned up.","section":"Throughout"},{"comment":"The header uses 'L VB' and 'ML VU' without expanding them in the caption or a table note. Please use full names or define the abbreviations directly below the table.","section":"Table 1"},{"comment":"The 3.77 pp average treats the four benchmarks equally, despite their very different sizes and task mixtures. Please also report per-benchmark averages over budgets, since the gains are much larger on LongVideoBench and MLVU than on Video-MME.","section":"Section 4.2"},{"comment":"The 11-budget comparison is dense; adding numerical labels or a small supplementary table would make the per-budget gains easier to read and verify.","section":"Figure 4 (left)"},{"comment":"The comparison is limited to uniform sampling, AKS, and WFS-SB. Please justify the omission of other recent selectors mentioned in the related work (e.g., BOLT, Nar-KFC), or include at least one at a subset of budgets.","section":"Table 1 / Baselines"},{"comment":"The loop 'for k = 2, . . . , M' assumes M ≤ |C(q)|; this restriction is stated in Section 3.3 but adding an explicit assertion in the algorithm pseudocode would prevent misuse.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The authors are transparent about a real failure mode, which is commendable. However, the paper's flagship claims—'one ranking, any budget' and the latency reduction—are not yet backed by a recall statistic or a clear amortization model. The absence of confidence intervals is also concerning given the small margins in Table 1. All three issues are addressable with additional analysis and reporting, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, practical method paper that deserves a real referee. The central ranking idea holds; the load-bearing risk is the sparse-discovery stage, and the paper's own Figure 7 shows it is a real failure mode. The 'any budget' claim needs a recall number.\n\nWhat is actually new: MEC is the first frame-selection method I know that builds a single nested ranking designed so that every prefix is good at its own budget. The Matryoshka angle is not just a label—early positions are weighted toward query evidence, later ones toward temporal coverage and diversity, and the position-adaptive schedule is what makes the prefixes work across K. The reusable sparse index (probes, local change, observability, segment activation, anchor zoom) is a sensible way to keep per-query cost down, and the efficiency numbers are meaningful: about 74% fewer candidates than AKS/WFS-SB, and 47–51% lower end-to-end latency under the cached setting. Experiments are broad: four benchmarks, three LMMs, up to eleven budgets, one shared config, plus ablations for the schedule and each component. The claimed 3.77 pp average gain over uniform is supported by the table; the gains are largest at tight budgets, which matches the design.\n\nThe soft spots are real but not fatal. The stress-test is correct: C(q) is built before ranking, and if a decisive frame never enters the pool, no prefix can recover it. Figure 7 is the paper's own counterexample—MEC fails at all six budgets on that question while Uniform succeeds from K=32 onward—and Section D honestly calls it a 'sparse-discovery blind spot.' What is missing is a frequency. A recall statistic for the candidate pool across the four benchmarks would tell us whether this is a rare edge case or a systematic limitation. Without it, 'one ranking, any budget' overpromises. Related complaints: no confidence intervals or significance tests; only two strong baselines in the main comparison; no code or data released. Those are addressable. The latency claim depends on a cached index, which is legitimate for a reusable selector but should be clearly flagged as such; it is in the table but easy to miss.\n\nThe ranking math itself looks sound: nesting is exact by construction, the greedy surrogate is a reasonable approximation, and the ablation shows the position schedule matters beyond any static mixing. I do not see circularity.\n\nThis paper is for people building long-video LMM pipelines who need one selector that adapts to different deployment budgets. It is not a paradigm shift, but it is a genuine step forward. I would accept it for peer review and push for the recall analysis, error bars, and a couple more baselines before publication. Worth bringing to a reading group.","headline":"MEC is a genuinely useful one-pass multi-budget frame selector; the ranking idea holds, but the sparse-discovery blind spot it documents needs a recall measurement before 'any budget' is credible.","tokens_in":21893,"tokens_out":2996,"would_cite":true,"duration_ms":25160,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Long-video frame selection can be reduced to a single Matryoshka ranking: one query-conditioned priority sequence whose prefixes serve any frame budget.","keywords":["long-video understanding","frame selection","Matryoshka ranking","large multimodal models","training-free selection","budget-independent ranking","video question answering","temporal coverage"],"falsifier":"The paper itself shows a 48-minute planter-comparison video where the final completed state is missed by sparse probing: MEC answers incorrectly at all six budgets, while uniform sampling succeeds from $K=32$ onward. Checking a set of such “brief endpoint” questions — where the answer depends on a single short state at a time boundary — and measuring how often MEC fails at all budgets while uniform succeeds would settle whether the central claim is conditional on discovery recall.","tokens_in":20910,"feed_emoji":"🎬","tokens_out":8855,"duration_ms":69275,"temperature":0.7,"pith_summary":"The paper is trying to establish that frame selection for long-video understanding does not need to be rerun for every frame budget. It claims that a single query-conditioned priority ranking, built once, can serve any budget simply by truncating it to the prefix of that length, provided the ranking is position-adaptive: early positions concentrate answer-critical evidence, later positions add temporal coverage and visual diversity. To support this, the paper introduces MEC, a training-free pipeline that builds a reusable sparse index of the video, discovers candidate frames by sparse probing and local zooming around high-evidence anchors, and greedily constructs the ranking with a position-dependent score. On four long-video benchmarks and six budgets, the paper reports that this one ranking improves accuracy over uniform sampling by an average of 3.77 percentage points, stays competitive with strong training-free selectors, and reduces end-to-end selection latency by 47.37–51.19%.","feed_headline":"One frame ranking serves every budget in long-video QA","feed_subtitle":"The same selector output is truncated to any budget, beating uniform sampling by 3.77 points on average.","key_machinery":"The central object is the Matryoshka ranking itself: a sequence of distinct original-frame indices, built once per query, whose every prefix is a valid selection for the corresponding budget. The construction is carried by a greedy algorithm that, at position $k$, scores each remaining candidate $n$ by $u_k(n)=w_e(k)E_n(q)+w_c(k)H_n(k)+w_dD_n(k)$, where $E_n(q)$ is the evidence score (a weighted combination of frame–query matching, visual change, and observability), $H_n(k)$ is temporal coverage relative to the already chosen prefix, $D_n(k)$ is visual diversity, and the weights move from evidence-dominated ($w_e=0.6$) to coverage-dominated ($w_e=0.2$) as $k$ grows. Because the greedy step never replaces a selected frame, every shorter prefix is literally the beginning of every longer one, which guarantees nesting by construction. Efficiency comes from a reusable sparse index: duration-adaptive probes, low-resolution grayscale appearance encodings, local change scores, and observability priors are computed once and reused across questions.","core_discovery":"The paper's central claim is that the right object to compute is not a budget-specific subset but a budget-independent Matryoshka ranking $R=\\Psi(V,q,M)=(r_1,\\dots,r_M)$, where the selected set for any budget $K\\le M$ is the prefix $F_K=\\{r_1,\\dots,r_K\\}$. The distinctive thesis is that rank positions carry different roles: the first ranks should be maximally evidence-bearing for the query, while later ranks should progressively fill the timeline and diversify appearance, so that longer prefixes preserve the evidence of shorter ones while adding context. MEC realizes this by scoring candidates with a mixture of query–frame relevance, local visual change, and observability, activating the highest-scoring temporal segments, zooming around anchors to add fine-grained candidates, and then greedily extending the ranking with a position-adaptive weight schedule. The reported outcome is that this single ranking, with one fixed configuration across models and benchmarks, improves accuracy over uniform sampling by 3.77 percentage points on average and matches the accuracy of strong selectors while cutting selection latency by roughly half.","pith_inferences":["If the nesting property holds in practice, the ranking can be computed once per video-query pair and then cached; any later change in the deployment budget, downstream model context window, or latency requirement costs nothing beyond truncation.","Because the failure mode is confined to the discovery stage, a natural extension is to add uncertainty-triggered refinement: if neighboring probes disagree sharply or the query includes temporal comparatives like “first and last,” increase local probing around inactive segment boundaries before ranking.","The framework separates two testable claims: sparse discovery recall and ranking quality. Datasets biased toward brief, isolated state changes can benchmark the first; datasets with many visible but redundant events can benchmark the second.","One could test near-optimality of the greedy ranking by comparing its prefixes against a beam-search or local-swap variant on a small subset; if large gaps appear, the position-adaptive surrogate is not the bottleneck."],"forward_implications":["Deployments can choose the frame budget at inference time: the same cached ranking serves $K=8$ and $K=256$ without rerunning the selector, so latency and accuracy can be traded on demand.","Tight budgets benefit most; the paper reports gains over uniform sampling that reach 8.30 points on one benchmark and 12.06 points on another at $K=8$, narrowing as the budget grows.","The one configuration transfers across downstream model families and scales, so a single selector can ship with a system without per-model calibration.","Selection cost drops sharply: candidates evaluated shrink by about 74%, and end-to-end latency falls by 47.37–51.19% relative to the compared selectors on the long-video benchmark used for the timing study.","Evidence selected under a tight budget is never replaced under a larger budget, addressing the instability of budget-conditioned selection."],"supporting_citations":[{"why":"Supplies the long-video benchmark with short, medium, and long duration splits used for the main accuracy table.","marker":"[13]"},{"why":"Supplies the benchmark with a grouped nonlinear score, testing whether prefixes preserve chains of evidence.","marker":"[14]"},{"why":"Supplies the long-context referring-reasoning benchmark used in the six-budget comparison.","marker":"[28]"},{"why":"Supplies the multi-task long-video benchmark with nine tasks and macro accuracy.","marker":"[35]"},{"why":"Supplies the image–text matching score used as the relevance term in evidence scoring.","marker":"[18]"},{"why":"Establishes the nested-prefix principle that the Matryoshka ranking adapts to frame selection.","marker":"[17]"},{"why":"Defines the fixed-budget adaptive keyframe baseline that must be matched or beaten on accuracy and latency.","marker":"[26]"},{"why":"Defines the wavelet-based semantic-boundary baseline used as the second training-free competitor.","marker":"[6]"},{"why":"Provides the main downstream large multimodal model and its direct-response mode used for most evaluations.","marker":"[24]"},{"why":"Supplies the evaluation harness that fixes prompts, scoring, and answer normalization across selectors.","marker":"[31]"}],"fun_headline_variants":["One ranking, every budget: 3.77-point gain in long-video QA","Budget-agnostic frame ranking: 3.77 pt better than uniform","Matryoshka ranking: one sequence, any truncation, big speedup","Train-free MEC: single priority list for all video frame budgets","Any frame budget, one ranking: MEC wins by 3.77, halves latency"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"If a question's decisive evidence never enters the sparse candidate pool — because it falls between probes, in an inactive segment, or outside the zoomed anchor intervals — then no frame budget, however large, can recover it, and the ranking claim collapses for that question.","fun_headline_variants_meta":{"raw":{"variants":["One ranking, every budget: 3.77-point gain in long-video QA","Budget-agnostic frame ranking: 3.77 pt better than uniform","Matryoshka ranking: one sequence, any truncation, big speedup","Train-free MEC: single priority list for all video frame budgets","Any frame budget, one ranking: MEC wins by 3.77, halves latency"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000735,"raw_usage":{"total_tokens":3345,"prompt_tokens":1062,"completion_tokens":2283,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":2178}},"tokens_in":678,"tokens_out":2283,"duration_ms":15326,"temperature":1.0,"reasoning_tokens":2178,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:35:11.796890+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The paper itself shows a 48-minute planter-comparison video where the final completed state is missed by sparse probing: MEC answers incorrectly at all six budgets, while uniform sampling succeeds from $K=32$ onward. Checking a set of such “brief endpoint” questions — where the answer depends on a single short state at a time boundary — and measuring how often MEC fails at all budgets while uniform succeeds would settle whether the central claim is conditional on discovery recall.","supporting_citations":[{"cited_title":"Video-MME: The first-ever comprehensive evaluation benchmark of multi-modal LLMs in video analysis","cited_arxiv_id":null,"evidence_quote":"Supplies the long-video benchmark with short, medium, and long duration splits used for the main accuracy table."},{"cited_title":"Adaptive keyframe sampling for long video un- derstanding","cited_arxiv_id":null,"evidence_quote":"Supplies the long-context referring-reasoning benchmark used in the six-budget comparison."},{"cited_title":"Matryoshka representation learning","cited_arxiv_id":null,"evidence_quote":"Establishes the nested-prefix principle that the Matryoshka ranking adapts to frame selection."},{"cited_title":"Qwen3.5: Towards native multimodal agents, February 2026","cited_arxiv_id":null,"evidence_quote":"Defines the fixed-budget adaptive keyframe baseline that must be matched or beaten on accuracy and latency."}],"review_version":2}