{"id":"33102041-3ea7-4a21-813c-66ff468c49f7","arxiv_id":"2607.25266","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A query-conditioned, training-free frame selector that unifies relevance and diversity into a single volume-maximization objective improves keyframe recall and long-video question-answering accuracy.","lead":"FORGE is a training-free method that picks video frames for a question-answering model by reshaping the frame embedding space so that frames covering different question-relevant content are far apart, then choosing the K frames that span the largest volume. The authors report consistent answer-accuracy gains over uniform sampling and five baselines on two long-video benchmarks across eight open-source multimodal models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FORGE's gains may stem from the relevance score rather than the geometry: no top-K-by-relevance baseline is reported, so the claimed contribution of orthogonalization is untested.","rationale":"The reader's weakest_assumption concerns the strength of the relevance signal. My concern is more specific and, I think, more directly load-bearing: the paper never compares FORGE to a pure top-K-by-relevance baseline using the same scores. This omission confounds the contribution of the geometry with the contribution of the relevance weighting itself. If a simple top-K baseline matches FORGE, then the claimed unification of relevance and diversity via projected volume maximization is not the source of the gains; if it does not, the geometry is essential. This is a concrete, checkable gap in the evidence, not just a reproducibility issue. It also connects to the unstated identity of the relevance model: without that model named and without the top-K control, neither the absolute quality of the scores nor the incremental value of the geometry can be assessed. I agree with the reader that the verdict should be CONDITIONAL, but my recommended condition is broader: release the relevance model/constants and add the top-K-by-relevance baseline. This would settle whether the central mechanism claim holds or whether the method reduces to a weighted re-ranking of an already strong relevance signal.","tokens_in":17927,"tokens_out":5290,"duration_ms":63678,"concrete_test":"Reproduce FORGE on Video-MME and LongVideoBench at K=16,32,64 using the same 1-FPS frame pool and the same named relevance-scoring model. Add a baseline that selects the K frames with the highest combined score w_t (or s_t) with no volume/orthogonalization step. Compare KFR, UKSS, and downstream VQA accuracy on at least Qwen2.5-VL 7B and Gemma3 12B. If FORGE's margin over Top-K is within standard error, the central claim of geometric unification is not supported. Additionally, vary the relevance model (e.g., CLIP vs. BLIP) to confirm the geometry's contribution is independent of the scorer.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"FORGE's query-conditioning enters only through scalar weights w_t (Eq. 4), which are a monotone transform of the relevance score s_t from an unnamed pretrained multimodal encoder. Algorithm 1 then performs greedy residual-norm selection in an SVD-projected, weighted embedding space. The natural control is selecting the K frames with the largest w_t (or s_t) directly. The paper mentions 'Top-k (reference)' in Table 4 but omits it from Tables 1–3. Without this baseline, the reported improvements over AKS/ASCS/MDP3/MaxInfo do not isolate the effect of the geometry: those baselines combine relevance with temporal or visual spreading, but none is the pure relevance ranking that FORGE's weighting approximates. If Top-K by w_t achieves similar KFR/accuracy, then the volume/orthogonalization step adds little and the central claim that 'relevance and diversity are unified into a single objective' is unsupported. If Top-K is much worse, the geometry matters. The ablation in Fig. 5 removes the query warp or the local-contrast term, but never removes the orthogonalization step, so the specific value of the geometry is never tested. This is load-bearing because the paper's theoretical contribution is precisely the geometry, not the relevance weighting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FORGE, a training-free and model-agnostic frame-selection method for long-form video understanding. Given a pretrained multimodal encoder that produces per-frame query-relevance scores, FORGE forms a query-conditioned embedding geometry by scaling frame embeddings with sigmoid-transformed relevance weights, projecting to an effective rank estimated from spectral entropy, and greedily selecting K frames that maximize the spanned volume via iterative orthogonalization. The authors claim that this unifies relevance and diversity into a single objective, and they report consistent gains in keyframe recall, scene hit rate, a unified selection score, and downstream VQA accuracy over several training-free baselines on Video-MME and LongVideoBench across eight MLLMs and budgets of 16, 32, and 64 frames.","tokens_in":18213,"tokens_out":5182,"duration_ms":59099,"significance":"If the reported results hold, FORGE would be a practically useful and low-cost improvement to inference-time frame selection for long-video MLLMs. The paper's strengths include a broad evaluation across multiple MLLM families and sizes, deterministic and reproducible-in-principle experiments, and a clear efficiency argument relative to agentic pipelines. The main empirical claims are supported by the tables, which show FORGE leading in every reported cell for the primary selection-quality and accuracy comparisons. However, the paper's central conceptual contribution — that the query-conditioned geometry, rather than the relevance weighting, drives the gains — is not cleanly isolated by the experiments as presented.","major_comments":[{"comment":"The key control is missing: selecting the top-K frames by the relevance score w_t (or s_t) alone is not reported in Tables 1–3. Table 4 includes 'Top-k (reference)' only for the property analysis, not for KFR, SHR, UKSS, or VQA accuracy. Since the query-conditioning in Eq. (4)–(5) enters only through scalar weights, a direct top-K-by-weight baseline is needed to attribute the gains to the orthogonalization/volume step. If top-K by w_t matches FORGE's numbers, the geometry contributes little; if it is much worse, the geometry matters. This baseline must be added to the main tables before the central claim can be assessed.","section":"§4, Tables 1–2; §4.4, Table 4"},{"comment":"The implementation constants c, λmin, λmax, W, τ, and ε are stated to be fixed but are not reported, and the code is not yet available. Similarly, the relevance model that produces s_t is only described as 'a pretrained multimodal encoder' without naming the exact model (e.g., CLIP or BLIP). These omissions make the experiments irreproducible and prevent readers from evaluating sensitivity of the method to the relevance signal. Please specify the encoder, list the constants, and release the implementation or a reproducible description.","section":"§3, Eqs. (3)–(4); §4, Implementation"},{"comment":"The ablation removes the local-contrast term (λ=0) and removes the query warp (w_t=1), but it never removes the greedy orthogonalization/volume-selection step while keeping the relevance weighting. Therefore it does not isolate the contribution of the geometry per se. A variant that selects top-K by w_t, or a variant that uses the same weighted, projected space but selects without volume maximization, is necessary to show that the unification of relevance and diversity is what drives the improvement. The MaxInfo baseline in Table 1 is related but uses a different (unweighted) embedding, so it does not serve as this control.","section":"§4.3, Fig. 5"},{"comment":"The claim that FORGE 'preserves the query's high-dimensional structure' is not directly supported by the formalism. The query text is never embedded into the geometry; the only query-dependent quantity is the scalar score s_t, which is transformed monotonically into w_t and applied as a diagonal scaling diag(w)·E. The SVD projection in Eq. (6) is computed on the weighted frame embeddings, not on any query vector. Thus the geometry depends on the query only through scalar relevance scores, and the paper should either temper the 'high-dimensional' claim or provide an explicit analysis of how the weighting plus spectral projection captures directional query structure.","section":"§3, Eqs. (5)–(7); §1, Contributions"}],"minor_comments":[{"comment":"The phrase 'infdicate' should be 'indicate'.","section":"§4.4, Table 4"},{"comment":"In the sentence 'Downstream video understating is assessed', 'understating' should be 'understanding'.","section":"§4, Setup"},{"comment":"There is no report of variance or confidence intervals. The paper states all numbers were confirmed over three independent runs, but because the method and decoding are deterministic, it would be useful to state explicitly whether the reported values are exact or means over the three runs.","section":"§4, Tables 1–3"},{"comment":"The legend text in the figure appears mangled ('Forge Forge ( =0)'); the notation should be cleaned to match the text's FORGE (λ=0) and FORGE (no query warp).","section":"Fig. 5"},{"comment":"For the LongV A and SeViLA rows, the comparison is between FORGE at 32 frames and the original method at a different frame count (e.g., 128 frames). This is informative but should be clearly labeled as a cross-budget comparison to avoid confusion.","section":"§4, Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is potentially publishable if the missing top-K-by-relevance baseline is added and the implementation details (encoder identity and constants) are disclosed. The conceptual framing currently overstates the role of the 'query-conditioned geometry' because all query information is mediated by scalar weights. I would advise the editor that the strong empirical results are encouraging, but the central mechanism claim needs a de-biasing experiment before acceptance. The code link in the manuscript points to a general lab page rather than a specific repository, so reproducibility is currently not verifiable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: FORGE is a clean, well-scoped frame selector with consistently positive results, but the paper never runs the one baseline that would prove its central claim. Without a top-K-by-relevance-weight comparison, the contribution of the geometry itself is untested.\n\nWhat's genuinely new is the specific construction: query relevance scalars, converted to per-frame weights, used to warp pretrained embeddings, followed by a spectral-entropy projection and greedy residual-volume selection. I don't see that exact pipeline in the prior work cited, and the related-work section does a fair job of mapping the design space. The empirical pattern is strong — every cell in Tables 1 and 2 favors FORGE, and the gains hold across eight MLLMs and three budgets. That consistency is hard to dismiss as noise. The method is training-free, deterministic, model-agnostic, and described precisely enough to reimplement. The \"scalar collapse\" framing is genuinely useful.\n\nThe soft spots are real. The biggest is the missing top-K-by-relevance baseline. The stress-test note is exactly right: FORGE's query conditioning enters only through the scalar weights w_t, so all the gains over AKS, ASCS, MDP3, and MaxInfo could come from those weights rather than from the volume/orthogonalization step. Table 4 lists \"Top-k (reference)\" but only for frame-set properties, never for KFR, UKSS, or accuracy. The ablation in Fig. 5 removes the query warp and the local-contrast term, but never removes the orthogonalization — it never compares FORGE against weighted top-K. That makes the headline claim, that relevance and diversity are unified in a single geometry, unsupported. This is load-bearing, not a minor omission.\n\nThere's also a reproducibility gap: six implementation constants are named but withheld, the visual and multimodal encoders are unnamed, and code is promised only on acceptance. The ablation numbers appear only in figures, not in a table. These are fixable, but they matter for trust.\n\nThe derivation is a chain of proxies — answer entropy swapped for feature entropy, then a Gaussian assumption producing log-volume. That's a reasonable heuristic, not a derivation, and it's assumption-heavy rather than circular.\n\nWho benefits: people working on efficient long-video QA and inference-time frame selection. The paper deserves a serious referee. Send it out, but ask for the weighted top-K baseline, the constants, the encoder names, and the ablation numbers. If the geometry survives that baseline, this is a genuinely useful contribution.\n\n","headline":"FORGE is a plausible, well-scoped frame selector with strikingly consistent wins, but it never runs the one baseline that would prove its central claim: selecting the K highest-relevance-weight frames.","tokens_in":18585,"tokens_out":3959,"would_cite":false,"duration_ms":45984,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free frame selector, FORGE, claims to improve long-video question answering by up to 8.7 accuracy points over uniform sampling.","keywords":["frame selection","long-form video understanding","video question answering","multimodal LLMs","query-conditioned geometry","training-free","keyframe selection","volume maximization"],"falsifier":"A concrete falsifier: run FORGE with a deliberately corrupted relevance score (e.g., a random vector or an image-only embedding) on Video-MME at K=16. If accuracy still exceeds uniform sampling, the geometry itself carries the information; if it drops to or below uniform, the relevance signal is the real driver. Also, a sweep of the lambda constant that controls the global/local blend would test whether the adaptive scale selection is essential.","tokens_in":17749,"feed_emoji":"🎬","tokens_out":4424,"duration_ms":43940,"temperature":0.7,"pith_summary":"FORGE is an inference-time, training-free method for choosing which frames of a long video to feed a multimodal LLM when answering a question. The paper argues that existing selectors suffer from 'scalar collapse': they reduce each frame's query-relevance to a single number, losing which dimensions of the query the frame actually covers. FORGE instead builds a query-conditioned geometry by weighting frame embeddings with a scale-adaptive relevance signal, then selects the K frames spanning the largest volume in that space, unifying relevance and diversity into one objective. On Video-MME and LongVideoBench, it reports consistent gains over five baselines and uniform sampling across eight open-source MLLMs, with up to 8.7-point accuracy gains. The authors claim this shows that aligning the embedding space with the query's high-dimensional structure is a promising direction for inference-time video understanding.","feed_headline":"Frame selector lifts long-video QA by 8.7 points","feed_subtitle":"Training-free method unifies relevance and diversity in one geometry, beating uniform sampling across eight open-source MLLMs","key_machinery":"The central object is the query-conditioned geometry: a weighted embedding space in which each frame's vector is scaled by a relevance weight (derived from a scale-adaptive blend of global score and local temporal contrast) and then projected onto the subspace of its effective rank. The selection mechanism is greedy volume maximization: repeatedly pick the frame with the largest residual norm after orthogonalizing against already-chosen frames, which greedily maximizes the log-volume of the selected subset. Because the log-volume objective is submodular and monotone, greedy selection carries a (1-1/e) approximation guarantee.","core_discovery":"The central claim is that a single volume-maximization objective in a query-warped embedding space can jointly capture relevance and diversity, because frames that are individually relevant but redundant span little volume, and diverse but irrelevant frames are collapsed near the origin. FORGE constructs this space by scaling each frame embedding by a weight derived from a scale-adaptive blend of global relevance score and local temporal contrast, projecting onto the effective-rank subspace, and greedily selecting frames that maximize the log-volume. The paper reports that this outperforms five training-free baselines and uniform sampling on two long-video benchmarks across eight MLLMs, and","pith_inferences":["The method's gains likely hinge on the quality of the pretrained relevance scores; a systematic study varying the matching model would clarify how much of the improvement is due to the geometry versus the scorer.","The local-contrast term is effectively a temporal saliency prior; testing different window sizes or replacing it with other temporal cues could isolate its contribution.","The volume-maximization view could transfer to other subset-selection problems where both relevance and diversity matter, such as selecting frames for video captioning or evidence selection in retrieval-augmented generation.","One testable extension: applying FORGE to shorter videos or different frame pools (e.g., higher FPS) to see if the effective-rank projection is robust to pool density."],"forward_implications":["If correct, FORGE can replace uniform frame sampling as a drop-in inference-time improvement for long-video question answering with any multimodal LLM.","The result suggests that preserving the query's high-dimensional structure in the selection geometry matters more than using scalar relevance scores.","FORGE is complementary to token-compression and pruning methods, so it could be combined with them for further gains.","It offers a cheap alternative to agentic pipelines that repeatedly call an LLM, at a fraction of the compute.","The consistent gains across model sizes and architectures imply that the benefit comes from the selected frames themselves, not from a particular backbone."],"fun_headline_variants":["FORGE: query-aware geometry boosts video QA by 8.7 points","Training-free method unifies relevance and diversity in one goal","Keyframe recall doubles with FORGE's volume-max selection","Video QA accuracy up 8.7 pts with query-warped embeddings"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the per-frame relevance scores from a pretrained multimodal encoder correctly separate answer-bearing frames from distractors; if those scores are weak or misaligned, the query-conditioned geometry has no signal to amplify and FORGE's improvements would collapse.","fun_headline_variants_meta":{"raw":{"variants":["FORGE: query-aware geometry boosts video QA by 8.7 points","Training-free method unifies relevance and diversity in one goal","Keyframe recall doubles with FORGE's volume-max selection","Video QA accuracy up 8.7 pts with query-warped embeddings"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00028,"raw_usage":{"total_tokens":1524,"prompt_tokens":798,"completion_tokens":726,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":651}},"tokens_in":542,"tokens_out":726,"duration_ms":8348,"temperature":1.0,"reasoning_tokens":651,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T01:41:34.892948+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsifier: run FORGE with a deliberately corrupted relevance score (e.g., a random vector or an image-only embedding) on Video-MME at K=16. If accuracy still exceeds uniform sampling, the geometry itself carries the information; if it drops to or below uniform, the relevance signal is the real driver. Also, a sweep of the lambda constant that controls the global/local blend would test whether the adaptive scale selection is essential.","supporting_citations":[],"review_version":2}