{"id":"6fbd4d9e-f863-499f-8b31-281ec7e479ef","arxiv_id":"2508.03009","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"SLFG, a scene-localized frame grouping method, improves MLLMs on long video QA by reassembling frames into semantic scenes, validated on the new LVSQA benchmark and other tests.","lead":"This paper introduces SceneQA, a new long-video question answering scenario, and LVSQA, a dataset built from LVBench videos with new QA pairs, to test MLLMs on scene-level perception and reasoning. It also presents SLFG, a plug-and-play method that groups frames into scene-level units and reassembles them dynamically, claiming significant gains on long video benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract leaves unquantified how SLFG's scene grouping handles imperfect scene boundaries and whether any gain comes from scene semantics or merely from selecting fewer frames; this is the load-bearing dependency.","rationale":"The reader's weakest assumption names scene localization accuracy as pivotal; I agree that this is a key dependency, and I extend it with a second, equally load-bearing condition: the scene-frame representation must preserve question-relevant temporal information. Both conditions are concrete and falsifiable, and both are absent from the abstract. Because the full text is not available, I cannot confirm whether the paper includes the necessary ablations or failure analysis; the appropriate verdict is therefore UNCHANGED from UNVERDICTED rather than a move to rejection. The concern is not that the method is wrong, but that the central claim is empirically under-supported as presented. The proposed test would settle the issue directly by isolating the effect of semantic grouping from simple frame reduction.","tokens_in":754,"tokens_out":1883,"duration_ms":29165,"concrete_test":"Once the released code and dataset are available, run a three-way controlled comparison on the LVSQA benchmark: (A) SLFG with ground-truth scene boundaries from the video annotations; (B) SLFG with boundaries replaced by random segmentation into the same number of groups; (C) uniform frame sampling using the same total token budget. If (A) does not significantly beat (B) and (C), the scene-semantic grouping is not the cause of the gain. Additionally, build a small probe set of QA items requiring exact temporal order within a single scene; if SLFG performs worse on these probes than uniform sampling, the grouping mechanism is discarding necessary temporal detail.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SLFG's scene-localized frame grouping enhances long-video QA in a model-agnostic way. The argument depends on two conditions that the abstract does not support: (1) the scene localization step is accurate enough that grouping does not mix content from different scenes or drop important transitions, and (2) the resulting scene-level representation retains the question-relevant temporal information that many long-video QA benchmarks require. Because the scene localization method is cited but not specified, its failure modes are entirely unquantified. If boundaries are noisy, the MLLM will be fed synthesized frames that blend distinct scenes, which can degrade rather than improve answers. Even with perfect boundaries, condensing many frames into a single scene frame likely discards intra-scene temporal ordering; questions that ask about sequence or short-term actions within one scene would then be unanswerable from the scene frame alone. Furthermore, the abstract's 'performs exceptionally well' is not accompanied by baselines or ablations. A model that merely subsamples frames could show the same or better results; without an ablation that holds the token budget and frame count fixed while varying only the semantic grouping, there is no evidence that scene semantics, rather than frame reduction, causes the improvement. These are not internal inconsistencies, but they are empirically unverified conditions on which the headline result rests.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes SceneQA, a new evaluation scenario emphasizing scene-level perception and reasoning in long-video question answering, along with a corresponding dataset LVSQA built on a subset of LVBench with new question-answer pairs. The authors also introduce SLFG (Scene-Localized Frame Grouping), a model-agnostic preprocessing method that groups individual frames into semantically coherent scene frames using scene localization and dynamic frame reassembly, which is claimed to enhance the understanding capabilities of existing MLLMs in long videos without modifying their architectures. The abstract, which is the only text available for this review, reports no quantitative results, no baselines, no ablations, and no explicit list of evaluated models or benchmarks.","tokens_in":1003,"tokens_out":2514,"duration_ms":29712,"significance":"If the central claims are substantiated, SLFG would be a useful plug-and-play preprocessing module that could improve long-video QA across multiple MLLMs, and LVSQA would provide a new benchmark for scene-based video understanding. The idea of leveraging scene segmentation to reduce the frame burden on MLLMs is plausible and potentially practical. However, the abstract alone provides no evidence for these claims, and the potential value of the contribution is entirely conditional on the missing experimental validation.","major_comments":[{"comment":"The central claim that SLFG 'significantly enhances' MLLM understanding and 'performs exceptionally well' on benchmarks is unsupported by any quantitative comparison, baseline, or ablation. The abstract should report key metrics (e.g., accuracy on LVBench, LVSQA, or other long-video QA benchmarks) and specify the baselines, such as the same MLLM without SLFG or with simple frame subsampling, to demonstrate the incremental benefit.","section":"Abstract"},{"comment":"The method depends on 'scene localization methods' and 'dynamic frame reassembly mechanisms,' but the abstract does not identify the specific localization method or characterize its accuracy and failure modes. If scene boundaries are noisy, grouping frames across scene boundaries could mix content from different scenes or discard important transitions, potentially degrading MLLM performance. The paper should specify the localization approach and include an analysis or ablation of sensitivity to localization errors.","section":"Abstract"},{"comment":"The claim that SLFG is model-agnostic and 'enhances the understanding capabilities of existing MLLMs' is not supported by a list of evaluated models or a demonstration of generality. Without knowing which MLLMs were tested and whether the improvement holds across architectures, the 'existing MLLMs' claim is too broad. The abstract should name the evaluated models and report per-model results.","section":"Abstract"},{"comment":"The core idea of combining individual frames into semantically coherent scene frames raises a concern about preserving intra-scene temporal ordering. Condensing many frames into one scene frame may discard information about sequences or short-term actions within a scene, which are often required by video QA tasks. The paper should address how SLFG handles questions that depend on temporal order within a scene, or provide evidence that such questions are not adversely affected.","section":"Abstract"}],"minor_comments":[{"comment":"The phrase 'performs exceptionally well' is vague and should be replaced with precise metric names and numerical values to allow objective assessment.","section":"Abstract"},{"comment":"The URL 'http://www.slfg.pkuzwh.cn' should be checked for validity and preferably given as an HTTPS link; the domain name is also unusual and could be verified.","section":"Abstract"},{"comment":"The abstract introduces SceneQA and LVSQA but does not explain how LVSQA differs from LVBench beyond new question-answer pairs; a brief description of the dataset size, selection criteria, and type of annotations would be helpful.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"This review is based solely on the abstract, as the full text was not made available. The abstract lacks the experimental evidence necessary to support the paper's headline claims, so the revision should add quantitative results and ablations directly in the abstract or provide them in the full text. If the full text already contains such evidence, the abstract must be rewritten to reflect it accurately. I would re-evaluate after receiving the full manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead the abstract for arXiv:2508.03009. The pitch: introduce SceneQA, a scenario emphasizing scene-level reasoning in long-video QA, with a new dataset LVSQA built on LVBench, plus SLFG, a model-agnostic grouping that clusters frames into scene-level composites before feeding an MLLM. The combination of task, dataset, and method is new as far as I can tell. The plug-and-play framing is genuinely useful—no architecture changes, just a preprocessing module. If it works, it's a practical tool for the long-video QA crowd.\n\nWhat the abstract does well: it names a real gap (frame-level retrieval isn't the same as scene-level reasoning), and it commits to releasing code and data. That matters. The method has a plausible cognitive motivation: humans group scenes, so frame grouping is a reasonable inductive bias.\n\nThe soft spots are mostly about evidence. There are no numbers, no baselines, no ablations in the abstract. The phrase 'performs exceptionally well' is doing a lot of work. The key dependency is the scene localization step: if its boundaries are noisy, SLFG could feed blended or misleading frames to the MLLM and hurt rather than help. Even with clean boundaries, condensing many frames into one scene frame could discard intra-scene temporal order—questions about sequences within a scene would break. The abstract doesn't address this. Also, without an ablation that holds token budget fixed and compares semantic grouping against simple frame subsampling, we can't tell if the gain comes from scene reasoning or just from seeing fewer frames.\n\nThe stress-test note is fair, though it's a concern about what's missing, not a demonstrated flaw. The self-built dataset is a usual concern, but that's cultural; what matters is whether the QA pairs and video selection have rigorous filtering.\n\nRecommendation: this deserves a serious referee, not a desk reject. The novelty bar is cleared, and the checklist (dataset, code, ablations) is concrete. The referee should demand full experimental tables, an ablation isolating the grouping mechanism, and an honest failure analysis of the scene localization step. If the full text doesn't provide those, then it fails, but we can't know from the abstract. I'd bring it to reading group if the full text shows up; for now, maybe.","headline":"A plausible plug-and-play idea for long-video QA that is impossible to evaluate from the abstract alone; worth a referee's time if the full experiments deliver what the abstract promises.","tokens_in":1512,"tokens_out":1853,"would_cite":false,"duration_ms":19915,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that grouping video frames into semantically coherent scene units improves long-video question answering in existing multimodal LLMs without changing their architecture.","keywords":["long video understanding","video question answering","multimodal large language models","scene localization","frame grouping","SceneQA","LVSQA dataset"],"falsifier":"Take a set of long videos with human-annotated scene boundaries, run SLFG using its own scene localizer, and compare question-answering accuracy against a version of SLFG fed oracle scene boundaries. If accuracy is no better than random or fixed-interval frame grouping when the localizer is wrong, the claimed gain rests entirely on the localizer's accuracy.","tokens_in":585,"feed_emoji":"🎬","tokens_out":4889,"duration_ms":56830,"temperature":0.7,"pith_summary":"This paper tries to establish that a preprocessing step—grouping video frames by detected scene boundaries rather than sampling or sparsely picking frames—can make existing multimodal large language models substantially better at answering questions about long videos. It argues that current long-video benchmarks ask models to spot a single frame containing a core object, which does not match real video questions, so it proposes SceneQA, a scenario built on scene-level detail perception and reasoning, and introduces the LVSQA dataset to test it. The method, SLFG, localizes scenes and dynamically reassembles frames into scene-level units, then feeds those units to any multimodal LLM without retraining. A sympathetic reader would care because it promises an immediate, plug-and-play accuracy gain for long-video understanding while also steering evaluation toward a more realistic task.","feed_headline":"Grouping frames by scene sharpens long-video question answering","feed_subtitle":"A plug-in module groups a long video into scene-level frames so existing multimodal models can answer detail questions better.","key_machinery":"The load-bearing object is SLFG's two-step grouping pipeline. First, scene localization identifies where semantic scene boundaries occur in the long video. Second, dynamic frame reassembly packs frames inside each detected scene into a coherent scene-level representation, which is then passed to the multimodal LLM in place of raw frames or uniformly sampled frames. This machinery carries the argument because it changes only the input representation, leaving the model's weights untouched, so any measured improvement comes from better frame organization rather than added model capacity.","core_discovery":"The paper's central claim is that long-video question answering fails not because multimodal LLMs cannot reason over video, but because they are shown either too many redundant frames or the wrong sparse frames. To fix this, the paper defines SceneQA, a task centered on scene-level detail perception and reasoning, constructs the LVSQA benchmark with fresh question-answer pairs on selected long videos, and proposes SLFG, a model-agnostic module that detects scene boundaries and dynamically reassembles frames into semantically coherent scene frames. SLFG runs before the multimodal LLM, requires no architecture changes, and the reported experiments show large gains on several long-video benchmarks. If true, the contribution is an input-organization method plus a more realistic evaluation scenario, not a new model.","pith_inferences":["Inference: SLFG's gains probably depend on the quality of the scene localizer; benchmarking the method against oracle scene boundaries would separate grouping quality from localization quality.","Inference: The same scene-grouping idea could be adapted to online or streaming settings, where scene boundaries are detected incrementally and the grouped representation is updated as frames arrive.","Inference: If grouped scene frames are fed as single units, SLFG may also reduce the number of tokens per video, making it an efficiency tool for long-video processing, though the abstract does not report token counts."],"forward_implications":["Any existing multimodal LLM can be upgraded for long-video question answering by routing its inputs through SLFG, with no retraining and no change to model weights.","Benchmarks built around scene-level questions will expose failures that frame-spotting tasks hide, giving a more realistic score of real-world video understanding.","SLFG's dynamic reassembly lets the input adapt to video content, so a slow scene keeps more frames while a fast-cut scene is compressed into fewer scene units.","Combined with SceneQA and LVSQA, the method provides a recipe: localize scenes, group frames, ask questions, and compare models fairly on scene-level detail."],"supporting_citations":[],"fun_headline_variants":["Scene-grouped frames improve long-video QA accuracy","Frame grouping by scene clarifies long-video questions","SLFG: scene-aware frame assembly boosts video QA","Group frames by scene to sharpen long-video answers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the scene localization step reliably finds true scene boundaries in arbitrary long videos; if it mixes frames from different scenes or drops key transitions, the grouped input can hurt a multimodal LLM's answers instead of helping.","fun_headline_variants_meta":{"raw":{"variants":["Scene-grouped frames improve long-video QA accuracy","Frame grouping by scene clarifies long-video questions","SLFG: scene-aware frame assembly boosts video QA","Group frames by scene to sharpen long-video answers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000392,"raw_usage":{"total_tokens":2062,"prompt_tokens":945,"completion_tokens":1117,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":1055}},"tokens_in":561,"tokens_out":1117,"duration_ms":10756,"temperature":1.0,"reasoning_tokens":1055,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T04:42:21.484513+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of long videos with human-annotated scene boundaries, run SLFG using its own scene localizer, and compare question-answering accuracy against a version of SLFG fed oracle scene boundaries. If accuracy is no better than random or fixed-interval frame grouping when the localizer is wrong, the claimed gain rests entirely on the localizer's accuracy.","supporting_citations":[],"review_version":1}