{"id":"a167f7ba-72ad-48e3-975f-a35feb20870a","arxiv_id":"2504.16082","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage MapReduce pipeline, independent clip captioning followed by global reduction, lifts LVBench accuracy to 60.8 percent, over ten points higher than previous methods.","lead":"MR. Video applies the MapReduce idea to long-video understanding: first every short clip is described independently, then all descriptions are merged to answer a question. On LVBench, a hard benchmark with hour-long videos, it reports 60.8 percent accuracy, more than ten points above prior VLMs and video agents.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Uncontrolled comparison undermines the >10% MapReduce claim: MR. Video uses Gemini-2.0-Flash + GPT4o with heavy test-time compute, while prior agents use different backbones and leaderboard numbers.","rationale":"The paper is clearly written, releases code, and honestly discloses several limitations, including the caption-quality upper bound in Sec. 4.5.2 and the fact that different benchmarks emphasize different skills. Those disclosures are real evidence of good-faith reporting. However, the central scientific claim — that the MapReduce principle, not the particular VLM/LLM combination or extra compute, causes the >10% improvement — is not yet supported by controlled experiments. The reader's weakest-assumption focuses on information loss through captions; that is a genuine bottleneck and is acknowledged by the authors, but it is not the main threat to the headline number. The more load-bearing issue is that every baseline in Table 1 is either a raw VLM with a single forward pass or an agent built with different components and budgets, so the comparison conflates the MapReduce design with model strength, test-time compute, and evaluation-set differences. The ablation on 98 questions is underpowered and does not include a same-backbone agent baseline. A matched, same-budget comparison against a sequential key-segment agent would directly test whether the MapReduce architecture itself is responsible for the reported gain. Because the reader already issued a CONDITIONAL verdict based on overlapping concerns, my stress-test does not change that verdict; it sharpens the specific experiment needed.","tokens_in":26023,"tokens_out":4361,"duration_ms":42566,"concrete_test":"Run a matched non-MapReduce agent on the exact 1,492-question LVBench subset used by MR. Video: same VLM (Gemini-2.0-Flash), same LLM (GPT4o), and comparable per-question inference budget, but using sequential key-segment retrieval (e.g., a VCA/VideoAgent-style loop that selects key frames/clips before answering). If this matched agent scores within roughly 5 points of MR. Video's 60.8%, the >10% claim cannot be attributed to MapReduce; if it remains near 41-49%, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the >10% LVBench improvement (60.8% vs. previous best 48.7% among VLMs and 41.3% among agents, Table 1). This comparison is not controlled. MR. Video combines Gemini-2.0-Flash (VLM) with GPT4o (LLM), processes 500-2k captions per video, and runs multi-stage analysis per question (Sec. 3, Sec. 4.2). Baselines are taken from leaderboards or the VCA paper, so they use different VLMs, different LLMs, and different per-video compute budgets. The only same-backbone point is raw Gemini-2.0-Flash at 48.6% with 256 uniformly sampled frames; the 12.2-point gap over that baseline could come from additional test-time compute, the stronger GPT4o reasoner, denser frame sampling, or the captioning/analysis pipeline — not necessarily from the MapReduce structure as opposed to, say, exhaustive captioning followed by direct QA. Moreover, MR. Video is evaluated on 1,492 questions after dropping 4 unavailable videos, while leaderboard baselines are presumably on the full 1,543 questions (Sec. 4.1); if the removed videos are not equally hard for all methods, the relative margin shifts. The 98-question, 6-video ablation (Sec. 4.4) is far too small to establish that component-level differences (e.g., 62.2% vs. 59.2% without consistent characters) are real, and it does not include a same-backbone non-MapReduce agent. The paper's own limitation about caption quality (Sec. 4.5.2) is a separate, acknowledged issue; the load-bearing problem is that the headline improvement has not been isolated from confounded baselines.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MR. Video, an agentic long-video understanding framework built on a 'MapReduce' principle: a Map stage independently and densely perceives short video clips (scene captioning, then per-question segment analysis), and a Reduce stage aggregates the resulting information into a consistent video-level representation and a final answer (character/object merging, global intention analysis, answer generation). The system is implemented with Gemini-2.0-Flash as the VLM and GPT-4o as the LLM, and it is evaluated on LVBench, LongVideoBench, Video-MME (long subset), and EgoSchema. The authors report 60.8% on LVBench and claim a >10% improvement over prior VLMs and video agents, with additional ablations on consistent characters, question intention analysis versus MM-Embed retrieval, and goal-aware analysis.","tokens_in":26379,"tokens_out":6863,"duration_ms":63430,"significance":"If the quantitative claims hold, the paper makes a strong conceptual and practical contribution: it proposes a simple and interpretable principle for long-video understanding, ships code and a complete prompt suite, and openly documents failure modes such as caption information loss. The reported results on LVBench and LongVideoBench are substantially higher than prior methods, and the MapReduce framing is attractive for scalable, sequence-parallel video agents. However, the current verification does not control for the much stronger LLM (GPT-4o), the much larger test-time compute budget, or the difference in evaluation question sets, so the central claim that the MapReduce structure itself is responsible for the improvement is not yet established. The paper's honest limitations section and its detailed failure analysis partially mitigate this, but the main comparison and ablation evidence need hardening.","major_comments":[{"comment":"The headline comparison of MR. Video (60.8%) against prior VLMs (best 48.7%) and video agents (best 41.3%) is uncontrolled. MR. Video uses Gemini-2.0-Flash for visual perception and a separate GPT-4o LLM for reasoning, plus a dense captioning pass over the entire video and per-question VLM queries, whereas the listed baselines come from leaderboards or the VCA paper and use different backbones and much smaller test-time compute budgets. The only same-backbone baseline, Gemini-2.0-Flash with 256 uniformly sampled frames (48.6%), differs from MR. Video simultaneously in three variables: test-time compute, the presence of a separate LLM, and the MapReduce structure itself. The paper therefore cannot attribute the 12.2-point gap to the MapReduce principle on the basis of Table 1. The authors' own Fig. 6(a) shows that on the 98-question subset, replacing GPT-4o with Qwen-2.5-7B drops accuracy from 62.2% to 43.9% (18.3 points), a larger swing than any component ablation; this demonstrates that LLM choice is a major confound that must be controlled with a same-backbone non-MapReduce agent before the >10% claim can be accepted.","section":"§4.3.2, Table 1"},{"comment":"The main evaluation is not performed on the same question set as the baselines. Section 4.1 states that 4 of 103 LVBench videos are unavailable, leaving 1,492 questions, yet Section 4.3.2 refers to 'the complete set of LVBench' and Table 1 compares against leaderboard numbers presumably computed on the original full set. The manuscript is also internally inconsistent about the full size: Section 4.1 says LVBench curates 1,549 questions, while Appendix B.4 says 'we still have 1,492 out of 1,543 questions.' If the 4 missing videos contain 51–57 questions, the reported margin of >10% is measured against a different test set. The authors should either re-run available baselines on the same 1,492 questions or report how the margin changes if the removed videos are excluded from the baseline numbers; without this, the exact magnitude of the claimed improvement is not apples-to-apples.","section":"§4.1, §4.3.2, Appendix B.4"},{"comment":"The ablation study is based on only 98 questions from 6 videos, and no confidence intervals or significance tests are reported. The differences in Fig. 6(a) (62.2% vs. 59.2% for 'w/o Consistent Characters', 62.2% vs. 52.0% for 'w/o Goal-Aware Analysis') correspond to 3 and 10 questions respectively out of 98, which is within sampling noise for paired binary outcomes; a 3-question difference is effectively a tie. The key-frame recall comparison in Fig. 6(b) uses only 64 questions (as stated in Appendix B.5.2) and similarly lacks uncertainty quantification. More importantly, none of these ablations includes a same-backbone agent that uses sequential key-frame retrieval with the same VLM and LLM, so the experiments cannot isolate the MapReduce principle from the particular pipeline choices. The authors should either enlarge the ablation set, report binomial confidence intervals, or provide a direct controlled comparison with a comparable-compute non-MapReduce agent before claiming that the results validate the principle.","section":"§4.4, Fig. 6"},{"comment":"The paper itself acknowledges in Sec. 4.5.2 and in the Limitations section that 'the upper bound of MR. Video is limited by the quality of captions' and that information omitted by the VLM cannot be recovered by the Reduce stage. This is an honest caveat, but it directly qualifies the central claim that MapReduce is the principle for long video understanding. The conclusions currently state that the principle is validated without mentioning that it is only validated for questions whose required visual information survives the dense captioning step; for fine-grained detail questions such as the noodle example in Fig. 8, the method has a known failure mode. The paper should reword the conclusions to make this conditional explicit and should discuss the failure on detail-oriented questions in the main contribution rather than only in the supplementary failure analysis.","section":"§3.4, §4.5.2, Conclusions"}],"minor_comments":[{"comment":"The figure does not report the sample size; please add 'n=98' for (a) and 'n=64' for (b) in the captions or in the text, and ideally add error bars based on a binomial or paired estimate.","section":"Fig. 6"},{"comment":"There are several typographical errors: 'Th six selected videos' should be 'The six selected videos' in Appendix B.4; 'shoty' should be 'shot' in Fig. 7; 'paritipating' should be 'participating' in Table C; 'previoous' should be 'previous' in Table J; and 'a captal letter' should be 'a capital letter' in Table J. These should be corrected.","section":"Appendix B.4 and Fig. 7"},{"comment":"The breadth comparison in Table 2 also relies on numbers from different sources for different backbones; for example, LongVideoBench accuracies for baselines are from the LongVideoBench paper rather than re-evaluated under the same conditions. Please state explicitly in the text that these are not controlled comparisons and consider adding the evaluation setup (e.g., number of frames, prompting) for each baseline.","section":"§4.3.2, Table 2"},{"comment":"The captioning Reduce step merges repeated characters using VLM association, but the paper does not report the cost or failure rate of this association over the full LVBench set. Adding a brief quantitative statement (e.g., how often the merging step changes the final names) would improve reproducibility.","section":"§3.2.2 and Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The MapReduce framing is appealing and the paper is honest about its limitations, but the central quantitative claim currently rests on an uncontrolled comparison and a very small ablation. I would not reject on the idea alone; the issues are fixable with additional controlled experiments or a more careful wording of the contribution. The internal inconsistency in the LVBench question count (1,549 vs. 1,543) should also be resolved during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The punchline: MR. Video gives the field a genuinely new organizational principle for long-video understanding—dense, parallel clip-level captioning (the Map) followed by global aggregation and reasoning over all of that text (the Reduce). That framing is simple and testable, and the paper's reported 60.8% on LVBench is a big number. But the comparison behind that number is not controlled, so I'd treat the '>10% improvement' as a promising signal rather than an established fact.\n\nWhat's actually new: the paper does more than rename an agentic pipeline. It decouples perception from reasoning by converting the whole video into a large set of scene-level captions, then runs question intention analysis and goal-aware probing over those captions plus targeted frames. The character-consistency 'Reduce' step addresses a real failure mode of sequence-parallel captioning, and the head-to-head against MM-Embed key-frame retrieval (68.8% vs 34.4% recall) supports the claim that broad context helps localization. The paper is also honest about its main failure bottleneck—caption quality limits the upper bound (Sec. 4.5.2)—and it ships code and reports dollar costs, which is more than many agent papers do.\n\nThe soft spots are real but not fatal. The headline comparison mixes backbones and compute budgets: MR. Video uses Gemini-2.0-Flash as VLM plus GPT-4o as LLM, while prior agents use different VLMs and leaderboard numbers. The only same-backbone baseline is raw Gemini-Flash at 48.6% with 256 frames; the 12.2-point gap could come from extra test-time compute, denser sampling, the stronger reasoner, or the MapReduce structure itself. The ablation is too small to carry much weight—98 questions across 6 videos, so the 3-point effect from consistent characters is within noise. The evaluation on 1,492 of 1,543 LVBench questions (4 dead videos) is a minor confound, not a major one.\n\nIf the authors add a same-backbone agent baseline (e.g., VideoAgent with Gemini-Flash and GPT-4o on the same 1,492 questions) and report confidence intervals or significance on the ablations, the central claim would become much stronger. As it stands, this is a solid, well-written empirical paper with an important idea and a still-uncontrolled headline result.\n\nRecommendation: send it to serious peer review. The MapReduce idea deserves a rigorous controlled test, and the paper is good enough to get one. Major revision, not desk reject.","headline":"A plausible and clearly written new recipe for long-video QA, but the headline >10% LVBench gain is not yet an apples-to-apples comparison.","tokens_in":26939,"tokens_out":1639,"would_cite":true,"duration_ms":18561,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating long video understanding as MapReduce—caption every clip, then aggregate—outperforms prior models by 10 points on LVBench.","keywords":["long video understanding","MapReduce","video agents","vision-language models","dense video captioning","LVBench","question intention analysis","video question answering"],"falsifier":"For any LVBench question whose answer depends on a visual detail visible in only one short clip, edit the clip's caption to omit that detail and rerun MR. Video; if the model still answers correctly, the stated caption bottleneck is not load-bearing, whereas if accuracy drops, the bottleneck claim is confirmed. Equivalently, measure whether per-question accuracy is predicted by whether the correct object appears in the captions.","tokens_in":25792,"feed_emoji":"🎬","tokens_out":8167,"duration_ms":70748,"temperature":0.7,"pith_summary":"The paper sets out to show that long video understanding can be treated as large-scale data processing: break the video into short clips, perceive each clip independently and in detail (the Map step), then aggregate all clip-level results into a unified answer (the Reduce step). It instantiates this as MR. Video, a two-stage agent pipeline that first writes dense captions for every scene and then runs question-specific analysis over those captions. On LVBench, which contains hour-long videos with challenging questions, MR. Video scores 60.8% accuracy, more than 10 points above state-of-the-art VLMs and video agents. The authors argue that the MapReduce principle, not any single component, is what makes the improvement possible.","feed_headline":"MapReduce video agent beats long-video models by 10 points","feed_subtitle":"Caption every clip, then reason across all captions: 60.8% on LVBench, 10+ points over prior systems.","key_machinery":"The load-bearing mechanism is the MapReduce decomposition itself: independent, sequence-parallel perception of short clips (Map) followed by joint aggregation and reasoning over all clip-level outputs (Reduce). In MR. Video this takes concrete form as a Captioning stage—dense scene captions plus a Reduce step that merges repeated characters into standardized names—and an Analysis stage—question intention analysis over caption segments, goal-aware queries to a vision-language model on candidate scenes, and a final Reduce that composes an answer. This design sidesteps the context-length limit of sequence-to-sequence VLMs and replaces the sequential key-segment retrieval of earlier video agents with broader, parallel scene inspection and global reasoning.","core_discovery":"On the paper's own terms, the central claim is that the MapReduce principle is the route to long video understanding: every short segment is perceived thoroughly and independently, so no detail is skipped, and the results are then combined globally, so no context is lost to a limited window. MR. Video implements this in two stages: captioning maps clips to dense scene captions and reduces repeated characters and objects to consistent names; analysis maps the user's question onto candidate scenes and reduces scene-level evidence into a final answer. The reported result is 60.8% accuracy on LVBench, an improvement of more than 10% over previous VLMs and video agents, with ablations showing that consistent character names, question intention analysis, and goal-aware perception each contribute. The authors frame the principle as applicable to both VLMs and video agents, with the LLM-agent implementation serving as a convenient validation.","pith_inferences":["Extension: the same MapReduce recipe could be trained end-to-end inside a single VLM using segment-level attention blocks, so dense local perception and global aggregation happen in one forward pass rather than through an agent loop; the paper notes this as future work.","Extension: because the authors identify caption quality as the upper bound, a closed-loop system that detects when a question needs details absent from captions and re-queries the VLM on those clips is a natural next step that could push accuracy further.","Extension: the principle should transfer to other long-form multimodal settings—audio, egocentric video, or multi-camera streams—where local details must be accumulated over a long horizon, though that transfer is not tested in the paper."],"forward_implications":["A simple caption-then-analyze pipeline can outperform both context-limited VLMs and key-segment video agents on localization-heavy long-video benchmarks.","Because the Map steps are independent and sequence-parallel, the same recipe should scale to longer videos without hitting a context window, at the cost of more captioning calls.","Question intention analysis over captions localizes relevant scenes more reliably than embedding-based key-frame retrieval (68.8% vs. 34.4% recall on the paper's subset).","Character and object consistency in captions, and a final goal-aware perception step, each contribute measurable accuracy on the ablation subset, so the Reduce stages are doing real work, not just bookkeeping."],"supporting_citations":[{"why":"Supplies the MapReduce principle from big-data processing that the paper transfers to video understanding.","marker":"[7]"},{"why":"LVBench is the primary benchmark; its 1,492-question subset supplies the headline 60.8% accuracy.","marker":"[38]"},{"why":"VideoAgent is the representative sequential key-segment video agent that MapReduce is contrasted against.","marker":"[41]"},{"why":"VCA is another video-agent baseline whose reported accuracies are used in the comparison.","marker":"[53]"},{"why":"VideoChat-Flash is the strongest vision-language baseline on LVBench that MR. Video surpasses by more than 10 points.","marker":"[21]"},{"why":"The cited retriever supplies the embedding-based key-segment retrieval baseline in the localization ablation.","marker":"[24]"},{"why":"The cited technical report supplies the vision-language model used for captioning and frame perception in MR. Video.","marker":"[36]"}],"fun_headline_variants":["MapReduce for long videos: 10-point LVBench jump","See every clip, then think: MR. Video's 60.8%","Two-stage MapReduce video reasoning beats prior by 10","Dense clip captions, global reasoning: MR. Video wins","MapReduce video agent: 60.8% on LVBench, +10"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the dense captions and sparsely sampled frames produced in the Map stage preserve the visual details needed to answer each question; the authors state in Section 4.5.2 that information lost at captioning time, such as a missed object or face, cannot be recovered by the Reduce stage.","fun_headline_variants_meta":{"raw":{"variants":["MapReduce for long videos: 10-point LVBench jump","See every clip, then think: MR. Video's 60.8%","Two-stage MapReduce video reasoning beats prior by 10","Dense clip captions, global reasoning: MR. Video wins","MapReduce video agent: 60.8% on LVBench, +10"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1451,"prompt_tokens":976,"completion_tokens":475,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":381}},"tokens_in":592,"tokens_out":475,"duration_ms":4279,"temperature":1.0,"reasoning_tokens":381,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:10:34.420720+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For any LVBench question whose answer depends on a visual detail visible in only one short clip, edit the clip's caption to omit that detail and rerun MR. Video; if the model still answers correctly, the stated caption bottleneck is not load-bearing, whereas if accuracy drops, the bottleneck claim is confirmed. Equivalently, measure whether per-question accuracy is predicted by whether the correct object appears in the captions.","supporting_citations":[],"review_version":1}