{"id":"31820431-4dbb-49a8-b9f0-ca55eff12045","arxiv_id":"2411.14901","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A recursive vision-language model with sparse and dense video features reports state-of-the-art temporal grounding on hour-long videos.","lead":"ReVisionLLM is a vision-language model that finds events in hour-long videos by first scanning broad segments and then zooming in to refine the boundaries. It reports state-of-the-art results on two long-video grounding benchmarks, but several internal inconsistencies make the exact gains uncertain.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Coarse-stage recall is unmeasured and irrecoverable: if the top hierarchy drops the true event, no lower level can recover it, yet the paper never reports this recall.","rationale":"The reader's weakest_assumption identifies exactly the part of the argument least supported by evidence: the coarse stage's recall is never measured, and a miss there is structurally unrecoverable. This is the most load-bearing concern because the entire recursive mechanism, and the associated efficiency claim, depend on the top level reliably retaining the true event. The concern does not by itself falsify the SOTA claim; the reported final metrics already include any coarse misses, and the model still outperforms baselines. But it does mean the paper has not verified the core mechanism it proposes, and the efficiency figure (57% frames processed) could be misleading if the pruning is discarding real events. The reader's CONDITIONAL verdict is appropriate: the paper should measure coarse recall and either demonstrate that the top level nearly always retains the event or explicitly report the trade-off. I also considered the missing VLM baselines and the citation inconsistencies, but those are completeness issues that can be fixed without altering the central mechanism; the coarse-recall gap is the one place where the architecture could fail in a way that the final metric would silently absorb. The proposed test is a single, feasible computation using the authors' own model and data, and it would settle whether the concern lands.","tokens_in":19276,"tokens_out":10052,"duration_ms":98897,"concrete_test":"Run the default ReVisionLLM top-level stage alone on the MAD test split: for each query, extract the predicted coarse interval and compute the fraction of ground-truth moments whose center (or full span with IoU>=0.1) lies within that interval. Report this top-level recall@1. If it is below roughly 95%, quantify how many final errors originate from irrecoverable coarse misses and discuss the efficiency/performance trade-off explicitly; if it is near 100%, the concern is mitigated and the recursive mechanism is verified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The recursive design in Sec. 3.2 (Fig. 3) makes the top-level coarse selection structurally load-bearing: hour-long video is encoded into sparse features (one token per ~125s window on MAD), the LLM predicts a coarse interval, and only segments inside that interval receive dense-feature processing at the bottom hierarchy. Any event whose true span is not contained in the top-level interval is irrecoverable. The paper never measures recall of this coarse stage, so it cannot distinguish between (a) the hierarchy genuinely narrowing to the event and (b) the top level occasionally dropping the true segment, with the final metric silently absorbing those misses. Table 3 is consistent with this concern: default ReVisionLLM (top-down, 57% frames) scores 15.0 R1@.1, while ReVisionLLM-I (bottom-up, 100% frames) scores 17.4 R1@.1. The 2.4-point gap may be exactly the cost of coarse misses, but the paper does not decompose it. This matters because the abstract's efficiency claim (43% fewer frames than VTimeLLM) is only meaningful if the pruned segments are truly irrelevant; if coarse recall is low, the efficiency is purchased by discarding a nontrivial fraction of events.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ReVisionLLM, a recursive vision-language model for temporal grounding in hour-long videos. The model builds on a frozen CLIP encoder, a hierarchical adapter that produces dense and sparse temporal features, and a Vicuna-7B LLM with LoRA that predicts start/end boundaries progressively from coarse to fine. Training proceeds in stages: first on short contrastive segments, then on long videos with sparse features; inference ranks predictions by an inverse-entropy confidence score. The paper reports state-of-the-art claims on MAD and VidChapters-7M, ablations of the proposed modules and model variants, an ECE calibration comparison in the supplement, and a text-to-video retrieval generalization experiment on MSRVTT, with code released.","tokens_in":19548,"tokens_out":11192,"duration_ms":119446,"significance":"If the reported numbers are correct, ReVisionLLM is a practically relevant contribution: it demonstrates that a single instruction-following VLM can perform temporal grounding on hour-long inputs, and its recursive inference offers an accuracy/efficiency trade-off (57% of frames on MAD for the default variant). The paper has clear strengths: the staged ablations in Tables 2-4 isolate the contributions of contrastive segments, confidence-based ranking, and recursion; the supplementary ECE analysis directly addresses the calibration motivation; and the MAD headline number (+2.6% R1@0.1 over RGNet) is internally consistent with Table 1. However, several central claims and protocol details need additional support or correction before the results can be taken at face value, in particular the VidChapters-7M 'state-of-the-art' claim and the unmeasured recall of the top-level coarse selection.","major_comments":[{"comment":"The text states that ReVisionLLM 'sets a new state-of-the-art' on VidChapters-7M and 'significantly outperforms' M-DETR, but the comparison in Table 1 is not uniformly favorable: at R1@0.3 the model scores 33.8 versus M-DETR's 37.4, and at R1@0.5 the two are essentially tied (27.4 versus 27.3). The claim rests on the average and on the high-IoU thresholds R1@0.7 and R1@0.9. Please specify which metric is the primary endpoint, report the aggregated recall consistently, and state whether the 'new state-of-the-art' claim is based on the average rather than on the standard lower-IoU thresholds. Without this clarification, the VidChapters-7M claim is stronger than the table supports.","section":"§5, Table 1"},{"comment":"The recursive pipeline is structurally load-bearing: the top hierarchy consumes only sparse temporal features, and the lower hierarchies receive dense features only for segments inside the previously predicted interval. Any event not contained in the top-level interval is irrecoverable, yet the paper never reports the recall of this coarse stage. Table 3 makes the concern concrete: the top-down default ReVisionLLM uses 57% of frames and reaches R1@0.1 = 15.0, while the bottom-up ReVisionLLM-I uses 100% of frames and reaches R1@0.1 = 17.4. The 2.4-point gap may be exactly the cost of coarse misses, but the paper does not decompose it. Please report coarse-stage recall or per-hierarchy cumulative recall, and connect it to the efficiency claim of processing 43% fewer frames than VTimeLLM; otherwise the efficiency is purchased at an unquantified risk of discarding true events.","section":"§3.2, Fig. 3, Table 3"},{"comment":"The hierarchy count and indexing are inconsistent. Section 3.2 defines I^(1) as the dense bottom level and higher levels l = 2,...,L as sparse, while Section 3.4 refers to 'the bottom hierarchy (l = 0)'. Table 4 ablates 0 to 3 hierarchies, but the supplement says 'we employ two hierarchies with long videos' and then describes a top level, a second level, and a final hierarchy. These inconsistencies prevent readers from reproducing the exact inference protocol for the default model and the ReVisionLLM-I variant. Please align the definitions, state exactly how many levels are used for MAD and VidChapters-7M, and specify which levels consume dense versus sparse features.","section":"§3.2 and Supplementary S1"}],"minor_comments":[{"comment":"The sentence on ReVisionLLM-I states that it outperforms RGNet by '+4.9% in R1@.1 and +6.3% in R1@.3', but Table 1 gives R1@.3 = 12.7 for ReVisionLLM-I versus 9.5 for RGNet, a difference of +3.2, not +6.3. Please correct the number.","section":"§5"},{"comment":"Table 1 cites VTimeLLM* as [69], but reference [69] is not VTimeLLM; the correct reference is [18]. Similarly, the text refers to the CLIP encoder as [60], while [60] is CLIP-ViP; the CLIP paper is [45]. Please fix these reference mismatches.","section":"Table 1 and References"},{"comment":"The LoRA training protocol is described inconsistently: Section 3.3 Stage 2 says 'fine-tuning only the same LoRA module used in Stage 1', Section 4.2 says the default model trains two LoRAs (one for the bottom hierarchy, one for higher levels), and the supplement says two separate LoRA modules are used. Please clarify which LoRA is updated in each stage and which one is used at each hierarchy.","section":"§3.3, §4.2, Supp. S1"},{"comment":"For the MSRVTT text-to-video retrieval experiment, the paper says all videos are concatenated into a single long video and the model predicts the relevant video index. Please specify how many videos are concatenated per test example, how the input length is handled for the 1k-Test set, and whether the 'hour-long' framing applies to this setting.","section":"§5.3 and Supp. S3"},{"comment":"The video-length ablation in Figure 5 extends videos by repeating them multiple times. Please state explicitly whether the reported long-video behavior is measured on repeated content, since this is not equivalent to naturally longer videos with new scene structure.","section":"§5.1, Fig. 5"},{"comment":"The sentence on ReVisionLLM-U says it uses 'fewer trainable parameters (363M vs. 159M)', but Table 3 lists ReVisionLLM-U as 159M and ReVisionLLM as 363M. The numbers in parentheses appear to be in the wrong order.","section":"§5.1, Table 3"},{"comment":"The implementation details paragraph contains the phrase 'Training is conducted on the using a total batch size of 128'; please remove the stray words.","section":"§4.2.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is sound and the MAD result appears internally consistent, but the paper needs a careful revision round before it can be accepted: the VidChapters-7M claim must be reconciled with Table 1, the coarse-stage recall analysis should be added to justify the recursive design, and the hierarchy/LoRA/inference protocol inconsistencies need to be resolved. I would not reject the paper on these grounds, but they are not merely typographical."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this is a genuinely new architecture idea, not a routine VLM extension. The recursive hierarchy with sparse segment tokens, the contrastive 'not present' training, and LLM-internal confidence ranking are real contributions. The headline +2.6% R1@.1 over RGNet on MAD is consistent with Table 1, and the cumulative ablation (Table 2) actually shows each module pulling its weight. VidChapters-7M gains at strict IoU thresholds are also credible. I would send this to a serious referee.\n\nThe soft spots are mostly about measurement and presentation. The biggest is that the coarse stage's recall is never reported. The design makes the top-level selection load-bearing: if the first pass drops the true event, no lower level can recover it. The paper's efficiency claim (57% frames) is only meaningful if those pruned segments are truly irrelevant. Table 3 gives a hint: ReVisionLLM-I with 100% frames beats the default by 2.4 points in R1@.1, and the paper does not decompose that gap into coarse-stage misses versus other effects. That needs to be measured directly.\n\nThere are also several internal inconsistencies that are easy to fix but currently erode trust. The text says ReVisionLLM-I beats RGNet by +6.3% R1@.3, but Table 1 gives 12.7 vs 9.5, a 3.2 point gap. The main text reports ablations with 0–3 hierarchies while the supplement says two hierarchies are used for long videos. The reference [69] for VTimeLLM points to a different paper. And the code availability statement changes from 'is available' in the abstract to 'will release' in the supplement. Nothing fatal, but together they make the paper look rushed.\n\nOne more substantive gap: the comparison tables include only VTimeLLM and CLIP among VLM grounding models. TimeChat, LITA, and Hawkeye are discussed in related work but not benchmarked. Given the abstract claims to be the first VLM to handle hour-long grounding, leaving out the closest VLM baselines weakens that claim.\n\nBottom line: the core idea is solid and likely important for the subfield. The paper needs a revision that reports coarse-stage recall, fixes the numbers and references, and adds the missing VLM comparisons. Whoever reviews it should demand those, but this is not a desk-reject. I'd bring it to reading group.","headline":"Novel recursive coarse-to-fine VLM for hour-long temporal grounding with table-consistent SOTA on MAD, but unmeasured coarse-stage recall and several internal inconsistencies mean the efficiency/accuracy trade-off is not yet pinned down.","tokens_in":20101,"tokens_out":2701,"would_cite":true,"duration_ms":23957,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Recursively zooming from coarse segments to fine boundaries lets a single vision-language model ground events in hour-long videos, beating prior specialized methods on MAD and VidChapters-7M.","keywords":["temporal grounding","long-video understanding","vision-language models","recursive hierarchy","coarse-to-fine localization","confidence calibration","contrastive training","text-to-video retrieval"],"falsifier":"A direct test is to measure, on the MAD test split, how often the ground-truth moment falls inside at least one of the coarse segments selected by the top hierarchy; if that recall is close to 100 percent, the residual error is in refinement, while if it is far below the final R1@0.1, the coarse stage is the bottleneck and the recursive argument needs a better first scan.","tokens_in":19070,"feed_emoji":"🎬","tokens_out":8696,"duration_ms":81248,"temperature":0.7,"pith_summary":"The paper tries to establish that a vision-language model can locate a text-described event inside an hour-long video, something earlier VLMs could not do because they could only consume a small number of frames. Its solution, ReVisionLLM, mimics human search: scan the whole video with compact segment-level features to pick broad regions of interest, then zoom into those regions with frame-level features to predict exact start and end boundaries. If the claim holds, one instruction-following model could replace the multi-network pipelines used by specialized long-video grounding methods, and the same search-then-refine machinery transfers to text-to-video retrieval. The paper reports state-of-the-art results on MAD and VidChapters-7M, including a +2.6 point R1@0.1 gain over the previous best method on MAD.","feed_headline":"Recursive zoom lets one VLM ground events in hour-long video","feed_subtitle":"ReVisionLLM scans broadly, then zooms in, beating prior methods on MAD and VidChapters-7M.","key_machinery":"The load-bearing mechanism is the two-level temporal feature representation produced by the Hierarchical Adapter. Dense features keep every sampled frame's CLS token, while sparse features are produced by cross-attending each long segment to the query text and then condensing it with self-attention into one learnable embedding per segment; the LLM consumes sparse features at the upper hierarchies to select segments and dense features at the bottom hierarchy to refine boundaries. A progressive training schedule first teaches the model to localize events in short clips with contrastive absent-event segments, then freezes the adapter and fine-tunes a second LoRA on hour-long videos using sparse features. At inference, predictions are ranked by the inverse of the mean entropy of the generated words, a calibration step that replaces CLIP-similarity ranking.","core_discovery":"The central claim is that the frame-limit barrier for VLM temporal grounding can be removed by recursive hierarchical perception. Given an hour-long video, the model's top hierarchy uses sparse temporal features, each condensing a long window into one embedding, to identify which broad segments are relevant to the query; lower hierarchies then feed dense, frame-aligned features from only those segments into the language model, which outputs start and end times or \"Not Present.\" The authors show cumulative ablations where each component contributes, with the recursive process delivering the largest gain, and report that the full model outperforms the prior state of the art on both MAD and VidChapters-7M while processing fewer frames than the VTimeLLM baseline.","pith_inferences":["Beyond the paper, the top-level coarse recall is the pipeline's ceiling; measuring it directly would show how much of the remaining error comes from misses at the first scan rather than from boundary refinement.","One extension the authors leave implicit is applying the same hierarchy to dense video captioning or highlight detection, which share the coarse-to-fine search structure and could inherit the frame-efficiency gains.","The inverse-entropy confidence score could serve as a dynamic stopping rule for the recursion, skipping additional levels when the model is already certain."],"forward_implications":["If ReVisionLLM's central claim is correct, a single instruction-following VLM can replace the multi-network, post-processing pipelines used by prior long-video grounding methods.","The hierarchy lets the model adapt to any video length by changing the number of levels, so the same checkpoint handles short clips, half-hour footage, and multi-hour movies.","Because the top levels process sparse segment embeddings, the model sees fewer frames than a full-video baseline and still improves accuracy, suggesting recursive zooming is an efficient scaling strategy for long video.","The contrastive training and entropy-based confidence ranking reduce false positives, and the reported ECE drop indicates the model's confidence scores are more trustworthy for selecting top-k predictions.","The same model, applied to concatenated videos, also solves text-to-video retrieval, indicating the grounding capability transfers to a related retrieval task."],"supporting_citations":[{"why":"VTimeLLM, the VLM temporal-grounding baseline that ReVisionLLM extends, ablates against, and compares with for frame efficiency.","marker":"[18]"},{"why":"RGNet, the previous state-of-the-art method on MAD that ReVisionLLM claims to surpass by +2.6 R1@0.1.","marker":"[15]"},{"why":"CONE, the coarse-to-fine ranking framework used as a baseline and replaced by ReVisionLLM's confidence-based ranking.","marker":"[16]"},{"why":"M-DETR, the strongest prior method on VidChapters-7M that ReVisionLLM claims to beat at high IoU thresholds.","marker":"[26]"},{"why":"MAD dataset, the 1,200-hour movie benchmark with short moments that drives the main experiments.","marker":"[48]"},{"why":"VidChapters-7M dataset, the user-chaptered long-video benchmark used for the second main evaluation.","marker":"[62]"},{"why":"CLIP ViT-L/14 encoder that supplies the frame-level CLS tokens and text features used by the multimodal encoder.","marker":"[60]"},{"why":"Vicuna-7B, the LLM that serves as the temporal grounding decoder across hierarchies.","marker":"[9]"}],"fun_headline_variants":["Recursive zoom lets VLM ground events in hour-long video","First VLM to ground events in hour-long videos","Zoom-revise VLM localizes events in hour-long videos","Recursive attention grounds events in hour-long video","Recursive hierarchy zooms into hour-long video events"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The recursive pipeline depends on the first coarse scan having high recall: if the true event is not among the broad segments passed to the finer levels, no amount of refinement can recover it, and the paper does not report this coarse-stage recall.","fun_headline_variants_meta":{"raw":{"variants":["Recursive zoom lets VLM ground events in hour-long video","First VLM to ground events in hour-long videos","Zoom-revise VLM localizes events in hour-long videos","Recursive attention grounds events in hour-long video","Recursive hierarchy zooms into hour-long video events"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000921,"raw_usage":{"total_tokens":3919,"prompt_tokens":885,"completion_tokens":3034,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":2954}},"tokens_in":501,"tokens_out":3034,"duration_ms":20996,"temperature":1.0,"reasoning_tokens":2954,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:45:05.712120+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to measure, on the MAD test split, how often the ground-truth moment falls inside at least one of the coarse segments selected by the top hierarchy; if that recall is close to 100 percent, the residual error is in refinement, while if it is far below the final R1@0.1, the coarse stage is the bottleneck and the recursive argument needs a better first scan.","supporting_citations":[{"cited_title":"Vtimellm: Empower llm to grasp video moments","cited_arxiv_id":null,"evidence_quote":"VTimeLLM, the VLM temporal-grounding baseline that ReVisionLLM extends, ablates against, and compares with for frame efficiency."},{"cited_title":"RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos","cited_arxiv_id":"2312.06729","evidence_quote":"RGNet, the previous state-of-the-art method on MAD that ReVisionLLM claims to surpass by +2.6 R1@0.1."},{"cited_title":"Detecting mo- ments and highlights in videos via natural language queries","cited_arxiv_id":null,"evidence_quote":"M-DETR, the strongest prior method on VidChapters-7M that ReVisionLLM claims to beat at high IoU thresholds."},{"cited_title":"Vidchapters-7m: Video chapters at scale,","cited_arxiv_id":null,"evidence_quote":"VidChapters-7M dataset, the user-chaptered long-video benchmark used for the second main evaluation."},{"cited_title":"Clip-vip: Adapting pre- trained image-text model to video-language representation alignment","cited_arxiv_id":null,"evidence_quote":"CLIP ViT-L/14 encoder that supplies the frame-level CLS tokens and text features used by the multimodal encoder."},{"cited_title":"Gonzalez, Ion Stoica, and Eric P","cited_arxiv_id":null,"evidence_quote":"Vicuna-7B, the LLM that serves as the temporal grounding decoder across hierarchies."}],"review_version":1}