{"id":"a8b066b6-ee5c-420f-bff2-3aae3e890c2e","arxiv_id":"2411.10332","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Number-Prompt overlays frame numbers on video frames, improving temporal grounding in video LLMs and setting new state-of-the-art results on moment retrieval and highlight detection.","lead":"This paper shows that overlaying a unique number on each video frame makes video-language AI models much better at answering 'when does X happen' questions. The authors report gains on standard moment retrieval and highlight detection benchmarks, both with no training and after fine-tuning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The training-free gain may be driven by the added textual instruction, not the number overlay; the missing instruction-only control is the key unaddressed confound.","rationale":"The paper is well-executed in many respects: the CLIP-based design study in Section 3.3 is a principled way to choose overlay properties, Table 2 includes the important +FT baseline showing NumPro-FT outperforms conventional fine-tuning, the sampling analysis in Section 4.4 shows performance scales with labeled frames, and the attention analysis in Section 3.1 provides independent evidence that Vid-LLMs localize relevant content but fail to verbalize timestamps. These are real supporting data. However, the central training-free claim is that overlaying frame numbers is what enables temporal grounding, and the paper's protocol confounds the overlay with a textual instruction that is never tested alone. This is exactly the reader's weakest assumption, and it is not resolved by the design ablations, which only compare overlay variants under a fixed instruction. The missing control is cheap to run and directly decides the attribution. If the instruction alone explains the gain, the method is still a valid prompt-engineering trick, but the mechanism and novelty shift substantially. Therefore the reader's CONDITIONAL verdict is appropriate, and no further adjustment is needed beyond requiring the control experiment and the other requested clarifications (main-table +FT baseline, train/test exclusion, code release).","tokens_in":24478,"tokens_out":2808,"duration_ms":28198,"concrete_test":"Run Qwen2-VL-7B on Charades-STA and ActivityNet under four conditions: (A) baseline prompt as in the paper (no instruction, no overlay); (B) baseline prompt plus the instruction 'The red numbers on each frame represent the frame number' with no numbers overlaid; (C) NumPro overlay without the instruction; (D) NumPro overlay with the instruction (the paper's current setting). Compare mIoU and R@0.5 across conditions. If B approximately equals D, the instruction is the active ingredient; if C approximately equals D, the overlay suffices; if D substantially exceeds both B and C, the interaction is what matters. Reporting all four conditions would settle whether the overlay is causally responsible for the training-free gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 states that in the training-free setting, 'we prepend a simple instruction to each event query: The red numbers on each frame represent the frame number.' The NumPro condition therefore differs from the baseline in two ways: the overlay and the instruction. The paper never runs an instruction-only control (instruction on unnumbered frames), so the central causal claim that the number overlay itself is responsible for the large training-free gains (e.g., Qwen2-VL-7B mIoU rising from 7.9 to 38.5 on Charades-STA) is not isolated. The design ablations in Table 3 vary overlay properties while presumably holding the instruction fixed, showing that overlay details matter, but they do not quantify how much of the improvement comes from the instruction alone versus the overlay. If the instruction alone yields most of the gain, the 'flipping manga' mechanism is not supported, even though the combined method might still improve performance. This is the most load-bearing concern because the abstract's headline claim attributes the boost specifically to NumPro's number overlay, and the missing control leaves the attribution ambiguous.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Number-Prompt (NumPro), a visual-prompting method that overlays unique frame numbers onto video frames. In the training-free variant, the frame numbers are read by Vid-LLMs via OCR after a textual instruction is prepended; in the fine-tuned variant, a LoRA-tuned LLM is trained on a NumPro-augmented instruction dataset (NumPro-FT). The authors report large improvements on Charades-STA, ActivityNet, and QVHighlights across GPT-4o, Qwen2-VL-7B/72B, LLaVA-Video, LLaVA-OneVision, and LongVA, and claim new state-of-the-art results for moment retrieval and highlight detection. They also contribute a CLIP-based design analysis for font size, color, and position, an attention analysis of why Vid-LLMs struggle with temporal grounding, and ablations on sampling ratio and sampling strategy.","tokens_in":24735,"tokens_out":6607,"duration_ms":66376,"significance":"If the causal attribution holds, this is an elegant and potentially widely applicable idea: it converts timestamp prediction into a visual reading task, requiring no architectural changes, no new tokens, and no changes to the model vocabulary. The paper's strengths include the breadth of models and benchmarks tested, the transferable CLIP-based design methodology, the attention analysis in Section 3.1 and Appendix 8, and the planned code release. However, the central causal claim that the number overlay itself, rather than the accompanying textual instruction, produces the gains is not yet isolated, and the fine-tuning SOTA claim needs clarification about possible train/eval overlap. These issues are fixable but currently leave the headline conclusions under-supported.","major_comments":[{"comment":"The training-free comparison conflates two interventions. Section 3.2 states that with NumPro the authors prepend the instruction 'The red numbers on each frame represent the frame number.' The baseline receives neither the overlay nor this instruction, so the observed gains (Qwen2-VL-7B mIoU on Charades-STA from 7.9 to 38.5 in Table 1; LLaVA-Video-7B from 9.8 to 34.6 in Table 2) could be partly or wholly attributable to the instruction alone. An instruction-only control, that is, the same prompt applied to unnumbered frames, must be reported. Table 3's design ablations hold the instruction fixed and show that overlay properties matter, but they do not quantify the instruction's contribution. Without this control, the abstract's claim that the number overlay 'significantly boosts VTG performance' is not established.","section":"Section 3.2, Tables 1-2"},{"comment":"The NumPro-FT training set is built from DiDeMo and ActivityNet Caption data, while the headline moment-retrieval evaluation is on ActivityNet. The paper never states whether the ActivityNet evaluation videos were excluded from the 220k training samples. If the same video instances, or their temporal annotations, appear in both training and evaluation, the SOTA gains in Table 1 would be inflated. Please clarify the exact train/eval split, and if necessary retrain without the evaluation split and report the comparison. This is essential for the fine-tuning contribution, whose purpose is to show that number-augmented training data adds value beyond conventional fine-tuning.","section":"Section 4.1, Section 4.2, Tables 1-2"}],"minor_comments":[{"comment":"The '+FT' (fine-tuning without NumPro) baseline appears in Table 2 but is omitted from Table 1. Since Table 2 shows that +FT already reaches 40.2 mIoU on Charades-STA, the contribution of the number overlay in the fine-tuned setting should be made more visible in the main comparison table.","section":"Tables 1-2"},{"comment":"Please clarify whether the instruction 'The red numbers on each frame represent the frame number' is also used in the fine-tuning stage and in the '+FT' baseline. If the instruction differs between +FT and +NumPro-FT, that difference is an additional confound in the fine-tuning comparison.","section":"Section 3.2, Figure 3"},{"comment":"The tables report single runs without error bars or seed information. Given the very large reported gains for some models, such as Qwen2-VL-72B moving from near zero to 17.4 mIoU, reporting variance across at least three seeds would substantially strengthen the results.","section":"Tables 1-3 and Appendix 10"},{"comment":"The comparison between timestamp overlays and frame-number overlays is difficult to parse because the table rows are not labeled. Please specify which rows correspond to which overlay format and ensure the units (seconds, minutes, or frame indices) are stated in the table header.","section":"Appendix 12, Table 9"},{"comment":"There are several presentation issues: references [8] and [9] appear to be the same paper; 'Tabel 9' should be 'Table 9'; and the model names are inconsistently spaced (e.g., 'LLaV A-Video'). These should be corrected.","section":"References and notation"},{"comment":"The phrase 'without additional computational cost' should be qualified: the method requires the user to pre-render frame-number overlays and to treat the overlaid video as the model input. That is an input-modality assumption rather than a strictly cost-free zero-shot capability, and the paper should state this clearly.","section":"Abstract, Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the experiments are unusually broad, and I do not see evidence that the reported numbers are fabricated. However, the missing instruction-only control is a standard confound that must be resolved before the central attribution claim can be accepted, and the ActivityNet train/eval overlap question is important for the SOTA claim. Both are fixable with additional experiments or clarifications. If the authors provide the instruction-only control, clarify the split, and address the minor presentation issues, I would be willing to accept a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read NumPro with interest. The core idea is genuinely simple and, as far as the cited literature goes, new: overlay frame numbers onto video frames and let Vid-LLMs OCR their way to temporal boundaries. The paper does a good job showing the idea transfers across several models (GPT-4o, Qwen2-VL-7B/72B, LLaVA-Video, LongVA) and works in both training-free and fine-tuned settings. The design ablation using CLIP on COCO is a reasonable heuristic and the follow-up validation on Charades-STA matches it. The attention analysis is a nice concrete illustration of the failure mode: models attend to the right frames but can't verbalize the time.\n\nThe main soft spot is exactly what your stress-test flags. In the training-free setup, the authors prepend 'The red numbers on each frame represent the frame number' to the query. That means the NumPro condition differs from the baseline in two ways: the overlay and the instruction. Without an instruction-only control (instruction on unnumbered frames), the causal claim that the numbers themselves do the work isn't fully isolated. Table 3 shows that varying overlay properties changes performance, which argues the overlay matters, but it doesn't tell us how much of the gain comes from the instruction alone. I'd want that control before trusting the training-free numbers as evidence for the 'flipping manga' mechanism.\n\nThe fine-tuned results have their own caveats. The +FT baseline (fine-tuning without NumPro) appears in Table 2 but not in Table 1, which makes the headline SOTA claims harder to interpret. And the fine-tuning data includes ActivityNet Caption, which overlaps with the ActivityNet evaluation set; the paper doesn't state an exclusion split. That could inflate the fine-tuned ActivityNet numbers. QVHighlights and Charades-STA are cleaner. No error bars or seed variance is reported, which is a bit sloppy for gains this large.\n\nNone of this kills the idea. The overlay is cheap, transferable, and the broad consistent improvements across models and datasets are suggestive. But a serious referee should ask for the instruction-only control, the missing baseline in the main table, and clarification on split overlap. The code promised will help.\n\nWho is this for? Anyone working on VTG with Vid-LLMs will find it useful. It's a plausible 'free lunch' that could become a standard pre-processing step. I'd send it to review; the central claim is clear enough that peer review can sort out the confound.","headline":"A simple, plausible trick for VTG that deserves a careful referee; the missing instruction-only control is the one thing I'd ask for before trusting the training-free numbers.","tokens_in":25246,"tokens_out":3463,"would_cite":true,"duration_ms":28759,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Printing each frame's index onto the video lets Vid-LLMs read off event timestamps instead of hallucinating them, with no retraining.","keywords":["video temporal grounding","frame-number overlay","visual prompting","video large language models","moment retrieval","highlight detection","OCR","training-free"],"falsifier":"Run the Charades-STA moment-retrieval queries through Qwen2-VL-7B with the NumPro prompt sentence but with the frame numbers omitted or rendered invisible (for example, red digits on a red patch the model classes as background). If mIoU stays near the reported 38.5 instead of falling back toward the 7.9 baseline, the overlay itself is not the operative cause and the OCR-reading explanation would need to be revised.","tokens_in":24321,"feed_emoji":"🔢","tokens_out":10088,"duration_ms":82128,"temperature":0.7,"pith_summary":"Video-language models (Vid-LLMs) can say what happens in a video but not when: asked for an event's timing, they hallucinate frame ranges that are impossible for the actual footage. This paper claims the bottleneck is not visual recognition but verbalization — attention maps show the model already fixes its gaze on the right frames — and that a simple visual prompt closes the gap. NumPro prints each frame's index onto the frame itself, so the model's built-in optical character recognition can read the timeline like numbered manga panels and answer with exact frame numbers. The paper shows this overlay lifts temporal-grounding accuracy across several top Vid-LLMs with no training, and that fine-tuning on number-augmented data establishes a new state of the art on moment retrieval and highlight detection. A sympathetic reader would care because the fix is a two-line preprocessing step rather than a new architecture.","feed_headline":"Frame numbers let video AIs say exactly when events occur","feed_subtitle":"A zero-training overlay lifts moment retrieval and highlight detection on top video LLMs by up to 8.5 points.","key_machinery":"The load-bearing object is the numbered frame: each input image carries its index printed in red at the bottom right, making the video's timeline part of the visual signal the model already processes. The mechanism that makes it work is OCR — Vid-LLMs are trained on image-text data and can read printed digits, so the frame index is recovered by perception rather than synthesized from a learned temporal mapping. Three supporting pieces carry the argument: an attention-map analysis (Qwen2-VL attends to the correct frames yet emits impossible timestamps) that identifies verbalization, not localization, as the failure; a CLIP-based design search over font size, color, and position that balances Number Accuracy against Caption Accuracy on MSCOCO images; and a fine-tuning stage (NumPro-FT) that freezes the visual encoder, applies LoRA to the LLM, and trains on a number-augmented instruction dataset so the model learns to emit answers in the form 'From x to y' in frame units.","core_discovery":"The paper's central claim is that numerical identifiers overlaid on video frames convert Video Temporal Grounding from a generation problem into a reading problem. Qwen2-VL's attention maps already point at the query-relevant frames; what fails is translating that attention into textual start and end times, so the model emits boundaries like 'from 000 to 580' for a ten-frame video. NumPro adds one red frame number (font size 40, bottom-right, selected by a CLIP-based trade-off between number readability and caption preservation) to every frame and instructs the model that the red numbers are frame numbers. The model then reports boundaries directly from the visual input — answers such as 'from 3 to 12' — and in the training-free setting Qwen2-VL-7B's mIoU rises from 7.9 to 38.5 on Charades-STA. Fine-tuning LongVA-7B-DPO with LoRA on a 220k-sample number-augmented instruction dataset yields new state-of-the-art results, surpassing prior VTG-tuned methods by up to 6.9% in mIoU for moment retrieval and 8.5% in mAP for highlight detection. The paper concludes that the overlay, not model surgery, is the operative intervention: it works across open- and closed-source models, transfers to highlight detection and general video QA with minimal side effects, and needs no additional computational cost at inference.","pith_inferences":["A control the paper does not report — giving the same instruction sentence ('The red numbers on each frame represent the frame number') on frames with no numbers, or with numbers printed invisibly — would separate the overlay's contribution from the instruction's. If the instruction alone recovers much of the gain, the 'OCR reads the timeline' story is only part of the explanation.","If the OCR mechanism is really what carries the improvement, gains should track the model's text-reading ability: tiny fonts, low contrast, or non-digit glyphs should erode the effect, and models with weak OCR should benefit less. The paper's ablations touch readability but do not test this gradient directly.","The recipe is general: any quantity a video model knows implicitly but cannot verbalize — shot indices, saliency scores, object tracks — could be overlaid rather than injected into the token vocabulary, turning other 'cannot say when or where' failures into reading tasks."],"forward_implications":["Any current Vid-LLM with basic OCR can gain temporal grounding as a training-free upgrade: moment retrieval and highlight detection improve without new parameters or extra inference cost.","The gains transfer across model families — GPT-4o, Qwen2-VL-7B/72B, LLaVA-Video, LLaVA-OneVision — and across benchmarks (Charades-STA, ActivityNet, QVHighlights).","Fine-tuning on a number-augmented dataset (NumPro-FT) sets a new state of the art, surpassing prior VTG-tuned methods by up to 6.9% in mIoU for moment retrieval and 8.5% in mAP for highlight detection.","Numbering only a fraction of frames still helps: uniformly overlaying numbers on 80% of frames already beats previous state-of-the-art on moment retrieval.","General video QA is largely preserved (VideoInstruct, MVBench, VideoMME), with temporal-understanding sub-scores improving slightly, so the overlay can be left on in mixed conversational video systems."],"supporting_citations":[{"why":"Qwen2-VL is the case-study model whose hallucinated timestamps motivate NumPro, and the main model for the training-free evaluation and ablations.","marker":"[66]"},{"why":"Charades-STA is the primary moment-retrieval benchmark where the design ablations and most performance gains are measured.","marker":"[18]"},{"why":"ActivityNet is the second moment-retrieval benchmark and a source of the NumPro-FT instruction data.","marker":"[4]"},{"why":"QVHighlights is the benchmark for highlight detection, providing the mAP and HIT@1 metrics used in the main results.","marker":"[33]"},{"why":"VTimeLLM is a prior VTG-tuned baseline that NumPro-FT must beat, and its dataset stages contribute training samples.","marker":"[25]"},{"why":"TimeChat is a prior state-of-the-art baseline compared in the main tables and qualitative cases.","marker":"[58]"},{"why":"CLIP is the encoder used in the COCO-based design search that selects font size, color, and position for the number overlay.","marker":"[57]"},{"why":"ViP-LLaVA supplies evidence that multimodal LLMs respond to visual prompts such as circles and tags, motivating the numbered overlay as a prompt.","marker":"[5]"},{"why":"LongVA-7B-DPO is the base model fine-tuned in NumPro-FT and the model that produces the reported state-of-the-art results.","marker":"[87]"}],"fun_headline_variants":["Red frame numbers let video AI pinpoint events","NumPro flips video grounding into a manga read","Frame IDs make video LLMs accurate timekeepers","Zero-cost frame numbers sharpen video event timing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the improvement comes from the visible numbers, because every numbered-frame condition also adds the sentence 'The red numbers on each frame represent the frame number' that the unnumbered baseline never receives, and no control runs that sentence with unnumbered frames — so part or all of the measured gain could come from the instruction alone.","fun_headline_variants_meta":{"raw":{"variants":["Red frame numbers let video AI pinpoint events","NumPro flips video grounding into a manga read","Frame IDs make video LLMs accurate timekeepers","Zero-cost frame numbers sharpen video event timing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000321,"raw_usage":{"total_tokens":1859,"prompt_tokens":1048,"completion_tokens":811,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":664,"completion_tokens_details":{"reasoning_tokens":753}},"tokens_in":664,"tokens_out":811,"duration_ms":8132,"temperature":1.0,"reasoning_tokens":753,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:44:19.154464+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Charades-STA moment-retrieval queries through Qwen2-VL-7B with the NumPro prompt sentence but with the frame numbers omitted or rendered invisible (for example, red digits on a red patch the model classes as background). If mIoU stays near the reported 38.5 instead of falling back toward the 7.9 baseline, the overlay itself is not the operative cause and the OCR-reading explanation would need to be revised.","supporting_citations":[{"cited_title":"Timechat: A time-sensitive multimodal large lan- guage model for long video understanding","cited_arxiv_id":null,"evidence_quote":"TimeChat is a prior state-of-the-art baseline compared in the main tables and qualitative cases."}],"review_version":1}