{"id":"07bb71ba-7b48-4354-8c03-89bed6bf0287","arxiv_id":"2412.11391","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper claims that a video-language model trained with dynamic temporal prompts and temporal contrastive learning beats four published models on three self-defined VidSitu temporal reasoning tasks.","lead":"This paper proposes training video-language models with dynamic temporal prompts, a temporal contrastive loss, and a masked-frame prediction loss, and reports large gains over four published models on three self-defined reasoning tasks built on the VidSitu dataset.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The training objective in Eq. (7) contains only contrastive and masked-embedding losses, yet the method claims to generate temporally aligned text descriptions that are scored in Table III; no language-modeling or generation loss is defined, so the central claim lacks a specified training signal…","rationale":"The reader's weakest assumption precisely identifies the same load-bearing concern: Eq. (7) omits any loss that would train the text generation required by the method and by the human evaluation. My reading of the paper confirms this is not a minor omission but a structural gap. The method section defines only L_contrastive and L_mask, yet the inference section and Table III require the model to predict textual descriptions. Without a language-modeling objective or an explicitly stated frozen decoder, the described training procedure cannot explain the reported gains. This is an internal inconsistency, not merely a disagreement with external consensus. Therefore the central claim is unsupported as written, and rejection is appropriate. I do not base the verdict on citation issues or the self-defined metrics, though those further weaken the paper; the training-objective gap alone is sufficient.","tokens_in":9030,"tokens_out":1090,"duration_ms":12293,"concrete_test":"Inspect the training code or, if unavailable, re-derive the method with a standard video-language model (e.g., Video-LLaVA) using exactly Eq. (7) as the full objective with the paper's stated lambda1 and lambda2, and then evaluate generation on the reported VidSitu tasks. If the model cannot produce the described textual outputs because the text decoder receives no gradient from Eq. (7), the central claim fails as written; if a hidden language-modeling term or frozen-decoder assumption is needed to reproduce Table III, then the paper's stated objective is incomplete and must be corrected before the claimed results can be accepted.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that TSADP significantly outperforms state-of-the-art models on temporal video reasoning, with quantitative gains and human-evaluated text descriptions. For this to be true, the model must be trained to produce the textual descriptions that are scored. But the paper defines the total loss in Section III-D as L = lambda1*L_contrastive + lambda2*L_mask (Eq. 7). L_contrastive (Eq. 5) aligns visual and language embeddings via contrastive learning, and L_mask (Eq. 6) is an L2 regression on masked visual embedding predictions. Neither term is a language-modeling loss (e.g., cross-entropy over text tokens), and no such term appears anywhere in the method. Section III-E states that during inference the DPG 'informs the language generation process,' but the paper never specifies how the text decoder is trained or whether it is a frozen pretrained component. If Eq. (7) is the complete objective, gradients from the scored text outputs do not flow into any text-generation parameter, so either the objective is incomplete or the method silently relies on an undeclared pretrained decoder. This is a load-bearing internal inconsistency because Table III and the claimed human-evaluation improvements depend on generated descriptions that Eq. (7) cannot train. The paper provides no code, data, or detailed training configuration to resolve which of these two readings is correct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Temporal Semantic Alignment via Dynamic Prompting (TSADP), a framework for video temporal reasoning that combines a Dynamic Prompt Generator (DPG) with a Temporal Contrastive Loss (TCL) and a Masked Temporal Prediction (MTP) objective. The method is evaluated on a self-described enriched version of VidSitu through three author-defined tasks (IVEA, TRU, CP) and a human evaluation of generated temporal descriptions. The paper claims state-of-the-art accuracy on these tasks and superior human ratings for coherence, temporal alignment, and semantic accuracy.","tokens_in":9239,"tokens_out":7245,"duration_ms":62993,"significance":"If the framework were correctly specified and the evaluations were grounded in public benchmarks and data, the idea of using dynamic task-specific prompts and temporal contrastive learning to improve temporal reasoning in LVLMs would be interesting and potentially useful. The paper also makes a reasonable attempt at ablations (Table I) and at reporting human judgments. However, as it stands, the method's training objective (Eq. 7) contains no term that supervises the generated textual descriptions that are scored in Table III, and the evaluation is built on author-defined tasks and an unreleased dataset extension. These issues are load-bearing, so the paper cannot currently support its central claim of state-of-the-art temporal reasoning. The strengths of the manuscript are the clarity of the three-component idea and the inclusion of ablations; the weaknesses are the missing training signal for text generation and the lack of reproducible evaluation.","major_comments":[{"comment":"The total training objective in Eq. (7) is L = λ1L_contrastive + λ2L_mask; neither term is a language-modeling objective over generated text. Section III-E nevertheless states that the DPG prompts 'inform the language generation process' and Table III scores free-form temporal descriptions produced by the model. The paper never defines a text-decoder loss, a cross-entropy term, or a frozen pretrained decoder; without one, the gradients from the scored descriptions do not train any parameter that affects the text, and the human-evaluation improvements in Table III have no specified training signal. This is load-bearing: all reported text-generation gains depend on a mechanism that Eq. (7) cannot provide. The authors should either add the missing generation loss to Eq. (7) or explicitly state that the decoder is frozen and explain how the DPG is trained to influence it through the contrastive and masked losses alone.","section":"Section III-D, Eq. (7), and Section III-E"},{"comment":"IVEA, TRU, and CP are introduced in this paper, and the labels come from an 'enriched' version of VidSitu that is not released and whose annotation procedure is not described. The manuscript does not specify train/validation splits, number of videos or instances per task, evaluation prompts, or the protocol used to produce CLIP/CLIP-ViP/Video-LLaVA/EVA-CLIP predictions on these tasks. As a result, the absolute numbers in Tables I and II cannot be reproduced or compared against the literature, and the abstract's 'significant improvements over state-of-the-art models' is not verifiable. A public benchmark, a released annotation extension, or a detailed task description with full evaluation protocols is required.","section":"Section IV-A, Tables I and II"},{"comment":"The human evaluation reports scores for CLIP, CLIP-ViP, and EVA-CLIP, which are not trained to generate free-form textual descriptions; no procedure is described for generating descriptions from these models, nor are the prompts, sampling parameters, annotator instructions, number of annotations, or inter-annotator agreement reported. Under these conditions, the 4.7/4.5/4.6 scores cannot be interpreted as evidence that TSADP generates better temporal narratives than the baselines.","section":"Section IV-D, Table III"},{"comment":"The MTP loss is written as ∑_{t∈Masked} ||ẑ_t − z_t||₂², but z_t is never defined: it is not the input frame feature v_t (which has been masked) and no teacher or ground-truth embedding source is described. The selection rule for the masked subset is also left unspecified. Since Eq. (7) includes L_mask with a fixed weighting, the objective is not implementable as stated.","section":"Section III-C, Eq. (6)"},{"comment":"The dynamic prompt P_t is said to be 'constructed by integrating v̂_t into the model's language generation process', but no equation specifies how P_t conditions the text decoder, how the DPG's prompt tokens are concatenated or added, or how the visual embedding z^v_t in Eq. (5) is obtained from v_t and the DPG output. Together with the first major comment, this leaves the coupling between DPG/TCL and the generated text underspecified to the point that the method cannot be reimplemented from the paper.","section":"Section III-A, Eq. (1)"}],"minor_comments":[{"comment":"Reference [22] (Hermes et al., 'Electrophysiological responses in the ventral temporal cortex during reading of numerals and calculation') is unrelated to the claimed long-form video understanding method TESTA; this appears to be a citation error.","section":"References, [22]"},{"comment":"Equation (5) sums the denominator over all t′, including the anchor t; standard contrastive losses exclude the positive from the negative set. The paper should state the negative sampling strategy and the value of τ.","section":"Section III-B, Eq. (5)"},{"comment":"The 'Average Score' column in Table II is not defined; because CP MAE is a lower-is-better error, a raw arithmetic average of accuracy and MAE is not a meaningful aggregate. The authors should specify a normalization or remove the column.","section":"Section IV-B, Table II"},{"comment":"The text states that TSADP 'significantly outperforms' baselines, but no significance tests, confidence intervals, or standard deviations are reported for any of the quantitative results.","section":"Section IV-B and IV-E"}],"recommendation":"reject","confidential_remarks":"The manuscript is not ready for publication in its current form. The missing text-generation loss and the unreleased evaluation data are blockers. If the authors can provide code, data, and a corrected or clarified training objective, a resubmission may be worth considering."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know before reading: the reported gains are not anchored to a training signal that can produce the text they score. Section III-D defines the total loss as L = λ1·L_contrastive + λ2·L_mask (Eq. 7). L_contrastive aligns visual and language embeddings; L_mask is L2 regression on masked visual embeddings. Neither term trains a text decoder, and no cross-entropy or generation loss appears anywhere in the paper. Section III-E says the DPG prompts 'inform the language generation process,' but the decoder is never specified as trainable or frozen. That is a load-bearing gap: Table III and the human preference scores depend on generated descriptions that Eq. (7) cannot train. It is not a stylistic quibble; the method as written has no path from loss to the evaluated output.\n\nWhat the paper does well is narrow but real: the assembly of dynamic prompting via windowed self-attention (Eqs. 1–3), InfoNCE-style temporal contrast (Eq. 5), and masked prediction (Eq. 6) is clearly described. The ablation in Table I is consistent with the authors' intuition that both DPG and TCL matter. The writing is mostly readable, and the math is explicit enough to check.\n\nThe soft spots extend beyond the missing loss. The evaluation tasks (IVEA, TRU, CP) are author-defined on an unpublished 'enriched' VidSitu; no code, data, or training configuration is released; no error bars, run counts, or significance tests appear; the 'Average Score' column in Table II matches no obvious formula. The reference list includes an unrelated neuroscience paper (Ref. [22], Hermes et al., Cerebral Cortex 2017) cited for long-range video memory, and the EVA-CLIP baseline is never cited. These are mechanical signs of a rushed draft, but they are secondary to the missing generation loss.\n\nMy honest read: this is not a paper with a fixable flaw. The core claim—TSADP significantly improves temporal text generation—requires a training objective that is not present in the manuscript. A serious referee could not verify the numbers without code, and the method as written cannot generate the evaluated descriptions. I would desk reject, and I would not bring it to reading group or cite it.","headline":"The paper's central claim collapses on its own training objective: Eq. (7) has no language-modeling loss, yet the method is scored on generated text descriptions.","tokens_in":682,"tokens_out":829,"would_cite":false,"duration_ms":19234,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TSADP claims dynamic prompts and temporal contrastive learning give video-language models fine-grained temporal reasoning, beating four baselines on VidSitu.","keywords":["temporal reasoning","video-language understanding","contrastive learning","dynamic prompting","large vision-language models","VidSitu","masked temporal prediction"],"falsifier":"Inspect or reproduce the training loop: if the only loss terms are $\\lambda_1 \\mathcal{L}_{\\text{contrastive}} + \\lambda_2 \\mathcal{L}_{\\text{mask}}$, and no gradient flows from the generated text to the decoder, then the described procedure cannot learn to produce temporally coherent captions. A minimal experiment is to train with Eq. (7) exactly and measure whether task accuracy and caption quality improve at all; if the caption outputs do not change or deteriorate while the contrastive and masked losses improve, the central claim fails.","tokens_in":8650,"feed_emoji":"⏱️","tokens_out":5619,"duration_ms":51027,"temperature":0.7,"pith_summary":"The paper proposes a training recipe, Temporal Semantic Alignment via Dynamic Prompting (TSADP), for making vision-language models reason about time. The recipe has three pieces: a Dynamic Prompt Generator (DPG) that builds per-frame prompts from a sliding temporal window, a Temporal Contrastive Loss (TCL) that pulls visual and textual embeddings of the same moment together and pushes different moments apart, and a Masked Temporal Prediction (MTP) objective that makes the model robust to missing frames. The authors evaluate TSADP on an augmented VidSitu benchmark and report that it outperforms CLIP, CLIP-ViP, Video-LLaVA, and EVA-CLIP on Intra-Video Entity Association, Temporal Relationship Understanding, and Chronology Prediction, and that human annotators rate its descriptions higher on coherence, temporal alignment, and semantic accuracy. The point of the paper is that temporal reasoning can be taught to existing large vision-language models with these lightweight components rather than with heavy temporal attention or recurrent architectures.","feed_headline":"Temporal contrastive training beats four video baselines","feed_subtitle":"The proposed TSADP reports top accuracy on three VidSitu temporal-reasoning tasks plus higher human ratings.","key_machinery":"The machinery is a combination of three trainable pieces. The DPG is a multi-head self-attention module over a temporal window of $2k+1$ frames around each frame $v_t$, producing a prompt $P_t = \\text{DPG}(v_t, \\{v_{t-k},\\dots,v_t,\\dots,v_{t+k}\\})$ that is injected into the language generation process. TCL is a cross-modal softmax contrastive loss over projected visual and textual embeddings, so that same-time embeddings have high cosine similarity and different-time embeddings are pushed apart. MTP masks some frame features and predicts their embeddings from context with an $\\ell^2$ loss. The final training objective is $L = \\lambda_1 \\mathcal{L}_{\\text{contrastive}} + \\lambda_2 \\mathcal{L}_{\\text{mask}}$, and the argument hinges on the interaction of these terms: the ablation study shows that removing DPG drops IVEA from 85.7% to 78.4%, and removing TCL degrades TRU and chronology MAE, which is presented as evidence that each component carries part of the temporal reasoning ability.","core_discovery":"The central claim is that a model trained with the joint objective $L = \\lambda_1 \\mathcal{L}_{\\text{contrastive}} + \\lambda_2 \\mathcal{L}_{\\text{mask}}$ learns temporal semantics that transfer to downstream video-language tasks. Concretely, TSADP reaches 85.7% accuracy on Intra-Video Entity Association, 78.9% on Temporal Relationship Understanding, and 2.8 frames mean absolute error on Chronology Prediction, beating the best baseline EVA-CLIP by 6.1 points, 6.1 points, and 0.6 frames respectively, and the human evaluation gives TSADP 4.7, 4.5, and 4.6 out of 5 for coherence, temporal alignment, and semantic accuracy. The author would state it as: dynamic task-specific prompts plus time-contrastive alignment is a sufficient and efficient way to give large vision-language models fine-grained temporal understanding.","pith_inferences":["A natural reading is that a language-modeling term, such as cross-entropy on output caption tokens, is missing from Eq. (7); adding it explicitly would make the framework reproducible and is the most direct way to test the reported gains.","TCL could plausibly be applied as a regularizer on top of existing video-LLMs without the DPG, since the contrastive term only needs frame-level embeddings, so a cheap experiment could isolate where the improvement actually comes from.","The augmented VidSitu annotations and task splits are not presented as a released public benchmark, so publishing them with exact prompts would let other groups verify the claimed margins and the human-evaluation protocol."],"forward_implications":["If the recipe is correct, the same DPG plus TCL plus MTP combination should transfer to other video-language backbones beyond the four tested, giving a generic plug-in for temporal reasoning.","The three proposed tasks and the augmented VidSitu annotations would become a practical evaluation suite for fine-grained temporal understanding in future video-LLM training.","Temporal understanding would not require heavy recurrent layers or excessive temporal attention; the paper reports runtime per frame comparable to CLIP-ViP and faster than Video-LLaVA, which matters for real-time video analytics and autonomous systems.","Masked temporal prediction would make models robust to occluded or skipped frames, a property that carries over to noisy video streams in deployed settings."],"supporting_citations":[{"why":"Supplies the VidSitu dataset with semantic role annotations that the authors extend with temporal event sequences and use for all three evaluation tasks.","marker":"[2]"},{"why":"Provides the CLIP baseline whose static performance anchors the lower bound of the reported comparisons.","marker":"[3]"},{"why":"Provides the CLIP-ViP temporal baseline that TSADP is compared against on all three tasks.","marker":"[4]"},{"why":"Provides the Video-LLaVA video-LLM baseline used for both automatic metrics and human evaluation.","marker":"[5]"}],"fun_headline_variants":["TSADP: dynamic prompts plus contrastive training for video reasoning","Temporal contrastive learning lifts LVLM video reasoning by 6 points","Video AI gets better at timing: TSADP beats four baselines","85.7% entity association: TSADP improves temporal reasoning","Coherent video descriptions: TSADP tops human ratings"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the objective in Eq. (7), which contains only contrastive and masked-prediction losses, actually trains the textual descriptions that the three tasks and the human evaluation score; the paper never specifies a language-modeling or generation loss, so if the model's text outputs are not supervised, the reported gains and human scores have no training signal to come from.","fun_headline_variants_meta":{"raw":{"variants":["TSADP: dynamic prompts plus contrastive training for video reasoning","Temporal contrastive learning lifts LVLM video reasoning by 6 points","Video AI gets better at timing: TSADP beats four baselines","85.7% entity association: TSADP improves temporal reasoning","Coherent video descriptions: TSADP tops human ratings"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1467,"prompt_tokens":937,"completion_tokens":530,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":439}},"tokens_in":553,"tokens_out":530,"duration_ms":5772,"temperature":1.0,"reasoning_tokens":439,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:00:12.033849+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect or reproduce the training loop: if the only loss terms are $\\lambda_1 \\mathcal{L}_{\\text{contrastive}} + \\lambda_2 \\mathcal{L}_{\\text{mask}}$, and no gradient flows from the generated text to the decoder, then the described procedure cannot learn to produce temporally coherent captions. A minimal experiment is to train with Eq. (7) exactly and measure whether task accuracy and caption quality improve at all; if the caption outputs do not change or deteriorate while the contrastive and masked losses improve, the central claim fails.","supporting_citations":[{"cited_title":"Visual semantic role labeling for video understanding,","cited_arxiv_id":null,"evidence_quote":"Supplies the VidSitu dataset with semantic role annotations that the authors extend with temporal event sequences and use for all three evaluation tasks."},{"cited_title":"Video-llava: Learning united visual representation by al ignment before projection,","cited_arxiv_id":null,"evidence_quote":"Provides the Video-LLaVA video-LLM baseline used for both automatic metrics and human evaluation."}],"review_version":1}