{"id":"be036550-e5dc-41be-93dc-862b7cbacc63","arxiv_id":"2504.17365","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"TimeSoccer introduces an end-to-end multimodal language model that jointly predicts event timestamps and generates soccer commentary for full matches, reporting state-of-the-art results on the SoccerNet-Caption benchmark.","lead":"TimeSoccer is a new AI system that watches full 45-minute soccer matches and writes commentary, predicting when key events happen and describing them in a single step. It claims to outperform prior systems using motion-aware frame selection and progressive training on long videos.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparison is confounded: TimeSoccer is evaluated on MatchTime's refined timestamps, while SoccerNet-Caption's Table 1 numbers are not shown to use the same labels and protocol; the SoTA claim depends on an apples-to-apples comparison that the paper does not establish.","rationale":"The reader's REJECT verdict is well supported. I identify a more fundamental protocol concern than the reader's weakest_assumption: the paper changes the timestamp annotation labels for its own training and evaluation, but does not show that the main baseline, SoccerNet-Caption, was scored under the same labels. If the baseline numbers are taken from the original protocol, the headline temporal and caption gains could be artifacts of label refinement rather than model quality. This is prior to the question of whether Qwen2.5-VL-judge scores are valid, since the entire comparison table is suspect if the evaluation conditions differ. The reader's LLM-judge concern is a serious secondary issue, and the ablation-table contradictions noted by the reader further reduce confidence. No code, data, or evaluation scripts are provided, so the protocol issue cannot be resolved without rerunning the baseline. The verdict therefore remains REJECT.","tokens_in":13973,"tokens_out":10406,"duration_ms":98294,"concrete_test":"Re-run SoccerNet-Caption's published inference on the exact 49 test videos and MatchTime-refined timestamp labels used for TimeSoccer, with the same 5-second extension and IoU evaluation code. If SN-Caption's P@0.3 or CIDEr under the shared protocol differs materially from 12.5 or 11.0, respectively, then Table 1 is not an apples-to-apples comparison and the claimed SoTA gains are not established. Report the recomputed numbers in a revised table.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that TimeSoccer beats SoccerNet-Caption (SN-Caption) on full-match SDVC requires both methods to be scored on the same videos, timestamp annotations, and evaluation code. Section 4.2 states that the authors 'replace the original timestamp annotations with the refined temporal labels provided by MatchTime' and fine-tune/evaluate their model on that version. Yet Table 1 reports SN-Caption's P@0.3/0.5/0.7/0.9 (12.5/7.2/3.0/1.1), F1 8.9, and CIDEr 11.0 without any statement that SN-Caption was re-run on the same refined labels and 49-video split. The text explicitly claims temporal gains of '+4.5, +3.8, +3.0, and +2.3' over SN-Caption; if those baseline numbers come from the original annotation protocol while TimeSoccer's come from MatchTime's refined labels, the comparison is not valid and the superiority could be an artifact of label refinement. The same confound affects the caption metrics, because CIDEr and SODA_c are computed against the same ground-truth timestamps and captions. A correct comparison protocol is a minimal condition for the strongest claim, and the paper does not document it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TimeSoccer, an end-to-end multimodal large language model for Single-anchor Dense Video Captioning (SDVC) over full 45-minute soccer matches. The model jointly predicts timestamps and generates commentary in a single pass, building on TimeChat and introducing a training-free motion-aware frame compression module (MoFA-Select) together with progressive training and position-embedding extrapolation. Experiments on the SoccerNet-Caption dataset with MatchTime-refined temporal labels report temporal grounding metrics, CIDEr/METEOR/SODA_c, and LLM-judge scores (M-S and C-S), and the paper claims state-of-the-art performance in the end-to-end setting.","tokens_in":14255,"tokens_out":6776,"duration_ms":59622,"significance":"If the experimental claims are validated, the end-to-end formulation for long-form soccer commentary is a useful contribution: it removes the two-stage dependence on a separate spotting module, enables global-context modeling across a full half, and the proposed frame-selection module is training-free and architecture-agnostic. The paper also provides a broad comparison set and explicit ablations of its components. However, the central performance claim currently rests on a potentially confounded baseline comparison and on unvalidated LLM-judge metrics, since on standard caption metrics the 45-minute model does not beat SoccerNet-Caption. The strengths are real but the evidence is not yet sufficient to support the stated SoTA conclusion.","major_comments":[{"comment":"The evaluation-protocol mismatch with SoccerNet-Caption is load-bearing for the SoTA claim. The text states that the authors 'replace the original timestamp annotations with the refined temporal labels provided by MatchTime' and use the 422/49 split from MatchTime, yet Table 1 reports SoccerNet-Caption's P@0.3/0.5/0.7/0.9 = 12.5/7.2/3.0/1.1, F1 = 8.9, and CIDEr = 11.0 without stating that SoccerNet-Caption was re-run on the same refined labels and the same 49-video split. Because CIDEr and SODA_c are computed against the same ground-truth timestamps and captions, a comparison across differently labeled test sets is not apples-to-apples. The claimed temporal gains (+4.5, +3.8, +3.0, +2.3) and the caption-quality comparison may therefore be artifacts of label refinement or split differences. Please specify the exact evaluation protocol for every baseline, or re-run SoccerNet-Caption under the identical protocol used for TimeSoccer.","section":"Section 4.2 / Table 1"},{"comment":"The 45-minute superiority claim relies on the unvalidated LLM-judge metrics M-S and C-S. On the standard caption metrics, TimeSoccer (45-minute) scores CIDEr 8.3 vs. 11.0, F1 8.8 vs. 8.9, and SODA_c 2.7 vs. 3.9 compared with SoccerNet-Caption; only the Qwen2.5-VL-72B-Instruct scores (M-S 5.03 vs. 3.41, C-S 5.14 vs. 4.30) favor TimeSoccer. No human evaluation, judge-versus-human correlation, prompt template, or scoring rubric is reported for these 1–10 ratings. Without such validation, the headline claim that TimeSoccer generates higher-quality 45-minute commentary is not supported. Please add a human study or provide evidence that the LLM judge agrees with human judgments.","section":"Section 4.2 / Table 1"},{"comment":"The ablation statements are contradicted by the reported numbers. Table 2 shows that removing the Motion-Aware component raises SODA_c from 2.7 (full) to 4.1 while lowering METEOR from 6.2 to 2.9, so the text's claim that 'removing any of them leads to a performance drop' is false for SODA_c. In Table 3, the full model (Progressive + MoFA-Select) has CIDEr 8.3, which is lower than the progressive-training-with-repeated-PosEnc row at 9.2, suggesting that MoFA-Select may hurt CIDEr in that configuration. These inconsistencies must be explained, or the component-necessity claims should be revised.","section":"Section 4.4 / Tables 2 and 3"}],"minor_comments":[{"comment":"The term 'Single-anchor Dense Video Captioning' is used in the abstract and introduction but is never formally defined; please define it and explain how it differs from the standard dense video captioning setting.","section":"Abstract and Section 1"},{"comment":"The module name is spelled inconsistently as both 'MoFA-Select' and 'MoFa-Select'; please use one spelling consistently.","section":"Throughout"},{"comment":"The first contribution bullet contains the typo 'We propos'; please correct it.","section":"Section 1, Contributions"},{"comment":"Several hyperparameters needed for reproducibility are missing: the number of sampled frames N_o, the compressed length N_p, the sliding-window length M_w and stride S, and the details of the progressive training schedule (number of epochs or steps at each video length).","section":"Section 4.1"},{"comment":"The prompts and scoring instructions used with Qwen2.5-VL-72B-Instruct for the M-S and C-S metrics are not reported; please include them in the appendix.","section":"Section 4.2"},{"comment":"The paper uses the 3-minute via 15x-inference setting for the 'surpassing the previous SoTA by +2.8 CIDEr' claim, while also stating that this setting is not fully end-to-end; the framing should clearly separate the end-to-end 45-minute results from the non-end-to-end 3-minute results so that the SoTA claim is not ambiguous.","section":"Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"The experimental-protocol and LLM-judge issues are substantial but, in my view, fixable within the manuscript's scope: re-running all baselines under the identical refined-label protocol and adding a human evaluation would directly address the main concerns. If the authors cannot provide such evidence, the SoTA claim should be removed and the paper reframed as a system demonstration of end-to-end long-form commentary generation. I therefore recommend major revision rather than rejection, despite the current weaknesses."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take on TimeSoccer (arXiv:2504.17365). The genuinely new thing here is the end-to-end formulation: they take a 45-minute match video and produce both timestamps and commentary in a single pass, building on TimeChat but adding a training-free frame selection module (MoFA-Select) that clusters frames, allocates budget by motion, and merges redundant features. That is a real step for long-video sports captioning, and the qualitative examples in Fig. 4 show plausible context-aware commentary. Credit where due: the idea is clearly motivated and the system design is reasonable.\n\nThe trouble is the evidence. On the standard CIDEr metric in the 45-minute setting, TimeSoccer scores 8.3, below SoccerNet-Caption's 11.0, so the SoTA caption-quality claim rests entirely on the Qwen2.5-VL LLM-judge scores (M-S, C-S), which are unvalidated against human judgment. The temporal gains over SN-Caption (e.g., +4.5 P@0.3) also depend on an evaluation protocol that isn't fully documented: the authors say they use MatchTime's refined timestamps and split, but the SN-Caption row doesn't state explicitly that it was re-run on the same labels. That may be an oversight rather than a cheat, but it's a minimal condition for the comparison.\n\nMore serious are the internal inconsistencies in the ablations. Table 2 says removing Motion-Aware drops performance, but 'w/o Motion-Aware' has SODA_c 4.1 vs. 2.7 for the full model—that's an improvement. Table 3 has similar issues: Progressive training without Repeated PosEnc has CIDEr 8.6, higher than the full model's 8.3, and lower METEOR/SODA_c than Direct 45-min. The text claims every component contributes positively; the tables don't support that. These aren't typos in one place—they appear in several rows, and they cut against the central design justification.\n\nThere's also no code or data release, and no error bars, which makes it hard to know how much of the 1–3 point gaps are noise.\n\nSo my read: the idea and architecture are worth pursuing, but the paper as written doesn't reliably establish its SoTA claim. It's a solid borderline case for peer review—send it out, but flag that the ablations need to be corrected and the evaluation protocol clarified before acceptance. I'd want to see a re-run of the baselines on the same labels, human validation of the LLM judge, and a full corrected ablation table.\n\nRecommendation: worthwhile to referee, but expect major revision. Not something I'd cite yet.","headline":"A promising end-to-end soccer captioning system undermined by inconsistent ablations and a possibly confounded SoTA comparison; worth a second look after major fixes.","tokens_in":14800,"tokens_out":5218,"would_cite":false,"duration_ms":47037,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TimeSoccer generates soccer commentary in one pass over full matches.","keywords":["soccer commentary generation","dense video captioning","multimodal large language model","temporal localization","long video understanding","motion-aware frame selection","end-to-end video captioning"],"falsifier":"Run a blinded human evaluation on the test matches, with expert soccer viewers comparing TimeSoccer's single-pass 45-minute commentary with the two-step baseline's commentary for temporal accuracy and informativeness; if the baseline is rated equal or better, the paper's full-match quality claim is refuted.","tokens_in":13753,"feed_emoji":"⚽","tokens_out":7923,"duration_ms":66799,"temperature":0.7,"pith_summary":"TimeSoccer aims to establish that soccer commentary can be generated end-to-end from a full 45-minute match video, without any pre-detected event timestamps or a separate timestamping stage. The paper proposes a multimodal large language model that predicts when events happen and writes the commentary in a single forward pass, while compressing the long video with a motion-aware frame-selection module. It argues that this single-pass design preserves global match context, so commentary can refer back to earlier events, and reports state-of-the-art performance on the Single-anchor Dense Video Captioning task. A sympathetic reader should care because current soccer captioning systems either need ground-truth timestamps or split the match into short clips and lose the global picture.","feed_headline":"TimeSoccer generates soccer commentary in one pass over full matches","feed_subtitle":"TimeSoccer predicts timestamps and captions in one pass, keeping the whole match in context.","key_machinery":"The load-bearing component is MoFA-Select, a training-free, motion-aware frame compression module with a coarse-to-fine strategy. It first clusters frames with a time-constrained variant of K-Means, then allocates the fixed frame budget to clusters according to the variance of their features, and finally merges adjacent redundant frames while protecting high-motion segments. This reduces a 45-minute match to a fixed-length representation that can pass through the sliding video Q-former and the language model. Two complementary training mechanisms—progressive lengthening of training videos and position-embedding extrapolation by periodic replication—let the pretrained model generalize beyond its original temporal range.","core_discovery":"The paper's central claim is that the two-step paradigm for soccer dense video captioning—first localize events, then caption each short clip—can be replaced by one end-to-end model that jointly outputs timestamps and captions for full 45-minute matches. On the SoccerNet-Caption benchmark, TimeSoccer is reported to exceed the previous best method in temporal localization precision, for example Precision@0.3 of 17.0 versus 12.5 in the 45-minute setting, and to score higher when a large vision-language model judges match-level semantic alignment and commentary quality. The authors attribute the gains to jointly modeling the whole match, so a caption can use context from earlier moments, and to training procedures that extend the model to match length.","pith_inferences":["If the single-pass design transfers, the same joint timestamp-and-caption objective could be applied to other long sports broadcasts, where events are sparse and global context matters.","Because the headline quality gains come from a vision-language model judge rather than n-gram overlap, a direct human comparison of TimeSoccer and SoccerNet-Caption outputs on the same test matches would be a natural next check.","The motion-aware allocation rule is simple enough to be reused as a preprocessing front-end for any long-form video multimodal model that has a frame-token budget, independent of the soccer task."],"forward_implications":["Full-match soccer commentary can be produced in a single pass, removing the need for a separate event-spotting stage or ground-truth timestamps at inference.","Commentary can use global context, so a caption can explain a substitution by referring to an earlier injury or continue an attacking narrative after a prior corner.","Motion-aware frame compression is training-free, so it can be dropped into other long-video multimodal models without retraining the compression module.","On strict temporal overlap thresholds such as IoU 0.7 and 0.9, the reported gains are larger, suggesting the model localizes key moments more sharply rather than only coarsely.","The same single-pass objective could make full-length sports commentary systems practical for broadcast settings, where events are sparse and long-range context matters."],"supporting_citations":[{"why":"Defines the dataset and the two-step SoccerNet-Caption pipeline that TimeSoccer replaces and compares against.","marker":"[29]"},{"why":"Supplies the train/test split and refined timestamp labels used in all experiments.","marker":"[33]"},{"why":"Provides the time-aware baseline architecture, with its frame encoder and sliding video Q-former, that TimeSoccer extends.","marker":"[34]"},{"why":"Supplies the adjacent-frame merging idea used inside the fine-grained stage of MoFA-Select.","marker":"[35]"},{"why":"Serves as a training-free token-pruning baseline that MoFA-Select outperforms in the ablation study.","marker":"[17]"},{"why":"Provides the large vision-language model used as the automated judge for match-level semantic alignment and commentary quality scores.","marker":"[3]"},{"why":"Provides the vision encoder used to extract frame features for the matches.","marker":"[37]"},{"why":"Provides the language backbone used with LoRA fine-tuning for the final commentary generation.","marker":"[39]"}],"fun_headline_variants":["TimeSoccer: end-to-end commentary for full soccer matches","One-pass soccer commentary that keeps the whole match in context","TimeSoccer predicts timestamps and captions in a single pass","Whole-match soccer commentary, no two-step pipeline","Soccer commentary with global context, from start to finish"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The strongest claim rests on the assumption that the automated language-model judge gives valid, unbiased quality scores, because on the standard CIDEr metric TimeSoccer's full-match output trails the two-step baseline it challenges.","fun_headline_variants_meta":{"raw":{"variants":["TimeSoccer: end-to-end commentary for full soccer matches","One-pass soccer commentary that keeps the whole match in context","TimeSoccer predicts timestamps and captions in a single pass","Whole-match soccer commentary, no two-step pipeline","Soccer commentary with global context, from start to finish"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1356,"prompt_tokens":949,"completion_tokens":407,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":325}},"tokens_in":565,"tokens_out":407,"duration_ms":3781,"temperature":1.0,"reasoning_tokens":325,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:41:21.703758+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a blinded human evaluation on the test matches, with expert soccer viewers comparing TimeSoccer's single-pass 45-minute commentary with the two-step baseline's commentary for temporal accuracy and informativeness; if the baseline is rated equal or better, the paper's full-match quality claim is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the dataset and the two-step SoccerNet-Caption pipeline that TimeSoccer replaces and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the adjacent-frame merging idea used inside the fine-grained stage of MoFA-Select."}],"review_version":1}