{"id":"2bc0f285-ae6b-4552-ad48-192d5b5e1876","arxiv_id":"2508.01699","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"TimeExpert applies dynamic mixture-of-experts routing to video temporal grounding, reporting small state-of-the-art gains over TRACE on dense video captioning, moment retrieval, and highlight detection.","lead":"TimeExpert is a video-language model that routes timestamps, saliency scores, and captions to different expert subnetworks inside a mixture-of-experts decoder. It reports state-of-the-art results on three video temporal grounding benchmarks, but the gains over prior work are small and mostly vanish when compared with a fixed expert count.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 5 shows static top-k MoE with k=6 matches TimeExpert on all seven metrics, and TimeExpert activates more experts on average; without seed-level error bars the central accuracy and efficiency claims are unsupported.","rationale":"I agree with the reader's weakest-assumption analysis and with the CONDITIONAL verdict. The paper's architectural story is coherent, but Table 5 is the load-bearing evidence for the claim that dynamic task-token routing and expert addition/removal improve VTG performance. That evidence shows near-exact parity with a plain static top-k MoE at k=6, with deltas of 0.1 to 0.4 points across all benchmarks. The manuscript provides no standard deviations, no number of seeds, and no statistical test, so the observed differences cannot be distinguished from training noise. In addition, Section 4.2's reported average activated expert counts (14.5/9.1/11.8) make the efficiency narrative questionable: the model activates more experts than the k=6 control while achieving the same scores. The absence of a static-k control for the 'TRACE's data' variant means the comparison to TRACE is also confounded by backbone and data differences. None of this is evidence of misconduct; it is an attribution problem. The paper could be accepted conditionally if the authors add seed-level variance estimates, a properly matched static-k baseline with all other modules held fixed, and compute-cost measurements. Without those, the central claim is not established, so I recommend keeping the CONDITIONAL verdict unchanged.","tokens_in":14932,"tokens_out":7326,"duration_ms":91077,"concrete_test":"Re-run the Table 5 comparison with 5 random seeds for TimeExpert and for static top-k MoE with k=6 (and k=8), holding all other components identical: same ARIA backbone, same task encoders and decoding heads, same auxiliary loss, same data order and training stages. Report per-metric mean plus/minus standard deviation and total activated parameters or FLOPs. If the mean TimeExpert advantage over k=6 is within one standard deviation on every metric, or if TimeExpert uses more activated parameters while at parity, then the central accuracy and efficiency claims should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive counterfactual is Table 5's static MoE control, and it does not support the paper's central attribution. 'Vanilla MoE (k=6)' achieves 2.5/8.2/23.5/42.8/20.2/29.6/46.9 across the seven reported metrics (YouCook2 SODAc/CIDEr/F1, Charades-STA R@1 IoU=.5/.7, QVHighlights mAP/HIT@1), while TimeExpert achieves 2.5/8.2/23.6/42.8/20.3/29.6/46.9. Every difference is 0.1 to 0.4 points, no error bars or multiple seeds are reported, and the k=8 row (mAP 29.8) actually numerically exceeds TimeExpert on one metric. Section 4.2 states the adaptive-k variant averages 14.5, 9.1, and 11.8 activated experts on the three benchmarks, i.e., more than the k=6 control on every task, so the 'increased computational efficiency' claim is also not supported by the comparison actually run. The upper ablation 'w/o token-adaptive routing' is not an adequate substitute for this control because it is a different configuration; the paper never reports a static-k model trained under the identical recipe with the same task encoders, auxiliary loss, and data. The 'TRACE's data' variant in Table 2 likewise lacks a static-k control, so even the apparent win over TRACE could come from the ARIA backbone or data preprocessing rather than dynamic routing. The central claim therefore rests on differences that could easily be seed noise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes TimeExpert, a Mixture-of-Experts Video-LLM for video temporal grounding. The model builds on the ARIA backbone and introduces task-aware dynamic gating (Eq. 6), token-adaptive routing with expert addition/removal (Eqs. 7-8), separate encoders and decoding heads for timestamps, saliency scores, and captions, and a task-dependent auxiliary loss (Eq. 9). The authors evaluate zero-shot and fine-tuned performance on Dense Video Captioning (YouCook2), Moment Retrieval (Charades-STA), Video Highlight Detection (QVHighlights), and ActivityNet Captions, reporting state-of-the-art results over TRACE and other VTG-specific Video-LLMs. The central claim is that dynamic, task-token-aware expert allocation improves accuracy and computational efficiency.","tokens_in":15323,"tokens_out":5480,"duration_ms":58745,"significance":"If the attribution to dynamic routing were established, TimeExpert would be a useful demonstration that MoE specialization can be applied to structured output tokens in VTG, and the benchmark results over TRACE are potentially interesting. The paper has strengths: it evaluates across three VTG tasks and several benchmarks, includes a data-recipe variant to control for training data, and reports a number of ablations including frame count and number of experts. However, the key ablation in Table 5 shows that a vanilla top-k MoE with k=6 matches TimeExpert to within 0.1-0.4 points on all seven metrics, and no error bars or multiple seeds are reported. Because the paper's central scientific claim is that adaptive routing is the cause of the gains, the current evidence does not support that claim; the contribution is therefore currently more architectural than empirical.","major_comments":[{"comment":"The bottom block of Table 5 directly undercuts the central attribution. Vanilla MoE with k=6 produces 2.5/8.2/23.5/42.8/20.2/29.6/46.9 on the seven metrics, while TimeExpert produces 2.5/8.2/23.6/42.8/20.3/29.6/46.9; the k=8 row is higher than TimeExpert on mAP (29.8 vs 29.6). No standard deviations, confidence intervals, or multiple-seed results are reported anywhere. Since the only control that isolates the adaptive mechanism is indistinguishable from static top-k MoE, the paper's claim that 'dynamic gating and adaptive routing' (Section 4.2) drive the improvements is unsupported. The 'w/o token-adaptive routing' row is not an adequate substitute: it changes the gating recipe rather than comparing a static-k model trained under the identical task encoders, auxiliary loss, and data recipe.","section":"§4.3, Table 5"},{"comment":"The abstract and Section 4.2 claim 'increased computational efficiency' and fewer activated parameters. Section 4.2 reports that the adaptive-k variant activates on average 14.5, 9.1, and 11.8 experts on YouCook2, Charades-STA, and QVHighlights, respectively, which is more than the k=6 static-MoE control that matches its accuracy in Table 5. No FLOPs, inference latency, or memory measurements are reported. The only defensible efficiency statement is that TimeExpert activates fewer parameters than dense 7B baselines; the comparison to a static MoE of the same family, which is the relevant control, shows no efficiency benefit and possibly a cost.","section":"§4.2, Table 2 (efficiency claim)"},{"comment":"The 'TimeExpert (TRACE's data)' row is intended to rule out data-recipe effects, but it does not isolate the routing mechanism. Relative to TRACE, the model still differs in the base LLM (ARIA), the separate task encoders/heads, and the MoE decoder; no static-k MoE trained on the same data and encoders is reported. Given that the gains over TRACE are small (0.0 to 2.3 points on the seven metrics in Table 2) and no variance estimates are given, the improvements over TRACE could be due to the backbone or data preprocessing rather than dynamic routing. I would ask for seed-level results and a static-k baseline under the identical recipe before accepting the attribution.","section":"Table 2, 'TimeExpert (TRACE's data)' row"}],"minor_comments":[{"comment":"The 'w/o separate encoders' row is reported only as dashes, with the text saying the model 'fails to follow instructions.' A qualitative statement is not a quantitative ablation; please report the actual failure mode or a measurable proxy (e.g., formatting accuracy) and state whether evaluation was impossible.","section":"Table 5, 'w/o separate encoders' row"},{"comment":"The notation 'AePKj=1Aj' and 'NePKj=1Nj' is ambiguous; it appears to intend (Ae / ΣAj) and (Ne / ΣNj), but the fraction is not typeset. Please clarify.","section":"Eq. (9)"},{"comment":"The text states that data are 'systematically processed, filtered, and re-annotated' but no details, statistics, or filtering criteria are given; this makes the data recipe hard to reproduce.","section":"§3.5, Training Data"},{"comment":"The table reports three different approximate values (≈5.9B/3.5B/4.8B) for the same model, and it is unclear why these differ across benchmarks and how they were computed; please clarify the accounting and consider reporting FLOPs or inference latency.","section":"Table 2, 'No. of Activated Parameters'"},{"comment":"The hyperparameters α, λ1, λ2, τmin, and the initial number of experts K are never given values, and no sensitivity analysis is reported for them; since the proposed gating mechanism depends on these, at least the chosen values should be stated.","section":"§3.2-§3.4, hyperparameters"},{"comment":"No code or checkpoints are provided, and the training recipe uses roughly 5.1M samples; releasing the model or code would materially aid reproducibility and would also help the community verify the small reported gains.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical system paper whose central scientific claim is not yet supported by the ablations as presented. I do not see a fundamental correctness error that would force rejection; the missing seed-level experiments and a static-k control under the identical recipe are within scope. I would like the editor to require those experiments and a revised efficiency analysis before any acceptance, and to verify that the small gains in Tables 2-4 are stable under different seeds."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"TimeExpert is a well-engineered system, and the writing is honest enough to include the ablation that undercuts its own pitch. The core idea—route task tokens (timestamps, saliency, captions) through an MoE with dynamic expert addition/removal, plus a task-dependent auxiliary loss—is a reasonable adaptation of dynamic MoE ideas from text LLMs to VTG, and the empirical sweep across YouCook2, Charades-STA, QVHighlights, and ActivityNet is broader than most VTG work.\n\nBut the paper's main claim is not supported by its own Table 5. Vanilla MoE with k=6 produces 2.5/8.2/23.5/42.8/20.2/29.6/46.9; TimeExpert gets 2.5/8.2/23.6/42.8/20.3/29.6/46.9. Those deltas are 0.1 to 0.4 points across seven metrics, with no error bars or seeds reported. That is noise territory. The k=8 row even exceeds TimeExpert on QVHighlights mAP (29.8 vs 29.6). The abstract's 'increased computational efficiency' also doesn't survive contact: Section 4.2 says the adaptive-k variant averages 14.5, 9.1, and 11.8 activated experts on the three benchmarks, versus exactly 6 for the k=6 control. So you are paying more compute for the same number.\n\nThe other soft spots are second-order. The 'w/o separate encoders' ablation is reported as a failure to follow instructions rather than with numbers, which is fine but weak. Hyperparameters (alpha, lambda1/2, tau_min, initial K) are not given, so the dynamic mechanism is hard to reproduce. The 'TRACE's data' row in Table 2 shows TimeExpert still beats TRACE, but without a static-k control under that same recipe, the win could come from ARIA backbone or data preprocessing.\n\nWhat holds up: the fine-tuned results in Table 3 are more convincing (+3.5 CIDEr, +2.4 R@0.5 over TRACE), and the ActivityNet results in Table 4 are consistent. The system is coherent, the ablations are mostly complete, and the authors included the vanilla-MoE comparison even though it hurts their narrative—that is a mark in their favor.\n\nWho this is for: anyone working on VTG Video-LLMs or MoE routing in multimodal models. It is a serious engineering contribution that deserves referee time, but the authors need to add seed-level variance and a matched static-k baseline trained under the identical recipe. Without that, the central claim is empirically unestablished.\n\nRecommendation: send to peer review, but flag the missing statistical validation and the unmatched control as the main revision points.","headline":"Table 5 undercuts the paper's own central claim: TimeExpert's dynamic routing matches a static top-6 MoE on every metric, so the reported gains are plausibly noise.","tokens_in":15860,"tokens_out":2157,"would_cite":false,"duration_ms":22078,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TimeExpert claims that routing timestamp, saliency, and caption tokens to different experts in a mixture-of-experts decoder improves video temporal grounding across three tasks while activating fewer parameters.","keywords":["video temporal grounding","mixture of experts","video large language models","moment retrieval","dense video captioning","video highlight detection","dynamic routing","task token specialization"],"falsifier":"Rerun the same three-stage training with a fixed top-6 router and the same data across several random seeds, and compare the distributions of the seven metrics; if the 0.1 to 0.4 point differences fall inside one standard error, the central claim that dynamic expert allocation helps is not supported on these benchmarks.","tokens_in":14740,"feed_emoji":"🎬","tokens_out":10570,"duration_ms":98981,"temperature":0.7,"pith_summary":"Video temporal grounding asks a model to turn a video and a text query into a structured list of events, each with a timestamp, a saliency score, and a caption. TimeExpert's thesis is that these three output types are different enough that a single shared parameter path hurts performance, so it routes each type of token to a different specialist inside a mixture-of-experts decoder. The paper reports state-of-the-art zero-shot results on dense video captioning (YouCook2), moment retrieval (Charades-STA), and highlight detection (QVHighlights), and improved fine-tuned results on the first two, all with fewer activated parameters than dense 7B baselines. A sympathetic reader would take the contribution to be evidence that task-token-aware expert routing is a useful design axis for fine-grained video understanding.","feed_headline":"Task-specific expert routing tops video grounding benchmarks","feed_subtitle":"TimeExpert splits timestamps, scores, and captions across specialists and beats denser models with fewer active parameters.","key_machinery":"The load-bearing object is the task-token-aware dynamic MoE decoder. Its gating computes $s(x) = \\cos(x, W_g)$ and then applies a sigmoid to $(s(x) + \\alpha A_t)/(1+\\alpha)$ before comparing with a learnable threshold $G$; the term $A_t$ is the historical activation rate of the token type, so timestamp tokens can develop a preference for experts that have served timestamps well. The routing layer records per-expert activation rates $R_E$ and the average embedding $R_S$ of tokens that activate no expert; when too many tokens go unmatched, a new expert is inserted with $W_{g,K+1} = R_S/\\|R_S\\|$ and $G_{K+1}=0$, and experts with activation below $\\tau_{\\min}$ are pruned. Finally, the auxiliary loss $L_{\\text{aux}}$ combines a task-aware concentration term, which matches the fraction of activations an expert receives to the fraction of task tokens assigned to it, with a weight-norm regularizer that discourages one expert from dominating. Together these pieces are what let the model specialize timestamps, saliency scores, and captions instead of processing all tokens identically.","core_discovery":"The paper's central claim is that token type—timestamp, saliency score, or caption text—should determine which experts process a token, rather than all tokens flowing through the same static pathway. To realize this, TimeExpert replaces the decoder of a multimodal mixture-of-experts Video-LLM with three coordinated mechanisms: a task-aware dynamic gating function that activates an expert only if the cosine similarity between token and expert exceeds a learnable threshold adjusted by that token type's historical activation rate; a token-adaptive routing scheme that logs per-type activation statistics, creates a new expert initialized from the mean embedding of tokens no expert will accept, and deletes experts whose activation stays below $\\tau_{\\min}$; and a task-dependent auxiliary loss that pulls frequently activated experts toward their preferred token types while regularizing expert weights. On the three VTG tasks, the paper reports the best zero-shot numbers among the compared Video-LLMs—for example 23.6 F1 on YouCook2, 42.8 R@1 IoU=0.5 on Charades-STA, and 46.9 HIT@1 on QVHighlights—using an average of roughly 9 to 15 activated experts per layer rather than a full 7B-parameter dense pass. The paper also shows the same data-recipe variant still beats the strongest prior baseline on all seven zero-shot metrics, which it reads as evidence that the architecture, not extra data, drives the improvement.","pith_inferences":["The gap between TimeExpert and a vanilla MoE with $k=6$ is 0.1 to 0.4 points on every one of the seven metrics, and the paper reports no error bars; the dynamic routing may be buying efficiency and stability more than accuracy, so the state-of-the-art claim should be read as tentative until repeated-seed variance is reported.","The paper's implicit prediction is that the benefit of task-token-aware routing grows with the share of non-text tokens in the output; a direct test would be to compare routing-aware versus fixed-$k$ MoE on tasks with very long timestamp or score sequences.","Because removing the separate time and score encoders makes the model fail to follow instructions, the structured-encoding component is validated by collapse rather than by graded ablation; an informative follow-up would be to train with shared encoders but separate heads to isolate the contribution of token-type separation.","The adaptive add-and-prune scheme generalizes beyond video: any structured generation task whose outputs mix coordinates, confidence scores, and natural language (for example GUI grounding or embodied instruction following) could use the same task-token-aware routing policy."],"forward_implications":["On the three VTG benchmarks (YouCook2, Charades-STA, QVHighlights), TimeExpert reports the best zero-shot numbers among the compared Video-LLMs, with adaptive expert counts averaging 14.5, 9.1, and 11.8 and fewer activated parameters than dense 7B baselines.","Fine-tuned on YouCook2 and Charades-STA, it improves over the prior best model on both dense video captioning (CIDEr 39.0 vs 35.5, F1 33.5 vs 31.8) and moment retrieval (R@1 IoU=0.5 64.1 vs 61.7).","The same data-recipe variant, trained only on the prior model's data, still outperforms that baseline on all seven zero-shot metrics, so the paper argues the gains are not simply from a larger training set.","Ablation results show the token-adaptive routing and the task-dependent auxiliary loss each contribute, with the auxiliary loss's removal costing the most on highlight detection (mAP 28.1 vs 29.6, HIT@1 45.2 vs 46.9).","Increasing input frames from 64 to 128 improves moment retrieval and highlight detection, indicating that longer temporal context helps even with expert routing."],"supporting_citations":[{"why":"Supplies the causal event model and the strongest baseline that TimeExpert extends and compares against.","marker":"[16]"},{"why":"Provides the differentiable dynamic routing mechanism that the paper adapts for task-aware gating.","marker":"[14]"},{"why":"Motivates routing harder tokens to more experts, the idea behind adaptive expert addition.","marker":"[21]"},{"why":"The multimodal MoE base model whose decoder TimeExpert replaces with task-aware routing.","marker":"[25]"},{"why":"Defines timestamp tokens and the structured-output approach that TimeExpert builds on.","marker":"[15]"},{"why":"Provides a primary Video-LLM baseline for zero-shot and fine-tuned comparisons.","marker":"[40]"},{"why":"Charades-STA benchmark used for moment retrieval evaluation.","marker":"[13]"},{"why":"QVHighlights benchmark used for highlight detection and moment retrieval evaluation.","marker":"[24]"},{"why":"YouCook2 benchmark used for dense video captioning evaluation.","marker":"[64]"}],"fun_headline_variants":["Token-type expert routing tops video grounding benchmarks","TimeExpert splits tokens by type to beat dense video LLMs","Task-aware gating routes each token to a specialized expert","Video grounding gains from dynamic expert selection per token"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the improvements come from the task-aware dynamic routing and auxiliary loss rather than from extra expert capacity or chance, but the paper's own ablation shows a vanilla mixture-of-experts model with six activated experts matching TimeExpert within 0.1 to 0.4 points on all seven metrics, with no error bars reported.","fun_headline_variants_meta":{"raw":{"variants":["Token-type expert routing tops video grounding benchmarks","TimeExpert splits tokens by type to beat dense video LLMs","Task-aware gating routes each token to a specialized expert","Video grounding gains from dynamic expert selection per token"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1513,"prompt_tokens":1026,"completion_tokens":487,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":642,"completion_tokens_details":{"reasoning_tokens":424}},"tokens_in":642,"tokens_out":487,"duration_ms":6188,"temperature":1.0,"reasoning_tokens":424,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:25:04.288412+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the same three-stage training with a fixed top-6 router and the same data across several random seeds, and compare the distributions of the seven metrics; if the 0.1 to 0.4 point differences fall inside one standard error, the central claim that dynamic expert allocation helps is not supported on these benchmarks.","supporting_citations":[{"cited_title":"Trace: Temporal grounding video llm via causal event modeling","cited_arxiv_id":null,"evidence_quote":"Supplies the causal event model and the strongest baseline that TimeExpert extends and compares against."},{"cited_title":"Dynamic mixture of experts: An auto- tuning approach for efficient transformer models","cited_arxiv_id":null,"evidence_quote":"Provides the differentiable dynamic routing mechanism that the paper adapts for task-aware gating."},{"cited_title":"Harder tasks need more experts: Dynamic routing in moe models","cited_arxiv_id":null,"evidence_quote":"Motivates routing harder tokens to more experts, the idea behind adaptive expert addition."},{"cited_title":"Vtg-llm: Integrating timestamp knowledge into video llms for enhanced video temporal grounding","cited_arxiv_id":null,"evidence_quote":"Defines timestamp tokens and the structured-output approach that TimeExpert builds on."},{"cited_title":"Timechat: A time-sensitive multimodal large language model for long video understanding","cited_arxiv_id":null,"evidence_quote":"Provides a primary Video-LLM baseline for zero-shot and fine-tuned comparisons."},{"cited_title":"Tall: Temporal activity localization via language query","cited_arxiv_id":null,"evidence_quote":"Charades-STA benchmark used for moment retrieval evaluation."},{"cited_title":"Detecting mo- ments and highlights in videos via natural language queries","cited_arxiv_id":null,"evidence_quote":"QVHighlights benchmark used for highlight detection and moment retrieval evaluation."},{"cited_title":"Towards automatic learning of procedures from web instructional videos","cited_arxiv_id":null,"evidence_quote":"YouCook2 benchmark used for dense video captioning evaluation."}],"review_version":1}