Pith. sign in

REVIEW 4 major objections 5 minor 71 references

MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that keeping temporal motion and spatial semantic features in separate cross-attention streams, then exchanging task information between moment retrieval and highlight detection, pushes both tasks past prior…

desk verdict Competent incremental MR/HD paper with a real reproducibility gap over auxiliary-data split restriction; worth reviewing, needs clarification. read the letter →

arxiv 2507.12062 v1 pith:Y4VA2S5E submitted 2025-07-16 cs.CV

Hongxu Ma , Guanshuo Wang , Fufu Yu , Qiong Jia , Shouhong Ding This is my paper
classification cs.CV
keywords videomomentretrievalhighlightdetectionDETRmotion-semanticsdisentanglementcross-modaltransformercontrastivedenoisingSlowFastCLIP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that jointly solving video moment retrieval and highlight detection is best done by keeping temporal motion and spatial semantic information separate until late in the model, instead of concatenating pre-extracted features up front. The proposed MS-DETR feeds SlowFast motion features and CLIP semantic features into two distinct cross-attention towers, each guided by the text query, and only then fuses them for the encoder; a task-collaborated decoder then lets highlight salience seed moment-retrieval queries and lets retrieved boundaries refine salience scores. To overcome sparse annotations, the authors generate auxiliary video-text pairs from captions and rewrites and train with contrastive denoising. If the results are right, this is a practical recipe: state-of-the-art performance on four benchmarks, with the headline gap on QVHighlights val MR mAP of 67.19 versus 64.07 for the prior CG-DETR.

What carries the argument

The load-bearing mechanism is the Motion-Semantics Disentangled Encoder (MSDE), which replaces feature concatenation with two cross-attention towers: TMCT attends from temporal motion features to text, SSCT attends from spatial semantic features to text, and their outputs are concatenated and mapped back to dimension $d$ by a learnable projection (Eq. 1). The Mutual Task-Collaborated Decoder (MTCD) then computes salience scores from a learned salience token via a dot-product (Eq. 3), selects top-$K$ encoder outputs as content queries, turns them into positional queries via sinusoidal encoding, and feeds the two task heads. The auxiliary-data branch generates caption and query pairs and trains with a contrastive denoising loss that adds scaled noise to ground-truth moments before decoding.

What would settle it

Probe each tower's learned representation with labeled action and scene classifications: if the motion tower classifies scenes as accurately as the semantic tower, or the semantic tower classifies actions as accurately as the motion tower, the features are not disentangled and the central explanation is false. A cheaper check is to mask one tower at test time, since true disentanglement predicts verb-focused queries fail when TMCT is masked while noun-focused queries fail when SSCT is masked.

Watch

Extended reading notes

Core claim

The paper's central claim is that motion-semantics disentanglement, rather than more parameters or a stronger backbone, is what drives the gains. Concretely, it reports that MS-DETR outperforms all compared state-of-the-art joint MR/HD models: on QVHighlights val, MR mAP rises to 67.19 and R1@0.75 to 46.00; on test, MR mAP is 66.41; and on TVSum, highlight mAP reaches 88.43. The ablation in Table 5 attributes the improvement to the three modules in combination, and Table 6 argues the MSDE gain is not simply from added layers.

Load-bearing premise

The method's gains are explained as disentanglement, but this relies on the unverified assumption that SlowFast features encode motion only and CLIP features encode appearance only; if both encode mixed information, the two towers are not truly disentangled and the claimed mechanism reduces to extra capacity.

Editorial extensions

If this is right

  • Disentangled motion and semantic streams can be reused as a drop-in encoder upgrade for existing DETR-based MR/HD pipelines, since it only changes how pre-extracted features are combined.
  • Because HD salience is computed from encoder outputs and MR from decoder outputs, the mutual-collaboration loss should make both tasks improve together; Table 1 shows simultaneous gains on both metrics.
  • Auxiliary generated pairs act as hard negatives, implying that annotation sparsity can be mitigated at the data level without new human labels.
  • The method's reported TVSum HD mAP of 88.43 surpasses even audio-augmented models, suggesting motion-plus-semantics video features alone can carry highlight detection when separated properly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the architecture's real test is whether TMCT and SSCT capture genuinely different information; probing the two tower outputs with action and scene classifiers would show whether the semantic tower also encodes motion, which would require revising the stated explanation even if the scores stand.
  • Editorial extension: since the losses rely on LLM-generated captions and rewrites, the method inherits any bias or hallucination in those generators; ablating the generation source (human captions versus generated captions) would separate data richness from data noise.
  • Editorial extension: replacing SlowFast with another motion feature extractor, or CLIP with another semantic extractor, would reveal how much of the gain comes from the separation principle rather than from the specific pre-trained features.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes MS-DETR, a DETR-based joint architecture for video moment retrieval (MR) and highlight detection (HD). The encoder uses two cross-modal transformer towers over pre-extracted SlowFast and CLIP features to model temporal-motion and spatial-semantic dimensions separately, and the decoder lets the HD task seed MR queries while MR boundaries refine HD salience. To address annotation sparsity, the authors generate auxiliary captions and rewritten queries, then train with contrastive denoising losses. Experiments on QVHighlights, TACoS, Charades-STA, and TVSum report gains over QD-DETR, CG-DETR, and TR-DETR, with ablations in Table 5 attributing improvements to each proposed component.

Significance. The paper has several strengths: it evaluates on four standard public benchmarks with pre-extracted features, it includes a parameter-matched comparison in Table 6, and the ablations in Table 5 show positive contributions from the proposed modules. The reported improvements over CG-DETR and TR-DETR are plausible and of interest to the MR/HD community. However, the central 'disentangled motion-semantics' explanation is not directly verified, and the auxiliary-data generation procedure leaves a potential train/validation/test leakage ambiguity that is load-bearing because Table 5(d) credits auxiliary data with a large part of the gain.

major comments (4)
  1. [Section 3.4, Motion-Semantics Corpus Generation] The corpus-generation text never states that the generated captions, rewritten queries, and selected video-text pairs are derived only from the training split. The captioning step starts from 'each video clip in the ground truth' and assigns intervals by CLIP cosine similarity; if any validation or test video appears in the generated corpus, its ground-truth annotations leak into training. This is not a marginal issue because ablation Table 5(d) attributes +3.60 MR mAP to auxiliary data alone, and the full-model gain over CG-DETR on the same QVHighlights val split is +3.12 mAP. Please state explicitly that all generation is restricted to the training split, and verify that the released code enforces this restriction.
  2. [Section 3.4, Eq. (13)] The loss L_enc_cont is not implementable as written. The phrase 'the samples with higher salience scores than n' is undefined because n is a loop index, not a score; the construction of X_pos^n and X_neg^n is therefore ambiguous. In addition, the denominator sums over x in X_pos^n ∪ X_neg^n of exp(S(x_i)/ξ), mixing the summation variable x with x_i. Please specify the exact ranking/partition procedure (for example, rank thresholds within each video, within the ground-truth moments, or across the batch) and rewrite Eq. (13) with consistent variables.
  3. [Section 3.2, Eq. (1)] The claimed disentanglement rests on the assumption that SlowFast features represent temporal motion and CLIP features represent spatial semantics, but the manuscript provides no evidence for this separability. If both pre-trained feature sets contain substantial amounts of both kinds of information, the two towers may reduce to two-stream cross-modal attention without genuinely disentangled dimensions, which would undermine the stated explanation for the gains. Please add a diagnostic test, such as probing each tower's output or ablating by feeding the same feature type to both towers, or temper the disentanglement claim to match what the experiments actually support.
  4. [Sections 3.4 and 4] The training setup is under-specified. The loss weights λ1 and λ2, the margin δ, the temperature ξ, the noise scale δ2, the number of decoder queries K, and the standard training hyperparameters (batch size, learning rate, schedule) are not reported. Since Eq. (14) combines several weighted losses and the auxiliary-data pipeline introduces additional thresholds, reproduction from the paper alone is currently impossible. Please add a training-details subsection or a supplementary document with these values and the selection procedure for the captioning thresholds.
minor comments (5)
  1. [Section 3.2, Eq. (1)] The notation is inconsistent: the text defines the second tower's query as Q_sv = F_sv, but Eq. (1) writes SSCT(Q_tv, K_t, V_t). Please use Q_sv consistently.
  2. [Section 3.4, Motion corpus generation] The sentence 'we replace the verbs in text queries with their corresponding original video moment intervals to produce positive and hard-negative video-text pairs' appears garbled; it should describe replacing verbs with synonyms or antonyms, mirroring the noun rewriting described for the semantics corpus.
  3. [Section 4.2, Table 1 paragraph] The sentence 'improving the mAP metric by 2% and 3.77% respectively' does not say which split and which task the percentages refer to. Please specify (e.g., test MR mAP and val HD mAP) so the claim is verifiable.
  4. [Section 4.2, Table 4] The claim of outperforming all existing SOTAs should be qualified: on TVSum, MS-DETR is below TR-DETR on several categories (for example DS 76.88 vs. 81.6, and PR 88.52 vs. 88.6). The statement should be phrased in terms of average mAP or the majority of metrics rather than universal superiority.
  5. [Section 3.3, Eq. (9)] It is unclear how Φ(m_noise) is computed when m_noise is a (center, span) pair rather than a scalar reference position. A sentence explaining how noisy moments are converted into decoder position queries would improve reproducibility.

Circularity Check

1 steps flagged · score 2.0 of 10

SOTA claim is externally benchmarked; only minor self-reference in the contrastive loss's rank-based positives/negatives.

  1. self definitional [Sec. 3.4, Eq. (13), Contrastive Metric Learning]
    "We perform N iterations of the data within a batch, and in each iteration n∈{1,2,...,N}, the samples with higher salience scores than n are used as X^p_pos and the lower are set as X^p_neg. Note that the output of negative video-text pairs, i.e., X^neg are also be included in {X^p_neg,n∈{1,...,N}} (Eq. 13 context)."

    The contrastive objective L_enc_cont in Eq. (13) optimizes salience scores S(x_i) against a positive/negative partition defined by the ranking of those same scores S(x_i). Thus the training target is a function of the model being trained, so the loss can be reduced by preserving the model's own score ordering without any new external video-text signal. This is a self-referential training regularizer rather than an independent supervision source. It is not the source of the reported benchmark numbers, because the final metrics in Tables 1-4 are computed on held-out splits against published baselines, and this term is only one weighted component in L_total alongside GT-based losses. Severity is therefore minor and does not make the central SOTA claim circular.

full rationale

The main derivation chain of MS-DETR is self-contained and externally evaluated. MSDE consumes frozen SlowFast and CLIP features through standard cross-attention towers; MTCD builds decoder queries from HD salience predictions and supervises them with GT moment losses; the auxiliary corpus is generated from ground-truth intervals and used as training augmentation, which is not circular reasoning. The SOTA comparisons in Tables 1-4 use established external benchmarks and metrics against published methods, so the reported gains are not constructed from the model's own definitions. No load-bearing self-citation or imported uniqueness theorem appears; the only self-citation [29] is motivational. The single self-referential element is the rank-based positive/negative sampling in Eq. (13), which is a self-consistency regularizer, not a source of test predictions. Separately, Section 3.4 does not state that generated captions, rewritten queries, and new video-text pairs are restricted to the training split, leaving a possible leakage risk; this is a data-integrity and reproducibility concern, not an in-paper circularity. Overall score 2 reflects the minor self-reference only.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The method rests on standard pretrained features and benchmark assumptions, but the central novelty (disentangled motion/semantics learning) depends on an unverified separability assumption about those features. Training also relies on several unreported hyperparameters and on generated labels whose quality is not measured. No invented entities are introduced.

free parameters (7)
  • loss weight λ1 (L_cont_dn) = not reported
    In Eq. 14, controls denoising reconstruction loss; no value given in text.
  • loss weight λ2 (contrastive losses) = not reported
    In Eq. 14, controls L_enc_neg + L_margin + L_enc_cont; no value given.
  • margin δ = 0.2
    Default margin in Eq. 12; chosen by hand without sensitivity study.
  • temperature ξ = not reported
    Temperature in Eq. 13; no value or sensitivity reported.
  • noise scale δ2 = not reported
    Controls denoising noise magnitude in Section 3.4; no value reported.
  • number of decoder queries K = not reported
    Top-K selection in Eq. 2; K not stated.
  • caption selection thresholds = interval length < 3 filtered; top-2 selected
    Ad hoc thresholds in corpus generation (Section 3.4) may affect auxiliary data quality.
assumptions (4)
  • domain assumption Pre-extracted CLIP and SlowFast features separate spatial semantics from temporal motion
    Section 3.1 assigns all motion information to SlowFast features and all semantic information to CLIP features; this separability is not verified.
  • ad hoc to paper Generated auxiliary video-text pairs are sufficiently accurate to provide training signal
    Section 3.4 uses LLaVA captions and synonym/antonym rewrites as positives and hard negatives without manual validation.
  • domain assumption Standard benchmark evaluation protocol (features, splits, metrics) is directly comparable across methods
    Section 4.1 states adherence to prior protocols, but no code or standardized evaluation harness is provided.
  • domain assumption The contrastive losses improve generalization rather than overfitting to training set quirks
    Section 3.4 introduces L_enc_cont and L_margin to exploit auxiliary data; no analysis of overfitting or gradient behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learning." pith.science (2026). https://pith.science/paper/Y4VA2S5E

@misc{pith2026250712062,
  author       = {Pith},
  title        = {Pith review of: MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y4VA2S5E}},
  note         = {Machine review of arXiv:2507.12062}
}
read the original abstract

Video Moment Retrieval (MR) and Highlight Detection (HD) aim to pinpoint specific moments and assess clip-wise relevance based on the text query. While DETR-based joint frameworks have made significant strides, there remains untapped potential in harnessing the intricate relationships between temporal motion and spatial semantics within video content. In this paper, we propose the Motion-Semantics DETR (MS-DETR), a framework that captures rich motion-semantics features through unified learning for MR/HD tasks. The encoder first explicitly models disentangled intra-modal correlations within motion and semantics dimensions, guided by the given text queries. Subsequently, the decoder utilizes the task-wise correlation across temporal motion and spatial semantics dimensions to enable precise query-guided localization for MR and refined highlight boundary delineation for HD. Furthermore, we observe the inherent sparsity dilemma within the motion and semantics dimensions of MR/HD datasets. To address this issue, we enrich the corpus from both dimensions by generation strategies and propose contrastive denoising learning to ensure the above components learn robustly and effectively. Extensive experiments on four MR/HD benchmarks demonstrate that our method outperforms existing state-of-the-art models by a margin. Our code is available at https://github.com/snailma0229/MS-DETR.git.

Figures

Figures reproduced from arXiv: 2507.12062 by the authors.

Figure 1
Figure 1. Examples from the QVHighlight dataset [18] illustrating our motivation: the performance by MR/HD may be determined by fine-grained temporal motion and spatial semantic clues within video-text pairs, so more accurate results require learning finer-grained motion-semantics representations and effectively utilizing deeper intrinsic associations between them. (Best viewed zoomed in on screen) deeper intrinsic associatio… view at source ↗
Figure 2
Figure 2. Overview of the MS-DETR’s architecture, which consists of three core modules: (a) Motion-Semantics Disentangled [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of Contrastive Metric Learning on Auxiliary Data. We sufficiently utilize the richness of auxiliary data in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visualization of prediction comparisons on video clips from the QVHighlight dataset [ [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 52 canonical work pages

  1. [1]

    Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan Russell. 2017. Localizing moments in video with natural language. In Proceedings of the IEEE international conference on computer vision . 5803–5812

    work page 2017

  2. [2]

    Taivanbat Badamdorj, Mrigank Rochan, Yang Wang, and Li Cheng. 2021. Joint visual and audio learning for video highlight detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 8127–8137

    work page 2021

  3. [3]

    Taivanbat Badamdorj, Mrigank Rochan, Yang Wang, and Li Cheng. 2022. Con- trastive learning for unsupervised video highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 14042–14052

    work page 2022

  4. [4]

    Sijia Cai, Wangmeng Zuo, Larry S Davis, and Lei Zhang. 2018. Weakly-supervised video summarization using variational encoder-decoder and web prior. In Pro- ceedings of the European conference on computer vision (ECCV) . 184–200

    work page 2018

  5. [5]

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexan- der Kirillov, and Sergey Zagoruyko. 2020. End-to-end object detection with transformers. In European conference on computer vision . Springer, 213–229

    2020

  6. [6]

    Jingyuan Chen, Xinpeng Chen, Lin Ma, Zequn Jie, and Tat-Seng Chua. 2018. Tem- porally grounding natural sentence in video. In Proceedings of the 2018 conference on empirical methods in natural language processing . 162–171

    work page 2018

  7. [7]

    Victor Escorcia, Mattia Soldan, Josef Sivic, Bernard Ghanem, and Bryan Rus- sell. 2019. Temporal localization of moments in video collections with natural language. (2019)

    work page 2019

  8. [8]

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slow- fast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision . 6202–6211

    work page 2019

Show all 71 references
  1. [9]

    Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. 2017. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on computer vision . 5267–5275

    2017

  2. [10]

    Runzhou Ge, Jiyang Gao, Kan Chen, and Ram Nevatia. 2019. Mac: Mining activity concepts for language-based temporal localization. In 2019 IEEE winter conference on applications of computer vision (W ACV). IEEE, 245–253

    2019

  3. [11]

    Soham Ghosh, Anuva Agarwal, Zarana Parekh, and Alexander Hauptmann. 2019. Excl: Extractive clip localization using natural language descriptions. arXiv preprint arXiv:1904.02755 (2019)

    2019 arXiv

  4. [12]

    Michael Gygli, Yale Song, and Liangliang Cao. 2016. Video2gif: Automatic gen- eration of animated gifs from video. In Proceedings of the IEEE conference on computer vision and pattern recognition . 1001–1009

    2016

  5. [13]

    Fa-Ting Hong, Xuanteng Huang, Wei-Hong Li, and Wei-Shi Zheng. 2020. Mini- net: Multiple instance ranking network for video highlight detection. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16 . Springer, 345–360

    2020

  6. [14]

    Jinhyun Jang, Jungin Park, Jin Kim, Hyeongjun Kwon, and Kwanghoon Sohn

  7. [15]

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision- language representation learning with noisy text supervision. In International conference on machine learning . PMLR, 4904–4916

    2021

  8. [16]

    Aditya Khosla, Raffay Hamid, Chih-Jen Lin, and Neel Sundaresan. 2013. Large- scale video summarization using web-image priors. In Proceedings of the IEEE conference on computer vision and pattern recognition . 2698–2705

    2013

  9. [17]

    Sukanya Kudi and Anoop M Namboodiri. 2017. Words speak for actions: Us- ing text to find video highlights. In 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR). IEEE, 322–327

    2017

  10. [18]

    Jie Lei, Tamara L Berg, and Mohit Bansal. 2021. Detecting moments and highlights in videos via natural language queries. Advances in Neural Information Processing Systems 34 (2021), 11846–11858

    2021

  11. [19]

    Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. 2020. Tvr: A large-scale dataset for video-subtitle moment retrieval. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI 16 . Springer, 447–463

    2020

  12. [20]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning . PMLR, 12888–12900

    2022

  13. [21]

    Mingxiao Li, Fang Qu, Zhanpeng Chen, Na Su, Zhizhou Zhong, Ziyang Chen, Nan Du, and Xiaolong Li. 2025. From Visuals to Vocabulary: Establishing Equivalence Between Image and Text Token Through Autoregressive Pre-training in MLLMs. arXiv preprint arXiv:2502.09093 (2025)

    2025 arXiv

  14. [22]

    Mingxiao Li, Na Su, Fang Qu, Zhizhou Zhong, Ziyang Chen, Yuan Li, Zhaopeng Tu, and Xiaolong Li. 2025. VISTA: Enhancing Vision-Text Alignment in MLLMs via Cross-Modal Mutual Information Maximization. arXiv preprint arXiv:2505.10917 (2025)

    2025 arXiv

  15. [23]

    Kevin Qinghong Lin, Pengchuan Zhang, Joya Chen, Shraman Pramanick, Difei Gao, Alex Jinpeng Wang, Rui Yan, and Mike Zheng Shou. 2023. Univtg: To- wards unified video-language temporal grounding. InProceedings of the IEEE/CVF International Conference on Computer Vision . 2794–2804

    2023

  16. [24]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruc- tion tuning. Advances in neural information processing systems 36 (2024)

    2024

  17. [25]

    Meng Liu, Xiang Wang, Liqiang Nie, Qi Tian, Baoquan Chen, and Tat-Seng Chua

  18. [26]

    Wu Liu, Tao Mei, Yongdong Zhang, Cherry Che, and Jiebo Luo. 2015. Multi-task deep visual-semantic embedding for video thumbnail selection. In Proceedings of the IEEE conference on computer vision and pattern recognition . 3707–3715

    2015

  19. [27]

    Ye Liu, Siyuan Li, Yang Wu, Chang-Wen Chen, Ying Shan, and Xiaohu Qie. 2022. Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3042–3051

    2022

  20. [28]

    Yanzuo Lu, Manlin Zhang, Andy J Ma, Xiaohua Xie, and Jianhuang Lai. 2024. Coarse-to-fine latent diffusion for pose-guided person image synthesis. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 6420–6429

    2024

  21. [29]

    Hongxu Ma, Kai Tian, Tao Zhang, Xuefeng Zhang, Han Zhou, Chunjie Chen, Han Li, Jihong Guan, and Shuigeng Zhou. 2025. Generative Regression Based Watch Time Prediction for Short-Video Recommendation. arXiv:2412.20211 [cs.LG]

    2025 arXiv

  22. [30]

    Kaijing Ma, Xianghao Zang, Zerun Feng, Han Fang, Chao Ban, Yuhan Wei, Zhongjiang He, Yongxiang Li, and Hao Sun. 2023. LLaViLo: Boosting Video Moment Retrieval via Adapter-Based Multimodal Modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 2798–2803

    2023

  23. [31]

    Behrooz Mahasseni, Michael Lam, and Sinisa Todorovic. 2017. Unsupervised video summarization with adversarial lstm networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition . 202–211

    2017

  24. [32]

    Yuxi Mi, Zhizhou Zhong, Yuge Huang, Qiuyang Yuan, Xuan Zhao, Jianqing Xu, Shouhong Ding, Shaoming Wang, Rizen Guo, and Shuigeng Zhou. 2025. Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusion. In Proceedings of the Computer Vision and Pattern Reco...

    2025

  25. [33]

    WonJun Moon, Sangeek Hyun, SuBeen Lee, and Jae-Pil Heo. 2023. Correlation- guided Query-Dependency Calibration in Video Representation Learning for Temporal Grounding. arXiv preprint arXiv:2311.08835 (2023)

    2023 arXiv

  26. [34]

    WonJun Moon, Sangeek Hyun, SangUk Park, Dongchan Park, and Jae-Pil Heo

  27. [35]

    Jonghwan Mun, Minsu Cho, and Bohyung Han. 2020. Local-global video-text interactions for temporal grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 10810–10819

    2020

  28. [36]

    Rameswar Panda, Abir Das, Ziyan Wu, Jan Ernst, and Amit K Roy-Chowdhury

  29. [37]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Query-dependent video representation for moment retrieval and highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 23023–23033

  30. [38]

    Michaela Regneri, Marcus Rohrbach, Dominikus Wetzel, Stefan Thater, Bernt Schiele, and Manfred Pinkal. 2013. Grounding action descriptions in videos. Transactions of the Association for Computational Linguistics 1 (2013), 25–36

    2013

  31. [39]

    Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. 2019. Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 658–666

    2019

  32. [40]

    Mrigank Rochan, Linwei Ye, and Yang Wang. 2018. Video summarization using fully convolutional sequence networks. In Proceedings of the European conference on computer vision (ECCV) . 347–363

    2018

  33. [41]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  34. [42]

    Yale Song, Jordi Vallmitjana, Amanda Stent, and Alejandro Jaimes. 2015. Tvsum: Summarizing web videos using titles. In Proceedings of the IEEE conference on computer vision and pattern recognition . 5179–5187

    2015

  35. [43]

    Hao Sun, Mingyao Zhou, Wenjing Chen, and Wei Xie. 2024. TR-DETR: Task- Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection. arXiv preprint arXiv:2401.02309 (2024)

    2024 arXiv

  36. [44]

    Min Sun, Ali Farhadi, and Steve Seitz. 2014. Ranking domain-specific highlights by analyzing edited videos. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13. Springer, 787–802

    2014

  37. [45]

    Yale Song, Miriam Redi, Jordi Vallmitjana, and Alejandro Jaimes. 2016. To click or not to click: Automatic selection of beautiful thumbnails from videos. In Proceed- ings of the 25th ACM international on conference on information and knowledge management. 659–668

    2016

  38. [46]

    Lezi Wang, Dong Liu, Rohit Puri, and Dimitris N Metaxas. 2020. Learning trailer moments in full-length movies with co-contrastive attention. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, MM ’25, October 27–31, 2025, Dublin, Ireland Ho...

    2020

  39. [47]

    Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, and Xiu Li. 2023. Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection. arXiv preprint arXiv:2311.16464 (2023)

    2023 arXiv

  40. [48]

    Bo Xiong, Yannis Kalantidis, Deepti Ghadiyaram, and Kristen Grauman. 2019. Less is more: Learning highlight detection from video duration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 1258–1267

    2019

  41. [49]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  42. [50]

    Minghao Xu, Hang Wang, Bingbing Ni, Riheng Zhu, Zhenbang Sun, and Changhu Wang. 2021. Cross-category video highlight detection via set-based learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 7970– 7979

    2021

  43. [51]

    Yifang Xu, Yunzhuo Sun, Yang Li, Yilei Shi, Xiaoxiang Zhu, and Sidan Du. 2023. Mh-detr: Video moment and highlight detection with cross-modal transformer. arXiv preprint arXiv:2305.00355 (2023)

    2023 arXiv

  44. [52]

    Zunnan Xu, Jiaqi Huang, Ting Liu, Yong Liu, Haonan Han, Kehong Yuan, and Xiu Li. 2024. Enhancing fine-grained multi-modal alignment via adapters: a parameter-efficient training framework for referring image segmentation. In 2nd Workshop on Advancing Neural Network Training: Co...

    2024

  45. [53]

    Huijuan Xu, Kun He, Bryan A Plummer, Leonid Sigal, Stan Sclaroff, and Kate Saenko. 2019. Multilevel language and vision integration for text-to-clip retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 9062–9069

    2019

  46. [54]

    Shen Yan, Xuehan Xiong, Arsha Nagrani, Anurag Arnab, Zhonghao Wang, Weina Ge, David Ross, and Cordelia Schmid. 2023. Unloc: A unified framework for video localization tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 13623–13633

    2023

  47. [55]

    Qinghao Ye, Xiyue Shen, Yuan Gao, Zirui Wang, Qi Bi, Ping Li, and Guang Yang

  48. [56]

    Yitian Yuan, Lin Ma, Jingwen Wang, Wei Liu, and Wenwu Zhu. 2019. Semantic conditioned dynamic modulation for temporal sentence grounding in videos. Advances in Neural Information Processing Systems 32 (2019)

    2019

  49. [57]

    Zunnan Xu, Zhentao Yu, Zixiang Zhou, Jun Zhou, Xiaoyu Jin, Fa-Ting Hong, Xiaozhong Ji, Junwei Zhu, Chengfei Cai, Shiyu Tang, et al. 2025. Hunyuanportrait: Implicit condition control for enhanced portrait animation. In Proceedings of the Computer Vision and Pattern Recognition ...

    2025

  50. [58]

    Delong Zhang, Qiwei Huang, Yuanliu Liu, Yang Sun, Wei-Shi Zheng, Pengfei Xiong, and Wei Zhang. 2024. Learning Implicit Features with Flow Infused Attention for Realistic Virtual Try-On. arXiv preprint arXiv:2412.11435 (2024)

    2024 arXiv

  51. [59]

    Delong Zhang, Yi-Xing Peng, Xiao-Ming Wu, Ancong Wu, and Wei-Shi Zheng

  52. [60]

    Hao Zhang, Aixin Sun, Wei Jing, Liangli Zhen, Joey Tianyi Zhou, and Rick Siow Mong Goh. 2021. Natural language video localization: A revisit in span- based question answering framework. IEEE transactions on pattern analysis and machine intelligence 44, 8 (2021), 4252–4266

    2021

  53. [61]

    Hao Zhang, Aixin Sun, Wei Jing, and Joey Tianyi Zhou. 2020. Span-based localizing network for natural language video localization. arXiv preprint arXiv:2004.13931 (2020)

    2020 arXiv

  54. [62]

    Da Zhang, Xiyang Dai, Xin Wang, Yuan-Fang Wang, and Larry S Davis. 2019. Man: Moment alignment network for natural language moment retrieval via iterative graph adjustment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1247–1257

    2019

  55. [63]

    Songyang Zhang, Houwen Peng, Jianlong Fu, and Jiebo Luo. 2020. Learning 2d temporal adjacent networks for moment localization with natural language. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 12870–12877

    2020

  56. [64]

    Yue Zhang, Zhizhou Zhong, Minhao Liu, Zhaokang Chen, Bin Wu, Yubin Zeng, Chao Zhan, Junxin Huang, Yingjie He, and Wenjiang Zhou. 2024. Musetalk: Real-time high quality lip synchronization with latent space inpainting. arXiv preprint arXiv:2410.10122 (2024)

    2024 arXiv

  57. [65]

    Zhu Zhang, Zhijie Lin, Zhou Zhao, and Zhenxin Xiao. 2019. Cross-modal inter- action networks for query-based moment retrieval in videos. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 655–664

    2019

  58. [68]

    Ke Zhang, Wei-Lun Chao, Fei Sha, and Kristen Grauman. 2016. Video summariza- tion with long short-term memory. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII 14. Springer, 766–782

    2016

  59. [2017]

    In Proceedings of the IEEE international conference on computer vision

    Weakly supervised summarization of web videos. In Proceedings of the IEEE international conference on computer vision . 3657–3666

  60. [2018]

    In Proceedings of the 26th ACM international conference on Multimedia

    Cross-modal moment localization in videos. In Proceedings of the 26th ACM international conference on Multimedia . 843–851

  61. [2021]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Temporal cue guided video highlight detection with low-rank audio-visual fusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7950–7959

  62. [2023]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Knowing where to focus: Event-aware transformer for video grounding. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 13846– 13856

  63. [2024]

    InProceedings of the 32nd ACM International Conference on Multimedia

    PixelFade: Privacy-preserving Person Re-identification with Noise-guided Progressive Replacement. InProceedings of the 32nd ACM International Conference on Multimedia. 6326–6334

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.