Pith. sign in

REVIEW 3 major objections 6 minor 60 references

Grounding-MD: Grounded Video-language Pre-training for Open-World Moment Detection

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a single video-language model can detect any user-described moment in an untrimmed video, unifying temporal action detection and moment retrieval in open-world settings.

desk verdict Solid grounded pre-training framework for unified video moment detection with credible supervised gains, but the THUMOS14 open-vocabulary evidence leaks pre-training categories through HACS, so the headline open-world claim needs re-earning. read the letter →

arxiv 2504.14553 v1 pith:RJTFPRNF submitted 2025-04-20 cs.CV

Weijun Zhuang , Qizhang Li , Xin Li , Ming Liu , Xiaopeng Hong , Feng Gao , Fan Yang , Wangmeng Zuo This is my paper · ORCID
classification cs.CV
keywords open-worldmomentdetectiontemporalactionretrievalvideo-languagepre-traininggroundedstructuredpromptszero-shotlocalizationcross-modalfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Grounding-MD argues that temporal action detection and moment retrieval are the same underlying problem: given a video and any number of open-ended natural language queries, output every segment that matches each query, and do so in open-world settings where the queries are not restricted to a predefined label set. The paper builds a video-language pre-training framework that feeds the whole set of queries as one structured prompt, aligns video and text with early fusion in an encoder and late fusion in a decoder, and balances short action labels against long event descriptions with a query-wise pooler. After pre-training on action detection and moment retrieval corpora, it reports new state-of-the-art results in zero-shot and supervised settings on ActivityNet, THUMOS14, ActivityNet-Captions, and Charades-STA. The result matters because the model achieves these numbers with roughly 0.5 billion parameters, claiming zero-shot moment retrieval on par with or better than 7-billion-parameter video-language models.

What carries the argument

The mechanism that carries the argument is the structured prompt paired with early-late cross-modal fusion. A structured prompt concatenates any number of open-ended natural language queries, and a query-wise splitter isolates a per-query text representation after fusion, so each query can be scored against every candidate video moment. The Cross-Modality Fusion Encoder supplies the first interaction between otherwise unrelated VideoMAE video features and BERT text features, while the Text-Guided Fusion Decoder refines the alignment at multiple temporal scales produced by the FPN. A contrastive loss matches video moments to their query representations and a distance regression loss pins down segment boundaries, which together let an unseen query written at test time be grounded without retraining.

What would settle it

Remove HACS (or explicitly exclude every 'unseen' THUMOS14 category) from the pre-training corpus and rerun the Table 4 open-vocabulary zero-shot evaluation. If the mAP on unseen categories collapses to the non-pre-trained baseline, the open-world generalization claim fails; if it survives, the claim is supported.

Watch

Extended reading notes

Core claim

The central claim is that open-world moment detection can be cast as conditional grounding: a structured prompt made of an arbitrary number of open-ended queries acts as the condition, and the model detects temporal segments for every query in a single forward pass. Grounding-MD realizes this with two fusion stages. The Cross-Modality Fusion Encoder first runs self-attention inside each modality and cross-attention in both directions, creating an initial semantic link between video frames and text tokens; the Text-Guided Fusion Decoder then re-attends pooled query representations against multi-scale video features to refine boundaries. The Query-Wise Pooler prevents the longer natural-language event descriptions from overwhelming short action labels during joint training. Large-scale pre-training on HACS, FineAction, and a filtered subset of InternVid supplies the alignment that the VideoMAE and BERT backbones do not natively share. On this basis the paper reports state-of-the-art open-vocabulary zero-shot results on THUMOS14, zero-shot domain transfer on ActivityNet, ActivityNet-Captions, and Charades-STA, and supervised gains across all four benchmarks.

Load-bearing premise

The open-vocabulary zero-shot evaluation assumes the 'unseen' THUMOS14 action categories were never encountered during pre-training, yet the paper notes that the pre-training dataset HACS covers all categories in ActivityNet, and THUMOS14 categories form a subset of ActivityNet categories, so the reported open-world gains may instead be same-category transfer.

Editorial extensions

If this is right

  • A single detector can serve arbitrary user queries at inference time, so video search and action monitoring need no longer train a new classifier head for each closed category set.
  • Grounding-based detection appears to be a far cheaper route to open-world temporal localization than generative video-LLM pipelines, since a 0.5B-parameter model matches or beats 7B-parameter baselines on zero-shot moment retrieval.
  • Adding a new action category becomes a purely textual operation: append the query to the structured prompt, and the existing model can localize it without fine-tuning.
  • The reported gains at high IoU thresholds, including a 3.8-percentage-point improvement over AdaTAD on THUMOS14 at tIoU 0.7, suggest the text-guided decoder sharpens temporal boundaries rather than only improving semantic matching.
  • The query-wise pooling design makes text-length imbalance a first-class training concern, a lesson that carries over to any multi-query detection or retrieval setting where query lengths vary widely.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The structured-prompt formulation is the temporal analogue of open-vocabulary object detection: the same recipe that made object categories promptable in image grounding could make dense video captioning, action segmentation, and highlight detection promptable in video.
  • If the efficiency comparison holds under matched pre-training data, video-LLM temporal grounding may shift toward hybrid designs in which a small grounded detector proposes segments and a large language model reranks them, cutting generation cost.
  • The paper caps prompts at 35 queries during training; behavior beyond that cap is untested, so a natural extension is measuring per-query accuracy as the query count grows.
  • Section 4.7 concedes that the pre-training phase demands substantial compute and defers masked video modeling to future work, so the open-world advantage is currently demonstrated only at the 0.5B scale with a fixed pre-training budget.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Grounding-MD, a unified video-language pre-training framework for moment detection that bridges temporal action detection (TAD) and moment retrieval (MR) in an open-world setting. The model accepts a structured prompt containing an arbitrary number of natural-language queries, encodes video and text with VideoMAE and BERT, and uses a Cross-Modality Fusion Encoder, a Query-Wise Pooler, and a Text-Guided Fusion Decoder to align queries with temporal segments. Training combines HACS, FineAction, and a filtered InternVid subset; evaluation covers ActivityNet, THUMOS14, ActivityNet-Captions, and Charades-STA under both zero-shot and supervised protocols. The paper reports state-of-the-art or competitive results in most of these settings and emphasizes the model's efficiency relative to 7B video-LLMs.

Significance. If the claims hold, Grounding-MD would be a valuable step toward open-vocabulary moment detection: it is substantially smaller than video-LLM baselines, shows strong zero-shot transfer on moment retrieval, and produces consistent gains in supervised fine-tuning. The design ideas—early and late cross-modal fusion, query-wise pooling to balance TAD/MR text lengths, and structured prompts with sampled negatives—are clearly presented, and the ablations in Table 8 support the contribution of the proposed modules. However, the central evidence for open-vocabulary temporal action detection (THUMOS14, Table 4) is undermined by category overlap with the HACS pre-training corpus, so that portion of the open-world claim is not currently supported. The external zero-shot MR results (Table 6) and the supervised results remain informative and should be credited.

major comments (3)
  1. [Section 4.4, Table 4; Sections 3.3 and 4.3] The claimed open-vocabulary zero-shot evaluation on THUMOS14 is not category-disjoint from pre-training. The model is pre-trained on HACS (Section 3.3, Table 1), and Section 4.3 states that HACS 'encompasses all categories present in ActivityNet.' THUMOS14 action categories are a subset of ActivityNet categories, so the categories marked 'unseen' in the 75/25 and 50/50 splits have very likely been seen with temporal annotations during pre-training. The no-pre-training row (G-MD-S w/o pt) versus the pre-trained row (G-MD-S (B) w/ pt) in Table 4 shows a jump from 19.7 to 27.1 Avg (75/25 split) and from 11.9 to 26.0 Avg (50/50 split), exactly the pattern one would expect from memorizing the visual and textual patterns of the 'unseen' classes. The paper reports no check that the seen/unseen split is disjoint from HACS, FineAction, or InternVid-MR categories. As a result, Table 4 cannot support the paper's open-world zero-shot generalization claim. Please re-run the evaluation on categories verified absent from all pre-training data, or explicitly remove or reword the open-vocabulary TAD claim.
  2. [Section 4.5, Table 6] The text states that Grounding-MD 'achieved superior performance compared to Video-LLM-based methods,' but Table 6 shows VideoChat2 achieving 27.8 R@0.5 on ActivityNet-Captions versus 25.3 for G-MD-L; G-MD-L is better on the other three reported metrics. The comparison is therefore selective, and the claim of overall superiority should be qualified to specific metrics and datasets. Please report the full set of metrics with an explicit per-metric comparison, and if variance across evaluation subsets is available, include it.
  3. [Section 4.3, Table 3] The 'zero-shot domain transfer' evaluation on ActivityNet is not an open-vocabulary test: because HACS shares the full ActivityNet category set, no category is unseen. The comparison with AdaTAD* is a valid cross-dataset transfer comparison with seen categories, but it should not be described as evidence for open-world category generalization. The paper should either relabel this setting as seen-category domain transfer or provide an additional evaluation on a dataset whose categories are disjoint from all pre-training data.
minor comments (6)
  1. [Section 4.4] The average results across the 10 random samplings are reported without standard deviation; given the sensitivity of category-split results, please include variance or confidence intervals.
  2. [Equation 9] The notation in Eq. (9) is ambiguous: the sum over t is divided by T^+, and the role of the indicator P_t in the denominator is unclear. Please clarify the normalization.
  3. [Section 3.2] The Query-Wise Pooler relies on a 'query-wise split' of the text features, but the paper does not specify how the token boundaries for each query are determined after the queries are concatenated into a structured prompt. Please clarify this step.
  4. [Section 4.2] The maximum text token length is 512 and the prompt is restricted to 35 queries during training; the phrase 'arbitrary number of open-ended natural language queries' should be qualified accordingly.
  5. [Table 5] The TriDet row in Table 5 contains a formatting error ('62.447.469.3') that should be corrected to separate mAP values.
  6. [Section 4.7] The limitations section discusses computational cost but does not mention the category-overlap issue with THUMOS14 and HACS; this limitation should be disclosed.

Circularity Check

1 steps flagged · score 5.0 of 10

THUMOS14 'unseen' categories overlap with HACS pre-training data, so the open-vocabulary zero-shot claim in Table 4 is partially circular; the MR zero-shot and supervised results remain independent.

  1. fitted input called prediction [Section 4.4 (Open-Vocabulary Zero-Shot on THUMOS14), relying on Section 3.3/Table 2 pre-training on HACS and Section 4.3's admission that HACS encompasses all ActivityNet categories.]
    "Given that HACS [58] encompasses all categories present in ActivityNet [4] ... open-vocabulary zero-shot, which assesses the ability to generalize to unseen action categories"

    Grounding-MD is pre-trained on HACS (Section 3.3, Table 2), and Section 4.3 states that HACS encompasses all ActivityNet categories. THUMOS14 action categories are a subset of ActivityNet categories, so the THUMOS14 categories designated 'unseen' in the Section 4.4 split were very likely present in the HACS pre-training data, including temporal annotations. The 'open-vocabulary zero-shot' evaluation in Table 4 therefore measures transfer to categories already seen during pre-training, not generalization to never-seen categories. The no-pre-training row (19.7/11.9 Avg) versus pre-trained (27.1/26.0 Avg) jump is exactly the signature of category leakage.

full rationale

The core architecture and training objective of Grounding-MD are not circular: the structured-prompt formulation, the Cross-Modality Fusion Encoder, the Text-Guided Fusion Decoder, and the contrastive-plus-DIoU losses are independent design choices, and they are evaluated on external benchmarks in supervised fine-tuning and in zero-shot moment-retrieval transfer to ActivityNet-Captions and Charades-STA. The load-bearing circularity is limited to the open-vocabulary zero-shot THUMOS14 evaluation: because pre-training includes HACS, and HACS is said to encompass all ActivityNet categories, while THUMOS14 categories overlap ActivityNet, the categories labeled 'unseen' in Section 4.4 were likely seen during pre-training. This makes the open-world TAD zero-shot claim in Table 4 partially circular by construction. There are no self-citation chains, no imported uniqueness theorems, and no ansatz smuggled via citation. Score 5 reflects partial circularity in one headline claim, with substantial independent content elsewhere.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the proposed modules (CMFE, TGFD, QWP) are architectural components, not postulates requiring independent evidence.

free parameters (5)
  • UMT-SIM filtering threshold = 0.4
    Chosen by hand to discard low-quality automated annotations in InternVid-10M-FLT; directly determines the composition of InternVid-MR.
  • Keyword set size = over 1000 keywords
    Curated by the authors to filter InternVid-MR toward daily human activities; no keyword list is released.
  • Maximum queries per structured prompt = 35
    Set so that prompts fit within BERT's 512 token limit; affects the contrastive training distribution.
  • Loss balancing coefficient lambda = 1
    Fixed at 1 to combine classification (focal) and regression (DIOU) losses; no sensitivity analysis is reported.
  • Layer counts for CMFE, FPN, TGFD = 3, 5, 6
    Chosen by hand as architectural hyperparameters; no ablation on depth is provided.
assumptions (4)
  • domain assumption VideoMAE with TIA provides a strong video representation backbone whose features are transferable across datasets.
    The model's performance depends on the backbone features; no analysis of backbone pretraining leakage is provided.
  • domain assumption BERT text embeddings are sufficient to represent open-ended natural language queries for moment detection.
    The text backbone is not trained on video-text pairs beyond the fusion modules; no ablation on text encoder choice is given.
  • domain assumption InternVid-10M-FLT automated video-text annotations, after UMT-SIM and keyword filtering, constitute reliable supervision for open-world moment detection.
    The filtering thresholds are heuristic; no manual verification of annotation quality is reported.
  • domain assumption The open-vocabulary evaluation protocol of EffPrompt, applied to THUMOS14, measures open-world generalization.
    This assumes that unseen categories in the split are not seen during pre-training, which is violated by HACS coverage.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Grounding-MD: Grounded Video-language Pre-training for Open-World Moment Detection." pith.science (2026). https://pith.science/paper/RJTFPRNF

@misc{pith2026250414553,
  author       = {Pith},
  title        = {Pith review of: Grounding-MD: Grounded Video-language Pre-training for Open-World Moment Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RJTFPRNF}},
  note         = {Machine review of arXiv:2504.14553}
}
read the original abstract

Temporal Action Detection and Moment Retrieval constitute two pivotal tasks in video understanding, focusing on precisely localizing temporal segments corresponding to specific actions or events. Recent advancements introduced Moment Detection to unify these two tasks, yet existing approaches remain confined to closed-set scenarios, limiting their applicability in open-world contexts. To bridge this gap, we present Grounding-MD, an innovative, grounded video-language pre-training framework tailored for open-world moment detection. Our framework incorporates an arbitrary number of open-ended natural language queries through a structured prompt mechanism, enabling flexible and scalable moment detection. Grounding-MD leverages a Cross-Modality Fusion Encoder and a Text-Guided Fusion Decoder to facilitate comprehensive video-text alignment and enable effective cross-task collaboration. Through large-scale pre-training on temporal action detection and moment retrieval datasets, Grounding-MD demonstrates exceptional semantic representation learning capabilities, effectively handling diverse and complex query conditions. Comprehensive evaluations across four benchmark datasets including ActivityNet, THUMOS14, ActivityNet-Captions, and Charades-STA demonstrate that Grounding-MD establishes new state-of-the-art performance in zero-shot and supervised settings in open-world moment detection scenarios. All source code and trained models will be released.

Figures

Figures reproduced from arXiv: 2504.14553 by the authors.

Figure 1
Figure 1. Illustration of UniMD [54] and Grounding-MD. UniMD operates under a closed-set assumption, limiting its appli￾cability in open-world scenarios. In contrast, Grounding-MD sup￾ports user-defined action categories and open-ended natural lan￾guage event descriptions, enabling it to adapt to diverse and dy￾namic user queries in open-world environments. namics. Two prominent tasks addressing this challenge are temporal ac… view at source ↗
Figure 2
Figure 2. Advantages of Grounding-MD. (a) The early and late fusion strategies achieve optimal video-text alignment, enabling the model to gain a deeper understanding of the video-text data. Moreover, the structured prompt design allows the model to handle an arbitrary number of open-ended natural language queries, demonstrating excellent cross-task collaboration; (b) Through pre-training on large-scale temporal action detect… view at source ↗
Figure 3
Figure 3. Overview of the Grounding-MD framework. The Cross-Modality Fusion Encoder performs early fusion of video and text features, establishing initial cross-modal alignment, while the Text-Guided Fusion Decoder conducts late fusion, refining the alignment through deeper interaction between modalities. Additionally, the Query-Wise Pooler addresses the training instability caused by the disparity in text lengths between act… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 35 canonical work pages

  1. [1]

    Localizing mo- ments in video with natural language

    Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan Russell. Localizing mo- ments in video with natural language. In Proceedings of the IEEE international conference on computer vision, pages 5803–5812, 2017. 3

    2017

  2. [2]

    Boundary content graph neural network for temporal action proposal generation

    Yueran Bai, Yingying Wang, Yunhai Tong, Yang Yang, Qiyue Liu, and Junhui Liu. Boundary content graph neural network for temporal action proposal generation. In Com- puter Vision–ECCV 2020: 16th European Conference, Glas- gow, UK, August 23–28, 2020, Proceedings, Part XXVIII 16, pages 121–137. Springer, 2020. 3

    work page 2020

  3. [3]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, G ¨ul Varol, and Andrew Zisser- man. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 1728–1738,

  4. [4]

    Activitynet: A large-scale video benchmark for human activity understanding

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In Proceed- ings of the ieee conference on computer vision and pattern recognition, pages 961–970, 2015. 2, 6, 8

    work page 2015

  5. [5]

    Conceptual 12m: Pushing web-scale image-text pre- training to recognize long-tail visual concepts

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre- training to recognize long-tail visual concepts. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3558–3568, 2021. 8

    work page 2021

  6. [6]

    Tallformer: Temporal ac- tion localization with a long-memory transformer

    Feng Cheng and Gedas Bertasius. Tallformer: Temporal ac- tion localization with a long-memory transformer. In Eu- ropean Conference on Computer Vision , pages 503–521. Springer, 2022. 6, 7

    work page 2022

  7. [7]

    Vindlu: A recipe for ef- fective video-and-language pretraining

    Feng Cheng, Xizi Wang, Jie Lei, David Crandall, Mohit Bansal, and Gedas Bertasius. Vindlu: A recipe for ef- fective video-and-language pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10739–10750, 2023. 3

    work page 2023

  8. [8]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the asso- ciation for computational linguistics: human language tech- nologies, volume 1 (long and short papers) , pages 4171– 4186, 2019. 6

    work page 2019

Show all 60 references
  1. [9]

    End-to-end learning of motion representation for video understanding

    Lijie Fan, Wenbing Huang, Chuang Gan, Stefano Ermon, Boqing Gong, and Junzhou Huang. End-to-end learning of motion representation for video understanding. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 6016–6025, 2018. 1

    2018

  2. [10]

    Tall: Temporal activity localization via language query

    Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on com- puter vision, pages 5267–5275, 2017. 2, 6, 8

    2017

  3. [11]

    in the wild

    Haroon Idrees, Amir R Zamir, Yu-Gang Jiang, Alex Gorban, Ivan Laptev, Rahul Sukthankar, and Mubarak Shah. The thumos challenge on action recognition for videos “in the wild”. Computer Vision and Image Understanding , 155:1– 23, 2017. 2, 6, 7

    2017

  4. [12]

    T-rex2: Towards generic object detec- tion via text-visual prompt synergy

    Qing Jiang, Feng Li, Zhaoyang Zeng, Tianhe Ren, Shilong Liu, and Lei Zhang. T-rex2: Towards generic object detec- tion via text-visual prompt synergy. InEuropean Conference on Computer Vision, pages 38–57. Springer, 2024. 3

    2024

  5. [13]

    Prompting visual-language models for efficient video understanding

    Chen Ju, Tengda Han, Kunhao Zheng, Ya Zhang, and Weidi Xie. Prompting visual-language models for efficient video understanding. In European Conference on Computer Vi- sion, pages 105–124. Springer, 2022. 7

    2022

  6. [14]

    Dense-captioning events in videos

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense-captioning events in videos. In Proceedings of the IEEE international conference on com- puter vision, pages 706–715, 2017. 2, 6, 8

    2017

  7. [15]

    Videochat: Chat-centric video understanding

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355, 2023. 8

    2023 arXiv

  8. [16]

    Unmasked teacher: Towards training-efficient video foundation models

    Kunchang Li, Yali Wang, Yizhuo Li, Yi Wang, Yinan He, Limin Wang, and Yu Qiao. Unmasked teacher: Towards training-efficient video foundation models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 19948–19960, 2023. 5

    2023

  9. [17]

    Mvbench: A comprehensive multi-modal video understand- ing benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understand- ing benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195–...

    2024

  10. [18]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre-training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, page...

    2022

  11. [19]

    Detal: open-vocabulary temporal action lo- calization with decoupled networks

    Zhiheng Li, Yujie Zhong, Ran Song, Tianjiao Li, Lin Ma, and Wei Zhang. Detal: open-vocabulary temporal action lo- calization with decoupled networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 7

    2024

  12. [20]

    Learning salient boundary feature for anchor- free temporal action localization

    Chuming Lin, Chengming Xu, Donghao Luo, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Yan- wei Fu. Learning salient boundary feature for anchor- free temporal action localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern reco...

    2021

  13. [21]

    Tsm: Temporal shift module for efficient video understanding

    Ji Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7083–7093, 2019. 1

    2019

  14. [22]

    Univtg: Towards unified video- language temporal grounding

    Kevin Qinghong Lin, Pengchuan Zhang, Joya Chen, Shra- man Pramanick, Difei Gao, Alex Jinpeng Wang, Rui Yan, and Mike Zheng Shou. Univtg: Towards unified video- language temporal grounding. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2794–...

    2023

  15. [23]

    Single shot tempo- ral action detection

    Tianwei Lin, Xu Zhao, and Zheng Shou. Single shot tempo- ral action detection. In Proceedings of the 25th ACM inter- national conference on Multimedia , pages 988–996, 2017. 3

    2017

  16. [24]

    Bmn: Boundary-matching network for temporal action pro- posal generation

    Tianwei Lin, Xiao Liu, Xin Li, Errui Ding, and Shilei Wen. Bmn: Boundary-matching network for temporal action pro- posal generation. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 3889–3898,

  17. [25]

    Focal loss for dense object detection

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ar. Focal loss for dense object detection. In Pro- ceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017. 5

    2017

  18. [26]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision , pages 38–55. Springe...

    2024

  19. [27]

    End-to-end temporal action detection with 1b parameters across 1000 frames

    Shuming Liu, Chen-Lin Zhang, Chen Zhao, and Bernard Ghanem. End-to-end temporal action detection with 1b parameters across 1000 frames. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18591–18601, 2024. 1, 3, 6, 7

    2024

  20. [28]

    An empirical study of end-to-end temporal action detection

    Xiaolong Liu, Song Bai, and Xiang Bai. An empirical study of end-to-end temporal action detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20010–20019, 2022. 6, 7

    2022

  21. [29]

    End-to-end temporal ac- tion detection with transformer.IEEE Transactions on Image Processing, 31:5427–5441, 2022

    Xiaolong Liu, Qimeng Wang, Yao Hu, Xu Tang, Shiwei Zhang, Song Bai, and Xiang Bai. End-to-end temporal ac- tion detection with transformer.IEEE Transactions on Image Processing, 31:5427–5441, 2022. 6, 7

    2022

  22. [30]

    Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection

    Ye Liu, Siyuan Li, Yang Wu, Chang-Wen Chen, Ying Shan, and Xiaohu Qie. Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3042–3051, 2022. 3

    2022

  23. [31]

    Fineaction: A fine-grained video dataset for temporal action localization

    Yi Liu, Limin Wang, Yali Wang, Xiao Ma, and Yu Qiao. Fineaction: A fine-grained video dataset for temporal action localization. IEEE transactions on image processing , 31: 6937–6950, 2022. 5

    2022

  24. [32]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 6

    2017 arXiv

  25. [33]

    Towards generalisable video moment retrieval: Visual-dynamic injection to image-text pre-training

    Dezhao Luo, Jiabo Huang, Shaogang Gong, Hailin Jin, and Yang Liu. Towards generalisable video moment retrieval: Visual-dynamic injection to image-text pre-training. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23045–23055, 2023. 8

    2023

  26. [34]

    Valley: Video assistant with large language model enhanced ability

    Ruipu Luo, Ziwang Zhao, Min Yang, Junwei Dong, Da Li, Pengcheng Lu, Tao Wang, Linmei Hu, Minghui Qiu, and Zhongyu Wei. Valley: Video assistant with large language model enhanced ability. arXiv preprint arXiv:2306.07207 ,

    arXiv

  27. [35]

    Local- global video-text interactions for temporal grounding

    Jonghwan Mun, Minsu Cho, and Bohyung Han. Local- global video-text interactions for temporal grounding. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 10810–10819, 2020. 3

    2020

  28. [36]

    Zero-shot temporal action detection via vision-language prompting

    Sauradip Nag, Xiatian Zhu, Yi-Zhe Song, and Tao Xi- ang. Zero-shot temporal action detection via vision-language prompting. In European conference on computer vision , pages 681–697. Springer, 2022. 7

    2022

  29. [37]

    Momen- tor: Advancing video large language model with fine-grained temporal reasoning

    Long Qian, Juncheng Li, Yu Wu, Yaobo Ye, Hao Fei, Tat- Seng Chua, Yueting Zhuang, and Siliang Tang. Momen- tor: Advancing video large language model with fine-grained temporal reasoning. arXiv preprint arXiv:2402.11435, 2024. 1, 7, 8

    2024 arXiv

  30. [38]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  31. [39]

    Action sensitivity learning for temporal action localization

    Jiayi Shao, Xiaohan Wang, Ruijie Quan, Junjun Zheng, Jiang Yang, and Yi Yang. Action sensitivity learning for temporal action localization. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 13457–13469,

  32. [40]

    Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning. In Pro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    2018

  33. [41]

    Tridet: Temporal action detection with relative boundary modeling

    Dingfeng Shi, Yujie Zhong, Qiong Cao, Lin Ma, Jia Li, and Dacheng Tao. Tridet: Temporal action detection with relative boundary modeling. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 18857–18866, 2023. 1, 3, 6, 7

    2023

  34. [42]

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural information processing systems, 35:10078–10093, 2022. 6

    2022

  35. [43]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 4

    2017

  36. [44]

    Actionclip: A new paradigm for video action recognition

    Mengmeng Wang, Jiazheng Xing, and Yong Liu. Actionclip: A new paradigm for video action recognition. arXiv preprint arXiv:2109.08472, 2021. 3

    2021 arXiv

  37. [45]

    Internvideo: General video foundation models via generative and discriminative learning

    Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.03191, 2022. 6, 7

    2022 arXiv

  38. [46]

    Internvid: A large-scale video-text dataset for multimodal understanding and generation

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942, 2023. 1, 3, 5, 8

    2023 arXiv

  39. [47]

    Learning to refactor action and co-occurrence fea- tures for temporal action localization

    Kun Xia, Le Wang, Sanping Zhou, Nanning Zheng, and Wei Tang. Learning to refactor action and co-occurrence fea- tures for temporal action localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13884–13893, 2022. 3

    2022

  40. [48]

    Unloc: A unified framework for video localiza- tion tasks

    Shen Yan, Xuehan Xiong, Arsha Nagrani, Anurag Arnab, Zhonghao Wang, Weina Ge, David Ross, and Cordelia Schmid. Unloc: A unified framework for video localiza- tion tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13623–13633, 2023. 8

    2023

  41. [49]

    Basictad: an astounding rgb-only baseline for tem- poral action detection

    Min Yang, Guo Chen, Yin-Dong Zheng, Tong Lu, and Limin Wang. Basictad: an astounding rgb-only baseline for tem- poral action detection. Computer Vision and Image Under- standing, 232:103692, 2023. 6, 7

    2023

  42. [50]

    Detclipv3: To- wards versatile generative open-vocabulary object detection

    Lewei Yao, Renjie Pi, Jianhua Han, Xiaodan Liang, Hang Xu, Wei Zhang, Zhenguo Li, and Dan Xu. Detclipv3: To- wards versatile generative open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27391–27401, 2024. 3

    2024

  43. [51]

    Semantic conditioned dynamic modulation for tempo- ral sentence grounding in videos

    Yitian Yuan, Lin Ma, Jingwen Wang, Wei Liu, and Wenwu Zhu. Semantic conditioned dynamic modulation for tempo- ral sentence grounding in videos. Advances in Neural Infor- mation Processing Systems, 32, 2019. 3

    2019

  44. [52]

    Graph con- volutional networks for temporal action localization

    Runhao Zeng, Wenbing Huang, Mingkui Tan, Yu Rong, Peilin Zhao, Junzhou Huang, and Chuang Gan. Graph con- volutional networks for temporal action localization. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 7094–7103, 2019. 3

    2019

  45. [53]

    Dense regression network for video grounding

    Runhao Zeng, Haoming Xu, Wenbing Huang, Peihao Chen, Mingkui Tan, and Chuang Gan. Dense regression network for video grounding. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 10287–10296, 2020. 3

    2020

  46. [54]

    Unimd: Towards unifying moment retrieval and temporal ac- tion detection

    Yingsen Zeng, Yujie Zhong, Chengjian Feng, and Lin Ma. Unimd: Towards unifying moment retrieval and temporal ac- tion detection. In European Conference on Computer Vision, pages 286–304. Springer, 2024. 1, 2, 3, 6

    2024

  47. [55]

    Actionformer: Lo- calizing moments of actions with transformers

    Chen-Lin Zhang, Jianxin Wu, and Yin Li. Actionformer: Lo- calizing moments of actions with transformers. In European Conference on Computer Vision , pages 492–510. Springer,

  48. [56]

    Video-llama: An instruction-tuned audio-visual language model for video un- derstanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video un- derstanding. arXiv preprint arXiv:2306.02858, 2023. 8

    2023 arXiv

  49. [57]

    Learning 2d temporal adjacent networks for moment local- ization with natural language

    Songyang Zhang, Houwen Peng, Jianlong Fu, and Jiebo Luo. Learning 2d temporal adjacent networks for moment local- ization with natural language. In Proceedings of the AAAI conference on artificial intelligence , pages 12870–12877,

  50. [58]

    Hacs: Human action clips and segments dataset for recognition and temporal localization

    Hang Zhao, Antonio Torralba, Lorenzo Torresani, and Zhicheng Yan. Hacs: Human action clips and segments dataset for recognition and temporal localization. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 8668–8678, 2019. 5, 6

    2019

  51. [59]

    Distance-iou loss: Faster and bet- ter learning for bounding box regression

    Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren. Distance-iou loss: Faster and bet- ter learning for bounding box regression. In Proceedings of the AAAI conference on artificial intelligence, pages 12993– 13000, 2020. 5

    2020

  52. [60]

    Enriching local and global contexts for temporal action localization

    Zixin Zhu, Wei Tang, Le Wang, Nanning Zheng, and Gang Hua. Enriching local and global contexts for temporal action localization. In Proceedings of the IEEE/CVF international conference on computer vision, pages 13516–13525, 2021. 3

    2021

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.