Pith. sign in

REVIEW 2 major objections 4 minor 51 references

Sparse-Dense Side-Tuner for efficient Video Temporal Grounding

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new sparse-dense side-tuner for video temporal grounding, SDST, is anchor-free and matches or beats state-of-the-art DETR-style methods on QVHighlights, TACoS, and Charades-STA while using only about 27% of the parameters.

desk verdict A genuinely useful side-tuning architecture, but the headline QVHighlights val numbers are internally inconsistent, so the SOTA claim is currently unverifiable. read the letter →

arxiv 2507.07744 v1 pith:3Z3LKA2C submitted 2025-07-10 cs.CV

David Pujol-Perich , Sergio Escalera , Albert Clapés This is my paper · ORCID
classification cs.CV
keywords VideotemporalgroundingMomentretrievalHighlightdetectionSide-tuningDeformableattentionParameter-efficientfine-tuningInternVideo2Recurrentdecoderqueries
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video temporal grounding (VTG) asks a model to find the moments a text query describes in a video, both by retrieving the time interval (moment retrieval, MR) and by scoring each frame's relevance (highlight detection, HD). Most competitive models use DETR-style decoders over final-layer features of a frozen image-text backbone, which limits adaptation to video domains and is parameter-heavy. The paper introduces SDST, a side-tuning architecture that is anchor-free—the first for VTG—and that refines frozen intermediate features from InternVideo2 through a shared dual stream: a dense stream for frame-level highlights and a sparse stream of learnable recurrent moment queries. It claims SDST beats prior side-tuning methods and matches or surpasses state-of-the-art DETR methods on QVHighlights, TACoS, and Charades-STA while using about 27% of the parameters of the strongest competitor. The central insight is that deformable attention only works naturally as self-attention, not cross-attention, because cross-attention queries lack local context of the key space—a limitation fixed by the paper's Reference-based Deformable Self-Attention.

What carries the argument

Reference-based Deformable Self-Attention (RDSA), a deformable-attention variant that uses reference-derived query embeddings instead of the raw decoder queries. For each moment proposal, a learned CNN gives the dense frame embeddings local context, then bilinear sampling extracts three action embeddings—left-most, center, right-most—which are concatenated and linearly projected to predict the deformable offsets and attention scores. Because these query embeddings are derived from the same dense features as the keys, the query and key spaces coincide, turning the deformable cross-attention into a deformable self-attention and providing the local context the standard mechanism lacks. The paper's other load-bearing mechanism is the cross-layer reuse of InternVideo2's frozen AdaptivePool module: the same last-layer pooling function is applied at every sampled intermediate layer, avoiding backprop and extra memory at the cost of a distribution shift between intermediate and last-layer features, a trade-off the authors call the 'depth-pooling trade-off'.

What would settle it

Take the trained SDST and replace the frozen AdaptivePool with a per-layer learned pooling module (or with no pooling reuse at all, e.g., average pooling) on the same three benchmarks; if the resulting accuracy does not drop below the reported numbers, the claim that AdaptivePool reuse is the enabling choice would be falsified. A more direct check is to measure the feature distribution distance between intermediate-layer pooled outputs and last-layer pooled outputs and test whether the performance gap across the paper's pooling ablation correlates with that distance.

Watch

Extended reading notes

Core claim

The paper's central claim is that a parameter- and memory-efficient side-tuner can reach the accuracy of conventional DETR-style VTG systems. SDST recursively applies a weight-shared dual-stream module to K intermediate visual and textual embeddings of a frozen InternVideo2 backbone. The dense stream produces frame-level embeddings for highlight detection, while the sparse stream maintains M learnable center-width moment proposals with latent embeddings, refined across levels for moment retrieval. The key discovery is diagnostic: in the deformable attention used by anchor-free methods, the query is fully independent of the key/value space, so when used as cross-attention its offset predictions collapse near initialization and cannot look beyond the current estimated boundaries. SDST replaces the queries with bilinearly sampled left-most, center, and right-most action embeddings extracted from a CNN-refined dense embedding, converting deformable cross-attention into deformable self-attention while keeping the efficiency of sparse key sampling. The paper also shows that InternVideo2's frozen AdaptivePool—trained only for the last layer—can be reused across intermediate layers to pool spatio-temporal tokens, and that this pooling choice is worth several points over CLS pooling. With these pieces, SDST reports highly competitive or state-of-the-art results on QVHighlights, TACoS, and Charades-STA at 4.1M trainable parameters, about 27% of the parameters of the SG-DETR baseline.

Load-bearing premise

The reported accuracy depends on reusing InternVideo2's frozen AdaptivePool, which was trained only for the last layer, on intermediate layers; if that pooling transfer causes a harmful distribution shift, the gains attributed to the backbone integration would shrink or vanish.

Editorial extensions

If this is right

  • On QVHighlights val, SDST improves the previous side-tuning method R2-Tuning by 3.82% average mAP on moment retrieval and 2.21% mAP on highlight detection, and is statistically indistinguishable from the SG-DETR state of the art while using about a quarter of its parameters.
  • On Charades-STA and TACoS, SDST sets a new state of the art: for example, R1@0.7 improves by 2.71% on Charades-STA and 2.39% on TACoS over the previous best results.
  • The paper shows RDSA's offsets move beyond the currently estimated moment boundaries, which makes it especially effective for long actions: mAP on long actions improves by 2.84% over the standard deformable cross-attention.
  • The dual-stream interaction is bidirectional in practice: conditioning the sparse stream on the dense stream's non-linear output improves both moment retrieval and highlight detection compared to conditioning on raw video features.
  • Reusing InternVideo2's frozen AdaptivePool across intermediate layers outperforms CLS pooling by 5.07% average mAP on MR and 7.74% HIT@1 on HD, establishing that pooling strategy is decisive for side-tuning with spatio-temporal backbones.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's diagnosis of deformable attention—that it is naturally suited to self-attention and loses context in cross-attention—is a generic mechanism claim, not limited to VTG; a direct test is to swap deformable cross-attention for an RDSA-style reference-derived self-attention in an image detection decoder and measure whether offset collapse disappears.
  • The depth-pooling trade-off points to a concrete follow-up: distilling the last-layer AdaptivePool into lightweight per-layer pooling heads on a small proxy set could remove the distribution shift and let SDST exploit shallower layers without full backpropagation.
  • Because the paper's Fig. 3 shows last-layer-only features with two or three refinement steps can beat intermediate features, part of the improvement attributed to 'intermediate features' may actually come from the number of recurrent refinement steps; separating these two variables is a natural ablation the paper does not fully isolate.
  • The authors leave cross-domain generalization for future work; a straightforward experiment would be training SDST on Charades-STA and testing on TACoS (and vice versa) to see whether the parameter savings and sparse-dense multitask training also improve robustness to domain shift.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes SDST, a side-tuning (ST) architecture for video temporal grounding (VTG) that combines a dense stream for highlight detection (HD) with a sparse stream for moment retrieval (MR). Three contributions are claimed: (1) the first anchor-free ST architecture for VTG, based on recurrent decoder queries; (2) a new Reference-based Deformable Self-Attention (RDSA) mechanism that addresses a context limitation of standard deformable cross-attention; and (3) the first effective integration of the InternVideo2 backbone into an ST framework, via reusing its frozen AdaptivePool module across intermediate layers. Experiments on QVHighlights, TACoS, and Charades-STA report highly competitive or state-of-the-art results with a parameter count of 4.1M, roughly 27% of SG-DETR's 15M parameters.

Significance. If the reported numbers are reproducible, the paper makes a useful contribution to parameter- and memory-efficient VTG. The efficiency claim is substantial and well supported by the parameter counts and memory measurements in Tables 1, D, and H. The RDSA idea is technically interesting and is backed by a careful ablation (Tables 5, M, N) and an offset analysis (Fig. 4). The paper also ships public code and includes an extensive ablation study, a statistical significance analysis, and a discussion of a previously underappreciated depth-pooling trade-off. These strengths make the work of potential interest to the VTG and efficient-transfer-learning communities, provided the main results are consistent across the paper.

major comments (2)
  1. [§5.2 (Table 1) vs. §E (Table B)] The QVHighlights val results for the proposed method are mutually inconsistent between the main text and the supplementary material. Table 1 reports Ours† with R1@0.5 = 73.68, R1@0.7 = 60.90, and mAP = 55.60. Table B, which evaluates the same model on three seeds with InternVideo2-1B features, reports R1@0.5 = 73.20 ± 0.23, R1@0.7 = 56.76 ± 0.53, and mAP = 55.31 ± 0.23. The R1@0.7 gap of 4.14 is over ten times the reported standard error of the mean (0.53/√3 ≈ 0.31) and cannot be explained by seed variation. The SG-DETR row matches exactly between the two tables (57.91), so the inconsistency is specific to the proposed method. The central claims that the method 'surpasses on several metrics' and is 'statistically on par' with SG-DETR rely directly on the Table 1 numbers. The authors must clarify which numbers are canonical, reconcile the two tables, and re-evaluate the conclusions if the lower numbers from Table B are the correct ones.
  2. [§M.1] The Friedman test section contains an inverted statistical statement. The text reports χ²_F = 5.640 with p = 0.933 and then states 'Hence, we can reject the null hypothesis, and conclude that there is no significant difference.' With p > 0.05, the correct conclusion is that the null hypothesis of no difference cannot be rejected. As written, the sentence is contradictory: rejecting the null would imply the opposite of the stated conclusion. This error undermines the credibility of the statistical analysis and should be corrected. The Nemenyi test in §M.2 appears to be computed correctly, but the Friedman test explanation needs to be fixed.
minor comments (4)
  1. [§3.3.4] The sentence 'which is our goal in this this work' contains a duplicated word 'this' and should be corrected.
  2. [Table 1 and Table B] The main results table carries no error bars, while the supplementary includes mean±std over three seeds. Given the large discrepancy between the two, the authors should report error bars in the main text or add a clear footnote explaining which run the main-table numbers correspond to.
  3. [§H] The comparison with other PEFT methods is limited to side-tuning baselines; adapter-, LoRA-, and prompt-based methods are not evaluated due to computational constraints. The paper would be strengthened by citing existing experimental comparisons against such methods (e.g., from R2-Tuning) to justify the claim that ST methods are competitive.
  4. [Throughout] The spelling 'R 2-Tuning' is used inconsistently; the paper should use a single consistent form, preferably matching the reference title 'R2-Tuning'.

Circularity Check

0 steps flagged · score 2.0 of 10

No equation-level circularity: the SDST architecture and losses are defined from external backbone features and trainable parameters, and benchmark results come from external datasets; the only self-citation is minor and not load-bearing.

full rationale

The paper's derivation is self-contained. The architecture is defined by Eqs. (1)-(15) from frozen InternVideo2 intermediate features and trainable side-tuner parameters; the RDSA module (Eqs. 10-11) is an architectural modification trained with the losses in Eqs. (12)-(14), and all benchmark numbers come from external datasets compared against external baselines. The only self-citation is [33] (SADA, same authors), used to motivate the general claim that large distribution shifts hurt transfer learning; that claim is not a premise of any equation or of the reported results, so the citation is not load-bearing and does not constitute circularity. The re-use of InternVideo2's frozen AdaptivePool across intermediate layers (Eq. 15) is explicitly presented as a hypothesis and is empirically tested against CLS and average pooling (Sec. 6.1, Tab. 3), so no component is defined in terms of the target result. The selection of K, loss weights, and learning rate on the QVHighlights val split is a model-selection caveat, not a reduction of a prediction to its input. The discrepancy between Table 1 (val R1@0.7=60.90) and Table B (56.76±0.53) is a reproducibility concern for the SOTA claim, but it is not a circular derivation. Accordingly, no circular step is identified; the score of 2 reflects only the presence of a minor, non-load-bearing self-citation per the scoring rubric.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard dataset and backbone assumptions plus one internal assumption: re-using the frozen AdaptivePool on intermediate layers. No new physical or independent entity is introduced.

free parameters (6)
  • Number of refinement levels K = 4
    Selected by ablation in Fig. 3 and Table K; central to the recurrent design.
  • Decoder queries M = 30 for QVHighlights and Charades-STA, 5 for TACoS
    Set per dataset and controls the capacity of the sparse stream.
  • Loss weights lambda = lambda1...lambda7 set to 1.0, 0.1, or tuned during grid search
    Final weights in Table G; robustness is checked with random permutations.
  • Learning rate = 1e-4 for QVHighlights and TACoS, 2.5e-4 for Charades-STA
    Optimized by grid search, as stated in Sec. A.
  • Hidden dimension = 256
    Architectural capacity choice following prior side-tuning work.
  • Deformable sampling points = 4
    Number of selected keys per query in RDSA, from Table A.
assumptions (6)
  • standard math Quadratic bilinear sampling is differentiable and sufficient for extracting reference point embeddings.
    Used in Eq. 10 to sample left, center, and right embeddings; standard in deformable attention literature.
  • standard math Hungarian matching produces a valid one-to-one assignment between queries and ground-truth moments.
    Used in Sec. 3.4 and C.2 for the DETR-style matching loss.
  • domain assumption The annotations in QVHighlights, TACoS, and Charades-STA are correct and representative of the task.
    All experiments and conclusions depend on these three datasets, described in Sec. B.
  • domain assumption Synthetic saliency labels for TACoS are a valid proxy for highlight detection.
    Sec. B assigns saliency 1 to action frames and 0 otherwise to create HD supervision.
  • ad hoc to paper InternVideo2's frozen AdaptivePool module, trained on the last layer, can be re-used on intermediate layers.
    Eq. 15 in Sec. 4; the paper calls this a hypothesis and supports it empirically, but it is load-bearing.
  • ad hoc to paper Shallower layers would add complementary information but suffer a distribution shift with the frozen AdaptivePool.
    Sec. 6.1.3 introduces this 'depth-pooling trade-off' conjecture to explain the layer sampling results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sparse-Dense Side-Tuner for efficient Video Temporal Grounding." pith.science (2026). https://pith.science/paper/3Z3LKA2C

@misc{pith2026250707744,
  author       = {Pith},
  title        = {Pith review of: Sparse-Dense Side-Tuner for efficient Video Temporal Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3Z3LKA2C}},
  note         = {Machine review of arXiv:2507.07744}
}
read the original abstract

Video Temporal Grounding (VTG) involves Moment Retrieval (MR) and Highlight Detection (HD) based on textual queries. For this, most methods rely solely on final-layer features of frozen large pre-trained backbones, limiting their adaptability to new domains. While full fine-tuning is often impractical, parameter-efficient fine-tuning -- and particularly side-tuning (ST) -- has emerged as an effective alternative. However, prior ST approaches this problem from a frame-level refinement perspective, overlooking the inherent sparse nature of MR. To address this, we propose the Sparse-Dense Side-Tuner (SDST), the first anchor-free ST architecture for VTG. We also introduce the Reference-based Deformable Self-Attention, a novel mechanism that enhances the context modeling of the deformable attention -- a key limitation of existing anchor-free methods. Additionally, we present the first effective integration of InternVideo2 backbone into an ST framework, showing its profound implications in performance. Overall, our method significantly improves existing ST methods, achieving highly competitive or SOTA results on QVHighlights, TACoS, and Charades-STA, while reducing up to a 73% the parameter count w.r.t. the existing SOTA methods. The code is publicly accessible at https://github.com/davidpujol/SDST.

Figures

Figures reproduced from arXiv: 2507.07744 by the authors.

Figure 1
Figure 1. Comparison of our proposed method and the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our method (left) first processes the video and textual inputs using [40], and then recursively applies a shared [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Ablation of the number of refinement levels and [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Average of the weighted offsets across M de￾coder queries and N batch elements for the K = 4 refine￾ment levels. Here head 0 (left) is initialized near the left boundary, and head 1 (right) near the right boundary. ferent decoder-query initialization strategies [48] fo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 35 canonical work pages

  1. [1]

    Joint visual and audio learning for video highlight detection

    Taivanbat Badamdorj, Mrigank Rochan, Yang Wang, and Li Cheng. Joint visual and audio learning for video highlight detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8127–8137, 2021

    work page 2021

  2. [2]

    Flashvtg: Feature layering and adaptive score handling network for video temporal grounding

    Zhuo Cao, Bingqing Zhang, Heming Du, Xin Yu, Xue Li, and Sen Wang. Flashvtg: Feature layering and adaptive score handling network for video temporal grounding. arXiv preprint arXiv:2412.13441, 2024

    arXiv 2024

  3. [3]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020

    work page 2020

  4. [4]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211, 2019

    2019

  5. [5]

    The use of ranks to avoid the assumption of normality implicit in the analysis of variance

    Milton Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the american statistical association, 32(200):675–701, 1937

    work page 1937

  6. [6]

    Tall: Temporal activity localization via language query

    Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on com- puter vision, pages 5267–5275, 2017

    work page 2017

  7. [7]

    Clip-adapter: Better vision-language models with fea- ture adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with fea- ture adapters. International Journal of Computer Vision , 132(2):581–595, 2024

    work page 2024

  8. [8]

    Saliency-guided detr for mo- ment retrieval and highlight detection

    Aleksandr Gordeev, Vladimir Dokholyan, Irina Tolstykh, and Maksim Kuprashevich. Saliency-guided detr for mo- ment retrieval and highlight detection. arXiv preprint arXiv:2410.01615, 2024

    arXiv 2024

Show all 51 references
  1. [9]

    Losa: long-short-range adapter for scaling end-to-end temporal action localization

    Akshita Gupta, Gaurav Mittal, Ahmed Magooda, Ye Yu, Graham W Taylor, and Mei Chen. Losa: long-short-range adapter for scaling end-to-end temporal action localization. arXiv preprint arXiv:2404.01282, 2024

    2024 arXiv

  2. [10]

    Unleash the potential of clip for video highlight detection

    Donghoon Han, Seunghyeon Seo, Eunhwan Park, Seong-Uk Nam, and Nojun Kwak. Unleash the potential of clip for video highlight detection. arXiv preprint arXiv:2404.01745, 2024

    2024 arXiv

  3. [11]

    Parameter-efficient fine-tuning for large models: A comprehensive survey

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608 , 2024

    2024 arXiv

  4. [12]

    Mini-net: Multiple instance ranking network for video highlight detection

    Fa-Ting Hong, Xuanteng Huang, Wei-Hong Li, and Wei- Shi Zheng. Mini-net: Multiple instance ranking network for video highlight detection. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16 , pages 345–360. Springer, 2020

    2020

  5. [13]

    Fifth berkeley symposium on mathematical statistics and probability

    Peter J Huber. Fifth berkeley symposium on mathematical statistics and probability. University of California, 1967

    1967

  6. [14]

    Knowing where to focus: Event-aware transformer for video grounding

    Jinhyun Jang, Jungin Park, Jin Kim, Hyeongjun Kwon, and Kwanghoon Sohn. Knowing where to focus: Event-aware transformer for video grounding. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 13846–13856, 2023

    2023

  7. [15]

    Vi- sual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. In European Conference on Computer Vision, pages 709–727. Springer, 2022

    2022

  8. [16]

    Fractalnet: Ultra-deep neural networks without residuals

    Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Fractalnet: Ultra-deep neural networks without residuals. arXiv preprint arXiv:1605.07648, 2016

    2016 arXiv

  9. [17]

    Bam-detr: Boundary- aligned moment detection transformer for temporal sentence grounding in videos

    Pilhyeon Lee and Hyeran Byun. Bam-detr: Boundary- aligned moment detection transformer for temporal sentence grounding in videos. In European Conference on Computer Vision, pages 220–238. Springer, 2025

    2025

  10. [18]

    Detecting moments and highlights in videos via natural language queries

    Jie Lei, Tamara L Berg, and Mohit Bansal. Detecting moments and highlights in videos via natural language queries. Advances in Neural Information Processing Sys- tems, 34:11846–11858, 2021

    2021

  11. [19]

    Dn-detr: Accelerate detr training by intro- ducing query denoising

    Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, and Lei Zhang. Dn-detr: Accelerate detr training by intro- ducing query denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 13619–13627, 2022

    2022

  12. [20]

    Univtg: Towards unified video- language temporal grounding

    Kevin Qinghong Lin, Pengchuan Zhang, Joya Chen, Shra- man Pramanick, Difei Gao, Alex Jinpeng Wang, Rui Yan, and Mike Zheng Shou. Univtg: Towards unified video- language temporal grounding. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2794–...

    2023

  13. [21]

    Focal loss for dense object detection

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ar. Focal loss for dense object detection. In Pro- ceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017

    2017

  14. [22]

    Frozen clip models are efficient video learners

    Ziyi Lin, Shijie Geng, Renrui Zhang, Peng Gao, Gerard De Melo, Xiaogang Wang, Jifeng Dai, Yu Qiao, and Hong- sheng Li. Frozen clip models are efficient video learners. In European Conference on Computer Vision, pages 388–404. Springer, 2022

    2022

  15. [23]

    Dab-detr: Dynamic anchor boxes are better queries for detr

    Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. Dab-detr: Dynamic anchor boxes are better queries for detr. arXiv preprint arXiv:2201.12329, 2022

    2022 arXiv

  16. [24]

    End-to-end temporal ac- tion detection with transformer.IEEE Transactions on Image Processing, 31:5427–5441, 2022

    Xiaolong Liu, Qimeng Wang, Yao Hu, Xu Tang, Shiwei Zhang, Song Bai, and Xiang Bai. End-to-end temporal ac- tion detection with transformer.IEEE Transactions on Image Processing, 31:5427–5441, 2022

    2022

  17. [25]

    arXiv preprint arXiv:2404.00801, 2024

    Ye Liu, Jixuan He, Wanhua Li, Junsik Kim, Donglai Wei, Hanspeter Pfister, and Chang Wen Chen.r2-tuning: Efficient image-to-video transfer learning for video temporal ground- ing. arXiv preprint arXiv:2404.00801, 2024

    2024 arXiv

  18. [26]

    Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection

    Ye Liu, Siyuan Li, Yang Wu, Chang-Wen Chen, Ying Shan, and Xiaohu Qie. Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3042–3051, 2022

    2022

  19. [27]

    Llava-mr: Large language-and- vision assistant for video moment retrieval

    Weiheng Lu, Jian Li, An Yu, Ming-Ching Chang, Sheng- peng Ji, and Min Xia. Llava-mr: Large language-and- vision assistant for video moment retrieval. arXiv preprint arXiv:2411.14505, 2024

    2024 arXiv

  20. [28]

    The surprising effectiveness of multimodal large language models for video moment retrieval

    Boris Meinardus, Anil Batra, Anna Rohrbach, and Marcus Rohrbach. The surprising effectiveness of multimodal large language models for video moment retrieval. arXiv preprint arXiv:2406.18113, 2024

    2024

  21. [29]

    Correlation-guided query-dependency calibration in video representation learning for temporal grounding.CoRR, 2023

    WonJun Moon, Sangeek Hyun, Su Been Lee, and Jae-Pil Heo. Correlation-guided query-dependency calibration in video representation learning for temporal grounding.CoRR, 2023

    2023

  22. [30]

    Query-dependent video representa- tion for moment retrieval and highlight detection

    WonJun Moon, Sangeek Hyun, SangUk Park, Dongchan Park, and Jae-Pil Heo. Query-dependent video representa- tion for moment retrieval and highlight detection. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23023–23033, 2023

    2023

  23. [31]

    Repre- sentation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  24. [32]

    St-adapter: Parameter-efficient image-to-video transfer learning

    Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hong- sheng Li. St-adapter: Parameter-efficient image-to-video transfer learning. Advances in Neural Information Process- ing Systems, 35:26462–26477, 2022

    2022

  25. [33]

    Sada: Semantic adversarial unsupervised domain adaptation for temporal action localization

    David Pujol-Perich, Albert Clap ´es, and Sergio Escalera. Sada: Semantic adversarial unsupervised domain adaptation for temporal action localization. In 2025 IEEE/CVF Win- ter Conference on Applications of Computer Vision (WACV), pages 9237–9247. IEEE, 2025

    2025

  26. [34]

    Disentangling spatial and temporal learning for efficient image-to-video transfer learning

    Zhiwu Qing, Shiwei Zhang, Ziyuan Huang, Yingya Zhang, Changxin Gao, Deli Zhao, and Nong Sang. Disentangling spatial and temporal learning for efficient image-to-video transfer learning. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 13934–1...

    2023

  27. [35]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  28. [36]

    Coherent multi-sentence video description with variable level of detail

    Anna Rohrbach, Marcus Rohrbach, Wei Qiu, Annemarie Friedrich, Manfred Pinkal, and Bernt Schiele. Coherent multi-sentence video description with variable level of detail. In Pattern Recognition: 36th German Conference, GCPR 2014, M¨unster, Germany, September 2-5, 2014, Proceedi...

    2014

  29. [37]

    Ranking domain- specific highlights by analyzing edited videos

    Min Sun, Ali Farhadi, and Steve Seitz. Ranking domain- specific highlights by analyzing edited videos. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13 , pages 787–802. Springer, 2014

    2014

  30. [38]

    Lst: Lad- der side-tuning for parameter and memory efficient transfer learning

    Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. Lst: Lad- der side-tuning for parameter and memory efficient transfer learning. Advances in Neural Information Processing Sys- tems, 35:12991–13005, 2022

    2022

  31. [39]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  32. [40]

    Internvideo2: Scaling foundation models for mul- timodal video understanding

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al. Internvideo2: Scaling foundation models for mul- timodal video understanding. In European Conference on Computer Vision, pages 396–416. Springer, 2025

    2025

  33. [41]

    Internvideo: General video foundation models via generative and discriminative learning

    Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.03191, 2022

    2022 arXiv

  34. [42]

    Vision transformer with deformable attention

    Zhuofan Xia, Xuran Pan, Shiji Song, Li Erran Li, and Gao Huang. Vision transformer with deformable attention. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4794–4803, 2022

    2022

  35. [43]

    Bridging the gap: A unified video comprehension framework for mo- ment retrieval and highlight detection

    Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Heng- wei Bian, Yatai Ji, Yujiu Yang, and Xiu Li. Bridging the gap: A unified video comprehension framework for mo- ment retrieval and highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  36. [44]

    Cross-category video high- light detection via set-based learning

    Minghao Xu, Hang Wang, Bingbing Ni, Riheng Zhu, Zhen- bang Sun, and Changhu Wang. Cross-category video high- light detection via set-based learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7970–7979, 2021

    2021

  37. [45]

    Unloc: A unified framework for video localiza- tion tasks

    Shen Yan, Xuehan Xiong, Arsha Nagrani, Anurag Arnab, Zhonghao Wang, Weina Ge, David Ross, and Cordelia Schmid. Unloc: A unified framework for video localiza- tion tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13623–13633, 2023

    2023

  38. [46]

    Understanding negative sampling in graph representation learning

    Zhen Yang, Ming Ding, Chang Zhou, Hongxia Yang, Jin- gren Zhou, and Jie Tang. Understanding negative sampling in graph representation learning. In Proceedings of the 26th ACM SIGKDD international conference on knowledge dis- covery & data mining, pages 1666–1676, 2020

    2020

  39. [47]

    Parameter-efficient is not sufficient: Exploring parame- ter, memory, and time efficient adapter tuning for dense pre- dictions

    Dongshuo Yin, Xueting Han, Bin Li, Hao Feng, and Jing Bai. Parameter-efficient is not sufficient: Exploring parame- ter, memory, and time efficient adapter tuning for dense pre- dictions. In Proceedings of the 32nd ACM International Con- ference on Multimedia, pages 1398–1406, 2024

    2024

  40. [48]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022

    2022 arXiv

  41. [49]

    Conditional prompt learning for vision-language mod- els

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language mod- els. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 16816–16825, 2022

    2022

  42. [50]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. Supplementary Material In this supplementary material, we provide additional details and v...

    2010 arXiv

  43. [51]

    Notably, these losses are applied to all the intermediate layers inde- pendently

    video-level alignment 2) layer-wise alignment. Notably, these losses are applied to all the intermediate layers inde- pendently. C.3.1 Video-level contrastive loss At a given level ℓ, this loss enforces similarity be- tween action-relevant frames and their corresponding tex- t...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.