Pith. sign in

REVIEW 3 major objections 1 minor 59 references

GIRL-DETR: Gradient-Isolated Reinforcement Learning for Video Moment Retrieval

T0 review · 3 major / 1 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GIRL-DETR freezes the backbone after supervised training so reinforcement learning can directly optimize tIoU in the detection head of lightweight video moment retrieval models.

desk verdict GIRL-DETR shows a workable way to add RL post-training to a small VMR model by freezing the backbone and tuning only the head, but the gains hinge on whether those frozen features are already good enough. read the letter →

arxiv 2606.00775 v1 pith:EAU3UKR4 submitted 2026-05-30 cs.CV cs.AI

Shihang Zhang , Mingjin Kuai , Ye Wei , Zhen Zhang , Wei Ji This is my paper
classification cs.CVcs.AI
keywords videomomentretrievalreinforcementlearninggradientisolationDETRtemporallocalizationcross-modalinteractiontIoUoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video moment retrieval suffers when continuous surrogate losses fail to align with non-differentiable evaluation metrics such as tIoU, causing late-stage training to stall in suboptimal boundary predictions. The paper demonstrates that a DETR-style architecture first learns aligned video-text features through cross-modal interaction and text-guided gating, then isolates gradients by freezing the backbone and applying three-stage progressive reinforcement learning only to the detection head. This orthogonal separation prevents RL updates from damaging the established feature manifold while still allowing the head to improve localization accuracy. The approach yields measurable gains on Charades-STA, QVHighlights, and TACoS using only minimal additional parameters.

What carries the argument

Gradient isolation via backbone freezing combined with three-stage progressive reinforcement learning (TPRL) applied solely to the detection head after cross-modal interaction and text-guided gating in a DETR framework.

What would settle it

An experiment that applies RL to the full unfrozen model and obtains equal or higher tIoU gains without feature degradation would falsify the necessity of gradient isolation.

Watch

Extended reading notes

Core claim

After supervised convergence, freezing the backbone protects the feature manifold while the detection head applies three-stage progressive reinforcement learning to directly maximize the non-differentiable tIoU metric, thereby resolving surrogate loss degradation through an orthogonal decoupling of state representation and metric optimization.

Load-bearing premise

Freezing the backbone after supervised convergence fully protects the feature manifold while still allowing the detection head alone to produce meaningful improvements through RL without requiring any further adaptation of the earlier cross-modal representations.

Editorial extensions

If this is right

  • Resolves optimization stagnation caused by misalignment between surrogate losses and non-differentiable metrics in late training stages.
  • Enables stable RL post-training on lightweight VMR networks without disrupting fragile cross-modal feature representations.
  • Delivers substantial accuracy gains on Charades-STA, QVHighlights, and TACoS while updating only a small fraction of parameters.
  • Establishes a practical pathway for applying RL to other lightweight detection architectures where direct metric optimization is desired.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same isolation principle may generalize to other multimodal detection tasks where pretrained representations are fragile and evaluation metrics are non-differentiable.
  • Adaptive or partial freezing schedules could be tested to determine whether complete isolation is always optimal or if limited backbone updates become beneficial after the RL phase stabilizes.
  • The method highlights a broader design choice between joint end-to-end optimization and staged training when combining supervised pretraining with reinforcement learning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 1 minor

Summary. The paper introduces GIRL-DETR for video moment retrieval, which performs supervised pre-training with cross-modal interaction (CMI), text-guided gating (TGG), and a transformer encoder-decoder, then freezes the backbone and applies three-stage progressive reinforcement learning (TPRL) via gradient isolation to the detection head alone in order to directly maximize the non-differentiable tIoU metric. The central claim is that this orthogonal decoupling resolves surrogate-loss degradation, protects the learned feature manifold, and yields substantial accuracy gains on Charades-STA, QVHighlights, and TACoS with only minimal parameter updates.

Significance. If the empirical claims hold, the work supplies a practical route for applying RL post-training to lightweight VMR architectures without catastrophic disruption of cross-modal representations, addressing a recognized mismatch between surrogate objectives and evaluation metrics. The explicit separation of representation learning from metric optimization is a clear methodological contribution.

major comments (3)
  1. [Abstract] Abstract: the claim that 'freezing the backbone network ... while the detection head directly optimizes ... tIoU' produces meaningful gains rests on the untested assumption that the supervised cross-modal features already supply high-SNR inputs sufficient for head-only RL; no ablation comparing frozen vs. unfrozen backbones or measuring residual misalignment after the supervised phase is referenced, rendering the orthogonal-decoupling argument unverifiable from the given text.
  2. [Abstract] Abstract: the statement that the method 'achieves substantial accuracy improvements' is presented without any numerical results, tables, or error bars; in the absence of concrete metrics (e.g., R@1, mIoU deltas) or comparisons against the supervised baseline, the magnitude and reliability of the reported gains cannot be assessed.
  3. [Abstract] Abstract: the TPRL strategy is described only at a high level ('three-stage progressive'); the manuscript supplies neither the stage-transition thresholds, the precise reward formulation, nor the gradient-isolation implementation details that would allow verification that the RL update truly leaves the frozen CMI+TGG+encoder manifold untouched.
minor comments (1)
  1. [Abstract] The abstract repeatedly uses the phrase 'lightweight temporal localization framework' without defining the parameter count or architectural constraints that distinguish it from prior DETR-style VMR models.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address each major comment below and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the claim that 'freezing the backbone network ... while the detection head directly optimizes ... tIoU' produces meaningful gains rests on the untested assumption that the supervised cross-modal features already supply high-SNR inputs sufficient for head-only RL; no ablation comparing frozen vs. unfrozen backbones or measuring residual misalignment after the supervised phase is referenced, rendering the orthogonal-decoupling argument unverifiable from the given text.

    Authors: We acknowledge that an explicit ablation comparing frozen versus unfrozen backbones during the RL phase would strengthen the orthogonal-decoupling claim. While the current results show gains with minimal parameter updates after freezing, we agree the assumption requires direct verification. We will add this ablation study in the revised manuscript. revision: yes

  2. Referee: [Abstract] Abstract: the statement that the method 'achieves substantial accuracy improvements' is presented without any numerical results, tables, or error bars; in the absence of concrete metrics (e.g., R@1, mIoU deltas) or comparisons against the supervised baseline, the magnitude and reliability of the reported gains cannot be assessed.

    Authors: We will revise the abstract to include concrete numerical results, such as the specific R@1 and mIoU deltas on Charades-STA, QVHighlights, and TACoS relative to the supervised baseline. revision: yes

  3. Referee: [Abstract] Abstract: the TPRL strategy is described only at a high level ('three-stage progressive'); the manuscript supplies neither the stage-transition thresholds, the precise reward formulation, nor the gradient-isolation implementation details that would allow verification that the RL update truly leaves the frozen CMI+TGG+encoder manifold untouched.

    Authors: We agree that additional details are required for full verification. We will expand the methods section to include the stage-transition thresholds, precise reward formulation, and gradient-isolation implementation details. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected in derivation chain

full rationale

The paper presents a two-phase procedure: supervised training of the full model (CMI + TGG + encoder + head) to convergence, followed by freezing the backbone and applying TPRL only to the detection head to directly maximize the external non-differentiable metric tIoU. No equations, self-definitions, or fitted-parameter renamings are shown that would make the reported accuracy gains tautological with the method's own inputs. The decoupling is an explicit design choice rather than a derived necessity, and the central claim rests on benchmark results rather than load-bearing self-citations or uniqueness theorems imported from prior author work. This is a standard non-circular empirical method description.

Assumptions & free parameters 1 free parameters · 2 assumptions · 0 invented entities

The approach rests on several domain assumptions about feature stability and RL applicability that are not independently evidenced in the provided abstract; no explicit free parameters or new entities are named but the three-stage RL strategy implies tunable components.

free parameters (1)
  • TPRL stage transition thresholds
    The three-stage progressive RL requires unspecified criteria or hyperparameters to move between stages, which must be chosen or fitted.
assumptions (2)
  • domain assumption Freezing the backbone after supervised training protects the established feature manifold from disruption by subsequent RL updates.
    Invoked to justify gradient isolation; if false, the orthogonal decoupling premise collapses.
  • domain assumption The detection head alone can meaningfully optimize tIoU via RL without further updates to cross-modal features.
    Central to the claim that minimal parameter updates suffice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GIRL-DETR: Gradient-Isolated Reinforcement Learning for Video Moment Retrieval." pith.science (2026). https://pith.science/paper/EAU3UKR4

@misc{pith2026260600775,
  author       = {Pith},
  title        = {Pith review of: GIRL-DETR: Gradient-Isolated Reinforcement Learning for Video Moment Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EAU3UKR4}},
  note         = {Machine review of arXiv:2606.00775}
}
read the original abstract

Video Moment Retrieval (VMR) task requires accurately localizing temporal boundaries aligned with natural language queries, but many models suffer from a misalignment between continuous surrogate losses and non-differentiable metrics, leading to optimization stagnation during the late stages of training and trapping boundary predictions in suboptimal solutions. Although Reinforcement Learning (RL) post-training successfully optimizes localization results for large models, applying it directly to lightweight networks easily disrupts the fragile feature representations established during the supervised phase. To overcome this optimization bottleneck, we propose Gradient-Isolated Reinforcement Learning for DETR (GIRL-DETR), introducing RL post-training into a lightweight temporal localization framework for the first time. The input video and text features first establish early alignment through Cross-Modal Interaction (CMI) before entering the transformer encoder. Subsequently, a Text-Guided Gating (TGG) mechanism dynamically injects semantic priors into the queries before the transformer decoder generates candidate proposals, providing high signal-to-noise ratio inputs for temporal prediction. After the supervised training reaches convergence, the backbone network is frozen to protect the feature manifold, while the detection head directly optimizes the non-differentiable evaluation metric tIoU to enhance localization accuracy through a Three-stage Progressive Reinforcement Learning (TPRL) strategy. This approach achieves an orthogonal decoupling of state representation and metric optimization. Experiments on Charades-STA, QVHighlights, and TACoS demonstrate that GIRL-DETR effectively resolves surrogate loss degradation and achieves substantial accuracy improvements with minimal parameter updates, providing a robust new pathway for RL applications in lightweight VMR models.

Figures

Figures reproduced from arXiv: 2606.00775 by the authors.

Figure 1
Figure 1. Loss Degradation and Ranking Collapse. (a) In early training, proxy loss optimization effectively drives metric improvements, despite gradient updates deviating from the optimal path. (b) Near the optimal solution, loss degradation prevents the model from finding effective update directions aligned with the localization metric. This misalignment triggers ranking collapse, erroneously prioritizing suboptimal predicti… view at source ↗
Figure 2
Figure 2. The overall architecture of the proposed GIRL-DETR framework. The pipeline processes video and text inputs through sequential stages of multimodal encoding and semantic-gated decoding to generate segment candidates. After achieving supervised convergence, the backbone is frozen while the detection head is refined via progressive reinforcement learning to optimize the tIoU. or boundary adjustments [20], [42]. For ins… view at source ↗
Figure 3
Figure 3. The gradient isolation strategy during post-training. The parameter space is partitioned into a frozen backbone Θback and an active detection head Θhead to ensure training stability. This configuration prevents feature manifold degradation when the model is subjected to high-variance policy gradients during the reinforcement learning phase. and q1. This provides each query vector with a rich repre￾sentation q2 ∈ R N… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The TPRL paradigm. This strategy aligns the training objective with the evaluation metric by transitioning from proxy losses to direct optimization. The process includes anchor refinement for calibration, reward-weighted regression for precision, and policy gradient op…
Figure 5
Figure 5. Figure 5: Performance evolution of TPRL on QVHighlights. The steady improvement of the core metric mIoU across the three stages demonstrates the effectiveness of TPRL in enhancing temporal localization. our TGG module further boosts R1@0.5 and mAP to 72.13% and 49.38%, respectiv…
Figure 6
Figure 6. Figure 6: Qualitative comparison of localization results on QVHighlights and Charades-STA. GIRL-DETR effectively rectifies the severe boundary misalignment, observed in the baseline without TPRL, achieving highly accurate boundary regression and improving the confidence score di…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 5 canonical work pages

  1. [1]

    Tall: Temporal activity localization via language query,

    J. Gao, C. Sun, Z. Yang, and R. Nevatia, “Tall: Temporal activity localization via language query,” inProc. IEEE Int. Conf. Comput. Vis. (ICCV), 2017, pp. 5267–5275

    2017

  2. [2]

    Localizing moments in video with natural language,

    L. Anne Hendricks, O. Wang, E. Shechtman, J. Sivic, T. Darrell, and B. Russell, “Localizing moments in video with natural language,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2017, pp. 5803–5812

    2017

  3. [3]

    Maban: Multi-agent boundary-aware network for natural language moment retrieval,

    X. Sun, H. Wang, and B. He, “Maban: Multi-agent boundary-aware network for natural language moment retrieval,”IEEE Trans. Image Process., vol. 30, pp. 5589–5599, 2021

    2021

  4. [4]

    Temporal sentence grounding in videos: A survey and future directions,

    H. Zhang, A. Sun, W. Jing, and J. T. Zhou, “Temporal sentence grounding in videos: A survey and future directions,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 8, pp. 10 443–10 465, 2023

    2023

  5. [5]

    Video moment localization via deep cross-modal hashing,

    Y . Hu, M. Liu, X. Su, Z. Gao, and L. Nie, “Video moment localization via deep cross-modal hashing,”IEEE Trans. Image Process., vol. 30, pp. 4667–4677, 2021

    2021

  6. [6]

    Local correspondence network for weakly supervised temporal sentence grounding,

    W. Yang, T. Zhang, Y . Zhang, and F. Wu, “Local correspondence network for weakly supervised temporal sentence grounding,”IEEE Trans. Image Process., vol. 30, pp. 3252–3262, 2021

    2021

  7. [7]

    Video moment retrieval with cross-modal neural architecture search,

    X. Yang, S. Wang, J. Dong, J. Dong, M. Wang, and T.-S. Chua, “Video moment retrieval with cross-modal neural architecture search,”IEEE Trans. Image Process., vol. 31, pp. 1204–1216, 2022. IEEE TRANSACTIONS ON IMAGE PROCESSING 12

    2022

  8. [8]

    Bam-detr: Boundary-aligned moment detection transformer for temporal sentence grounding in videos,

    P. Lee and H. Byun, “Bam-detr: Boundary-aligned moment detection transformer for temporal sentence grounding in videos,” inProc. Eur. Conf. Comput. Vis. (ECCV). Springer, 2024, pp. 220–238

    2024

Show all 59 references
  1. [9]

    Bmrn: Boundary matching and refine- ment network for temporal moment localization with natural language,

    M. Seol, J. Kim, and J. Moon, “Bmrn: Boundary matching and refine- ment network for temporal moment localization with natural language,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 5571–5579

    2023

  2. [10]

    Time-r1: Post-training large vision language model for temporal video grounding,

    Y . Wang, Z. Wang, B. Xu, Y . Du, K. Lin, Z. Xiao, Z. Yue, J. Ju, L. Zhang, D. Yanget al., “Time-r1: Post-training large vision language model for temporal video grounding,”Adv. Neural Inf. Process. Syst. (NIPS), vol. 38, pp. 83 330–83 364, 2026

    2026

  3. [11]

    Datasets and recipes for video temporal grounding via reinforcement learning,

    R. Chen, T. Luo, Z. Fan, H. Zou, Z. Feng, G. Xie, H. Zhang, Z. Wang, Z. Liu, and Z. Huaijian, “Datasets and recipes for video temporal grounding via reinforcement learning,” inProc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), 2025, pp. 983–992

    2025

  4. [12]

    Detecting moments and highlights in videos via natural language queries,

    J. Lei, T. L. Berg, and M. Bansal, “Detecting moments and highlights in videos via natural language queries,”Adv. Neural Inf. Process. Syst. (NIPS), vol. 34, pp. 11 846–11 858, 2021

    2021

  5. [13]

    Correlation-guided calibration of query dependency for video temporal grounding,

    W. Moon, S. Hyun, S. Lee, and J. pil Heo, “Correlation-guided calibration of query dependency for video temporal grounding,” Pattern Recognit., vol. 174, p. 112984, 2025. [Online]. Available: https://api.semanticscholar.org/CorpusID:284322691

    2025

  6. [14]

    Tr-detr: Task-reciprocal transformer for joint moment retrieval and highlight detection,

    H. Sun, M. Zhou, W. Chen, and W. Xie, “Tr-detr: Task-reciprocal transformer for joint moment retrieval and highlight detection,” inProc. AAAI Conf. Artif. Intell., vol. 38, no. 5, 2024, pp. 4998–5007

    2024

  7. [15]

    Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection,

    Y . Liu, S. Li, Y . Wu, C.-W. Chen, Y . Shan, and X. Qie, “Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 3042–3051

    2022

  8. [16]

    Bridging the gap: A unified video comprehension framework for moment retrieval and highlight detection,

    Y . Xiao, Z. Luo, Y . Liu, Y . Ma, H. Bian, Y . Ji, Y . Yang, and X. Li, “Bridging the gap: A unified video comprehension framework for moment retrieval and highlight detection,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 18 709–18 719

    2024

  9. [17]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Rayet al., “Training language models to follow instructions with human feedback,”Adv. Neural Inf. Process. Syst. (NIPS), vol. 35, pp. 27 730–27 744, 2022

    2022

  10. [18]

    Aligning large multimodal models with factually augmented rlhf,

    Z. Sun, S. Shen, S. Cao, H. Liu, C. Li, Y . Shen, C. Gan, L. Gui, Y .-X. Wang, Y . Yanget al., “Aligning large multimodal models with factually augmented rlhf,” inFindings Assoc. Comput. Linguist. (ACL), 2024, pp. 13 088–13 110

    2024

  11. [19]

    Video-r1: Reinforcing video reasoning in mllms,

    K. Feng, K. Gong, B. Li, Z. Guo, Y . Wang, T. Peng, J. Wu, X. Zhang, B. Wang, and X. Yue, “Video-r1: Reinforcing video reasoning in mllms,” Adv. Neural Inf. Process. Syst. (NIPS), vol. 38, pp. 99 114–99 137, 2026

    2026

  12. [20]

    Simple statistical gradient-following algorithms for connectionist reinforcement learning,

    R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,”Mach. Learn., vol. 8, no. 3, pp. 229–256, 1992

    1992

  13. [21]

    Policy gradient methods for reinforcement learning with function approximation,

    R. S. Sutton, D. McAllester, S. Singh, and Y . Mansour, “Policy gradient methods for reinforcement learning with function approximation,”Adv. Neural Inf. Process. Syst. (NIPS), vol. 12, 1999

    1999

  14. [22]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,”Proc. Natl. Acad. Sci., vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  15. [23]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 12, pp. 2935–2947, 2017

    2017

  16. [24]

    Context-aware biaffine localizing network for temporal sentence grounding,

    D. Liu, X. Qu, J. Dong, P. Zhou, Y . Cheng, W. Wei, Z. Xu, and Y . Xie, “Context-aware biaffine localizing network for temporal sentence grounding,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2021, pp. 11 235–11 244

    2021

  17. [25]

    Grounding action descriptions in videos,

    M. Regneri, M. Rohrbach, D. Wetzel, S. Thater, B. Schiele, and M. Pinkal, “Grounding action descriptions in videos,”Trans. Assoc. Comput. Linguist., vol. 1, pp. 25–36, 2013

    2013

  18. [26]

    To find where you talk: Temporal sentence localization in video with attention based location regression,

    Y . Yuan, T. Mei, and W. Zhu, “To find where you talk: Temporal sentence localization in video with attention based location regression,” inProc. AAAI Conf. Artif. Intell., vol. 33, no. 01, 2019, pp. 9159–9166

    2019

  19. [27]

    Learning 2d temporal adjacent networks for moment localization with natural language,

    S. Zhang, H. Peng, J. Fu, and J. Luo, “Learning 2d temporal adjacent networks for moment localization with natural language,” inProc. AAAI Conf. Artif. Intell., vol. 34, no. 07, 2020, pp. 12 870–12 877

    2020

  20. [28]

    Span-based localizing network for natural language video localization,

    H. Zhang, A. Sun, W. Jing, and J. T. Zhou, “Span-based localizing network for natural language video localization,” inProc. 58th Annu. Meet. Assoc. Comput. Linguist. (ACL), 2020, pp. 6543–6554

    2020

  21. [29]

    Dense regression network for video grounding,

    R. Zeng, H. Xu, W. Huang, P. Chen, M. Tan, and C. Gan, “Dense regression network for video grounding,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2020, pp. 10 287–10 296

    2020

  22. [30]

    Learning modality interaction for temporal sentence localization and event captioning in videos,

    S. Chen, W. Jiang, W. Liu, and Y .-G. Jiang, “Learning modality interaction for temporal sentence localization and event captioning in videos,” inProc. Eur. Conf. Comput. Vis. (ECCV). Springer, 2020, pp. 333–351

    2020

  23. [31]

    Multi-modal interaction graph convolutional network for temporal language localization in videos,

    Z. Zhang, X. Han, X. Song, Y . Yan, and L. Nie, “Multi-modal interaction graph convolutional network for temporal language localization in videos,”IEEE Trans. Image Process., vol. 30, pp. 8265–8277, 2021

    2021

  24. [32]

    On pursuit of designing multi-modal transformer for video grounding,

    M. Cao, L. Chen, M. Z. Shou, C. Zhang, and Y . Zou, “On pursuit of designing multi-modal transformer for video grounding,” inProc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), 2021, pp. 9810–9823

    2021

  25. [33]

    Query-dependent video representation for moment retrieval and highlight detection,

    W. Moon, S. Hyun, S. Park, D. Park, and J.-P. Heo, “Query-dependent video representation for moment retrieval and highlight detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 23 023–23 033

    2023

  26. [34]

    Knowing where to focus: Event-aware transformer for video grounding,

    J. Jang, J. Park, J. Kim, H. Kwon, and K. Sohn, “Knowing where to focus: Event-aware transformer for video grounding,” inProc. IEEE Int. Conf. Comput. Vis. (ICCV), 2023, pp. 13 846–13 856

    2023

  27. [35]

    Deep reinforcement learning from human preferences,

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,”Adv. Neural Inf. Process. Syst., vol. 30, 2017

    2017

  28. [36]

    Learning to summarize with human feedback,

    N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. V oss, A. Rad- ford, D. Amodei, and P. F. Christiano, “Learning to summarize with human feedback,”Adv. Neural Inf. Process. Syst., vol. 33, pp. 3008– 3021, 2020

    2020

  29. [37]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  30. [38]

    Direct preference optimization: Your language model is secretly a reward model,

    R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,”Adv. Neural Inf. Process. Syst. (NIPS), vol. 36, pp. 53 728–53 741, 2023

    2023

  31. [39]

    Qlora: Efficient finetuning of quantized llms,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,”Adv. Neural Inf. Process. Syst., vol. 36, pp. 10 088–10 115, 2023

    2023

  32. [40]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” Proc. 10th Int. Conf. Learn. Represent. (ICLR), 2022

    2022

  33. [41]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” inProc. Int. Conf. Mach. Learn. (ICML). PMLR, 2019, pp. 2790–2799

    2019

  34. [42]

    Active object localization with deep reinforcement learning,

    J. C. Caicedo and S. Lazebnik, “Active object localization with deep reinforcement learning,” inProc. IEEE Int. Conf. Comput. Vis. (ICCV), 2015, pp. 2488–2496

    2015

  35. [43]

    Read, watch, and move: Reinforcement learning for temporally grounding natural language descriptions in videos,

    D. He, X. Zhao, J. Huang, F. Li, X. Liu, and S. Wen, “Read, watch, and move: Reinforcement learning for temporally grounding natural language descriptions in videos,” inProc. AAAI Conf. Artif. Intell., vol. 33, no. 01, 2019, pp. 8393–8400

    2019

  36. [44]

    Tree-structured policy based pro- gressive reinforcement learning for temporally language grounding in video,

    J. Wu, G. Li, S. Liu, and L. Lin, “Tree-structured policy based pro- gressive reinforcement learning for temporally language grounding in video,” inProc. AAAI Conf. Artif. Intell., vol. 34, no. 07, 2020, pp. 12 386–12 393

    2020

  37. [45]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wuet al., “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,”arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  38. [46]

    Spatiotem- poral contrastive modeling for video moment retrieval,

    Y . Wang, K. Li, G. Chen, Y . Zhang, D. Guo, and M. Wang, “Spatiotem- poral contrastive modeling for video moment retrieval,”World Wide Web, vol. 26, no. 4, pp. 1525–1544, 2023

    2023

  39. [47]

    Momentdiff: Generative video moment retrieval from random to real,

    P. Li, C.-W. Xie, H. Xie, L. Zhao, L. Zhang, Y . Zheng, D. Zhao, and Y . Zhang, “Momentdiff: Generative video moment retrieval from random to real,”Adv. Neural Inf. Process. Syst. (NIPS), vol. 36, pp. 65 948– 65 966, 2023

    2023

  40. [48]

    Univtg: Towards unified video-language temporal grounding,

    K. Q. Lin, P. Zhang, J. Chen, S. Pramanick, D. Gao, A. J. Wang, R. Yan, and M. Z. Shou, “Univtg: Towards unified video-language temporal grounding,” inProc. IEEE Int. Conf. Comput. Vis. (ICCV), 2023, pp. 2794–2804

    2023

  41. [49]

    Prior knowledge integration via llm encoding and pseudo event regulation for video moment retrieval,

    Y . Jiang, W. Zhang, X. Zhang, X.-Y . Wei, C. W. Chen, and Q. Li, “Prior knowledge integration via llm encoding and pseudo event regulation for video moment retrieval,” inProc. 32nd ACM Int. Conf. Multimedia, 2024, pp. 7249–7258

    2024

  42. [50]

    Diversifying query: Region-guided transformer for temporal sentence grounding,

    X. Sun, L. Shi, L. Wang, S. Zhou, K. Xia, Y . Wang, and G. Hua, “Diversifying query: Region-guided transformer for temporal sentence grounding,” inProc. AAAI Conf. Artif. Intell., vol. 39, no. 7, 2025, pp. 7131–7139

    2025

  43. [51]

    Maskable retentive network for video moment retrieval,

    J. Hu, D. Guo, K. Li, Z. Si, X. Yang, and M. Wang, “Maskable retentive network for video moment retrieval,” inProc. 32nd ACM Int. Conf. Multimedia, 2024, pp. 1476–1485. IEEE TRANSACTIONS ON IMAGE PROCESSING 13

    2024

  44. [52]

    Sa-detr: Span aware detection transformer for moment retrieval,

    T. Xiong, W. Wei, K. Xu, and D. Chen, “Sa-detr: Span aware detection transformer for moment retrieval,” inProc. 31st Int. Conf. Comput. Linguist. (COLING), 2025, pp. 7634–7647

    2025

  45. [53]

    Query-aware video encoder for video moment retrieval,

    J. Hao, H. Sun, P. Ren, J. Wang, Q. Qi, and J. Liao, “Query-aware video encoder for video moment retrieval,”Neurocomputing, vol. 483, pp. 72–86, 2022

    2022

  46. [54]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gellyet al., “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  47. [55]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inProc. Int. Conf. Mach. Learn. (ICML). PMLR, 2021, pp. 8748–8763

    2021

  48. [56]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” inProc. Int. Conf. Mach. Learn. (ICML). PMLR, 2022, pp. 12 888– 12 900

    2022

  49. [57]

    Internvideo2: Scaling video foundation mod- els for multimodal video understanding. arxiv 2024,

    Y . Wang, K. Li, X. Li, J. Yu, Y . He, G. Chen, B. Pei, R. Zheng, J. Xu, Z. Wanget al., “Internvideo2: Scaling video foundation mod- els for multimodal video understanding. arxiv 2024,”arXiv preprint arXiv:2403.15377, vol. 2, 2024

    2024

  50. [58]

    Slowfast networks for video recognition,

    C. Feichtenhofer, H. Fan, J. Malik, and K. He, “Slowfast networks for video recognition,” inProc. IEEE Int. Conf. Comput. Vis. (ICCV), 2019, pp. 6202–6211

    2019

  51. [59]

    Video- lights: Feature refinement and cross-task alignment transformer for joint video highlight detection and moment retrieval,

    D. Paul, M. R. Parvez, N. Mohammed, and S. Rahman, “Video- lights: Feature refinement and cross-task alignment transformer for joint video highlight detection and moment retrieval,”arXiv preprint arXiv:2412.01558, 2024. Shihang Zhangis currently pursuing the B.E. de- gree with...

    2024

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.