Pith. sign in

REVIEW 3 major objections 5 minor 60 references

HIPPO-Video: Simulating Watch Histories with Large Language Models for Personalized Video Highlighting

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A viewer's watch history, not a short text query, is the better signal for highlighting video segments, and the history-conditioned HiPHer model beats all generic and query-based baselines on the new HIPPO-Video benchmark, including on…

desk verdict A new task and large simulated dataset for personalized video highlighting, but the main benchmark rests on LLM-generated labels and the human validation is too thin to confirm the gains transfer. read the letter →

arxiv 2507.16873 v1 pith:KERMQL5M submitted 2025-07-22 cs.CV cs.AI

Jeongeun Lee , Youngjae Yu , Dongha Lee This is my paper
classification cs.CVcs.AI
keywords personalizedvideohighlightingwatchhistoryLLM-basedusersimulationsaliencyscorepredictionhighlightdetectionmomentretrievalpreferencemodelingsummarization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a new task, personalized video highlighting: given a target video and a viewer's recent watch history, score every segment by how much that specific viewer would want to see it. To get data at scale without invading privacy, the authors built an LLM that plays the role of a viewer, browsing YouTube, updating its stated likes and dislikes after each video, and finally rating the segments of a target video from 1 to 10, producing 2,040 (history, scores) pairs over 20,400 videos. They then introduce HiPHer, which turns the watch history into a single preference embedding and conditions the target video's segments on it through cross-attention, trained with a contrastive ranking loss. HiPHer outperforms generic highlight detectors and query-based moment-retrieval and summarization methods on every reported metric, and keeps the lead on a small real-user test set, supporting the central claim that a history carries more personalization signal than a short query. If the claim holds, video platforms could replace one-size-fits-all highlights with per-user highlights that shift with the viewer's evolving interests.

What carries the argument

The load-bearing object is a two-stage pipeline. First, an LLM-based user simulator generates the data: initialized from one of 170 topic/sub-topic seeds plus an intent label, it iterates through video candidate retrieval (related videos or a fresh search query), engagement (it picks the most-wanted and least-wanted video from the candidate pool, watches the chosen one, and writes a preference-grounded review), and preference update (its long-term likes and dislikes are rewritten in natural language after each video). The last video watched is segmented by scene detection and each segment is scored 1–10 from the accumulated preferences. Second, HiPHer consumes that data: each watched video is mean-pooled into a single embedding, the history embeddings are averaged into a preference embedding $e_p$, projection layers stack LayerNorm and dropout, cross-attention uses the target video's segment features as queries with $e_p$ as key and value, and a transformer encoder refines the sequence before a margin contrastive loss $\max(0, \gamma - (y^+ - y^-))$ enforces that preference-relevant segments outscore irrelevant ones.

What would settle it

Have a larger group of real users record their own watch histories and score every segment (not just the top clip) of the target videos on the same 1-10 scale, then train and evaluate HiPHer and the baselines on those human labels; if the human segment rankings diverge from the LLM rankings, or if HiPHer's margin over the query-based baselines shrinks or reverses on that data, the paper's central claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes personalized video highlighting as a task in which the input is a target video plus the viewer's recent watch history and the output is a 1–10 relevance score for every segment. It claims that watch histories encode a richer, more accurate signal about what a person finds worth watching than the single short query used in moment retrieval or query-focused summarization, and that a model which ingests the whole history can exploit that signal well enough to beat all generic and query-based baselines on every reported metric, both on the simulated benchmark and on a 40-history real-user test set. The authors frame the result as evidence that watch-history-driven preference modeling, rather than query-based or history-free approaches, is the right route to user-centric video highlight detection.

Load-bearing premise

Every main result is measured against saliency scores written by an LLM playing a viewer rather than by real viewers, and the paper assumes those synthetic judgments are a valid proxy for human preferences; the human checks cover only the single top-scored clip of each video plus a 40-history real-user sample.

Editorial extensions

If this is right

  • On HIPPO-Video, HiPHer tops every baseline on all headline metrics (RMSE 0.301, mAP 0.766, Hit1@7 0.507, Hit1@9 0.166, Recall1@0.5 0.452, Recall1@0.7 0.245), so a history-conditioned scorer is directly competitive as a highlight detector and moment retriever.
  • Longer watch histories improve prediction, since performance climbs steadily as the history grows from 1 to 10 videos; a platform that tracks more of a user's viewing can expect sharper highlights.
  • Watch history beats text as a preference container: F1@5 rises from 0.566 with a phrase query and 0.658 with a sentence query to 0.726 with history, and history also beats a genre-based personalized summarizer (VSL, 0.466).
  • Both visual and transcribed-audio features are needed: fusing them yields mAP 0.77 versus 0.67 for visual only and 0.74 for text only, so multi-modal input is part of the method's effectiveness.
  • The margin persists on the real-user subset HIPPO-Video+, where HiPHer keeps the best RMSE, Hit1@7, Hit1@9, and F1@0.5, indicating the approach transfers beyond the simulated setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the simulator couples browsing behavior with saliency labeling, so the same generation pipeline could cheaply produce preference-conditioned labels for other media, such as personalized news digests, search-result ranking, or recommendation feedback, wherever a watch-history analogue exists.
  • Editorial inference: the paper's own design hints at an untested upgrade path, namely replacing mean pooling with attention-weighted aggregation of history videos by their relevance to the target video; the margin ablation further suggests very small margins near $\gamma = 0.1$ are better, implying fine-grained ranking rather than separation is what the loss should optimize.
  • Editorial inference: the validation is asymmetric in a way the paper does not dwell on, since human annotators judged only the single top-scored clip per video rather than the full 1–10 segment scores, and the real-user test set contains only 40 histories, so the fine-grained saliency scale and the reported margins on it are not yet directly confirmed by human opinion.
  • Editorial inference: the GPT-4-based indistinguishability test (40% accuracy, below chance) compares an LLM judge against LLM-generated histories, so the two systems could share systematic biases; a sterner test would have human annotators attempt the same simulated-versus-real discrimination.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces HIPPO-Video, a benchmark for personalized video highlighting in which each target video is paired with a 10-video watch history and per-segment saliency scores. The watch histories and scores are produced by an LLM-based user simulator over 2,040 profiles derived from 170 topic/sub-topic seeds and four intents. The authors also propose HiPHer, which mean-pools CLIP features of the watched videos into a preference embedding and conditions target-segment scoring through cross-attention, trained with a contrastive saliency loss. Experiments on HIPPO-Video show HiPHer outperforming generic and query-based baselines; a smaller real-user collection (HIPPO-VIDEO+, 40 histories) shows mostly better but not uniformly better results. Human evaluations validate the plausibility of simulated queries, video choices, and top-scored highlight clips.

Significance. If the simulated labels faithfully reflect human preferences, the dataset is a substantial resource: 20,400 videos across 170 categories, with public code and data, and the task formulation (watch history as a preference signal) is well motivated. The paper also ships useful ablations and a rare real-user check. The strength of the contribution, however, is conditional on the label-validity evidence: the main benchmark is generated by the same LLM-based simulation pipeline that defines the task, and the human validation covers only the top-scored clip, not the segment-level ranking used by all metrics. The real-user set is too small to support strong generalization claims by itself.

major comments (3)
  1. [3.2, 5.2, Table 3] The central results in Table 3 are all measured against saliency scores generated by the LLM simulator. Because HiPHer is trained on those same synthetic labels, its gains show that history-mean pooling correlates with the simulator's scoring function; they do not by themselves establish that the model captures human preferences. Section 3.3's saliency validation is too narrow to close this gap: annotators were shown only the highest-scored clip (or tied clips) and asked to agree or disagree, which checks one extreme of the ranking rather than the full 1-10 ordering used by mAP, Hit@1, Recall@1, and RMSE. The realism checks for watch histories (GPT-4 accuracy 40%, Fast-DetectGPT Hit@1 0.350) speak to history plausibility, not to the validity of the saliency labels. I ask the authors to add a human study that annotates segment-level saliency on a subset, to report correlation or ranking agreement between human and simulator scores, and to re-run the main comparison against those human labels.
  2. [5.3, Table 4] On HIPPO-VIDEO+ (40 real histories), HiPHer's RMSE is 0.427, which is worse than Moment-DETR's 0.419; its H1@9 gain is only 0.400 versus 0.389, and no confidence intervals or significance tests are reported. The text's statement that HiPHer 'consistently outperforms the baselines across most metrics' is therefore overstated, and 'strong robustness beyond simulated settings' is not supported at n=40. Please report paired per-user bootstrap confidence intervals or significance tests, and state explicitly which metric differences are statistically distinguishable from zero.
  3. [4, Eq. (2)] The contrastive saliency loss in Eq. (2) is central to training, but the paper never specifies how positive and negative segment pairs (v+, v-) are sampled from the 1-10 synthetic scores. Possible choices such as thresholding, top/bottom quantiles, random pairs, or margin-based mining lead to different training behavior and different sensitivity to the synthetic labels. Without this specification, the method is not fully reproducible and the loss's contribution cannot be assessed. Please state the sampling procedure explicitly.
minor comments (5)
  1. [3.3 vs Appendix A.4] The main text reports a video-selection agreement of 71.42%, while Appendix A.4 reports 68.42%; these numbers should be reconciled. The main text also says '85% inter-annotator agreement' whereas Appendix A.4 reports Fleiss' kappa of 0.85, which are different quantities. Please clarify and report the number of items judged in each human evaluation.
  2. [5.4, Table 6] The sentence 'textual features (HiPHer-V) being more informative than visual ones (HiPHer-T)' is inconsistent with Table 6, where HiPHer-T has higher mAP, H1@7, and R1@0.5 than HiPHer-V. Please fix the labels or the sentence so the naming and the results agree.
  3. [Appendix B.3] The appendix states, 'We will include these ablation results in the final version' regarding the margin gamma ablation. In the present manuscript the claim that smaller margins 'consistently yield better results' is therefore asserted without the supporting experiment; please include the actual ablation or remove the claim.
  4. [Table 3] The table header contains a typo, 'Recall@@α', which should be 'Recall@α'. Also, the mapping from the prior 4-out-of-5 threshold to the 7 and 9 thresholds used here should be justified more explicitly.
  5. [Figure 5] The history-length ablation shows a monotone trend, but no error bars or significance testing are reported; with a small test set, the trend could be driven by a few histories. Please add variance information or a paired test.

Circularity Check

2 steps flagged · score 6.0 of 10

Main 'history-driven' result is partially circular: the watch history used for conditioning includes the target video itself, and the benchmark labels are produced by the authors' own LLM simulator.

  1. self definitional [Section 3.2, Section 4 Eq. (1), Figure 5]
    "After simulation, the last video in each watch history is set as the target video for saliency annotation. ... Given a video V ... and a watch history consisting of m videos, H = {H1, H2, . . ., Hm} ... ep = Aggh({h(i) : h(i) = Aggs(s(i)1, . . . ,s(i)n)}m i=1)"

    The dataset has 2,040 pairs and 20,400 videos total, so each 10-video watch history includes the target video (the last history item). Equation (1) aggregates all m watched videos into the preference embedding ep, so when m=10 the preference context is a function of the target video's own pooled features. The history-length ablation in Figure 5 runs to 10, so the reported gain from adding history videos can be driven by adding the target itself to the conditioning signal. The central 'history-driven preference modeling' claim is therefore partly self-referential by construction and does not cleanly measure external viewing history.

  2. other [Section 3.2, Section 3.3, Table 3]
    "The simulator then assigns relevance scores ranging from 1 to 10 to each segment. These scores are determined based on two primary sources of information: (1) final long-term preferences ... and (2) personal reviews generated each time after watching the video. ... The results show that nearly 98% of pairs are deemed reasonable by majority agreement, confirming that the saliency scores accurately capture personalized preferences."

    The main benchmark's ground-truth saliency scores are generated by the authors' LLM-based simulation pipeline, and the human verification only checks the single highest-scored clip per video. Thus Table 3's superiority measures agreement with the simulator's scoring function rather than with independently established human preferences on the full segment ordering. The real-user set (HIPPO-VIDEO+) contains only 40 histories, and on RMSE HiPHer is not the best (0.427 vs Moment-DETR 0.419), so the real-world prediction is not independently grounded at the scale needed to support the abstract's 'real-world scenarios' claim.

full rationale

The most concrete circular step is the construction of the watch-history input: Section 3.2 states that the target video is the last video of each 10-video watch history, and Eq. (1) aggregates the entire watch history (including that target) into the preference embedding. Therefore, when the model is evaluated with full history length, the 'preference' signal contains the target video's own content, making the history-driven advantage partially self-referential. This is not a formal fit-to-label equation, but it is a by-construction reduction of the claimed input-output separation. The LLM-generated saliency labels add a second, validity-level circularity: the main benchmark measures how well HiPHer reproduces the authors' simulator, not independent human preferences, since human validation covers only the top-scored clip. These issues are load-bearing because the headline claim (Table 3) and the history-length ablation are the paper's main evidence. I do not find self-citation chains, imported uniqueness theorems, or renamed known results; the method is self-contained as a supervised model on the proposed benchmark. The score of 6 reflects that the central preference-modeling claim partially reduces to its own input construction and to the authors' label-generation pipeline.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that LLM-simulated watch histories and saliency scores approximate human preferences. This is a domain assumption validated only partially. The model itself introduces two hand-chosen hyperparameters (history length m=10, contrastive margin gamma=1) that affect results; the ablation shows gamma=1 may be suboptimal. No new physical or conceptual entities are introduced.

free parameters (2)
  • contrastive margin gamma = 1 (initial), ablation suggests 0.1-0.2
    Margin in the saliency loss (Eq. 2). The ablation in Appendix B.3 shows smaller margins improve performance, but the main tables use gamma=1, so the reported results may not reflect the best setting.
  • watch history length m = 10
    Fixed to 10 for all simulated histories. The ablation in Figure 5 shows performance improves with more history videos, so this design choice may limit the method's achievable accuracy.
assumptions (4)
  • domain assumption LLM-simulated watch histories and saliency scores are a valid proxy for real user preferences and behavior.
    This is the central premise of HIPPO-Video. It is validated by small MTurk studies and 40 real-user histories, but the full segment-level scoring is not human-validated.
  • domain assumption The 170 topic and sub-topic pairs from Qiu et al. (2024) cover the diversity of user video interests.
    Adopted from MMSum and used as initialization seeds for all 2,040 simulated profiles (Section 3.1 Initialization and Appendix A.1).
  • domain assumption CLIP ViT-B/32 image and text features provide sufficient semantic representations for video segments.
    Used for all input representations in HiPHer and for baseline feature extraction (Section 4.2 Input Representations).
  • domain assumption Scene change detection produces segments that are coherent semantic units appropriate for scoring.
    Used in both the simulation and the method to define segment boundaries (Section 3.1 and Section 4.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of HIPPO-Video: Simulating Watch Histories with Large Language Models for Personalized Video Highlighting." pith.science (2026). https://pith.science/paper/KERMQL5M

@misc{pith2026250716873,
  author       = {Pith},
  title        = {Pith review of: HIPPO-Video: Simulating Watch Histories with Large Language Models for Personalized Video Highlighting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KERMQL5M}},
  note         = {Machine review of arXiv:2507.16873}
}
read the original abstract

The exponential growth of video content has made personalized video highlighting an essential task, as user preferences are highly variable and complex. Existing video datasets, however, often lack personalization, relying on isolated videos or simple text queries that fail to capture the intricacies of user behavior. In this work, we introduce HIPPO-Video, a novel dataset for personalized video highlighting, created using an LLM-based user simulator to generate realistic watch histories reflecting diverse user preferences. The dataset includes 2,040 (watch history, saliency score) pairs, covering 20,400 videos across 170 semantic categories. To validate our dataset, we propose HiPHer, a method that leverages these personalized watch histories to predict preference-conditioned segment-wise saliency scores. Through extensive experiments, we demonstrate that our method outperforms existing generic and query-based approaches, showcasing its potential for highly user-centric video highlighting in real-world scenarios.

Figures

Figures reproduced from arXiv: 2507.16873 by the authors.

Figure 1
Figure 1. A video can produce varying highlights based on user interests, showing how [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall process of our LLM-based user simulation to collect video watch [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Dataset analysis results. (a–b) Exploration patterns and watch history embeddings [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The architecture of HiPHer consists of two modules: (1) a preference modeling [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Performance com￾parison with varying num￾bers of history videos used for preference modeling. 5.2 Main Results [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Case study on saliency (preference) scores between HiPHer and Moment-DETR. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: The process of converting YouTube video metadata into a structured JSON format. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Performance of vary￾ing γ. While we initially set γ = 1 following prior works (Lei et al., 2021; Moon et al., 2023), we conducted an ablation study to assess its impact on HiPHer ’s performance. We observed that smaller margins (γ= 0.1–0.2) consistently yield better re…
Figure 9
Figure 9. Figure 9: Qualitative Case Studies: “Ours” refers to HiPHer, and “Baselines” refers to [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 42 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Video summarization using deep neural networks: A survey

    Evlampios Apostolidis, Eleni Adamantidou, Alexandros I Metsai, Vasileios Mezaris, and Ioannis Patras. Video summarization using deep neural networks: A survey. Proceedings of the IEEE, 109 0 (11): 0 1838--1863, 2021

    work page 2021

  3. [3]

    Towards automated movie trailer generation

    Dawit Mureja Argaw, Mattia Soldan, Alejandro Pardo, Chen Zhao, Fabian Caba Heilbron, Joon Son Chung, and Bernard Ghanem. Towards automated movie trailer generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 7445--7454, 2024 a

    work page 2024

  4. [4]

    Scaling up video summarization pretraining with large language models

    Dawit Mureja Argaw, Seunghyun Yoon, Fabian Caba Heilbron, Hanieh Deilamsalehy, Trung Bui, Zhaowen Wang, Franck Dernoncourt, and Joon Son Chung. Scaling up video summarization pretraining with large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 8332--8341, 2024 b

    work page 2024

  5. [5]

    Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature

    Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. arXiv preprint arXiv:2310.05130, 2023

    arXiv 2023

  6. [6]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pp.\ 213--229. Springer, 2020

    2020

  7. [7]

    Personalized video summarization by multimodal video understanding

    Brian Chen, Xiangyuan Zhao, and Yingnan Zhu. Personalized video summarization by multimodal video understanding. In CIKM, 2024

    work page 2024

  8. [8]

    Chatbot arena: An open platform for evaluating llms by human preference

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference on Machine Learning, 2024

    2024

Show all 60 references
  1. [9]

    Tall: Temporal activity localization via language query

    Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on computer vision, pp.\ 5267--5275, 2017

    2017

  2. [10]

    Creating summaries from user videos

    Michael Gygli, Helmut Grabner, Hayko Riemenschneider, and Luc Van Gool. Creating summaries from user videos. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VII 13, pp.\ 505--520. Springer, 2014

    2014

  3. [11]

    Video2gif: Automatic generation of animated gifs from video

    Michael Gygli, Yale Song, and Liangliang Cao. Video2gif: Automatic generation of animated gifs from video. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 1001--1009, 2016

    2016

  4. [12]

    Shot2story20k: A new benchmark for comprehensive understanding of multi-shot videos

    Mingfei Han, Linjie Yang, Xiaojun Chang, and Heng Wang. Shot2story20k: A new benchmark for comprehensive understanding of multi-shot videos. arXiv preprint arXiv:2312.10300, 2023

    2023 arXiv

  5. [13]

    V2xum-llm: Cross-modal video summarization with temporal prompt instruction tuning

    Hang Hua, Yunlong Tang, Chenliang Xu, and Jiebo Luo. V2xum-llm: Cross-modal video summarization with temporal prompt instruction tuning. arXiv preprint arXiv:2404.12353, 2024

    2024

  6. [14]

    Movienet: A holistic dataset for movie understanding

    Qingqiu Huang, Yu Xiong, Anyi Rao, Jiaze Wang, and Dahua Lin. Movienet: A holistic dataset for movie understanding. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part IV 16, pp.\ 709--727. Springer, 2020

    2020

  7. [15]

    Video summarization with attention-based encoder--decoder networks

    Zhong Ji, Kailin Xiong, Yanwei Pang, and Xuelong Li. Video summarization with attention-based encoder--decoder networks. IEEE Transactions on Circuits and Systems for Video Technology, 30 0 (6): 0 1709--1717, 2019

    2019

  8. [16]

    Mdetr-modulated detection for end-to-end multi-modal understanding

    Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr-modulated detection for end-to-end multi-modal understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 1780--1790, 2021

    2021

  9. [17]

    Self-attentive sequential recommendation

    Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), pp.\ 197--206. IEEE, 2018

    2018

  10. [18]

    Tvr: A large-scale dataset for video-subtitle moment retrieval

    Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. Tvr: A large-scale dataset for video-subtitle moment retrieval. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXI 16, pp.\ 447--463. Springer, 2020

    2020

  11. [19]

    Detecting moments and highlights in videos via natural language queries

    Jie Lei, Tamara L Berg, and Mohit Bansal. Detecting moments and highlights in videos via natural language queries. Advances in Neural Information Processing Systems, 34: 0 11846--11858, 2021

    2021

  12. [20]

    Hero: Hierarchical encoder for video+ language omni-representation pre-training

    Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, and Jingjing Liu. Hero: Hierarchical encoder for video+ language omni-representation pre-training. arXiv preprint arXiv:2005.00200, 2020

    2005 arXiv

  13. [21]

    Univtg: Towards unified video-language temporal grounding

    Kevin Qinghong Lin, Pengchuan Zhang, Joya Chen, Shraman Pramanick, Difei Gao, Alex Jinpeng Wang, Rui Yan, and Mike Zheng Shou. Univtg: Towards unified video-language temporal grounding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 2794--2804, 2023

    2023

  14. [22]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024

    2024

  15. [23]

    Attentive moment retrieval in videos

    Meng Liu, Xiang Wang, Liqiang Nie, Xiangnan He, Baoquan Chen, and Tat-Seng Chua. Attentive moment retrieval in videos. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp.\ 15--24, 2018

    2018

  16. [24]

    Multi-task deep visual-semantic embedding for video thumbnail selection

    Wu Liu, Tao Mei, Yongdong Zhang, Cherry Che, and Jiebo Luo. Multi-task deep visual-semantic embedding for video thumbnail selection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 3707--3715, 2015

    2015

  17. [25]

    Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection

    Ye Liu, Siyuan Li, Yang Wu, Chang-Wen Chen, Ying Shan, and Xiaohu Qie. Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 3042--3051, 2022

    2022

  18. [26]

    Debug: A dense bottom-up grounding approach for natural language video localization

    Chujie Lu, Long Chen, Chilie Tan, Xiaolin Li, and Jun Xiao. Debug: A dense bottom-up grounding approach for natural language video localization. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference...

    2019

  19. [27]

    Videoautoarena: An automated arena for evaluating large multimodal models in video analysis through user simulation

    Ziyang Luo, Haoning Wu, Dongxu Li, Jing Ma, Mohan Kankanhalli, and Junnan Li. Videoautoarena: An automated arena for evaluating large multimodal models in video analysis through user simulation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.\ 846...

    2025

  20. [28]

    Howto100m: Learning a text-video embedding by watching hundred million narrated video clips

    Antoine Miech, Dimitri Zhukov, Jean-Baptiste Alayrac, Makarand Tapaswi, Ivan Laptev, and Josef Sivic. Howto100m: Learning a text-video embedding by watching hundred million narrated video clips. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 2...

    2019

  21. [29]

    Detectgpt: Zero-shot machine-generated text detection using probability curvature

    Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International Conference on Machine Learning, pp.\ 24950--24962. PMLR, 2023

    2023

  22. [30]

    Query-dependent video representation for moment retrieval and highlight detection

    WonJun Moon, Sangeek Hyun, SangUk Park, Dongchan Park, and Jae-Pil Heo. Query-dependent video representation for moment retrieval and highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 23023--23033, 2023

    2023

  23. [31]

    Clip-it! language-guided video summarization

    Medhini Narasimhan, Anna Rohrbach, and Trevor Darrell. Clip-it! language-guided video summarization. Advances in neural information processing systems, 34: 0 13988--14000, 2021

    2021

  24. [32]

    Sumgraph: Video summarization via recursive graph modeling

    Jungin Park, Jiyoung Lee, Ig-Jae Kim, and Kwanghoon Sohn. Sumgraph: Video summarization via recursive graph modeling. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXV 16, pp.\ 647--663. Springer, 2020

    2020

  25. [33]

    Mmsum: A dataset for multimodal summarization and thumbnail generation of videos

    Jielin Qiu, Jiacheng Zhu, William Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, et al. Mmsum: A dataset for multimodal summarization and thumbnail generation of videos. In Proceedings of the IEEE/CVF Conference on Computer...

    2024

  26. [34]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...

    2021

  27. [35]

    Bpr: Bayesian personalized ranking from implicit feedback

    Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pp.\ 452--461, 2009

    2009

  28. [36]

    Adaptive video highlight detection by learning from user history

    Mrigank Rochan, Mahesh Kumar Krishna Reddy, Linwei Ye, and Yang Wang. Adaptive video highlight detection by learning from user history. In European conference on computer vision, pp.\ 261--278. Springer, 2020

    2020

  29. [37]

    Query-focused extractive video summarization

    Aidean Sharghi, Boqing Gong, and Mubarak Shah. Query-focused extractive video summarization. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14, pp.\ 3--19. Springer, 2016

    2016

  30. [38]

    Query-focused video summarization: Dataset, evaluation, and a memory network based approach

    Aidean Sharghi, Jacob S Laurel, and Boqing Gong. Query-focused video summarization: Dataset, evaluation, and a memory network based approach. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 4788--4797, 2017

    2017

  31. [39]

    Tvsum: Summarizing web videos using titles

    Yale Song, Jordi Vallmitjana, Amanda Stent, and Alejandro Jaimes. Tvsum: Summarizing web videos using titles. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 5179--5187, 2015

    2015

  32. [40]

    To click or not to click: Automatic selection of beautiful thumbnails from videos

    Yale Song, Miriam Redi, Jordi Vallmitjana, and Alejandro Jaimes. To click or not to click: Automatic selection of beautiful thumbnails from videos. In Proceedings of the 25th ACM international on conference on information and knowledge management, pp.\ 659--668, 2016

    2016

  33. [41]

    Jinhwan Sul, Jihoon Han, and Joonseok Lee. Mr. hisum: A large-scale dataset for video highlight detection and summarization. Advances in Neural Information Processing Systems, 36: 0 40542--40555, 2023

    2023

  34. [42]

    Tr-detr: Task-reciprocal transformer for joint moment retrieval and highlight detection

    Hao Sun, Mingyao Zhou, Wenjing Chen, and Wei Xie. Tr-detr: Task-reciprocal transformer for joint moment retrieval and highlight detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 4998--5007, 2024

    2024

  35. [43]

    Ranking domain-specific highlights by analyzing edited videos

    Min Sun, Ali Farhadi, and Steve Seitz. Ranking domain-specific highlights by analyzing edited videos. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pp.\ 787--802. Springer, 2014

    2014

  36. [44]

    Query-adaptive video summarization via quality-aware relevance estimation

    Arun Balajee Vasudevan, Michael Gygli, Anna Volokitin, and Luc Van Gool. Query-adaptive video summarization via quality-aware relevance estimation. In Proceedings of the 25th ACM international conference on Multimedia, pp.\ 582--590, 2017

    2017

  37. [45]

    Videoagent: Long-form video understanding with large language model as agent

    Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pp.\ 58--76. Springer, 2024

    2024

  38. [46]

    Query-biased self-attentive network for query-focused video summarization

    Shuwen Xiao, Zhou Zhao, Zijian Zhang, Ziyu Guan, and Deng Cai. Query-biased self-attentive network for query-focused video summarization. IEEE Transactions on Image Processing, 29: 0 5889--5899, 2020 a

    2020

  39. [47]

    Convolutional hierarchical attention network for query-focused video summarization

    Shuwen Xiao, Zhou Zhao, Zijian Zhang, Xiaohui Yan, and Min Yang. Convolutional hierarchical attention network for query-focused video summarization. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.\ 12426--12433, 2020 b

    2020

  40. [48]

    Bridging the gap: A unified video comprehension framework for moment retrieval and highlight detection

    Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, and Xiu Li. Bridging the gap: A unified video comprehension framework for moment retrieval and highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2024

  41. [49]

    Cross-category video highlight detection via set-based learning

    Minghao Xu, Hang Wang, Bingbing Ni, Riheng Zhu, Zhenbang Sun, and Changhu Wang. Cross-category video highlight detection via set-based learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 7970--7979, 2021

    2021

  42. [50]

    Mh-detr: Video moment and highlight detection with cross-modal transformer

    Yifang Xu, Yunzhuo Sun, Benxiang Zhai, Youyao Jia, and Sidan Du. Mh-detr: Video moment and highlight detection with cross-modal transformer. In 2024 International Joint Conference on Neural Networks (IJCNN), pp.\ 1--8. IEEE, 2024

    2024

  43. [51]

    Highlight detection with pairwise deep ranking for first-person video summarization

    Ting Yao, Tao Mei, and Yong Rui. Highlight detection with pairwise deep ranking for first-person video summarization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 982--990, 2016

    2016

  44. [52]

    Semantic conditioned dynamic modulation for temporal sentence grounding in videos

    Yitian Yuan, Lin Ma, Jingwen Wang, Wei Liu, and Wenwu Zhu. Semantic conditioned dynamic modulation for temporal sentence grounding in videos. Advances in Neural Information Processing Systems, 32, 2019

    2019

  45. [53]

    Hierarchical video-moment retrieval and step-captioning

    Abhay Zala, Jaemin Cho, Satwik Kottur, Xilun Chen, Barlas Oguz, Yashar Mehdad, and Mohit Bansal. Hierarchical video-moment retrieval and step-captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 23056--23065, 2023

    2023

  46. [54]

    Moment is important: Language-based video moment retrieval via adversarial learning

    Yawen Zeng, Da Cao, Shaofei Lu, Hanling Zhang, Jiao Xu, and Zheng Qin. Moment is important: Language-based video moment retrieval via adversarial learning. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 18 0 (2): 0 1--21, 2022

    2022

  47. [55]

    Span-based localizing network for natural language video localization

    Hao Zhang, Aixin Sun, Wei Jing, and Joey Tianyi Zhou. Span-based localizing network for natural language video localization. arXiv preprint arXiv:2004.13931, 2020

    2004 arXiv

  48. [56]

    Towards automatic learning of procedures from web instructional videos

    Luowei Zhou, Chenliang Xu, and Jason Corso. Towards automatic learning of procedures from web instructional videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018

    2018

  49. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  50. [58]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  51. [59]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  52. [60]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.