Pith. sign in

REVIEW 4 major objections 5 minor 41 references

REGen: Multimodal Retrieval-Embedded Generation for Long-to-Short Video Editing

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read REGen claims that a two-stage retrieval-embedded generation system can automatically produce a coherent short video that embeds exact source clips as supporting evidence, a capability neither extractive nor abstractive video summarization…

desk verdict A genuinely new hybrid video summarization idea, but the evaluation is too weak to support the broad claims, and the narrator-labeling heuristic is doing more load-bearing work than the experiments account for. read the letter →

arxiv 2505.18880 v1 pith:JEYWRT6E submitted 2025-05-24 cs.CV cs.AI

Weihan Xu , Yimeng Ma , Jingyue Huang , Yang Li , Wenye Ma , Taylor Berg-Kirkpatrick , Julian McAuley , Paul Pu Liang
show 1 more author
Hao-Wen Dong
This is my paper
classification cs.CVcs.AI
keywords videosummarizationlong-to-shorteditingretrieval-augmentedgenerationdocumentaryteasermultimodalquotingquoteretrievalspeakerdiarization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

REGen is a proposed answer to a missing capability in video summarization: a system that writes a coherent story while quoting exact clips from the source video. The paper argues that pure extractive systems keep authentic footage but lose narrative flow, and pure abstractive systems produce smooth narration but cannot insert original footage as evidence. REGen splits the task into a finetuned language model that writes a script with quote placeholders, followed by a trained retriever that fills each placeholder with the most supportive clip from a pool of quotable segments. If the approach works as claimed, automated teasers, trailers, and educational shorts can ground their claims in exact source footage rather than paraphrases or resynthesized content.

What carries the argument

The load-bearing mechanism is the quote placeholder plus a learned clip-fitness retriever. The finetuned language model writes the script with <QUOTE> markers, or with <SOQ> and <EOQ> for direct quotes, and an encoder-decoder model then decodes the missing quote content and appends a <SUM> token; the hidden state of that token serves as the query embedding. Candidate quotable clips are all non-narrator segments of the input video, and the chosen clip maximizes the clip fitness $\mathrm{cos\_sim}(h, e_m)$, where $h$ is the hidden state at the final <SUM> token and $e_m$ is a fused text-and-frame embedding of the candidate clip. This turns the task of quoting the video into a retrievable, optimizable operation rather than a generation-time gamble.

What would settle it

Take a set of documentaries in which the true narrator is known not to be the longest speaker and run REGen end-to-end; if quote-retrieval recall stays near the random baseline and human raters judge the inserted interviews as unsupportive, then the narrator heuristic, rather than the retrieval design, is what carries or sinks the claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that a language model can quote video footage exactly if quoting is decomposed into two stages: first generate the narrative with explicit quote placeholders, then retrieve the clip that best supports each placeholder. To do this, the authors finetune a language model to emit either direct quotes wrapped in the special tokens <SOQ> and <EOQ>, or indirect quote markers <QUOTE>, and they train an encoder-decoder model to decode the missing quotation content while producing a <SUM> token whose hidden state acts as a retrieval query. The inserted clip is the one maximizing cosine similarity between that query and a candidate clip's fused text-and-frame embedding, chosen from all non-narrator segments of the input video. On documentary teaser generation, this hybrid system achieves the top subjective ratings for coherence, alignment, and realism among the compared methods. The authors conclude that exact multimodal quoting and coherent abstractive narrative are compatible in a single editing pipeline.

Load-bearing premise

The whole system is trained on the assumption that the narrator is the speaker with the longest transcribed audio, and since that guess is wrong for roughly 28 percent of test teasers, much of the training supervision for where to place and retrieve quotes is mislabeled.

Editorial extensions

If this is right

  • Documentary teaser generation can be fully automated while keeping original interview footage as verifiable evidence for the narration.
  • Quote retrieval models outperform prompted general-purpose language models at filling quote placeholders, so learned retrieval is the better route for multimodal quoting.
  • Finetuning a language model with quote markers enables it to produce exact source quotes that a vanilla model cannot produce.
  • Viewer ratings place the hybrid teaser above both extractive and abstractive baselines on coherence, alignment, and realism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: fixing the narrator heuristic with human-verified labels would likely improve both script generation and retrieval more than any architectural change, because the heuristic currently mislabels roughly a quarter of test videos.
  • Editorial inference: the same placeholder-plus-retriever pattern could be used as a grounding mechanism for generated video answers, with each claim in a narration backed by a timestamped source clip.
  • Editorial inference: the generalizability study, where lecture viewers preferred the abstractive baseline 62% of the time, suggests the right quote density depends on domain; a lower-quote variant could be more competitive for educational content.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes REGen, a two-stage pipeline for editing long videos into short teasers. In the first stage, a fine-tuned LLaMA model generates a script containing explicit quotation placeholders (<SOQ>/<EOQ> for direct quotes and <QUOTE> for indirect quotes). In the second stage, a fine-tuned BART-based quote retriever fills each indirect-quote placeholder by selecting a quotable interview clip from the source video, using a retrieval embedding that combines textual and visual features. The system is trained and evaluated on the DocumentaryNet dataset, with WhisperX-based speaker diarization used to separate narrator speech from quotable interview segments. Objective and subjective evaluations compare REGen against extractive, abstractive, and GPT-4-based baselines, and the paper reports results on script generation, quote retrieval, and full teaser generation.

Significance. If the stated results are reliable, REGen would be a genuinely useful hybrid between extractive and abstractive long-to-short video editing: it produces a coherent narrative while embedding exact source footage as supporting evidence, which neither pure extractive nor pure abstractive methods achieve. The framework is clearly specified, the training losses are standard, and the paper includes useful ablations on context length, loss weighting, negative sampling, and retrieval-token position. The task itself, documentary teaser generation with quotable insertions, is well motivated and the comparison against multiple baselines is thoughtful. However, the evaluation has load-bearing weaknesses: the narrator-identification heuristic supplies all training and test labels despite a reported F1 of only 71.6% on the test teasers, the retrieval recall is low (best Recall@10 of 30%), and the subjective results rely on a small participant pool with overlapping confidence intervals.

major comments (4)
  1. [Appendix B; Section 4.1] The automatic narrator-identification heuristic is load-bearing: Section 4.1 defines the narrator as the speaker with the longest transcribed audio segment, and this single rule generates every <QUOTE>/<SOQ> target used to train the LLaMA script generator, the ground-truth clips used to train and evaluate the BART quote retriever, and the candidate pool of quotable non-narrator clips. Appendix B reports a narrator-prediction F1 of only 71.6% on the 49 test teasers, and the four-person validation provides corrected labels only for test-set clips, not for the 839 training examples used to finetune the models. The paper therefore never quantifies how narrator-label errors propagate into the trained models or the reported metrics. I ask for an explicit error-propagation analysis, for example by retraining or re-evaluating on a subset with human-corrected narrator labels or by reporting an oracle-narrator upper bound. Without this, the central claim that the generated teasers place quotes at contextually correct locations is not fully established.
  2. [Section 4.3; Table 3] The objective retrieval results are modest: Table 3 shows that the best proposed retriever, QuoteRetriever-TV, achieves Recall@1 of 5.00%, Recall@5 of 17.50%, and Recall@10 of 30.00% on a test set with an average of 66 candidate interviews per video. The claim that REGen can effectively insert supporting interview clips therefore rests largely on the subjective insertion-effectiveness scores, but those scores are based on only 21 participants split across two versions (11 and 10), and the reported 95% confidence intervals in Table 3 overlap substantially (e.g., QuoteRetriever-T 3.56 ± 0.22 vs. QuoteRetriever-TV 3.49 ± 0.26). The paper should report significance tests, inter-annotator agreement, and ideally a larger participant pool. It should also discuss why the exact-match recall is so low and what fraction of retrieval failures still yield semantically appropriate clips.
  3. [Section 5.3; Table 4] Table 4 reports F1 scores for the full teaser-generation task that are all near 1–2% for the proposed methods and baselines, while the ground-truth teaser achieves 69.00%. The paper does not explain this discrepancy or its implications for the claim that REGen outperforms extractive and abstractive baselines on teaser generation. If F1 measures overlap with the actual ground-truth teaser frames, then no automatic method reproduces the actual teaser to any meaningful degree, and the paper should state this limitation explicitly and support the superiority claim with the metrics that actually differentiate the methods, such as the subjective ratings in Table 5.
  4. [Section 3.2; Eq. (5)] In the retrieval loss definition, e* is described as 'the sentence embedding of the ground truth narration,' but the retrieval target is a quotable interview clip, not a narration segment. This terminology should be corrected to 'ground-truth quote segment' or similar. In addition, Appendix C.2 states that the all-mpnet-base-v2 sentence embedder is jointly fine-tuned, whereas Section 3.2 describes the visual/textual fusion module f as the learnable component; please clarify which parameters are updated in retrieval training and whether the sentence embedder is shared between the candidate representation and the query representation.
minor comments (5)
  1. [Table 1] The REGen row is cited as 'REGen [11]', but [11] is TeaserGen; this should be marked as the proposed method (for example, 'Ours') to avoid confusion with the TeaserGen baseline row.
  2. [Section 5.2; Figure 3] The KDE plots in Figure 3 are described qualitatively; please report the quantitative separation (for example, the difference between median top-1 similarity and median all-similarity) for all compared methods, since the visual comparison alone is hard to assess from the figure.
  3. [Appendix I.2; Table 10] The ablation on alpha reports Recall@1, Recall@5, and Recall@10 but does not report standard deviations or significance tests; adding these would make the comparison among alpha values more meaningful.
  4. [Appendix B] The validation results report narrator F1, start-time correctness, end-time correctness, and transcription accuracy, but the relation between these quantities and the 128 'interview segments with correct start times, end times, and transcriptions' should be stated more precisely, including how many total segments were evaluated.
  5. [Section 6] The limitation paragraph correctly notes the risk of misplacing quotes, but it does not connect this risk to the specific narrator-label error rate measured in Appendix B; adding that connection would strengthen the paper's discussion of its own limitations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: REGen is a standard supervised pipeline with held-out training and evaluation, and its self-citations are to an externally grounded dataset and baseline, not to an imported theorem or fitted label.

full rationale

REGen's two stages are ordinary supervised learning: LLaMA is fine-tuned on script/quote-placement targets and the BART quote retriever is trained with Eq. (5) against ground-truth clips, then evaluated with held-out Recall@k and teaser metrics. No equation defines the evaluation target as the model's own output, and no fitted parameter is renamed as a prediction. The use of the authors' prior DocumentaryNet/TeaserGen work [11] is a dataset and a comparison baseline, not an imported uniqueness theorem; the teasers come from DW Documentary, PBS, and National Geographic, so the resource is externally grounded. The main caveat is the Section 4.1/Appendix B heuristic that labels the speaker with the longest transcribed audio as narrator (71.6% F1 on test teasers), and Section 6 admits the method 'relies on successful segmentation of the input video' through speaker diarization. These are label-validity and generalization limitations, not circularity: the models are not defined in terms of the labels they predict, and the reported metrics do not collapse to the heuristic by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the labeling heuristics above, which are validated with limited accuracy, plus standard supervised training assumptions. No new physical or conceptual entities are introduced; the model uses standard transformer architectures and special tokens. The main free parameters are the loss weight, the hard-negative ratio, and the context window, all tuned on validation data.

free parameters (3)
  • alpha (retrieval loss weight in L = L_gen + alpha * L_ret) = 1
    Selected based on validation Recall@5 and Recall@10 in Ablation Table 10; alpha=1 gave highest scores.
  • GroupSampler hard-negative ratio = 30%
    Chosen after ablation in Table 11; 30% gives best Recall@5/10, while 0% gives higher Recall@1. The paper uses 30%.
  • BART maximum input tokens = 256 for training, 128 for teaser inference
    Appendix C.2 states training uses 256 and inference uses 128 to cut costs; ablation Table 9 shows similar results, so the choice is partly efficiency-driven.
assumptions (4)
  • domain assumption Narrator is the speaker with the longest total transcribed audio in each video.
    Section 4.1 and Appendix B. Validation F1 on teaser test set is 71.6%, so the assumption is wrong in roughly one quarter of cases.
  • domain assumption All non-narrator video segments are quotable interview candidates.
    Section 4.1. This may include music, sound effects, or mixed segments, which would pollute the retrieval pool.
  • domain assumption WhisperX ASR and speaker diarization are accurate enough for training labels.
    Appendix B reports human validation with transcription accuracy 94.9% and end-time correctness 94.9% on teaser part; some residual label noise is expected.
  • domain assumption Ground-truth teasers in DocumentaryNet are valid references for evaluating summarization quality.
    Used for ROUGE, F1, and related metrics; editorial differences between source and teaser could make strict frame matching misleading, as the paper itself notes in Table 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of REGen: Multimodal Retrieval-Embedded Generation for Long-to-Short Video Editing." pith.science (2026). https://pith.science/paper/JEYWRT6E

@misc{pith2026250518880,
  author       = {Pith},
  title        = {Pith review of: REGen: Multimodal Retrieval-Embedded Generation for Long-to-Short Video Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JEYWRT6E}},
  note         = {Machine review of arXiv:2505.18880}
}
read the original abstract

Short videos are an effective tool for promoting contents and improving knowledge accessibility. While existing extractive video summarization methods struggle to produce a coherent narrative, existing abstractive methods cannot `quote' from the input videos, i.e., inserting short video clips in their outputs. In this work, we explore novel video editing models for generating shorts that feature a coherent narrative with embedded video insertions extracted from a long input video. We propose a novel retrieval-embedded generation framework that allows a large language model to quote multimodal resources while maintaining a coherent narrative. Our proposed REGen system first generates the output story script with quote placeholders using a finetuned large language model, and then uses a novel retrieval model to replace the quote placeholders by selecting a video clip that best supports the narrative from a pool of candidate quotable video clips. We examine the proposed method on the task of documentary teaser generation, where short interview insertions are commonly used to support the narrative of a documentary. Our objective evaluations show that the proposed method can effectively insert short video clips while maintaining a coherent narrative. In a subjective survey, we show that our proposed method outperforms existing abstractive and extractive approaches in terms of coherence, alignment, and realism in teaser generation.

Figures

Figures reproduced from arXiv: 2505.18880 by the authors.

Figure 1
Figure 1. An overview of the proposed REGen system for long-to-short video editing. Given a long input video, we first transcribe the narrations and dialogues using a pretrained automatic speech recognition model, and then we use a finetuned large language model to generate the output story script with quote placeholders (i.e., the <QUOTE> token). For the generated narration, following [11], we first synthesize the narration … view at source ↗
Figure 2
Figure 2. An illustration of the proposed two-stage quote retriever REGen-IDQ. We finetune an [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of infilling methods. The dotted lines indicate the median values. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 24 canonical work pages

  1. [1]

    Unveiling the Impact of Multi-Modal Interactions on User Engagement: A Comprehensive Evaluation in AI-driven Conversations

    L. Zhang, J. Yu, S. Zhang, L. Li, Y . Zhong, G. Liang, Y . Yan, Q. Ma, F. Weng, F. Pan, J. Li, R. Xu, and Z. Lan, “Unveiling the impact of multi-modal interactions on user engagement: A comprehensive evaluation in ai-driven conversations,” 2024. [Online]. Available: https://arxiv.org/abs/2406.15000

    work page Pith review arXiv 2024

  2. [2]

    Eye tracking research on readers’ interactions with multimodal texts: a mini-review,

    A. Gatcho, J. P. Manuel, and R. Sarasua, “Eye tracking research on readers’ interactions with multimodal texts: a mini-review,”Frontiers in Communication, vol. 9, 12 2024

    work page 2024

  3. [3]

    Rec-gpt4v: Multimodal recommendation with large vision-language models,

    Y . Liu, Y . Wang, L. Sun, and P. S. Yu, “Rec-gpt4v: Multimodal recommendation with large vision-language models,” 2024. [Online]. Available: https://arxiv.org/abs/2402.08670

    arXiv 2024

  4. [4]

    Align and Attend: Multimodal Summarization with Dual Contrastive Losses

    B. He, J. Wang, J. Qiu, T. Bui, A. Shrivastava, and Z. Wang, “Align and attend: Multimodal summarization with dual contrastive losses,” 2023. [Online]. Available: https://arxiv.org/abs/2303.07284

    work page Pith review arXiv 2023

  5. [5]

    Plots to previews: Towards automatic movie preview retrieval using publicly available meta-data,

    B. Gaikwad, A. Sontakke, M. Patwardhan, N. Pedanekar, and S. Karande, “Plots to previews: Towards automatic movie preview retrieval using publicly available meta-data,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3205–3214

    work page 2021

  6. [6]

    Smart trailer: Automatic generation of movie trailer using only subtitles,

    M. Hesham, B. Hani, N. Fouad, and E. Amer, “Smart trailer: Automatic generation of movie trailer using only subtitles,” in 2018 First International Workshop on Deep and Representation Learning (IWDRL). IEEE, 2018, pp. 26–30

    work page 2018

  7. [7]

    Automated production of tv program trailer using electronic program guide,

    Y . Kawai, H. Sumiyoshi, and N. Yagi, “Automated production of tv program trailer using electronic program guide,” in Proceedings of the 6th ACM International Conference on Image and Video Retrieval, 2007, pp. 49–56

    work page 2007

  8. [8]

    User preferences for automated curation of snackable content,

    A. King, E. Zavesky, and M. J. Gonzales, “User preferences for automated curation of snackable content,” in 26th International Conference on Intelligent User Interfaces, 2021, pp. 270–274

    work page 2021

Show all 41 references
  1. [9]

    Semi-supervised learning towards computerized generation of movie trailers,

    X. Liu and J. Jiang, “Semi-supervised learning towards computerized generation of movie trailers,” in 2015 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, 2015, pp. 2990–2995

    2015

  2. [10]

    Harnessing ai for augmenting creativity: Application to movie trailer creation,

    J. R. Smith, D. Joshi, B. Huet, W. Hsu, and J. Cota, “Harnessing ai for augmenting creativity: Application to movie trailer creation,” in Proceedings of the 25th ACM International Conference on Multimedia, 2017, pp. 1799–1808

    2017

  3. [11]

    Teasergen: Generating teasers for long documentaries,

    W. Xu, P. P. Liang, H. Kim, J. McAuley, T. Berg-Kirkpatrick, and H.-W. Dong, “Teasergen: Generating teasers for long documentaries,” 2024. [Online]. Available: https://arxiv.org/abs/2410.05586

    2024 arXiv

  4. [12]

    One-minute video generation with test-time training,

    K. Dalal, D. Koceja, G. Hussein, J. Xu, Y . Zhao, Y . Song, S. Han, K. C. Cheung, J. Kautz, C. Guestrin, T. Hashimoto, S. Koyejo, Y . Choi, Y . Sun, and X. Wang, “One-minute video generation with test-time training,” 2025. [Online]. Available: https://arxiv.org/abs/2504.05298

    2025 arXiv

  5. [13]

    Survey of hallucination in natural language generation,

    Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y . Xu, E. Ishii, Y . J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” ACM Comput. Surv., vol. 55, no. 12, Mar. 2023. [Online]. Available: https://doi.org/10.1145/3571730

    2023 doi

  6. [14]

    Hallucination of multimodal large language models: A survey,

    Z. Bai, P. Wang, T. Xiao, T. He, Z. Han, Z. Zhang, and M. Z. Shou, “Hallucination of multimodal large language models: A survey,” 2025. [Online]. Available: https://arxiv.org/abs/2404.18930

    2025 arXiv

  7. [15]

    Hallucination is inevitable: An innate limitation of large language models,

    Z. Xu, S. Jain, and M. Kankanhalli, “Hallucination is inevitable: An innate limitation of large language models,” 2025. [Online]. Available: https://arxiv.org/abs/2401.11817

    2025 arXiv

  8. [16]

    Whisperx: Time-accurate speech transcription of long-form audio,

    M. Bain, J. Huh, T. Han, and A. Zisserman, “Whisperx: Time-accurate speech transcription of long-form audio,” 2023. [Online]. Available: https://arxiv.org/abs/2303.00747

    2023 arXiv

  9. [17]

    Attributed question answering: Evaluation and modeling for attributed large language models,

    B. Bohnet, V . Q. Tran, P. Verga, R. Aharoni, D. Andor, L. B. Soares, M. Ciaramita, J. Eisenstein, K. Ganchev, J. Herzig, K. Hui, T. Kwiatkowski, J. Ma, J. Ni, L. S. Saralegui, T. Schuster, W. W. Cohen, M. Collins, D. Das, D. Metzler, S. Petrov, and K. Webster, “Attributed que...

    2023 arXiv

  10. [18]

    Learning fine-grained grounded citations for attributed large language models,

    L. Huang, X. Feng, W. Ma, Y . Gu, W. Zhong, X. Feng, W. Yu, W. Peng, D. Tang, D. Tu, and B. Qin, “Learning fine-grained grounded citations for attributed large language models,” in Findings of the Association for Computational Linguistics: ACL 2024 , L.-W. Ku, A. Martins, and ...

    2024

  11. [19]

    Dense passage retrieval for open-domain question answering,

    V . Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage retrieval for open-domain question answering,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , B. Webber, T. Cohn, Y . He, and...

    2020

  12. [20]

    Retrieval-augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” in Advances in Neural Information Processing Systems, H. Larochelle...

    2020

  13. [21]

    Trends in integration of knowledge and large language models: A survey and taxonomy of methods, benchmarks, and applications,

    Z. Feng, W. Ma, W. Yu, L. Huang, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. liu, “Trends in integration of knowledge and large language models: A survey and taxonomy of methods, benchmarks, and applications,” 2024. [Online]. Available: https://arxiv.org/abs/2311.05876

    2024 arXiv

  14. [22]

    Lamda: Language models for dialog applications,

    R. Thoppilan, D. D. Freitas, J. Hall, N. Shazeer, A. Kulshreshtha, H.-T. Cheng, A. Jin, T. Bos, L. Baker, Y . Du, Y . Li, H. Lee, H. S. Zheng, A. Ghafouri, M. Menegali, Y . Huang, M. Krikun, D. Lepikhin, J. Qin, D. Chen, Y . Xu, Z. Chen, A. Roberts, M. Bosma, V . Zhao, Y . Zho...

    2022 arXiv

  15. [23]

    Evaluating verifiability in generative search engines,

    N. Liu, T. Zhang, and P. Liang, “Evaluating verifiability in generative search engines,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 7001–...

    2023

  16. [24]

    Clip-it! language-guided video summarization,

    M. Narasimhan, A. Rohrbach, and T. Darrell, “Clip-it! language-guided video summarization,” inAdvances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 13 988–14 00...

    2021

  17. [25]

    Scaling up video summarization pretraining with large language models,

    D. M. Argaw, S. Yoon, F. C. Heilbron, H. Deilamsalehy, T. Bui, Z. Wang, F. Dernoncourt, and J. S. Chung, “Scaling up video summarization pretraining with large language models,” 2024. [Online]. Available: https://arxiv.org/abs/2404.03398

    2024 arXiv

  18. [26]

    Towards automated movie trailer generation,

    D. M. Argaw, M. Soldan, A. Pardo, C. Zhao, F. C. Heilbron, J. S. Chung, and B. Ghanem, “Towards automated movie trailer generation,” 2024. [Online]. Available: https://arxiv.org/abs/2404.03477

    2024 arXiv

  19. [27]

    "previously on

    A. K. Singh, D. Srivastava, and M. Tapaswi, “"previously on ..." from recaps to story summarization,”

  20. [28]

    Videoxum: Cross-modal visual and textural summarization of videos,

    J. Lin, H. Hua, M. Chen, Y . Li, J. Hsiao, C. Ho, and J. Luo, “Videoxum: Cross-modal visual and textural summarization of videos,” IEEE Transactions on Multimedia, 2023

    2023

  21. [29]

    Gpt-4 technical report,

    OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, R. Avila, I. Babuschkin, S. Balaji, V . Balcom, P. Baltescu, H. Bao, M. Bavarian, J. Belgum, I. Bello, J. Berdine, G. Bernadett-Shapiro, C. Berner, L...

    2024 arXiv

  22. [30]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” 2023. [Online]. Available: https://arxiv.org/abs/2302.13971

    2023 arXiv

  23. [31]

    Univtg: Towards unified video-language temporal grounding,

    K. Q. Lin, P. Zhang, J. Chen, S. Pramanick, D. Gao, A. J. Wang, R. Yan, and M. Z. Shou, “Univtg: Towards unified video-language temporal grounding,” 2023

    2023

  24. [32]

    BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,

    M. Lewis, Y . Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V . Stoyanov, and L. Zettlemoyer, “BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” in Proceedings of the 58th Annual Meeting of the Associati...

    2020

  25. [33]

    Sentence-Transformers/all-mpnet-base-v2,

    Sentence Transformers, “Sentence-Transformers/all-mpnet-base-v2,” https://huggingface.co/ sentence-transformers/all-mpnet-base-v2, 2022, apache-2.0 license; accessed on 2025-04-28

    2022

  26. [34]

    ROUGE: A package for automatic evaluation of summaries,

    C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” inText Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013/

    2004

  27. [35]

    G-eval: Nlg evaluation using gpt-4 with better human alignment,

    Y . Liu, D. Iter, Y . Xu, S. Wang, R. Xu, and C. Zhu, “G-eval: Nlg evaluation using gpt-4 with better human alignment,” 2023. [Online]. Available: https://arxiv.org/abs/2303.16634

    2023 arXiv

  28. [36]

    DeepEval: The open-source llm evaluation framework,

    Confident AI Inc., “DeepEval: The open-source llm evaluation framework,” https://www.deepeval.com, 2025, accessed: 2025-04-29

    2025

  29. [37]

    Clipscore: A reference-free evaluation metric for image captioning,

    J. Hessel, A. Holtzman, M. Forbes, R. L. Bras, and Y . Choi, “Clipscore: A reference-free evaluation metric for image captioning,” 2022. [Online]. Available: https://arxiv.org/abs/2104.08718

    2022 arXiv

  30. [38]

    Screenwriter: Automatic screenplay generation and movie summarisation,

    L. Mahon and M. Lapata, “Screenwriter: Automatic screenplay generation and movie summarisation,”

  31. [39]

    Multimodal lecture presentations dataset: Understanding multimodality in educational slides,

    D. W. Lee, C. Ahuja, P. P. Liang, S. Natu, and L.-P. Morency, “Multimodal lecture presentations dataset: Understanding multimodality in educational slides,” 2022. [Online]. Available: https://arxiv.org/abs/2208.08080 12 A Broader Impacts We envision our proposed method to be a...

    2022 arXiv

  32. [40]

    Available: https://arxiv.org/abs/2410.19809

    [Online]. Available: https://arxiv.org/abs/2410.19809

    arXiv

  33. [2024]

    Available: https://arxiv.org/abs/2405.11487

    [Online]. Available: https://arxiv.org/abs/2405.11487

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.