{"id":"8dd3d677-ec5f-4f4c-a72b-9eb9d4c9b99b","arxiv_id":"2505.18880","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"REGen generates documentary teasers by fine-tuning an LLM to write a script with <QUOTE> markers, then a trained retriever fills each marker with the most relevant clip from the source video.","lead":"REGen turns a long documentary into a short teaser by having an AI write the narration and then inserting real short clips from the original footage to support the story. It combines the best of two older approaches: a coherent AI-written narrative plus exact video quotes that prior methods could not produce.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The automatic longestspeaker narrator heuristic supplies every training label and evaluation target; with a reported narrator-prediction F1 of only 71.6% on the test teasers (Appendix B), this is the most load-bearing assumption, and the paper never quantifies how its errors propagate into the…","rationale":"The reader's conditional verdict is well-founded, and my pass converged on the same load-bearing point: the automatic narrator heuristic is the foundation of both training and evaluation. I considered alternative issues, including the low Recall@1 in Table 3, the swapped T/TV rows between Table 3 and Appendix I.1, the unusual retrieval-loss formulation in Eq. (5), and the generalizability preference for TeaserGen in Appendix H.5. These are real weaknesses, but each could be a symptom, a typo, or a secondary limitation; the narrator heuristic cannot be dismissed because it defines the task itself. If the longest-speaker rule is wrong for a sizeable fraction of videos, then both the proposed models and the baselines are trained and measured against a corrupted notion of what a correct quote is. The authors deserve credit for reporting human validation and for including ablations and multiple baselines, but the validation is limited to test videos and reports a narrator F1 of only 71.6% (Appendix B). Section 6 acknowledges reliance on diarization, so the limitation is in the manuscript; what is missing is any estimate of its impact on the headline results. The proposed concrete test, retraining on human-corrected labels for a sample, would separate 'label noise is the bottleneck' from 'the model is genuinely weak.' If no change appears, the central claim can stand with better framing; if metrics shift, the current numbers are partly artifacts of the annotation heuristic. Therefore the reader's CONDITIONAL verdict should remain unchanged, with an explicit request to report this label-reliability analysis.","tokens_in":18868,"tokens_out":10444,"duration_ms":105191,"concrete_test":"Manually annotate narrator identity and quote/non-quote boundaries on a stratified random sample of roughly 100 training-set main-documentary/teaser pairs, using the same human protocol as Appendix B. Retrain the REGen-DQ script generator and QuoteRetriever-T on the corrected labels, then re-run the Table 2 and Table 3 objective metrics and the Table 5 subjective study on the existing test set. If Recall@1, QDI, and coherence scores are essentially unchanged, the narrator heuristic is not the dominant failure; if they move substantially, the published results conflate model quality with annotation noise. A cheaper complementary check is to stratify Table 3 recall by per-video narrator-prediction F1 and test whether retrieval errors concentrate in low-F1 videos.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that REGen can generate a coherent teaser that quotes exact interview footage supporting the narrative. This claim is only as reliable as the automatic identification of what counts as narration versus quotable interview. Section 4.1 and Appendix B define the narrator as the speaker with the longest transcribed audio segment, and Appendix B reports a narrator-prediction F1 of 71.6% on the 49-video test set. This single heuristic generates: (i) every <QUOTE>/<SOQ> target used to fine-tune the LLaMA script generator, (ii) the ground-truth clip used to train and evaluate the BART quote retriever, and (iii) the candidate pool of non-narrator 'quotable' clips. If the longest-speaker rule misidentifies the narrator, the system is trained to place quotes at wrong locations, the retriever is trained to fetch a wrongly labeled clip, and the objective metrics in Tables 2 and 3 are measured against a potentially wrong reference. The four-person validation in Appendix B covers only test teasers and reports start/end-time and transcription correctness separately; it does not provide corrected labels for the 839 training examples used to train the models. The paper's Section 6 acknowledges reliance on speaker diarization, but it never estimates how a substantial label-error rate propagates through training and evaluation. The low Recall@1 in Table 3 (5.00%) is consistent with this concern, though not proof of it; the concern must be settled before the central claim can be accepted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes REGen, a two-stage pipeline for editing long videos into short teasers. In the first stage, a fine-tuned LLaMA model generates a script containing explicit quotation placeholders (<SOQ>/<EOQ> for direct quotes and <QUOTE> for indirect quotes). In the second stage, a fine-tuned BART-based quote retriever fills each indirect-quote placeholder by selecting a quotable interview clip from the source video, using a retrieval embedding that combines textual and visual features. The system is trained and evaluated on the DocumentaryNet dataset, with WhisperX-based speaker diarization used to separate narrator speech from quotable interview segments. Objective and subjective evaluations compare REGen against extractive, abstractive, and GPT-4-based baselines, and the paper reports results on script generation, quote retrieval, and full teaser generation.","tokens_in":19175,"tokens_out":3512,"duration_ms":28455,"significance":"If the stated results are reliable, REGen would be a genuinely useful hybrid between extractive and abstractive long-to-short video editing: it produces a coherent narrative while embedding exact source footage as supporting evidence, which neither pure extractive nor pure abstractive methods achieve. The framework is clearly specified, the training losses are standard, and the paper includes useful ablations on context length, loss weighting, negative sampling, and retrieval-token position. The task itself, documentary teaser generation with quotable insertions, is well motivated and the comparison against multiple baselines is thoughtful. However, the evaluation has load-bearing weaknesses: the narrator-identification heuristic supplies all training and test labels despite a reported F1 of only 71.6% on the test teasers, the retrieval recall is low (best Recall@10 of 30%), and the subjective results rely on a small participant pool with overlapping confidence intervals.","major_comments":[{"comment":"The automatic narrator-identification heuristic is load-bearing: Section 4.1 defines the narrator as the speaker with the longest transcribed audio segment, and this single rule generates every <QUOTE>/<SOQ> target used to train the LLaMA script generator, the ground-truth clips used to train and evaluate the BART quote retriever, and the candidate pool of quotable non-narrator clips. Appendix B reports a narrator-prediction F1 of only 71.6% on the 49 test teasers, and the four-person validation provides corrected labels only for test-set clips, not for the 839 training examples used to finetune the models. The paper therefore never quantifies how narrator-label errors propagate into the trained models or the reported metrics. I ask for an explicit error-propagation analysis, for example by retraining or re-evaluating on a subset with human-corrected narrator labels or by reporting an oracle-narrator upper bound. Without this, the central claim that the generated teasers place quotes at contextually correct locations is not fully established.","section":"Appendix B; Section 4.1"},{"comment":"The objective retrieval results are modest: Table 3 shows that the best proposed retriever, QuoteRetriever-TV, achieves Recall@1 of 5.00%, Recall@5 of 17.50%, and Recall@10 of 30.00% on a test set with an average of 66 candidate interviews per video. The claim that REGen can effectively insert supporting interview clips therefore rests largely on the subjective insertion-effectiveness scores, but those scores are based on only 21 participants split across two versions (11 and 10), and the reported 95% confidence intervals in Table 3 overlap substantially (e.g., QuoteRetriever-T 3.56 ± 0.22 vs. QuoteRetriever-TV 3.49 ± 0.26). The paper should report significance tests, inter-annotator agreement, and ideally a larger participant pool. It should also discuss why the exact-match recall is so low and what fraction of retrieval failures still yield semantically appropriate clips.","section":"Section 4.3; Table 3"},{"comment":"Table 4 reports F1 scores for the full teaser-generation task that are all near 1–2% for the proposed methods and baselines, while the ground-truth teaser achieves 69.00%. The paper does not explain this discrepancy or its implications for the claim that REGen outperforms extractive and abstractive baselines on teaser generation. If F1 measures overlap with the actual ground-truth teaser frames, then no automatic method reproduces the actual teaser to any meaningful degree, and the paper should state this limitation explicitly and support the superiority claim with the metrics that actually differentiate the methods, such as the subjective ratings in Table 5.","section":"Section 5.3; Table 4"},{"comment":"In the retrieval loss definition, e* is described as 'the sentence embedding of the ground truth narration,' but the retrieval target is a quotable interview clip, not a narration segment. This terminology should be corrected to 'ground-truth quote segment' or similar. In addition, Appendix C.2 states that the all-mpnet-base-v2 sentence embedder is jointly fine-tuned, whereas Section 3.2 describes the visual/textual fusion module f as the learnable component; please clarify which parameters are updated in retrieval training and whether the sentence embedder is shared between the candidate representation and the query representation.","section":"Section 3.2; Eq. (5)"}],"minor_comments":[{"comment":"The REGen row is cited as 'REGen [11]', but [11] is TeaserGen; this should be marked as the proposed method (for example, 'Ours') to avoid confusion with the TeaserGen baseline row.","section":"Table 1"},{"comment":"The KDE plots in Figure 3 are described qualitatively; please report the quantitative separation (for example, the difference between median top-1 similarity and median all-similarity) for all compared methods, since the visual comparison alone is hard to assess from the figure.","section":"Section 5.2; Figure 3"},{"comment":"The ablation on alpha reports Recall@1, Recall@5, and Recall@10 but does not report standard deviations or significance tests; adding these would make the comparison among alpha values more meaningful.","section":"Appendix I.2; Table 10"},{"comment":"The validation results report narrator F1, start-time correctness, end-time correctness, and transcription accuracy, but the relation between these quantities and the 128 'interview segments with correct start times, end times, and transcriptions' should be stated more precisely, including how many total segments were evaluated.","section":"Appendix B"},{"comment":"The limitation paragraph correctly notes the risk of misplacing quotes, but it does not connect this risk to the specific narrator-label error rate measured in Appendix B; adding that connection would strengthen the paper's discussion of its own limitations.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid systems contribution with a clear pipeline and a well-defined task, and I do not see any circularity or result-rigging. The main risk is the narrator-identification heuristic: since it supplies all training and test labels, the 71.6% F1 in Appendix B is a serious concern that must be addressed with explicit error-propagation analysis rather than a brief acknowledgement in Section 6. The subjective evaluation is also too small to support the headline claim as currently written; either a larger study or more cautious claims are needed. I would be comfortable with acceptance after these issues are resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"REGen is worth a look: the two-stage design—LLM script with <QUOTE> placeholders, then a trained retriever filling them with exact interview clips—is a real, novel combination of extractive and abstractive video summarization. The paper is clearly written, the training setup is standard, and they ship code and a demo page, so the work is reproducible. I haven't run it, but the ablations cover the main knobs.\n\nThat said, the evaluation does not yet support the general claims. The best retriever gets 30% Recall@10 on a test set where chance is about 1.5%, so it's not nothing, but it's far from reliable. More importantly, the whole pipeline is trained and evaluated on labels produced by a single heuristic: the narrator is assumed to be the speaker with the longest transcribed segment. Appendix B reports narrator-prediction F1 of 71.6% on the test teasers. That heuristic creates every <QUOTE>/<SOQ> target, every ground-truth clip for the retriever, and the candidate pool of quotable segments. If the wrong speaker is flagged, the system learns to insert quotes in the wrong places and the metrics are measured against a wrong reference. The paper acknowledges reliance on diarization but never estimates how label error propagates. This is the load-bearing assumption and it needs a sensitivity analysis or a smaller human-validated set.\n\nThe subjective study is also thin: 21 participants, overlapping confidence intervals, and no significance tests. The generalizability A/B in Appendix H shows TeaserGen preferred over REGen on lectures (62%) and news (55%), which undercuts the broad 'framework' claim. There's also an inconsistency between Table 3 (QuoteRetriever-TV at 23.33% Recall@10) and the ablation Table 9 (30% for the same configuration) that needs explanation.\n\nIf I were the editor, I'd send it to review but with the expectation of major revision: the idea deserves a fair shot, and the code release means reviewers can probe the weak spots. The paper would be much stronger with a corrected-label subset and an analysis of how narrator errors affect both stages.","headline":"A genuinely new hybrid video summarization idea, but the evaluation is too weak to support the broad claims, and the narrator-labeling heuristic is doing more load-bearing work than the experiments account for.","tokens_in":19767,"tokens_out":2329,"would_cite":true,"duration_ms":20246,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"REGen claims that a two-stage retrieval-embedded generation system can automatically produce a coherent short video that embeds exact source clips as supporting evidence, a capability neither extractive nor abstractive video summarization…","keywords":["video summarization","long-to-short video editing","retrieval-augmented generation","documentary teaser generation","multimodal quoting","quote retrieval","speaker diarization"],"falsifier":"Take a set of documentaries in which the true narrator is known not to be the longest speaker and run REGen end-to-end; if quote-retrieval recall stays near the random baseline and human raters judge the inserted interviews as unsupportive, then the narrator heuristic, rather than the retrieval design, is what carries or sinks the claim.","tokens_in":18653,"feed_emoji":"🎬","tokens_out":10685,"duration_ms":79475,"temperature":0.7,"pith_summary":"REGen is a proposed answer to a missing capability in video summarization: a system that writes a coherent story while quoting exact clips from the source video. The paper argues that pure extractive systems keep authentic footage but lose narrative flow, and pure abstractive systems produce smooth narration but cannot insert original footage as evidence. REGen splits the task into a finetuned language model that writes a script with quote placeholders, followed by a trained retriever that fills each placeholder with the most supportive clip from a pool of quotable segments. If the approach works as claimed, automated teasers, trailers, and educational shorts can ground their claims in exact source footage rather than paraphrases or resynthesized content.","feed_headline":"System writes a script and fills quotes with exact source clips","feed_subtitle":"REGen beats extractive and abstractive baselines on coherence, alignment, and realism in documentary teasers.","key_machinery":"The load-bearing mechanism is the quote placeholder plus a learned clip-fitness retriever. The finetuned language model writes the script with <QUOTE> markers, or with <SOQ> and <EOQ> for direct quotes, and an encoder-decoder model then decodes the missing quote content and appends a <SUM> token; the hidden state of that token serves as the query embedding. Candidate quotable clips are all non-narrator segments of the input video, and the chosen clip maximizes the clip fitness $\\mathrm{cos\\_sim}(h, e_m)$, where $h$ is the hidden state at the final <SUM> token and $e_m$ is a fused text-and-frame embedding of the candidate clip. This turns the task of quoting the video into a retrievable, optimizable operation rather than a generation-time gamble.","core_discovery":"The paper's central claim is that a language model can quote video footage exactly if quoting is decomposed into two stages: first generate the narrative with explicit quote placeholders, then retrieve the clip that best supports each placeholder. To do this, the authors finetune a language model to emit either direct quotes wrapped in the special tokens <SOQ> and <EOQ>, or indirect quote markers <QUOTE>, and they train an encoder-decoder model to decode the missing quotation content while producing a <SUM> token whose hidden state acts as a retrieval query. The inserted clip is the one maximizing cosine similarity between that query and a candidate clip's fused text-and-frame embedding, chosen from all non-narrator segments of the input video. On documentary teaser generation, this hybrid system achieves the top subjective ratings for coherence, alignment, and realism among the compared methods. The authors conclude that exact multimodal quoting and coherent abstractive narrative are compatible in a single editing pipeline.","pith_inferences":["Editorial inference: fixing the narrator heuristic with human-verified labels would likely improve both script generation and retrieval more than any architectural change, because the heuristic currently mislabels roughly a quarter of test videos.","Editorial inference: the same placeholder-plus-retriever pattern could be used as a grounding mechanism for generated video answers, with each claim in a narration backed by a timestamped source clip.","Editorial inference: the generalizability study, where lecture viewers preferred the abstractive baseline 62% of the time, suggests the right quote density depends on domain; a lower-quote variant could be more competitive for educational content."],"forward_implications":["Documentary teaser generation can be fully automated while keeping original interview footage as verifiable evidence for the narration.","Quote retrieval models outperform prompted general-purpose language models at filling quote placeholders, so learned retrieval is the better route for multimodal quoting.","Finetuning a language model with quote markers enables it to produce exact source quotes that a vanilla model cannot produce.","Viewer ratings place the hybrid teaser above both extractive and abstractive baselines on coherence, alignment, and realism."],"supporting_citations":[{"why":"Defines the documentary teaser generation task and the narration-visual matching baseline that REGen extends with quote insertions.","marker":"[11]"},{"why":"Provides the speech transcription and speaker diarization used to construct all training scripts and quotable clip pools.","marker":"[16]"},{"why":"Supplies the base language model finetuned to emit quote placeholders in REGen-DQ and REGen-IDQ.","marker":"[30]"},{"why":"Provides the encoder-decoder model finetuned with generation and retrieval losses to decode quote content and produce the <SUM> query embedding.","marker":"[32]"},{"why":"Supplies sentence embeddings used for nearest-neighbor retrieval baselines and candidate clip text embeddings.","marker":"[33]"},{"why":"Generates the chunk summaries that feed the script generator and powers the GPT-4o direct-quote, screenplay, and infilling baselines.","marker":"[29]"},{"why":"Provides the pretrained video-language temporal grounding model that aligns generated narration with visuals.","marker":"[31]"},{"why":"The retrieval-augmented generation literature motivates conditioning the language model on retrieved evidence before decoding.","marker":"[19–21]"},{"why":"Serves as the extractive summarization baseline whose outputs REGen compares against for coherence and realism.","marker":"[4]"}],"fun_headline_variants":["REGen quotes exact clips into AI-written video narratives","Video quoting without copy-paste: REGen pulls exact clips from source","LLM writes narrative, retrieval fills in exact video quotes","Two-stage method: script with placeholders, then retrieve matching clip","REGen: AI scriptwriting that literally quotes raw video"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole system is trained on the assumption that the narrator is the speaker with the longest transcribed audio, and since that guess is wrong for roughly 28 percent of test teasers, much of the training supervision for where to place and retrieve quotes is mislabeled.","fun_headline_variants_meta":{"raw":{"variants":["REGen quotes exact clips into AI-written video narratives","Video quoting without copy-paste: REGen pulls exact clips from source","LLM writes narrative, retrieval fills in exact video quotes","Two-stage method: script with placeholders, then retrieve matching clip","REGen: AI scriptwriting that literally quotes raw video"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000658,"raw_usage":{"total_tokens":3017,"prompt_tokens":959,"completion_tokens":2058,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":1973}},"tokens_in":575,"tokens_out":2058,"duration_ms":10655,"temperature":1.0,"reasoning_tokens":1973,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:24:12.715662+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of documentaries in which the true narrator is known not to be the longest speaker and run REGen end-to-end; if quote-retrieval recall stays near the random baseline and human raters judge the inserted interviews as unsupportive, then the narrator heuristic, rather than the retrieval design, is what carries or sinks the claim.","supporting_citations":[{"cited_title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,","cited_arxiv_id":null,"evidence_quote":"Provides the encoder-decoder model finetuned with generation and retrieval losses to decode quote content and produce the <SUM> query embedding."},{"cited_title":"Sentence-Transformers/all-mpnet-base-v2,","cited_arxiv_id":null,"evidence_quote":"Supplies sentence embeddings used for nearest-neighbor retrieval baselines and candidate clip text embeddings."},{"cited_title":"Univtg: Towards unified video-language temporal grounding,","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained video-language temporal grounding model that aligns generated narration with visuals."},{"cited_title":"Align and Attend: Multimodal Summarization with Dual Contrastive Losses","cited_arxiv_id":"2303.07284","evidence_quote":"Serves as the extractive summarization baseline whose outputs REGen compares against for coherence and realism."}],"review_version":1}