{"id":"72e1951c-8414-47c1-b718-39efa73f6d18","arxiv_id":"2507.04036","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"PresentAgent chains LLM segmentation, slide rendering, TTS, and ffmpeg to turn documents into narrated presentation videos, but the human-level claim rests on five documents and an unvalidated VLM judge.","lead":"This paper introduces PresentAgent, a modular software pipeline that turns long documents into narrated slide videos with synchronized speech and visuals. It also introduces PresentEval, a vision-language-model scoring framework, but the reported 'human-level' results rest on only five test documents and an unvalidated machine judge.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim rests on a 30-pair evaluation that the paper itself contradicts: Table 3 reports a 5-document test set and §7 says evaluation was limited to five academic papers; the 'human-level' conclusion is not supported by the reported evidence.","rationale":"The paper makes a strong central claim and the evidence section is where it breaks. I am not objecting to the modular architecture, the task formulation, or the benchmark idea; the pipeline is clearly described and the demo suggests a working system. The problem is specifically that the paper's own limitation section and Table 3 undermine the abstract's 30-pair human-level claim. This is a mechanical contradiction, not a disagreement about evaluation philosophy. If the authors had run a full 30-pair evaluation with human calibration and released artifacts, the claim could be tested; as submitted, the evidence is insufficient. The reader's verdict of REJECT is appropriate; my concern overlaps with but is slightly more direct than the reader's weakest assumption about VLM validity, because the 5-versus-30 discrepancy is internally verifiable from the manuscript itself.","tokens_in":12995,"tokens_out":4483,"duration_ms":48264,"concrete_test":"Release per-pair PresentEval scores for all 30 document–presentation pairs and run a human rating study on the same five pairs with at least three independent raters using the exact 1–5 rubric from Table 2; then recompute Table 3 statistics on the full 30 pairs and compare VLM ratings with human ratings (e.g., ICC or rank correlation). If the full-30 means differ materially from Table 3, or if VLM-human agreement is at chance level, the 'approaches human-level quality across all metrics' claim fails. Also verify whether the five evaluated documents are exactly the five academic papers mentioned in Section 7.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim ('experimental validation on a curated dataset of 30 document–presentation pairs demonstrates that PresentAgent approaches human-level quality') requires that the reported evaluation actually covers those 30 pairs with a credible measure. The paper's own evidence contradicts both requirements. Section 5.1 describes a 30-pair test set spanning four document types, but Table 3 is captioned 'Detailed evaluation results on the 5-document test set,' and Section 7 states that 'evaluation was limited to five academic papers, potentially underrepresenting the document diversity shown in our benchmark.' Thus the abstract's headline finding is at best a claim about five academic papers, not the 30-pair benchmark, and the paper does not explain how those five were selected. Even for those five, the subjective scores come from Qwen-Omni-7B and the quiz scores from Qwen-VL-2.5-3B with no human rating calibration, no inter-annotator agreement, and no correlation check against human judgments (Section 5.1, Table 2). The load-bearing step is therefore the evidential chain from 'Table 3 numbers on five documents' to 'human-level quality on 30 pairs.' If either the sample-size mismatch or the unvalidated VLM proxy fails, the headline claim is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PresentAgent proposes a modular pipeline for converting long-form documents into narrated presentation videos with synchronized slides and audio, and introduces PresentEval, a VLM-based evaluation framework with objective quizzes and subjective scoring. The paper reports experiments on a curated benchmark and claims that PresentAgent approaches human-level quality across all metrics. The system itself is a four-stage pipeline (outline planning, slide composition, narration/TTS, video assembly), and experiments compare several LLM backends against human-created reference videos on quiz accuracy and 1–5 subjective scores.","tokens_in":13219,"tokens_out":4578,"duration_ms":50228,"significance":"If the claims were fully supported, this would be a useful step toward automatic generation of time-aligned, narrated presentation videos from long documents, a task that existing slide-generation and text-to-video systems do not directly address. The proposed benchmark of 30 document–video pairs and the two-path evaluation idea (factual quizzes plus VLM preference scoring) are potentially valuable resources for the community. The paper also has a sensible modular design with explicit control points, and the authors promise to release code. However, the current evidence does not substantiate the central 'human-level quality' claim: the evaluation is internally inconsistent about the test set size, the scoring models are not validated against human judgments, and no existing baselines are compared. The significance of the contribution is therefore currently contingent on substantial additional evaluation work.","major_comments":[{"comment":"The abstract and Section 5.1 state that the test set consists of 30 document–presentation pairs spanning four document types, but Table 3 is captioned 'Detailed evaluation results on the 5-document test set,' and Section 7 says 'evaluation was limited to five academic papers, potentially underrepresenting the document diversity shown in our benchmark.' These statements are irreconcilable as written. The headline claim of approaching human-level quality on the 30-pair curated dataset is not supported by the reported experiments, and the paper does not explain how the five documents were selected or why the other 25 were omitted. This is the single most load-bearing issue in the manuscript: the central claim must be either backed by results on all 30 pairs or explicitly restated to reflect the five-document evidence.","section":"§5.1 vs. Table 3 vs. §7"},{"comment":"PresentEval's subjective scores are produced entirely by Qwen-Omni-7B and the quiz answers by Qwen-VL-2.5-3B, with no human rating study, no inter-annotator agreement, and no correlation check against human judgments. The paper states that this enables 'human-aligned evaluation,' but that claim is not demonstrated. Moreover, Section 5.2 says Qwen-VL-2.5-3B-Instruct is also used inside PresentAgent for slide-layout self-critique, so the generator and the judge share the same model family and likely share systematic preferences. This circularity makes the 'approaches human-level' conclusion particularly fragile. The authors should validate PresentEval against human raters on at least a subset of videos, or report the correlation, before using it as the primary evidence for the paper's central claim.","section":"§5.1, §5.2, Table 2"},{"comment":"Even on the five documents, the reported differences are within the noise of the evaluation procedure. Quiz accuracy is computed from five questions per document (25 questions total for the five-document set), so a one-question difference changes accuracy by 0.04; the human reference scores 0.56 and the best system 0.64, which is a two-question difference on 25 items. The subjective scores are single VLM judgments on a 1–5 scale with no variance, no repeated sampling, and no statistical test, yet the text interprets differences of 0.1–0.2 as meaningful (e.g., 'Gemini-2.5-flash scores highest in visual quality but lower in comprehension'). These numbers do not support the conclusion that PresentAgent 'approaches human-level quality' or that one backend is better than another.","section":"Table 3, §5.3"},{"comment":"The paper claims in the contribution list that PresentAgent 'significantly outperforms competitive variants,' but the experiments contain no comparison to any existing document-to-slide or document-to-video baseline, and no ablation of the proposed modules. The only comparison in Table 3 is among different LLM backends inside the same PresentAgent pipeline. Section 5.1 explicitly says 'our focus is not on competing with existing baselines,' which contradicts the contribution claim. To support the stated contribution, the authors should compare against at least one strong prior method, such as PPTAgent or Doc2PPT adapted to video output, and include module ablations (e.g., with/without outline planning, with/without dynamic routing, with/without self-critique).","section":"§5, §8, contribution bullet"}],"minor_comments":[{"comment":"Section 4.3 says each narration is controlled to 'typically between 30 and 150 seconds,' but Section 3.1 states that the reference videos are 1–2 minutes long and contain 5–10 slides. Five slides at 30 seconds each would already exceed two minutes, and ten slides at 150 seconds each would be far longer. The target-duration range should be reconciled with the stated video length.","section":"§4.3 vs. §3.1"},{"comment":"The references list both Zheng et al. 2025a and Zheng et al. 2025b as the same PPTAgent paper; these should be merged into a single entry with consistent citation keys.","section":"References"},{"comment":"The sentence explaining the split evaluation strategy says the reason is that no single multimodal model can assess 'videos longer than two minutes,' yet the paper also says each synthesized video is approximately two minutes. This reasoning is confusing and should be clarified or removed.","section":"§5.1"},{"comment":"The table contains a typo in the header: 'Prensentation' should be 'Presentation.'","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the gap between the abstract's claim of a 30-pair validation with human-level quality and the actual five-document, VLM-only evaluation. I would ask the editor to require that the revision either run the full 30-pair evaluation with human raters and a baseline comparison, or substantially weaken the claims to match the evidence. The modular pipeline and benchmark idea are worth pursuing, but the current version overclaims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about this one. First, it formulates a genuinely new task—document-to-presentation-video generation—and backs it with a 30-pair benchmark and a VLM-based evaluation framework. That part is a real contribution to the multimodal generation subfield. Second, the paper's central claim—that PresentAgent 'approaches human-level quality' on those 30 pairs—is not supported by the evidence in the paper. Table 3 is explicitly captioned as a 5-document test set, and Section 7 admits evaluation was limited to five academic papers. The abstract and Section 5.1 describe a 30-pair validation. That is a direct internal contradiction, and it is load-bearing.\n\nWhat the paper does well: the modular pipeline is coherent and the writing is clear. The authors are honest about some limitations—they say the generation module re-implements PPTAgent's editing paradigm, and they explicitly flag the five-paper limitation. The quiz-based comprehension protocol is a reasonable idea, and the benchmark itself, Doc2Present, could be useful to the community even if only the metadata is released.\n\nSoft spots, in order of severity. First, the sample mismatch: 30 pairs claimed, 5 evaluated, no explanation of how the 5 were selected. That alone sinks the headline result. Second, the evaluation is entirely model-based: Qwen-Omni-7B scores subjective quality, Qwen-VL-2.5-3B answers the quizzes, and Qwen-VL is also used for self-critique in generation. No human ratings, no inter-annotator agreement, no correlation check against human judgment. So the 'human-level' language is doing work the numbers can't support. Third, the missing baseline against PPTAgent or Doc2PPT is a gap, since the slide generation is explicitly built on PPTAgent's approach. These are not minor stylistic issues; they affect reproducibility and the validity of every conclusion in the paper.\n\nWho this is for: someone working on document-to-multimodal generation, especially presentation slides or narrated video, will want to know the task exists and read the benchmark description. But they should not cite the system's performance as a result. With a full 30-pair evaluation, human calibration, and released code, this could be a solid paper. Right now, it is a good idea with an unsupported headline.\n\nFor peer review: I would send it out rather than desk reject—the task and benchmark deserve at least one round of serious comment—but the authors need to fix the evaluation before it can be accepted. The abstract and conclusion should be rewritten to match the actual evidence.","headline":"New task and benchmark, but the 'human-level' claim is contradicted by the paper's own 5-document evaluation; the central result is unsupported.","tokens_in":13786,"tokens_out":2565,"would_cite":true,"duration_ms":24343,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PresentAgent introduces a modular pipeline that converts long-form documents into fully narrated, time-aligned presentation videos and reports that it approaches human-level quality on a vision-language-model-based evaluation.","keywords":["document-to-presentation video generation","multimodal agent","slide generation","text-to-speech narration","audio-visual alignment","vision-language model evaluation","Doc2Present benchmark","presentation video evaluation"],"falsifier":"Give the same document–video pairs to human viewers: have them answer the same five multiple-choice questions and rate content, visual, and audio quality on the same 1–5 scales, then compare their scores and answer patterns to Qwen-Omni-7B and Qwen-VL-2.5-3B outputs. Low correlation between model and human scores, or human quiz accuracy on PresentAgent videos falling below the human-authored video reference, would refute the approaching-human-level-quality claim.","tokens_in":12768,"feed_emoji":"🎬","tokens_out":8526,"duration_ms":76851,"temperature":0.7,"pith_summary":"PresentAgent is a new system for a task the paper names document-to-presentation video generation: taking a long document, such as an academic paper, web page, technical blog, or slide deck, and producing a narrated video in which slide frames and spoken audio are aligned in time, the way a human presenter would deliver it. The paper argues that existing tools stop at static slides or text summaries, and that a modular pipeline of outline planning, layout-aware slide rendering, narration script generation, text-to-speech synthesis, and video assembly can close that gap. To back the claim, the authors introduce PresentEval, a two-part evaluation that scores generated videos with a multiple-choice quiz answered by a vision-language model and 1–5 preference ratings for content, visual, and audio quality. On their curated set of 30 document–presentation pairs the system is said to approach human-level quality; the reported head-to-head table covers a five-document subset, a scope limit the authors attribute to API cost. If the claim holds, the practical payoff is that dense written material can be turned into accessible spoken-and-visual presentations without manual scripting, voice recording, or slide design.","feed_headline":"Converts long documents into narrated presentation videos","feed_subtitle":"Modular pipeline pairs slides with TTS narration and is reported to approach human-level quality scores.","key_machinery":"The mechanism that carries the argument is the slide–audio pair. Each semantic block of the document yields one slide $S_k$ and one narration audio $T_k$, and the video is the time-aligned composition of these pairs; alignment works because the narration is generated first and each slide is displayed for the duration of its own audio segment. Around this core sit two supporting components: a slide planner that maps content blocks to HTML-like layout templates with editable operations (replace text, insert image, add list), and PresentEval, a two-path scoring protocol in which a vision-language model answers five quiz questions per document and rates content, visual, and audio quality on a 1–5 scale. The paper also uses a dynamic routing policy across six LLM backends for generation and a TTS system with prosody controls, but the pairing of slide and narration is what makes the output a presentation video rather than a slideshow with voiceover.","core_discovery":"PresentAgent's central claim is that a document-to-presentation video can be produced by a four-stage generative pipeline, formulated as $V = \\mathrm{Compose}(\\{(S_1,T_1),\\dots,(S_K,T_K)\\}) = g(D)$: the input document $D$ is segmented into semantic blocks through outline planning; each block is matched to a layout template and rendered as a slide frame $S_k$; a language model rewrites the block's key message into oral-style narration, which a text-to-speech system turns into audio $T_k$; and the final video $V$ displays each slide for exactly the duration of its audio. The paper claims this output closely mimics human-style delivery, and that PresentEval's two-path assessment, fixed multiple-choice questions for factual comprehension plus VLM preference scoring along content, visual, and audio dimensions, shows the system approaching human-level quality across all measured metrics. The authors position this as the first fully integrated solution to the task, in contrast to prior work that generates static slides or isolated audio summaries.","pith_inferences":["The paper's split evaluation scores visuals and audio separately, which suggests a natural next test it leaves open: a fusion-aware scorer that checks whether narration and slides agree semantically would likely re-rank the six model variants.","A direct way to extend the effective-communication claim beyond quiz accuracy would be to measure real audiences' retention or learning gain after watching the generated videos; the paper does not run that study.","The five-document evaluation bottleneck implies that the diversity claims about the 30-pair benchmark are untested; a cheaper open-model pipeline could run the same protocol at full scale and reveal where the system degrades.","If PresentEval's VLM scores were validated against human ratings, the same quiz-and-preference protocol could be reused as a general benchmark for any document-to-multimodal generation task, not just presentation video."],"forward_implications":["Document-to-slide systems that today stop at static decks can be extended to fully narrated, time-aligned video by adopting the slide–audio pairing step.","PresentEval gives future presentation-video generators a scalable, reference-free scoring protocol: quiz accuracy plus VLM ratings, with no human annotation except the quiz questions themselves.","Because the pipeline is modular, replacing any stage, whether outline planner, slide renderer, narration model, or TTS engine, should improve the final video without redesigning the whole system.","The reported results imply that for at least some document types, an automated pipeline can produce videos whose factual content survives a comprehension quiz as well as a human-authored video does.","If the approach transfers to business reports, policy briefs, and technical manuals, the main cost of producing accessible narrated presentations shifts from human labor to compute and API usage."],"supporting_citations":[{"why":"Supplies the editing-based slide-construction paradigm that PresentAgent re-implements and tailors to narration-ready slide frames.","marker":"Zheng et al. (2025b)"},{"why":"Supplies the quiz-style evaluation protocol and layout-planning methodology that PresentEval and the benchmark build upon.","marker":"Pang et al. (2025)"},{"why":"Technical report behind Qwen-Omni-7B, the vision-language model that performs all subjective scoring in PresentEval.","marker":"Xu et al. (2025)"},{"why":"Technical report behind Qwen-VL-2.5-3B, the model that answers the comprehension quizzes in the objective evaluation path.","marker":"Wang et al. (2024b)"},{"why":"Prior document-to-slide work that defines the static-output baseline the new task moves beyond.","marker":"Fu et al. (2022)"},{"why":"Model card for Claude-3.7-Sonnet, one of the six generation backends and the variant with the highest reported quiz accuracy.","marker":"Anthropic (2024)"},{"why":"Model card for Gemini-2.5-Pro and Gemini-2.5-Flash, two of the generation backends compared in the main results.","marker":"DeepMind (2024)"}],"fun_headline_variants":["Multimodal agent turns documents into narrated presentation videos","From long docs to synchronized slide-and-voice videos with one agent","PresentAgent weaves slides and speech from a single text source","Document to full presentation video: planning, rendering, and narration","Agentic pipeline that transforms text into spoken slideshow videos"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that PresentAgent approaches human-level quality rests on scores produced by two vision-language models answering quizzes and rating videos, with no human rating study or correlation check to confirm those scores match how real audiences perceive the presentations; if those model scores diverge from human judgment, the human-level conclusion fails.","fun_headline_variants_meta":{"raw":{"variants":["Multimodal agent turns documents into narrated presentation videos","From long docs to synchronized slide-and-voice videos with one agent","PresentAgent weaves slides and speech from a single text source","Document to full presentation video: planning, rendering, and narration","Agentic pipeline that transforms text into spoken slideshow videos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000123,"raw_usage":{"total_tokens":1101,"prompt_tokens":945,"completion_tokens":156,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":72}},"tokens_in":561,"tokens_out":156,"duration_ms":2663,"temperature":1.0,"reasoning_tokens":72,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:56:51.518742+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give the same document–video pairs to human viewers: have them answer the same five multiple-choice questions and rate content, visual, and audio quality on the same 1–5 scales, then compare their scores and answer patterns to Qwen-Omni-7B and Qwen-VL-2.5-3B outputs. Low correlation between model and human scores, or human quiz accuracy on PresentAgent videos falling below the human-authored video reference, would refute the approaching-human-level-quality claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Model card for Claude-3.7-Sonnet, one of the six generation backends and the variant with the highest reported quiz accuracy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Model card for Gemini-2.5-Pro and Gemini-2.5-Flash, two of the generation backends compared in the main results."}],"review_version":1}