{"id":"a280f973-ef43-41a9-bfbf-7e7d01fea132","arxiv_id":"2607.19038","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"FilmWorld generates multi-scene films from novels by materializing an explicit evolving world-state trajectory and rendering shots in parallel, beating five agents on its own FilmEval benchmark.","lead":"This paper presents FilmWorld, a multi-agent system that turns short novels into films by first building an explicit symbolic world state (characters, locations, props, plot events) and then rendering shots in parallel from that state. It also introduces a 15-novel benchmark with nine automatic metrics and reports large gains over five video-generation agents — though the evaluation is done largely by the same AI model family that powers the system.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FilmEval's Gemini-based nine-metric protocol is the load-bearing support for the 5.19-pt superiority claim; the cross-evaluator re-run does not fully remove same-family judge bias, and six-annotator human validation is too thin to independently confirm the margin.","rationale":"The reader's weakest assumption identifies the same load-bearing concern I find: the empirical superiority claim is evaluated by a protocol whose judge is drawn from the same model family that generates the films, with only a small human study as independent check. I agree this is the most serious risk to the central claim. I note, however, that Appendix B.4 is real supporting evidence—two additional MLLM judges reproduce FilmWorld's first-place ranking—so the concern is not that the results are fabricated, but that they may not generalize to an independent, non-MLLM evaluation. The human study's small annotator pool and lack of inter-rater reliability statistics make it insufficient to fully settle the question. I also observed a separate internal issue in Section 4.4: the claim that O(N/P) latency is 'substantially sub-linear in N' is mathematically incorrect for fixed P (it is linear in N), and the O(1) claim holds only under unbounded parallelism. However, this latency argument is secondary to the quality-superiority claim and would not change the verdict if the evaluator-bias concern were resolved. Therefore the reader's CONDITIONAL verdict remains appropriate; no verdict adjustment is needed.","tokens_in":42334,"tokens_out":9866,"duration_ms":117412,"concrete_test":"Release the 90 generated films and exact FilmEval prompt templates, and have an independent lab run a pre-registered, blinded human study with at least 15 annotators (not authors) rating a difficulty-stratified subset on the three FilmEval dimensions. Compute the 95% confidence interval for the mean FilmWorld-minus-VideoClaw difference and per-item inter-rater agreement (e.g., Krippendorff's α). If the interval includes 0 for Novel Fidelity or Film Consistency, or the system-level ranking changes, the same-model-judge bias concern lands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that FilmWorld consistently outperforms SOTA agentic systems, with a 5.19-point overall margin (89.38 vs 84.19 in Table 1)—rests entirely on FilmEval's nine automatic metrics. Section 5.3 states that all metrics are implemented by prompting Gemini 3.1 Pro, the same model family used as FilmWorld's planner and internal verifier. If Gemini's judgments are biased toward outputs that match its own planning conventions, checklist structure, or stylistic defaults, the reported margin may not transfer to an independent evaluator. Appendix B.4 partially mitigates this: re-running with Qwen3.5-Omni and Seed2.0-Lite still ranks FilmWorld first. However, all three judges are MLLMs with overlapping web-scale training priors, so this does not fully remove the concern. The only non-MLLM validation is a human study with six annotators and 540 judgments (§6.2), which reports Spearman ρ=1.0 for system ranking but no inter-annotator agreement or confidence intervals on the decisive FilmWorld-vs-VideoClaw margin. Every downstream conclusion—ablations, difficulty invariance, human alignment—is scored by the same protocol, making this evaluator-validity assumption the weakest link in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes novel-to-film generation as dynamic cinematic world modeling, decomposing the task into a Construction Phase that grounds literary prose into a persistent, stateful world and an Evolution Phase that renders shots from a pre-materialized state trajectory. It proposes FilmWorld, a six-agent system implementing this decomposition, and FilmEval, a difficulty-graded benchmark of 15 novels with nine automatic metrics across Cinematic Presentation, Film Consistency, and Novel Fidelity. The main claims are: (i) explicit externalization of world state improves long-range coherence, with FilmWorld outperforming five agentic baselines by 5.19 points overall (Table 1); (ii) state externalization reduces Evolution-Phase latency from O(N) to O(1) (Eqs. 8–10), with measured 5.62× speedup on keyframe generation; and (iii) FilmEval's metrics align with human judgment. The formalization and system design are coherent and the latency argument is internally sound under a bounded-context condition, but the empirical support for the headline margin is weakened by the fact that the automatic judge is the same model family as the generator, and the independent human validation is thin.","tokens_in":42680,"tokens_out":8690,"duration_ms":96531,"significance":"If the empirical claims hold, this is a substantial contribution: it reframes long-form video generation as a world-modeling problem rather than a per-shot synthesis problem, introduces an explicit and addressable state representation that can be shared across agents, and offers a plausible mechanism for parallelizing shot rendering. The paper's strengths include a clean formalization (Section 3), a detailed system design (Section 4), a public benchmark with per-novel scores (Appendix B.1), ablations isolating each component (Table 4), a cross-evaluator stability check (Appendix B.4), a human evaluation study (Section 6.2), a style generalization study (Appendix B.2), and a candid failure analysis (Figure 16, Section E.1). These are genuine assets. The main weakness is that the quantitative superiority claim rests on an automated protocol whose judge shares model ancestry with the system being evaluated; the cross-evaluator re-run mitigates but does not eliminate the concern, and the human validation is too small to independently certify the reported margin. The latency claim is theoretically sound but only partially validated experimentally. These issues are load-bearing for the a","major_comments":[{"comment":"The central empirical claim—a 5.19-point overall margin over VideoClaw (Table 1)—is measured entirely by FilmEval's nine automatic metrics, all implemented by prompting Gemini 3.1 Pro (§5.3), the same model family used as FilmWorld's planner and internal verifier. If the judge systematically prefers outputs matching its own planning conventions or stylistic defaults, the margin may not transfer to an independent evaluator. The cross-evaluator re-run in Appendix B.4 (Qwen3.5-Omni, Seed2.0-Lite) shows the system-level ranking is stable (ρ≥0.9), which is reassuring, but all three judges are web-scale MLLMs with overlapping training priors; this does not fully remove the self-preference risk. The only independent validation is the human study (Table 3), which has six annotators, no inter-annotator agreement, and no confidence interval on the decisive FilmWorld-vs-VideoClaw margin. Please rep","section":"§5.3, Table 1, Appendix B.4"},{"comment":"The State Decoupling Condition is stated as: 'This reduction in complexity holds iff the information set I_t consumed by the renderer at shot t contains no rendered outputs from other shots.' However, Eq. (10) immediately includes {k_j}_{j∈J_t}, where k_j is the keyframe image of shot j, and J_t is a window of other shots. In the actual pipeline, the sliding-window planner consumes keyframes of context shots, so the information set does contain rendered outputs of other shots at the planning stage. The O(1) latency argument is preserved if the keyframe context is bounded (w = g+2c << N), but the text and Eq. (10) are internally inconsistent as written. Please clarify what counts as a 'rendered output' (final video segment vs. keyframe), and state explicitly that bounded keyframe context is permitted because its size does not grow with N.","section":"§4.4, Eq. (10)"},{"comment":"The experimental efficiency validation measures only the keyframe-generation stage, which is described as the most cleanly parallelizable component. The dynamic-planning and video-rendering stages—which are also part of the claimed O(1) Evolution-Phase latency—are not measured end-to-end. The statement that the remaining stages 'share this shot-independent structure and should exhibit analogous gains' is a prediction, not a measurement. Because API rate limits, memory constraints, and batching behavior can make full-pipeline scaling worse than the theoretical O(N/P), please either measure the full Evolution Phase or explicitly qualify the experimental claim to the keyframe stage while keeping the O(1) bound as a theoretical result.","section":"§4.4, Figure 10"},{"comment":"The human evaluation is too thin to independently confirm the reported margin. Six annotators and 540 judgments are small, and no inter-annotator agreement (e.g., Fleiss' κ or ICC) is reported. The per-film correlations in Figure 9 use 90 points, but because the points come from only six systems, the effective sample for system-level inference is small; the reported p-values may be anti-conservative due to clustering. Please report the number of judgments per film per dimension, inter-annotator reliability, and cluster-aware or bootstrapped confidence intervals on the human ratings. This is important because the human evaluation is the only non-MLLM check on the central superiority claim.","section":"§6.2, Table 3, Figure 9"}],"minor_comments":[{"comment":"The nine FilmEval metrics are called 'objective' in the abstract and Section 5.3, but they are scored by a prompted MLLM. 'Automatic' or 'LLM-judged' would be more accurate, and would avoid overclaiming objectivity relative to human perceptual judgment.","section":"Abstract, §5.3"},{"comment":"The text says six annotators rated 90 films on three dimensions, 'yielding 540 judgments.' The arithmetic is unclear: 90 × 3 × 6 = 1620, while 90 × 3 × 2 = 540. Please state explicitly how many judgments each annotator produced per film and per dimension.","section":"§6.2, Table 3"},{"comment":"Reference [31] is cited for 'Gemini 3.1 Pro,' but the listed reference is the original Gemini 1.0/1.5 family paper (2023). Please cite the correct technical report or model card for the specific model version used, and verify all model-version citations in Section 6.1.","section":"References, §5.3"},{"comment":"The column header 'A VP' appears to be a typo for 'AVP' (Audio-Visual Performance Quality). Please correct for consistency with Table 1 and Section 5.3.","section":"Table 2"},{"comment":"Three benchmark entries (C4 from Conquer, C13 from Link Click, C15 from Scissor Seven) are 'screen adaptations' re-novelized from existing audiovisual works. The statement in Appendix A.1 that these were 'independently rewritten' and the claim in E.2 that FilmEval uses only 'original, public-domain, or substantially rewritten sources' may not be legally sufficient, since derivative works of copyrighted media can still infringe even after substantial rewriting. This is not a technical issue, but it should be clarified if the benchmark is released publicly.","section":"Appendix A.1, §E.2"}],"recommendation":"major_revision","confidential_remarks":"The evaluator-bias concern is the main gate. The cross-evaluator appendix is a good-faith attempt, but all three judges are MLLMs and the human panel is small; the 5.19-point margin should not be certified until independent validation or at least much stronger human evidence is provided. The Eq. (10) inconsistency in the decoupling condition is fixable by rewording, but it sits at the center of the latency contribution and should be corrected before acceptance. I do not see a fatal technical error in the formalization or system; the paper's scope is ambitious and the appendix material is unusually detailed, which I view as a strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper has a real idea and a real evaluation problem. The construction/evolution formalization is a genuinely different way to attack long-form narrative consistency: build an explicit world-state trajectory first, then render shots in parallel. The O(1) latency argument is internally clean, and the ablation showing character consistency collapsing from 82.1 to 54.5 when the explicit world state is removed is striking evidence that the mechanism does something. The benchmark design is also a step up from per-shot metrics.\n\nThe soft spot is exactly where the reader put it. All nine FilmEval metrics are implemented by prompting Gemini 3.1 Pro, and FilmWorld uses Gemini 3.1 Pro as its planner and verifier. That is a same-family judge. The cross-evaluator re-run with Qwen and Seed helps, but all three are MLLMs with overlapping training priors; it doesn't establish independence. Six annotators and 540 judgments is thin for a 5.19-point margin, and no inter-annotator agreement or confidence intervals are reported. The human study shows the same system ordering, which is consistent with the margin but doesn't confirm it. I also noticed the code release is for baseline adaptation, not the full FilmWorld system, so full reproduction isn't possible from the paper alone.\n\nThe latency claim is sound in principle, and the experimental scaling result on keyframe generation is a reasonable sanity check, but it's only one stage. The paper would be more convincing with detection-based consistency checks or a genuinely independent human panel of more than six people.\n\nThis is a substantial contribution with testable claims. The formalization alone is worth citing. It deserves a serious referee, but the referee should push hard on evaluator validity and reproducibility. My call: send to review, require stronger evaluation and full code release before acceptance.","headline":"A genuinely new idea in long-form generation—explicit world-state bookkeeping with parallel rendering—but the 5-point superiority claim is scored by the same model family that generates the films, so the margin is not yet credible.","tokens_in":43150,"tokens_out":1939,"would_cite":true,"duration_ms":24528,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Explicit world-state pre-materialization makes long-form film generation parallel and coherent","keywords":["novel-to-film generation","dynamic cinematic world modeling","multi-agent video generation","world state externalization","cross-scene consistency","narrative fidelity","latency reduction","automated evaluation"],"falsifier":"Run the FilmEval protocol with an independent, instruction-tuned evaluator (for instance a different multimodal model or a blind human panel) on the same 90 generated films; if the system no longer ranks first or the margin over baselines shrinks to near zero on novel fidelity, the central empirical claim is not stable.","tokens_in":42246,"feed_emoji":"🎬","tokens_out":3995,"duration_ms":35723,"temperature":0.7,"pith_summary":"The paper argues that the way to generate coherent long films from novels is to model the source as a dynamic cinematic world with persistent, addressable entity states, and to fully resolve how that world changes before any pixels are rendered. It proposes a two-phase agent system that constructs a structured, visually anchored world state from prose, then evolves that state causally through plot events, referencing precomputed state identifiers at every rendering step. The central claimed payoff is that this externalization removes the sequential dependency between shots, cutting the theoretical latency of the rendering-dominant phase from O(N) to O(1) while making cross-scene visual and narrative consistency a matter of state lookup rather than accumulated guesswork. The paper also contributes a difficulty-graded benchmark with nine automatic metrics and reports that its system outperforms five existing agentic video generation baselines, especially on narrative fidelity and cross-scene consistency.","feed_headline":"Pre-materialized world states make film generation parallel and coherent","feed_subtitle":"A two-phase agent system cuts rendering latency to O(1) in shot count and improves cross-scene narrative fidelity.","key_machinery":"The load-bearing object is the Dynamic Cinematic World tuple W=(E,X,Φ,T,R): an entity set E (characters, locations, props), a discretized state space X, a state mapping Φ that hashes each concrete state to a unique identifier, a transition function T that updates the world state from plot events, and a rendering function R that conditions each shot on the active state identifiers. The decisive step is the offline materialization of the state trajectory {W_t} via T during construction, so that at evolution time each shot's renderer consumes no other shot's rendered output, which is what yields the O(1)-in-N latency bound.","core_discovery":"On the paper's own terms, the discovery is that novel-to-film generation should be treated as dynamic cinematic world modeling, decomposed into construction (grounding prose into stateful entities) and evolution (causally updating those states across scenes), and that pre-materializing the entire world-state trajectory as a symbolic blueprint before rendering transforms long-form coherence from a sequential, memory-bound problem into a parallel, lookup-bound one. Concretely, every entity (character, location, prop) is assigned a deterministic state identifier at each shot, all visual assets are anchored to those identifiers at first appearance, and the transition of states under plot events","pith_inferences":["The O(1) latency claim is not tied to the specific agentic implementation: any long-form generator that pre-materializes a state trajectory and satisfies the decoupling condition It ⊆ {Wt, L, dt, keyframes} should inherit the same parallel speedup, making the principle broadly applicable.","Because the same multimodal language model is used both as the system's planner and as the judge for FilmEval's automatic metrics, the reported margins may partly reflect the judge's preference for outputs that match its own stylistic defaults; an independent, instruction-tuned evaluator would be a stronger test of the empirical claim.","The discrete world-state representation (age stage, costume, season, weather) makes consistency easy but quantizes continuous visual evolution such as gradual aging or subtle lighting shifts; blending continuous attributes into the state space is a natural extension that could preserve both consistency and realism.","The closed-loop verifier only repairs local deviations from the prescribed state; if a plot event was mis-assigned during construction, the error propagates silently, so upstream parsing correctness remains the dominant risk for faithfulness."],"forward_implications":["If the state trajectory is correct, cross-shot character and scene coherence become enforced by construction rather than emergent, so errors do not compound with film length.","Because rendering is decoupled from sequential state propagation, all keyframes, motion plans, and video segments within a scene can be synthesized in parallel, making the runtime-dominant phase O(1) in the number of shots (or O(N/P) with P parallel workers).","Narrative fidelity is improved because plot events are explicitly chained into state transitions, so omissions or hallucinations are detectable and correctable by closed-loop verification.","The proposed FilmEval benchmark (15 novels, three difficulty tiers, nine metrics) provides a standardized way to compare long-form generation systems, and the paper reports that its system ranks first on all metric groups and difficulty tiers.","Ablations show that removing explicit world-state modeling collapses character consistency, indicating that state externalization is the structural foundation of the observed gains."],"fun_headline_variants":["From prose to film: pre-built world states unlock parallel shots","Novel-to-film via world-state blueprint: coherence without sequential limits","FilmWorld: modeling worlds, not scenes, for coherent long-form films","Pre-computing entity states makes film generation O(1) and coherent","Dynamic world modeling turns novel-to-film into a parallel task"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper's measured advantage rests on the assumption that its nine automatic metrics, all implemented by prompting a single multimodal language model that is also used as the system's planner, genuinely track human-judged cinematic quality and fidelity; if that judge systematically favors outputs matching its own stylistic defaults, the reported margins could be inflated.","fun_headline_variants_meta":{"raw":{"variants":["From prose to film: pre-built world states unlock parallel shots","Novel-to-film via world-state blueprint: coherence without sequential limits","FilmWorld: modeling worlds, not scenes, for coherent long-form films","Pre-computing entity states makes film generation O(1) and coherent","Dynamic world modeling turns novel-to-film into a parallel task"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000607,"raw_usage":{"total_tokens":2690,"prompt_tokens":796,"completion_tokens":1894,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":1803}},"tokens_in":540,"tokens_out":1894,"duration_ms":12099,"temperature":1.0,"reasoning_tokens":1803,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T13:36:10.406008+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the FilmEval protocol with an independent, instruction-tuned evaluator (for instance a different multimodal model or a blind human panel) on the same 90 generated films; if the system no longer ranks first or the margin over baselines shrinks to near zero on novel fidelity, the central empirical claim is not stable.","supporting_citations":[],"review_version":1}