{"id":"c62f862d-31b8-4963-839e-b71c926d8f0f","arxiv_id":"2607.17675","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Learnable planning tokens with fractional positional timestamps let one diffusion pass generate multi-shot video with frame-accurate cuts and timed camera motion.","lead":"ShotPlan adds small trainable tokens to a video-generation model so a single generated clip can contain several shots that switch at user-chosen frame numbers. The idea matters because current cinematic tools force hard cuts or mask off parts of the model, while this method keeps the original architecture and also works for timed camera moves.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transition Deviation may reward hard-cut output rather than temporal control: TransNet V2 detection is confounded with transition abruptness, so the headline 0.64 vs 1.12/2.71 gap needs a human-annotation check.","rationale":"The reader's weakest assumption correctly highlights evaluation circularity and missing statistical support. I agree, but I want to pin the single most load-bearing version of that concern to the Transition Deviation metric specifically, because it directly underlies the strongest claimed quantitative advantage. The FRoPE ablation and attention analysis are genuine independent evidence that planning tokens localize attention and that fractional coordinates matter for the model's own training objective. However, the cross-method comparison in Table 1 is the evidence for the headline 'better than prior methods,' and that comparison is insecure if the detector is not transition-type-invariant. A human-annotation probe is the most direct way to settle whether the reported 0.64-frame accuracy reflects real temporal control or detector/abruptness bias. The reader's verdict was already CONDITIONAL; my concern reinforces the conditionality (code/data release plus metric auditing) rather than moving it to a different category, so UNCHANGED is appropriate.","tokens_in":12783,"tokens_out":7869,"duration_ms":91064,"concrete_test":"Run a controlled temporal-precision probe using the 100 existing prompts plus the 10 long-take prompts. For each requested hard cut, ask three human annotators to mark the first frame where the scene changes (or mark 'no clear boundary' for soft transitions). Compute mean absolute/signed frame error for ShotPlan, the w/o-FRoPE ablation, and for MultiShotMaster/HoloCine on the subset where a boundary is detectable. Also record TransNet V2 detection failure rates per model. If ShotPlan's human-marked error is ≈0.64 frames and TransNet agrees, the concern is resolved. If human-marked error is ≥2 frames (the VAE latent stride), the frame-level claim fails. If baselines have >20% 'no clear boundary' cases, Table 1's Transition Deviation is not comparable across methods.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing point is not the FRoPE mechanism itself — the internal ablation in Table 2 (removing Fractional RoPE raises Transition Deviation from 0.64 to 2.13) and the attention visualization in Fig. 4 are consistent with the mechanism working as described. The weak link is the headline comparison in Table 1. Transition Deviation is defined as the frame-level gap between TransNet V2 detections on the generated 81-frame video and user-specified timestamps (Sec. 4.1). TransNet V2 is a shot-boundary detector trained on real edited video and is sensitive to transition abruptness. ShotPlan is trained to emit hard cuts at specified frames, whereas baselines such as CineTrans and HoloCine are designed around soft/gradual cinematic transitions. For a soft transition there is no unique boundary frame, so measuring a single-frame deviation against a requested hard-cut timestamp conflates transition-style mismatch with temporal-control error. A detector miss, or a detection at the center of a dissolve, can produce penalties of several frames that have nothing to do with when the scene change was intended. The 0.64 vs 1.12/2.71 gap could therefore overstate ShotPlan's advantage in temporal control. The absence of error bars, detection-rate reporting, or a human-marked ground-truth set makes it impossible to tell whether the advantage is real or an artifact of the detector.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"ShotPlan proposes a framework for multi-shot cinematic video generation built on the Wan2.1-T2V-14B diffusion transformer. The method introduces learnable planning tokens that are concatenated to the visual token sequence and equipped with Fractional Temporal Rotary Position Embedding (FRoPE), which maps user-specified frame-level transition timestamps to fractional latent coordinates. Training data is curated from VideoEvent using TransNet V2 shot detection and Gemini 2.5 filtering/annotation. Experiments compare ShotPlan with CineTrans, EchoShot, HoloCine, and MultiShotMaster on transition deviation, text alignment, DINOv2-based inter-shot consistency, and Gemini-judged narrative coherence, plus a user study for temporally localized camera motion. The paper claims state-of-the-art transition accuracy and inter-shot consistency.","tokens_in":13178,"tokens_out":5342,"duration_ms":49263,"significance":"If the empirical claims are fully supported, ShotPlan would be a valuable contribution. The design is elegant: it preserves the pretrained DiT architecture and the original RoPE for video tokens, adds only a small set of learnable conditioning tokens, and FRoPE is a parameter-free extension that addresses VAE temporal compression. The internal ablation in Table 2 and the attention visualization in Fig. 4 are consistent with the proposed mechanism working as described. However, the headline comparison against baselines is currently not decisive because the main metric is confounded with transition style, no uncertainty quantification is provided, and parts of the evaluation rely on the same model used for data curation. The paper needs stronger, less circular validation before the central claim can be accepted.","major_comments":[{"comment":"Transition Deviation is computed as the frame-level gap between TransNet V2 detections and user-specified timestamps. TransNet V2 is trained on real edited footage and is sensitive to transition abruptness. ShotPlan is explicitly trained to emit hard cuts at precise frames, whereas CineTrans and HoloCine are designed around soft/gradual cinematic transitions. For a dissolve or other gradual transition there is no unique boundary frame, so a detector firing at the transition center or missing the transition incurs a penalty that does not reflect temporal-control error. The 0.64 vs 1.12/2.71 gap may therefore conflate transition-style mismatch with timing accuracy. Please add a human-annotated frame-level evaluation on the same generated videos, report detection rates and confidence, and break down results by requested transition type (hard cut vs soft transition). This is central to the '","section":"Sec. 4.1, 'Metrics' and Table 1"},{"comment":"No error bars, standard deviations, or significance tests are reported for any automatic metric. The benchmark consists of 100 prompts, but the number of generated videos per method and per metric is not stated. Differences such as Text Alignment 0.26 vs 0.28 and Character Consistency 0.46 vs 0.39 may be within sampling noise. Please report per-prompt paired results, confidence intervals, and a paired significance test (e.g., bootstrap or Wilcoxon) for each metric. Without this, the claim that ShotPlan 'significantly outperforms' is unsupported.","section":"Table 1 and Sec. 4.2"},{"comment":"Gemini 2.5 is used to construct the evaluation prompts, filter the training data, generate shot captions, and judge Narrative Coherence. This creates a risk that the method is tailored to a particular judge and that the reported narrative scores reflect agreement with Gemini rather than general cinematic quality. This is not by itself a fatal flaw, but it needs mitigation: report human agreement on evaluation samples, use an independent captioning model for text alignment, and make the judge prompts public. The manual screening protocol for the curated data should also be described in enough detail to assess selection bias.","section":"Sec. 3.4 and Sec. 4.1"},{"comment":"The statement 'For fair comparison, we adapt each prompt to the input format required by each baseline' is too vague to establish equivalence. It is unclear whether the baselines receive the same transition timestamps, the same shot count, the same structured per-shot captions, and the same hard-cut versus soft-transition specification. If a baseline cannot consume a frame-level hard-cut schedule, then the Transition Deviation comparison is unfair because the task is not identical. Please provide a concrete example of the adapted prompt for each method and document any constraints (e.g., minimum shot duration) imposed on baselines.","section":"Sec. 4.1, 'Baseline'"},{"comment":"The camera-movement user study lacks essential setup details: number of participants, number of videos per condition, whether the evaluation is paired within participants, the exact wording of the 'Timing Accuracy' question, and any statistical analysis. The reported differences (e.g., Timing 97% vs 96% for Wan 2.1; Type 89% vs 81% for Kling 2.6) are not accompanied by confidence intervals or significance tests. The Wan 2.1 baseline is also not described: how is the motion instruction provided to it? Please supply the full protocol and per-item results, or weaken the claim to 'competitive with' rather than 'the highest timing accuracy.'","section":"Sec. 4.6, Table 3"}],"minor_comments":[{"comment":"Typo: 'we introduceShotPlan' should read 'we introduce ShotPlan'.","section":"Abstract"},{"comment":"The text says 'With temporal compression ratios t' but the symbol is later s_t; please fix the notation. Also, the equation for f = 1 + (T - 1)/s_t is broken across lines in the PDF; please reformat.","section":"Sec. 3.3"},{"comment":"The paragraph for Global AdaLN says it 'shows little improvement over the uncontrolled baseline,' but no uncontrolled baseline is shown in Table 2. Either include that row or rephrase to refer to an earlier sentence.","section":"Sec. 4.4, Table 2"},{"comment":"The caption says 'the vertical axis represents the flag tokens' — this should be 'planning tokens.' Also, the x-axis label 'Latent Frames (Time)' could be clearer as 'Latent temporal index'.","section":"Fig. 4 caption"},{"comment":"Minor style: '8-th DiT block' should be '8th DiT block.'","section":"Sec. 4.5"},{"comment":"The model name appears as both 'SeedDance 1.5 Pro' and 'Seedance 1.5'; please use one spelling consistently.","section":"Table 3 and text"},{"comment":"Reference [17] (LoRA) is cited only in the supplementary material. If it is not needed in the main text, consider moving it to the supplementary reference list or citing it in the main text for completeness.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The proposed mechanism is interesting and the internal ablations suggest it is plausible, but the current evaluation does not yet support the central claims of frame-accurate control and significant improvement over baselines. The Transition Deviation metric is confounded with transition style, and the lack of uncertainty quantification, the vague baseline conditioning, and the reliance on Gemini for both curation and evaluation are load-bearing concerns. I would be willing to look at a revision that adds human-annotated timing evaluation, per-prompt statistical analysis, a detailed baseline-conditioning protocol, and fuller reporting of the camera-motion user study. The paper may also benefit from releasing code/data to enable reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read of arXiv:2607.17675. The core idea is genuinely new: instead of attention masks or RoPE discontinuities to separate shots, they insert learnable tokens with fractional temporal coordinates (FRoPE) into the DiT self-attention, leaving the video tokens' RoPE untouched. The internal ablation in Table 2 is the cleanest evidence: removing FRoPE degrades transition deviation from 0.64 to 2.13, and the attention visualization in Fig. 4 shows each token peaking at its assigned cut. That part holds up. The camera-movement extension is a nice bonus, though the user study is small.\n\nThe soft spots are mostly in the quantitative comparison, not the mechanism. Table 1 has no error bars, no significance tests, and the baselines are not necessarily conditioned the same way—the paper says prompts are adapted, but that's not the same as matching transition style. The bigger issue is the Transition Deviation metric. It uses TransNet V2, a detector trained on real edits. ShotPlan is trained to emit hard cuts at exact frames, while CineTrans and HoloCine produce soft transitions. For a dissolve there's no unique boundary frame, so any single-frame deviation metric will punish those baselines for not being hard cuts, not for being temporally off. The 0.64 vs 1.12/2.71 gap could be largely an artifact of transition abruptness, and without a human-annotation check or detection-rate reporting, I can't tell.\n\nThe data pipeline is also heavily filtered by Gemini 2.5, which serves as annotator and evaluation judge; that's a circularity risk, and there's no code or data released. The user study for camera motion lacks participant numbers and setup. So the empirical evidence is thin, even though the mechanism is plausible.\n\nBottom line: worth a serious referee, but not as is. The important revision is to verify the transition-deviation result against human-marked boundaries and to equalize transition style across baselines. If that survives, the FRoPE token idea is a real contribution.\n\nI'd maybe bring it to reading group. I'd cite it if I worked on controllable video generation, mainly for the FRoPE design.","headline":"ShotPlan makes a plausible new move—learnable FRoPE-positioned planning tokens for frame-accurate shot control—but the headline comparison is undercut by a metric confound.","tokens_in":13648,"tokens_out":2494,"would_cite":true,"duration_ms":22291,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A few learnable tokens, positioned at fractional latent coordinates, give a pretrained video diffusion model frame-accurate shot transitions and better cross-shot consistency than prior multi-shot approaches.","keywords":["multi-shot video generation","shot planning","learnable tokens","fractional rotary position embedding","diffusion transformer","cinematic video","temporal control","video diffusion"],"falsifier":"Have independent human annotators mark the perceived cut frames on a fresh set of generated videos (e.g., 50 prompts with two to four specified cut frames), then compare median absolute error in frames against the same annotators' ratings for the strongest prior baseline. If the median error for ShotPlan is not clearly lower, or if human consistency ratings do not track the reported automated feature-similarity scores, the central claim of frame-accurate, more consistent multi-shot generation is not supported. A cheaper check: replace the detector-based transition measure with manual frame lab","tokens_in":12708,"feed_emoji":"🎬","tokens_out":7257,"duration_ms":58728,"temperature":0.7,"pith_summary":"The paper tries to establish that shot planning in cinematic multi-shot video generation can be made explicit and frame-accurate without disturbing the pretrained architecture of a video diffusion transformer. It does this by inserting a small number of learnable planning tokens into the token sequence, one per transition, and positioning them with a fractional variant of rotary position embedding so each token points at a precise physical frame. The authors argue this beats prior approaches—which rely on attention masks or modified positional encodings—because it preserves free cross-shot information flow, yielding tighter transition timing and stronger character and scene consistency. If true, the idea gives creators a lightweight way to script cuts, fades, and temporally localized camera moves inside a single generation pass.","feed_headline":"Frame-accurate video cuts with learnable planning tokens","feed_subtitle":"Frame-accurate cuts keep characters and scenes consistent across shots without touching the model's attention.","key_machinery":"Learnable planning tokens plus Fractional Temporal Rotary Position Embedding (FRoPE). A planning token is a small trainable vector replicated according to the number of transition events, each copy receiving a real-valued (fractional) temporal coordinate instead of a discrete latent index. Since rotary embeddings are sinusoidal, FRoPE plugs the fractional coordinate directly into the existing RoPE computation, allowing the token to point at an exact frame even under VAE temporal compression. The tokens are concatenated with visual tokens, pass through all DiT blocks as ordinary sequence elements, act as in-context conditioning, and are removed before decoding.","core_discovery":"ShotPlan's central claim is that a transition is best represented as a learnable token that acts as an in-context conditioning anchor in the video transformer's own coordinate system. The planning tokens are concatenated with the patchified video tokens and processed through ordinary self-attention, so no attention structure or positional encoding of the backbone is modified. Fractional Temporal Rotary Position Embedding (FRoPE) maps a user-specified frame timestamp into a continuous latent coordinate, which matters because the video autoencoder compresses four physical frames into one latent step. On the paper's benchmark, the method reports a transition deviation of 0.64 frames versus 1.12","pith_inferences":["If the mechanism generalizes, the same token-anchoring idea could control other time-localized events in video (action beats, lighting changes, audio cues), since it does not depend on the semantics of a cut.","The evaluation leans on automated surrogates—an off-the-shelf detector for cut timing and a single multimodal judge for consistency—so an independent human study on a held-out prompt set would be the decisive test of whether the reported gains hold perceptually.","The camera-motion result suggests a cheaper path to motion control than training dedicated adapters, but the paper tests only six predefined moves; broader motion vocabularies would reveal the limits of token-propagation.","A targeted ablation of token count versus number of cuts could clarify whether the method scales to long videos with many transitions or degrades as token sequence grows."],"forward_implications":["Users can specify hard cuts (and fade start/end points) at exact frame numbers, and the model honors them within about a frame.","Because the base model's attention and positional encoding are left intact, cross-shot information flows freely, which the authors connect to better preservation of character identity and scene layout.","The same token mechanism extends to gradual transitions and to temporally localized camera motion, suggesting a single controller for discrete and continuous temporal events.","The approach needs only modest fine-tuning data (roughly 13,000 curated clips and a few thousand training steps) to adapt a large pretrained video diffusion transformer."],"fun_headline_variants":["Learnable tokens direct video cuts to the exact frame","ShotPlan: planning tokens frame precise shot transitions","Frame-accurate cinematic cuts with planning tokens","Tokens plan cuts frame-by-frame without altering attention"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that an off-the-shelf shot-boundary detector and a single multimodal model used as both data curator and evaluation judge are faithful proxies for human perception of cut timing and cross-shot consistency; if either is biased, the reported margins may not survive human evaluation.","fun_headline_variants_meta":{"raw":{"variants":["Learnable tokens direct video cuts to the exact frame","ShotPlan: planning tokens frame precise shot transitions","Frame-accurate cinematic cuts with planning tokens","Tokens plan cuts frame-by-frame without altering attention"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000166,"raw_usage":{"total_tokens":1037,"prompt_tokens":637,"completion_tokens":400,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":381,"completion_tokens_details":{"reasoning_tokens":340}},"tokens_in":381,"tokens_out":400,"duration_ms":4174,"temperature":1.0,"reasoning_tokens":340,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T17:17:27.240713+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have independent human annotators mark the perceived cut frames on a fresh set of generated videos (e.g., 50 prompts with two to four specified cut frames), then compare median absolute error in frames against the same annotators' ratings for the strongest prior baseline. If the median error for ShotPlan is not clearly lower, or if human consistency ratings do not track the reported automated feature-similarity scores, the central claim of frame-accurate, more consistent multi-shot generation is not supported. A cheaper check: replace the detector-based transition measure with manual frame lab","supporting_citations":[],"review_version":1}