{"id":"9ab28a58-1cda-427f-9246-01bd605b4bb1","arxiv_id":"2412.12223","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A labeled 2,000-video dataset and a CLIP-guided LoRA composition method give text-to-video models control over twenty cinematic shot types.","lead":"This paper adds cinematic language control to text-to-video generation. It introduces a 2,000-video labeled dataset and a CLIP-guided method that blends camera styles in one generated clip.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 'stable and precise' camera control is not isolated from content: per-category LoRAs are trained on ~100 stock clips without a content-controlled test, and CLIPLoRA's gains may partly reflect optimizing the same CameraCLIP evaluator used for scoring.","rationale":"Read in good faith, the paper describes a coherent pipeline: a curated cinematic dataset, CLIP fine-tuning specialized to cinematic captions, LoRA per category, and a genetic search over LoRA schedules. The R@1 of 0.83 is consistent with fine-tuning a CLIP model on the same-domain validation split, and the FVD improvement of CLIPLoRA (1837 vs. 2534 for the strongest baseline) is at least suggestive. The reader's CONDITIONAL verdict is appropriate because the decisive evidence is missing, not because the method is internally impossible. The most load-bearing gap is that the per-category LoRAs are trained on very small, stock-footage-biased clip sets, and the experiments never test whether the model controls the camera attribute independent of scene content. The paper's own footnote that code, data, and weights will only be released after acceptance makes independent verification impossible now. A blinded content-controlled study plus held-out-prompt CLIPLoRA evaluation would settle the concern. I did not find a need to move to REJECT: the concerns are addressable and the qualitative examples are suggestive. Minor discrepancies (e.g., text cites 0.81 for mean pooling while Table 4 reports 0.83) do not affect the central claim.","tokens_in":11750,"tokens_out":8839,"duration_ms":82007,"concrete_test":"Release Cinematic2K, the trained LoRAs, and generation code, then run a blinded content-controlled attribute test. Fix ten scene prompts with matched content but differing intended camera attributes (e.g., the same subject at eye level vs. bird angle, the same scene as long shot vs. close-up, the same footage described as rack focus vs. still), generate with each single-category LoRA in AnimateDiff, and have at least three cinematography-trained annotators label the dominant framing, angle, and movement without seeing the prompt; report per-category accuracy and confusion with scene content. Also compute inter-annotator agreement on a random sample of Cinematic2K labels. If per-category accuracy on unseen prompts is near chance or tracks scene content, the 'stable and precise control' claim is confounded.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim requires that a LoRA trained on roughly 95-100 Cinematic2K clips encodes the intended cinematic attribute (framing, angle, or movement) and transfers to arbitrary prompts in the frozen AnimateDiff backbone. Section 3.1 builds Cinematic2K from Pexels/Videvo stock footage; Step I classifies each video into one of 20 subcategories and Figure 3 reports ~100 clips per category, while Section 3.3 trains each LoRA on the 'classified videos from the 20 categories.' Stock libraries have strong content priors: bird-angle clips are usually aerial landscape shots, rack-focus clips are usually static close-ups, dolly-in clips often involve a walking subject. If category labels are exclusive, a LoRA can reproduce the category's typical scene composition instead of the camera operation; if labels are not exclusive, the paper never specifies how subclips for a given LoRA are selected. No experiment separates attribute from content: Figure 4 is qualitative, and the only quantitative generation metrics are CLIPSIM and FVD, neither of which measures whether the generated video actually exhibits the requested framing/angle/movement. The CLIPLoRA evaluation is also partially circular: Eq. (2) uses CameraCLIP as the genetic-algorithm fitness, and Table 5 then uses CameraCLIP as the guiding evaluator, so the CLIPSIM/FVD advantage over static baselines may reflect optimization to the evaluator rather than perceptible cinematic quality. R@1=0.83 is a retrieval result on Cinematic2K splits, not evidence of generation control. The paper also reports no inter-annotator agreement for its 'meticulous' labels and no held-out-prompt human study.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a three-part system for cinematic control in text-to-video (T2V) generation. First, Cinematic2K is introduced as a dataset of roughly 2,000 stock videos annotated across twenty subcategories spanning shot framing, shot angle, and camera movement. Second, CameraCLIP is presented as a video-text alignment model obtained by fine-tuning the last layers of CLIP on Cinematic2K and mean-pooling eight frame features. Third, CameraDiff trains one LoRA per cinematic category on the frozen AnimateDiff backbone for single-shot control, and CLIPLoRA uses a genetic algorithm with CameraCLIP as fitness to select a sequence of LoRA activations across diffusion steps for multi-shot composition. Experiments report an R@1 of 0.83 for CameraCLIP on a Cinematic2K validation split, qualitative single- and multi-shot generation results, and CLIPSIM/FVD comparisons showing CLIPLoRA outperforming static LoRA composition baselines.","tokens_in":12125,"tokens_out":3443,"duration_ms":32120,"significance":"If the claims held, the work would fill a real gap: current T2V models offer limited control over framing, angle, and camera movement, and the proposed taxonomy plus LoRA-composition recipe would be a useful practical contribution. The dataset addresses an annotation gap relative to RealEstate10K, CineScale, and CineScale2, and the idea of using a video-text retriever to guide LoRA scheduling is interesting. However, as presented the evidence does not yet support the central claims. The headline retrieval comparison is not apples-to-apples, the generation evaluation is largely qualitative and uses metrics that do not verify camera attributes, and the CLIPLoRA search is optimized with the same evaluator used to score it. The paper is best viewed as a system proposal with promising components that need substantially stronger validation before the claims of stable, precise, and seamless cinematic control are justified.","major_comments":[{"comment":"CameraCLIP is fine-tuned on the Cinematic2K training split, while all baselines (CLIP, CLIP4CLIP, ViCLIP, LongCLIP, VideoCLIP-XL) are evaluated zero-shot. The reported R@1 improvement from 0.77 to 0.83 may therefore reflect in-domain fine-tuning rather than superior cinematic understanding. The authors should fine-tune the baselines on the same training split or evaluate CameraCLIP on an external cinematic benchmark; otherwise the headline claim that CameraCLIP 'surpasses all baselines' is not established.","section":"§4.2, Table 2"},{"comment":"CLIPLoRA uses CameraCLIP as the genetic-algorithm fitness in Eq. (2) and then Table 5 compares CLIPLoRA against variants guided by other CLIP models. Since CameraCLIP was trained on the same Cinematic2K categories that the LoRAs are meant to express, the search is optimized toward the evaluator itself. The resulting CLIPSIM/FVD gains may reflect optimization to CameraCLIP rather than perceptible cinematic quality. The authors should report independent human judgments of framing, angle, and movement correctness, and should also test CLIPLoRA with a held-out evaluator or with an evaluator trained on disjoint categories to break the circularity.","section":"§3.3, Eq. (2); §4.4.2, Table 5"},{"comment":"Each cinematic LoRA is trained on roughly 95–100 stock clips per category (Figure 3). Stock footage categories carry strong content priors: for example, 'bird angle' clips are often aerial landscape shots, and 'dolly in' clips often contain a walking subject. The paper does not provide any content-controlled experiment showing that a LoRA changes the camera attribute while leaving scene content fixed, and Figure 4 is qualitative only. The claim that CameraDiff provides 'stable and precise cinematic control' therefore lacks support. A controlled generation study with identical or matched prompts across categories, or a per-attribute human rating study, is needed.","section":"§3.1, Steps I–III; §3.3"},{"comment":"The quantitative generation metrics are not sufficient for the claim. FVD is computed against WebVid10M reference videos and CLIPSIM measures overall text-video similarity; neither metric verifies that the generated video actually exhibits the requested shot framing, angle, or camera movement. Moreover, no error bars, repeated runs, or significance tests are reported for the CLIPSIM/FVD differences, so the margins in Table 3 (e.g., 0.2535 vs. 0.2394) and Figure 6 may be within noise. The authors should add a human study or an attribute-level automatic metric, and report variance across seeds.","section":"§4.3, Table 3 and Figure 6"},{"comment":"The temporal-modeling ablation is internally inconsistent: the text says mean pooling 'significantly outperformed' Transformer with an accuracy of 0.81, but Table 4 lists 0.83 for Mean Pooling and does not state which ViT backbone and input resolution are used for each row. Since Table 2 shows R@1 varies strongly with model size and resolution, the comparison must hold the backbone fixed and report the same configuration. This is needed to justify the mean-pooling design choice that underlies CameraCLIP.","section":"§4.4.1, Table 4"}],"minor_comments":[{"comment":"The notation for the video feature V as the mean of frame features I_i is clear, but the layer indices for the fine-tuned ViT layers (20–23) should be defined with respect to the specific CLIP variant (e.g., ViT-L/14) used in the main results.","section":"§3.2, Eq. (1)"},{"comment":"There is a typo in 'VAEs' and inconsistent capitalization of 'cameraCLIP' versus 'CameraCLIP' in the related-work section; the paper should use a single consistent name.","section":"§2"},{"comment":"The sentence 'consistent with prior work's conclusion [20]' is vague; the authors should state which conclusion from CLIP4CLIP they are referring to, or remove the citation.","section":"§4.4.1"},{"comment":"The FVD comparison would be more informative if the authors reported the reference-video distribution and the number of generated clips used, since FVD estimates are sensitive to sample size.","section":"§4.1"},{"comment":"The donut-chart labels are difficult to read; listing exact per-category counts in the caption or in a table would improve clarity.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a system paper with a useful dataset and an interesting LoRA-composition idea, but the current evaluation does not adequately support the abstract's claims. The R@1 comparison is biased by fine-tuning, the generation evaluation lacks a human study, and CLIPLoRA's optimization loop is circular with respect to its own evaluator. These issues are fixable within the manuscript's scope if the authors add controlled experiments and independent evaluation; hence major revision rather than rejection. I would also suggest the authors clarify the release status of the dataset and code, since reproducibility is central to the paper's contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe dataset is the real contribution here. Cinematic2K covers twenty subcategories across framing, angle, and movement, which is broader than CineScale/CineScale2's separate coverage, and the combination of dataset, a fine-tuned CLIP evaluator (CameraCLIP), and a genetic search over LoRA activation sequences (CLIPLoRA) is new. The writing is clear and the pipeline is sensible.\n\nThe main soft spot is the headline R@1 of 0.83. It compares fine-tuned CameraCLIP to zero-shot baselines, so the gain is at least partly a finetuning effect. There are no error bars, no significance tests, and no inter-annotator agreement for the 'meticulous' labels. The generation claims rest on qualitative stills and a few aggregate CLIPSIM/FVD numbers. The stress-test concern about content priors in stock footage is real: roughly 100 clips per category from Pexels/Videvo likely correlate camera movement with scene content, and no experiment separates attribute from content. The CLIPLoRA search uses CameraCLIP as fitness, which is partially circular, though the final metrics include independent CLIPSIM and FVD, so it is not fully circular. Still, selecting on a fixed prompt set S could overfit.\n\nWho this is for: researchers working on controllable text-to-video generation who want a cinematic-language benchmark and a baseline method. It deserves a serious referee, but a revision needs a content-controlled human study, confidence intervals, and either released artifacts or a clear plan for them. I would accept for review, not as-is.","headline":"A broad cinematic-language dataset and a sensible pipeline, but the headline numbers compare fine-tuned to zero-shot models and the generation claims lack content-controlled human evaluation.","tokens_in":12719,"tokens_out":1716,"would_cite":true,"duration_ms":16603,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a text-to-video pipeline can independently control shot framing, shot angle, and camera movement, and can smoothly blend several such controls within a single generated video.","keywords":["cinematic language","text-to-video generation","camera control","LoRA composition","CLIP","shot framing","shot angle","camera movement"],"falsifier":"Have professional cinematographers blind-rate CameraDiff outputs on prompts with scene content far outside the stock clips used for training, such as rack focus on a chess game in a library, and measure whether the requested shot attribute is reliably present; alternatively, measure CameraCLIP's R@1 on a held-out set where scene content is varied while the cinematic attribute is held fixed. If either the human ratings fall to chance or the retrieval score drops well below 0.83, the claimed stable, disentangled cinematic control collapses.","tokens_in":1774,"feed_emoji":"🎬","tokens_out":2215,"duration_ms":66303,"temperature":0.7,"pith_summary":"This paper tries to close the gap between text-to-video (T2V) generation and professional cinematography by giving T2V models explicit control over cinematic language. It introduces Cinematic2K, a dataset of about 2,000 real videos organized into twenty subcategories spanning shot framing, shot angles, and camera movements. On top of this dataset it builds CameraDiff, which trains one lightweight LoRA module per cinematic category on a frozen AnimateDiff backbone, and CLIPLoRA, which uses a fine-tuned CLIP model (CameraCLIP) as a fitness evaluator in a genetic search over LoRA activation sequences, enabling several cinematic attributes to be composed in one video. If the claim is right, a filmmaker could ask for a long shot, eye level, panning right, zoom in, tilt up view of a bear walking in snow near a lake and receive a single video that obeys all those directives, rather than just object motion.","feed_headline":"Text-to-video gains cinematic control: 20 shot types, angles, moves","feed_subtitle":"Fine-tuned CLIP scores cinematic alignment at 0.83, and multi-shot LoRA blends stay stable in one video.","key_machinery":"The load-bearing mechanism is the pairing of a frozen text-to-video diffusion backbone (AnimateDiff) with lightweight per-category LoRA adapters, one per cinematic language category, each trained on roughly 95–100 carefully verified video clips from Cinematic2K. The second half of the machinery is CameraCLIP, a fine-tuned CLIP model that scores how well a generated video matches a cinematic text description; CLIPLoRA turns that score into a fitness function for a genetic algorithm that searches over which LoRA should be active at each denoising step. The paper's core identity is therefore that a cinematic attribute corresponds to a LoRA module, and a cinematic sentence corresponds to a temporally scheduled composition of LoRA modules, selected by a CLIP-based evaluator.","core_discovery":"The central claim is that stable, precise cinematic control in T2V generation reduces to per-category LoRA training plus CLIP-guided LoRA composition. CameraDiff yields twenty distinct cinematic patterns by fine-tuning a separate LoRA for each of the twenty categories on the frozen AnimateDiff backbone. CameraCLIP, built by fine-tuning only the last two text-encoder layers and the last four vision-encoder layers of CLIP and mean-pooling eight frames per video, reaches an R@1 score of 0.83 on the Cinematic2K validation set, above all compared video-CLIP models. CLIPLoRA then uses CameraCLIP as the fitness function in a genetic algorithm that searches for the optimal sequence of LoRA activations across denoising steps; the paper reports that this yields an FVD of 1837 and CLIPSIM of 0.2535, better than static LoRA merging, switching, or simple composition baselines.","pith_inferences":["The same per-category-LoRA plus CLIP-evaluator recipe could plausibly extend to lighting, lens length, color grade, or scene mood, since the paper's taxonomy is one choice of cinematic vocabulary rather than a necessary set.","Because CameraCLIP's ablation shows simple mean pooling beating Transformers and LSTM on this small dataset, the result suggests that for narrow, label-like attributes, complex temporal aggregators may overfit; a testable extension is whether this survives dataset growth.","The paper's title asks whether video generation can replace cinematographers; read strictly, the work demonstrates control over a finite vocabulary of shot attributes, not over aesthetic judgment, narrative pacing, or emotional intent, so replacement would require additional evidence the paper does not provide.","CLIPLoRA's genetic search uses generated videos on a set of test prompts as fitness, which makes the search cost proportional to the number of generations; an implicit extension is caching or distillation of the learned scheduling rule into a direct predictor to avoid per-prompt search."],"forward_implications":["Natural-language prompts can specify shot framing, shot angle, and camera movement simultaneously, without requiring camera parameter inputs such as extrinsic matrices or motion trajectories.","A single generated video can contain smooth transitions between multiple cinematic attributes, such as starting with a long shot, zooming in, and tilting up, which static LoRA merge methods produce with visual artifacts.","Cinematic text-video retrieval on this vocabulary reaches R@1 0.83, suggesting automated tagging and search of footage by shot attributes is feasible.","The CLIPLoRA genetic-search recipe gives a concrete way to adaptively compose any set of LoRAs during diffusion, not only cinematic ones.","The combination lifts FVD from above 2300 for the static composition baselines to 1837, pointing toward more temporally coherent composite shots."],"supporting_citations":[{"why":"Supplies the frozen T2V backbone whose motion module CameraDiff fine-tunes with LoRA.","marker":"[12]"},{"why":"Defines the low-rank adaptation method used to train one lightweight module per cinematic category.","marker":"[16]"},{"why":"Provides the base CLIP model that CameraCLIP fine-tunes for cinematic text-video alignment.","marker":"[22]"},{"why":"Presents a prior camera and object motion control approach that requires extra camera parameters, positioning CameraDiff's flexible alternative.","marker":"[40]"},{"why":"A prior method decoupling object and camera motion, which CameraDiff extends to full shot framing and angle control.","marker":"[43]"},{"why":"A shot-scale dataset lacking camera movement annotations, highlighting the gap Cinematic2K fills.","marker":"[28]"},{"why":"A shot-angle dataset lacking detailed camera movement information, another gap Cinematic2K addresses.","marker":"[29]"},{"why":"A camera-trajectory dataset with irregular motions, showing why clean cinematic categories are missing from prior data.","marker":"[46]"},{"why":"Introduces LoRA Switch and Composite baselines that CLIPLoRA compares against and extends with dynamic, CLIP-guided adaptation.","marker":"[45]"}],"fun_headline_variants":["Per-style LoRA gives stable, precise cinematic T2V control","20 shot types, CLIP-scored blending: multi-shot T2V improved","CameraDiff: 20 LoRAs for stable cinematic video generation","CLIP-guided LoRA fusion yields seamless multi-shot video","Stable multi-shot T2V via CLIP-scored LoRA blending"],"cache_read_input_tokens":14720,"weakest_assumption_plain":"The argument assumes that the Cinematic2K annotations are accurate and that the twenty cinematic categories are separable enough that a LoRA trained on roughly 95–100 stock clips per category transfers to arbitrary prompts and produces visibly correct framing, angle, and movement in the frozen AnimateDiff backbone.","fun_headline_variants_meta":{"raw":{"variants":["Per-style LoRA gives stable, precise cinematic T2V control","20 shot types, CLIP-scored blending: multi-shot T2V improved","CameraDiff: 20 LoRAs for stable cinematic video generation","CLIP-guided LoRA fusion yields seamless multi-shot video","Stable multi-shot T2V via CLIP-scored LoRA blending"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000887,"raw_usage":{"total_tokens":3842,"prompt_tokens":974,"completion_tokens":2868,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":2774}},"tokens_in":590,"tokens_out":2868,"duration_ms":18574,"temperature":1.0,"reasoning_tokens":2774,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:47:27.888009+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have professional cinematographers blind-rate CameraDiff outputs on prompts with scene content far outside the stock clips used for training, such as rack focus on a chess game in a library, and measure whether the requested shot attribute is reliably present; alternatively, measure CameraCLIP's R@1 on a held-out set where scene content is varied while the cinematic attribute is held fixed. If either the human ratings fall to chance or the retrieval score drops well below 0.83, the claimed stable, disentangled cinematic control collapses.","supporting_citations":[{"cited_title":"Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever","cited_arxiv_id":null,"evidence_quote":"Provides the base CLIP model that CameraCLIP fine-tunes for cinematic text-video alignment."},{"cited_title":"Motionctrl: A unified and flexible motion controller for video generation","cited_arxiv_id":null,"evidence_quote":"Presents a prior camera and object motion control approach that requires extra camera parameters, positioning CameraDiff's flexible alternative."},{"cited_title":"Direct-a-video: Customized video generation with user- directed camera movement and object motion","cited_arxiv_id":null,"evidence_quote":"A prior method decoupling object and camera motion, which CameraDiff extends to full shot framing and angle control."},{"cited_title":"Cinescale: A dataset of cinematic shot scale in movies","cited_arxiv_id":null,"evidence_quote":"A shot-scale dataset lacking camera movement annotations, highlighting the gap Cinematic2K fills."},{"cited_title":"Cinescale2: a dataset of cinematic camera features in movies","cited_arxiv_id":null,"evidence_quote":"A shot-angle dataset lacking detailed camera movement information, another gap Cinematic2K addresses."},{"cited_title":"Stereo magnification: learning view synthesis using multiplane images","cited_arxiv_id":null,"evidence_quote":"A camera-trajectory dataset with irregular motions, showing why clean cinematic categories are missing from prior data."}],"review_version":1}