{"id":"d05b54ff-1b04-41fd-b7fa-d8bbf7723767","arxiv_id":"2502.05979","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A controllable diffusion transformer generates VFX videos from a reference image, text, and mask and timestamp conditions, with a new 675-video dataset and a temporal accuracy metric.","lead":"VFX Creator generates animated visual effects on still images from text commands, using masks for object selection and timestamps for effect timing. The paper introduces Open-VFX, a dataset of 675 effect videos, as a benchmark for this task.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Temporal controllability metric may be circular: Co-Tracker/SAM2 labels serve as both training targets and evaluation ground truth, so reported TIoU and timestamp errors can be optimistically biased; no human validation is provided.","rationale":"The paper's central claim of state-of-the-art spatial and temporal controllability depends on a valid evaluation of temporal control. The reader's weakest assumption identifies precisely the risk that the automatic annotation pipeline (Co-Tracker for timestamps, SAM2 for masks) supplies both the training targets and the evaluation ground truth, making the reported TIoU and timestamp errors potentially circular. My reading of Sections 3.3 and 5.2 confirms this gap: no manual verification or inter-annotator agreement is reported, and the paper does not specify how 'predicted segments' are extracted for the temporal metric. The temporal claim is the load-bearing part of the contribution because spatial control is supported only qualitatively and by a small user study, while the temporal metric is the only quantitative evidence for precise timing control and covers only three effects. This is an evidentiary gap that can be fixed by human validation and artifact release, not a demonstrated internal contradiction; the dataset and qualitative results give the paper some independent support. Therefore the appropriate verdict remains CONDITIONAL: accept only if the authors validate the automatic annotations against human labels, report confidence intervals, and release the dataset/code. This does not change the reader's verdict, so I mark the verdict as UNCHANGED.","tokens_in":15272,"tokens_out":5667,"duration_ms":59967,"concrete_test":"Select 30 test videos across the 15 effect categories (including all 3 used in Table 2). Have at least 3 annotators independently mark effect start/end frames and instance masks on both source videos and model-generated videos. Compute inter-annotator agreement; recompute Ef/Es/TIoU using human-consensus timestamps as ground truth while keeping the model's predicted segments as originally measured. If TIoU drops by more than 0.10 or Ef/Es increase by more than 50% relative to Table 2, the temporal-control claim is systematically optimistic. Also report whether the model with random timestamp tokens achieves similar TIoU; if yes, the timestamps are not the active control.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 derives start/end timestamps from Co-Tracker displacement and masks from SAM2, with no manual verification. Section 5.2 defines Ef/Es/TIoU as differences between 'predicted and ground truth segments,' but the paper never states how the predicted segments are obtained. The natural implementation is to run the same Co-Tracker motion-detection on generated videos; if so, the ground-truth labels and the predicted labels share the same motion-onset/offset bias, so the metric rewards models that reproduce Co-Tracker's biases rather than perceptually correct effect timing. The problem is aggravated for effects like Dissolve/Melt, where appearance changes may not produce point displacement. Because Table 2 is the only quantitative evidence for temporal controllability and covers only 3 of 15 effects, this circularity is load-bearing for the headline claim. No inter-annotator agreement or human sanity check on the annotations is reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Open-VFX, a dataset of 675 VFX videos collected from Pika and PixVerse across 15 effect categories, annotated with text prompts, SAM2-derived instance masks, and Co-Tracker-derived start/end timestamps. It also proposes VFX Creator, a controllable video diffusion framework built on CogVideoX, adding a spatial mask-control branch (ControlNet-style) and two temporal-conditioning strategies (timestamp tokens in text space or temporal masks in timestep space) via LoRA adapters. Experiments report FID-VID, FVD, Dynamic Degree, temporal error metrics (Ef, Es, TIoU), qualitative comparisons, and user studies, claiming state-of-the-art spatial and temporal controllability on the Open-VFX test set.","tokens_in":15364,"tokens_out":4462,"duration_ms":44633,"significance":"If the claims are substantiated, the Open-VFX dataset would be a useful new resource for controllable VFX generation, and the LoRA-based, few-shot adaptation approach is practically appealing because it lowers the data barrier for customizing video diffusion models. The paper also introduces a new task formulation (VFX generation as image animation with spatial/temporal control) and proposes a dedicated temporal-control metric. However, the current experimental evidence does not yet establish the headline claims: the evaluation design has several load-bearing weaknesses, including possible circularity in the temporal metric, uncontrolled baselines, and very small test sets without statistical support.","major_comments":[{"comment":"The temporal-control metrics are not fully defined, and their validity is questionable. TIoU is introduced in the text but no equation or computation procedure is provided. More importantly, the manuscript never states how the predicted start/end timestamps (t_hat_start,i, t_hat_end,i) are obtained from generated videos. If the same Co-Tracker displacement pipeline used to create ground-truth labels in §3.3 is applied to generated videos, then the ground truth and predictions share the same motion-onset/offset bias, so the metric may reward reproducing Co-Tracker's biases rather than perceptually correct timing. The paper reports no manual verification, inter-annotator agreement, or human sanity check for these labels. Because Table 2 covers only 3 of 15 effects, the headline temporal-controllability claim is not supported by the evidence as presented.","section":"§5.2, Eq. (4)-(5)"},{"comment":"The comparison protocol is not controlled. VFX Creator is fine-tuned on the Open-VFX training split, while DynamiCrafter, CogVideoX, and LTX-Video are evaluated off-the-shelf without any fine-tuning on the dataset. The reported FID-VID/FVD/Dynamic Degree gains may therefore reflect task-specific adaptation rather than an architectural advantage of the proposed method. Additionally, the 'pseudo ground-truth Pika or PixVerse' baseline comes from the same commercial platforms used to source the training videos, so it is not a clean external baseline for generalization. The authors should either fine-tune the open-source baselines under the same protocol or clearly frame the comparison as 'zero-shot baselines vs. fine-tuned method,' and ideally evaluate on independently produced VFX videos.","section":"§5.3, Table 1"},{"comment":"The quantitative evaluation rests on very small test sets (5 videos per effect, per §5.1) and no error bars, confidence intervals, or significance tests are reported. The shot-number ablation in Table 3 is also internally inconsistent with the claim that increasing the number of shots generally improves performance: for 'Ta-da it,' the 10-shot FVD (2861.18) is much worse than the 1-shot value (1432.40), and for 'Explode it,' the 10-shot FVD (2829.00) is worse than 1-shot (2667.72). These non-monotonic results and the lack of variance estimates mean the data-efficiency claim is not established.","section":"§5.1, Table 3"},{"comment":"Spatial controllability is only demonstrated qualitatively; there is no quantitative metric for whether the effect occurs on the specified instance and not elsewhere. The reader cannot judge from still frames alone whether mask guidance is precise. The authors should add a spatial-accuracy metric, such as mask IoU between the region where the effect is applied in the generated video and the target mask, computed over the test set. Similarly, the temporal-control evaluation in Table 2 should be extended beyond three effects to all categories to support the general claim.","section":"§5.4, §5.5, Fig. 6 and Fig. 8"}],"minor_comments":[{"comment":"The description of Co-Tracker-based timestamp labeling is underspecified: 'motion is deemed to begin when the coordinates start to shift' needs a concrete displacement threshold and a policy for handling tracker noise and multi-point aggregation.","section":"§3.3"},{"comment":"There are typos: 'Defalte it' should be 'Deflate it' in §5.4, and 'VF Creator' in Fig. 7 should be 'VFX Creator'.","section":"§5.4 and Fig. 7"},{"comment":"The appendix heading 'DEFINATIONS' should be 'DEFINITIONS'.","section":"Appendix A"},{"comment":"In Eqs. (4) and (5), the variables t_start,i and t_end,i are said to be 'normalized' but no normalization range is specified; also, the relationship between frame-level error and second-level error via FPS is stated but not derived with explicit units.","section":"§5.2"},{"comment":"The construction of the 'temporal mask' used in Strategy I is not formalized; the authors should state how the binary moving/static mask is derived from the start and end timestamps and how it is aligned with the training frames after random shifting.","section":"§4.2.1"},{"comment":"The paper claims the 'first high-quality VFX video dataset' but does not discuss existing related datasets (e.g., video object segmentation or effect datasets); a brief comparison would help position the contribution.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":"The dataset contribution is potentially valuable, but the evaluation falls short of the claims. The most serious issue is the temporal metric: the paper must clarify how predictions are obtained and provide human validation of the label quality. I would also ask the editor to consider whether the use of Pika/PixVerse outputs as both training data and evaluation baseline raises a scope or fairness concern that the authors should address in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about this paper is that Open-VFX is a real, new resource—675 VFX clips across 15 categories with mask and timestamp annotations—and the system is a sensible combination of CogVideoX, ControlNet-style mask conditioning, and timestamp tokens in the text space. If you work on controllable video generation, the dataset is the contribution. The model itself is more application engineering than new mechanism, but that's fine.\n\nWhat it does well: the data collection pipeline is clearly described, the spatial mask control is evaluated with qualitative examples that look convincing, and the user study, though small (20 users, 30 sets), gives the model edge over Pika/PixVerse and open baselines. The paper also honestly reports that a single unified model underperforms per-category fine-tuning—that's a useful negative result.\n\nThe soft spot is the temporal evaluation, and it's not minor. The ground-truth start/end timestamps come from Co-Tracker displacement, and the masks from SAM2, with no manual verification reported. The paper never states how the predicted timestamps in the temporal metrics are obtained. The natural implementation is to run the same Co-Tracker detection on generated videos. If that's what they did, the metric measures how well the model reproduces Co-Tracker's motion-onset/offset biases, not how well it matches perceptually correct timing. That, plus the fact that Table 2 covers only three of fifteen effects, makes the temporal controllability claim much weaker than the paper's tone suggests. The effect is worse for appearance-only effects like Dissolve or Melt, where point displacement is a poor proxy.\n\nOther weaknesses: the open-source baselines are not fine-tuned on VFX data, the test set is five videos per effect, there are no error bars, and code and data are not released. So the 'state-of-the-art' claim is premature. That said, the flaws are fixable: add human annotation validation, report inter-annotator agreement, describe the predicted-segment extraction, fine-tune the baselines, and release artifacts.\n\nWho gets value: researchers in video generation and anyone thinking about building controllable-effect datasets. It deserves a serious referee—an editor should send it out, with a clear request to scrutinize the temporal metric and ask for the missing details.\n\nMy own verdict: the dataset is worth citing, the system is plausible, but the quantitative evidence does not yet support the headline claim.","headline":"New VFX dataset is the real contribution; the temporal control metric is likely circular and the SOTA claim needs better evidence.","tokens_in":15947,"tokens_out":2631,"would_cite":true,"duration_ms":25405,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that visual effects can be generated as image animation from a still image, a text prompt, an instance mask, and start/end timestamps, and supports the claim with a new dataset and a diffusion-transformer framework.","keywords":["visual effects generation","controllable video generation","video diffusion transformer","spatial control","temporal control","low-rank adaptation","VFX dataset","image animation"],"falsifier":"Take a random sample of Open-VFX videos, have people mark the true start/end frames and affected regions by hand, and recompute the paper's temporal IoU and timestamp errors against those manual labels; a large drop would show the automatic labels inflated the reported accuracy.","tokens_in":15026,"feed_emoji":"🎬","tokens_out":7635,"duration_ms":68484,"temperature":0.7,"pith_summary":"This paper tries to establish that visual effects can be turned into an image-animation problem: instead of hand-authoring an effect, a user supplies a still image, a text prompt like 'explode it,' a mask marking the target object, and start/end timestamps, and a video diffusion transformer generates the effect on that object and within that time window. To make this learnable, the authors build Open-VFX, a dataset of 675 videos covering 15 effect categories with text, masks, and timestamps, and they design VFX Creator, which adapts a frozen video transformer with low-rank updates plus two control modules. On their test set the system beats three open-source baselines on FID-VID, FVD, and dynamic degree, and reports a temporal IoU around 0.84 on selected effects, with human raters preferring it over or tying it with the commercial platforms that supplied the source videos. If the results hold, VFX production could move from expensive per-shot manual work to promptable, instance- and rhythm-controllable generation.","feed_headline":"New system adds controllable VFX to any still image","feed_subtitle":"A diffusion transformer with mask and timestamp control generates effects like melting and exploding exactly where and when requested.","key_machinery":"The load-bearing machinery is a pair of control adapters bolted onto a frozen video diffusion transformer. For space, a mask-control branch runs in parallel with the main network and injects per-frame instance masks through a zero-initialized convolution, so the effect is confined to the user-selected object. For time, a timestamp encoder maps normalized start/end frames into token embeddings that are concatenated with the text-prompt tokens and fed to the transformer's cross-attention, so the requested rhythm is coupled to the prompt. Both adapters are trained as low-rank updates, which keeps the number of new parameters small and lets the model learn from tens of videos per effect; the paper's ablation shows the text-space timestamp injection beats timestep-embedding injection.","core_discovery":"VFX Creator recasts animated visual effects as an image-animation task. The central claim is that a video diffusion transformer, lightly adapted, can take a static reference image, a text prompt naming one of 15 effects, a mask selecting the instance to affect, and normalized start/end timestamps, and output a video in which the named effect occurs on the chosen instance within the requested interval. The paper argues the design is data-efficient: with a frozen base network, trainable low-rank adapters, a mask-conditioned control branch, and timestamp tokens fused into the text embeddings, tens of videos per effect suffice. On the Open-VFX test set the system reports FID-VID 29.92 and FVD 752.95, better than three open-source baselines; temporal IoU averages 0.84 across three effects; and a user study places it above or on par with the commercial platforms whose videos were used as pseudo ground truth.","pith_inferences":["Beyond the paper, we would test whether the timestamp-token mechanism can sync effects to an external clock, such as music beats or shot cuts; the paper does not report that experiment.","Beyond the paper, the mask-control branch looks portable to other video diffusion transformers, but the paper only demonstrates it on its chosen baseline; transfer is our speculation.","Beyond the paper, because the source videos come from two commercial generators, Open-VFX may carry their stylistic biases; collecting effects from real film footage would reveal whether the model generalizes beyond synthetic VFX aesthetics.","Beyond the paper, the temporal metric could be stressed by sampling start/end pairs over the full range instead of the restricted ranges used in evaluation, which would expose whether extreme timings degrade control."],"forward_implications":["A user can point at an instance in a single image, type an effect such as 'melt it' or 'squish it,' and give rough start/end times, and the model will produce a video that honors both the region and the timing.","Because training needs only tens of videos per effect, new effect types can be added without collecting large datasets; the 10-shot runs already give balanced quality, and 40-shot lowers FVD on some effects.","The new temporal metric (frame-level error, second-level error, and temporal IoU) gives a standard way to compare how precisely a generator obeys requested effect timing.","Training one unified model across all 15 effects lowers quality compared with per-effect training; the paper attributes this to effects confusing each other and suggests mixture-of-experts as a future fix.","Open-VFX gives the community a benchmark with masks and timestamps for 15 VFX categories, which the paper positions as a foundation for later VFX-generation work."],"supporting_citations":[{"why":"Provides the base Video Diffusion Transformer architecture that VFX Creator adapts.","marker":"[Yang et al. 2024]"},{"why":"Supplies the low-rank adaptation method that makes training with few videos feasible.","marker":"[Hu et al. 2021]"},{"why":"Defines the mask-injection control branch with zero convolution used for spatial control.","marker":"[Zhang et al. 2023a]"},{"why":"Generates the displacement signals from which start/end motion timestamps are derived.","marker":"[Karaev et al. 2024]"},{"why":"Produces the instance mask sequences used as spatial conditioning and training targets.","marker":"[Ravi et al. 2024]"},{"why":"Contributes the idea of casting numeric settings as tokens merged into the text space, applied here to timestamps.","marker":"[Fang et al. 2024]"},{"why":"Establishes the diffusion-transformer block design into which conditioning signals are injected.","marker":"[Peebles and Xie 2023]"},{"why":"Serves as an open-source baseline that animates open-domain images and is compared against.","marker":"[Xing et al. 2025]"},{"why":"Serves as an open-source baseline for real-time video generation and is compared against.","marker":"[HaCohen et al. 2024]"}],"fun_headline_variants":["Controllable diffusion transformer adds timed VFX to still images","VFX Creator: melt or explode any image with precise timing","Mask and timestamps steer a diffusion model to animate VFX","Turn static images into animated VFX with text and masks","Diffusion VFX: image animation with controllable effect timing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole evaluation leans on automatically produced labels for when each effect starts and ends and which pixels it covers; if those labels are biased, the training target and the timing metric share the same bias and the reported temporal accuracy could be flattering.","fun_headline_variants_meta":{"raw":{"variants":["Controllable diffusion transformer adds timed VFX to still images","VFX Creator: melt or explode any image with precise timing","Mask and timestamps steer a diffusion model to animate VFX","Turn static images into animated VFX with text and masks","Diffusion VFX: image animation with controllable effect timing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000714,"raw_usage":{"total_tokens":3252,"prompt_tokens":1025,"completion_tokens":2227,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":641,"completion_tokens_details":{"reasoning_tokens":2143}},"tokens_in":641,"tokens_out":2227,"duration_ms":14879,"temperature":1.0,"reasoning_tokens":2143,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:08:53.181179+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of Open-VFX videos, have people mark the true start/end frames and affected regions by hand, and recompute the paper's temporal IoU and timestamp errors against those manual labels; a large drop would show the automatic labels inflated the reported accuracy.","supporting_citations":[],"review_version":1}