{"id":"b607b0c0-dabb-4112-8afa-94dd8e172440","arxiv_id":"2502.01101","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"VidSketch generates coherent video animations from any number of hand-drawn sketches plus a text prompt, using an abstraction-level control strength and a temporal-spatial attention mechanism.","lead":"Sketch-driven video generation that creates animations from hand-drawn sketches plus a text prompt. It scores the abstraction level of the sketches and adjusts how strongly the model follows them, while a temporal-spatial attention ties frames to earlier frames for smoother motion.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8) continuity score is inverted relative to its stated meaning, so the Level-Based Sketch Control can mis-order abstraction levels and apply the wrong guidance strength.","rationale":"Good-faith reading: the paper's core idea is plausible and the qualitative figures suggest the pipeline can produce coherent sketch-guided videos. However, the quantitative evidence is not strong enough to establish the central claim on its own: Table 1 is internally inconsistent (three different 'OURS' rows, and removing LBSC improves MSE in one row), user-study data are withheld, and the closest sketch-animation baselines are not compared. The single most load-bearing issue for the claimed contribution is the LBSC scoring: if Eq. (8) has the wrong monotonicity, the 'automatic adjustment for varying drawing skills' fails mechanically, not merely through missing hyperparameters. The proposed check can be run analytically from the formulas alone, without code. I do not think this warrants outright rejection because the generation pipeline might still work with a corrected score or a fixed strength; but the paper should be conditioned on addressing this inversion, providing the missing weights, and validating SC against human abstraction ratings. This is consistent with the reader's conditional verdict, so no verdict change is needed.","tokens_in":12821,"tokens_out":10732,"duration_ms":117732,"concrete_test":"Take a fixed canvas and three binary images: a sparse line drawing, a filled silhouette, and a densely shaded sketch. Compute AC, AL, AT, and SC using Eq. (8)-(11) with equal weights (wC=wL=wT=1/3). Independently ask several raters to order these images by sketch abstraction. If the SC ordering does not match the human ordering — specifically, if the filled silhouette is scored as more abstract than the sparse line — Eq. (8) is inverted and the LBSC control mapping in Appendix A.4 is unreliable. This check requires only the formulas in Section 3.2 and a few binary images, not the missing weights.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (8) defines AC = S·Pmax/(Smax·P) and states that lower AC means stronger continuity and less abstraction. But the area-to-perimeter ratio S/P is large for compact filled contours and small for thin strokes, so a filled 5×5 square in a 10×10 canvas gives AC = 25·40/(100·20) = 0.5, while a 10-pixel line gives AC ≈ 0.18. The formula therefore assigns a higher abstraction score to the filled shape than to the sparse line, contrary to the paper's own definition and to ordinary sketch intuition. Since SC = wC·AC + wL·AL + wT·AT (Eq. 11) directly controls the adapter scale s and injection threshold τ via the Appendix A.4 mapping, an inverted continuity term can push SC in the wrong direction for exactly the low- vs high-abstraction sketches the strategy is meant to distinguish. The missing weights wC, wL, wT and the lack of any validation of SC against human abstraction ratings compound the issue: even the direction of the score cannot be verified from the text. The central claim of automatic accommodation of varying drawing skills is therefore not supported as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VidSketch, a video-generation method that takes hand-drawn sketch sequences and a text prompt as input and produces short video animations. The method is built on a pretrained diffusion video model, uses an inflated T2I-Adapter to inject sketch features, and introduces two new components: a Level-Based Sketch Control (LBSC) strategy, which estimates an abstraction score from continuity, connectivity, and texture statistics and maps that score to an adapter scale and injection threshold, and a TempSpatial Attention (TSA) mechanism, which mixes keys/values from the first, second, and previous frames into the attention computation for the current frame. The authors report quantitative comparisons using VBench metrics, PickScore, and MSE, plus a user study, and claim that VidSketch is the first method to generate high-quality video animation directly from arbitrary hand-drawn sketches and simple text prompts.","tokens_in":13089,"tokens_out":4667,"duration_ms":48854,"significance":"If the method performs as claimed, it would address a real gap: enabling non-expert users to animate rough sketches without professional skills. The paper describes a complete pipeline, and the qualitative examples across several styles are plausible. The TempSpatial Attention design is a reasonable adaptation of attention mechanisms to temporal consistency. However, the central LBSC component currently contains an internally inconsistent formula and unspecified parameters, and the quantitative evaluation has comparability gaps. The significance is therefore conditional on substantial revision and re-validation.","major_comments":[{"comment":"The continuity score AC = S·Pmax/(Smax·P) is inconsistent with the accompanying text. The text states that lower AC corresponds to stronger continuity and less abstraction, but the area-to-perimeter ratio S/P is large for compact filled shapes and small for thin stroke-like contours. For a filled 5×5 square in a 10×10 canvas, AC = (25·40)/(100·20) = 0.5, while a 10-pixel line yields AC ≈ 0.18, so the formula labels the more detailed filled shape as more abstract. Because Eq. (11) forms SC as a weighted sum and Appendix A.4 maps SC to s and τ, this inversion can push the guidance strength in the wrong direction for exactly the low- versus high-abstraction distinction the strategy is designed to make. The claim that LBSC automatically accommodates varying drawing skills is therefore not supported as written.","section":"Section 3.2.1, Eq. (8)"},{"comment":"The weights wC, wL, and wT in Eq. (11) are never specified, and Lmax in Eq. (9) is also undefined. Consequently SC cannot be computed from the paper, the piecewise mapping to s and τ in Appendix A.4 cannot be reproduced, and the direction of SC's effect on guidance cannot be verified. In addition, no experiment compares SC to human judgments of sketch abstraction, so the central control mechanism is an unvalidated and non-reproducible heuristic.","section":"Section 3.2.4 and Appendix A.4"},{"comment":"The caption states that the ablation rows were evaluated with different samples, so the rows OURS (w/o LBSC), OURS, OURS (w/o TSA), and OURS are not directly comparable to one another. For example, OURS (w/o LBSC) reports MSE 13.83 and SubC 98.8, whereas the first OURS row reports MSE 17.24 and SubC 97.4, and the w/o-TSA row shows a catastrophic drop on yet another set of samples. Without matched evaluation sets and significance tests, the quantitative claims that LBSC and TSA improve performance are not established.","section":"Table 1"},{"comment":"The user-study raw data and video examples are explicitly withheld, with the statement that the authors are 'temporarily unable to publicly share the specific video examples and their scoring details.' Table 2 reports only aggregate values with no significance tests or confidence intervals. Furthermore, the numbers in Table 2 appear to be percentages, but the protocol described in Appendix B collects 1–5 Likert scores and sums averaged scores; the relation between the two is unexplained. This makes the user-study evidence impossible to audit and insufficient to support the claim of significant superiority.","section":"Appendix B and Table 2"}],"minor_comments":[{"comment":"There are typographical and naming inconsistencies: Appendix B contains 'user stduy' and 'CONTRONET,' and ControlNet is attributed to Chen et al. (2023) in Table 1 but to Zhang et al. (2023) in Table 2 and the references.","section":"Appendix B, Table 1, Table 2"},{"comment":"The symbol t is overloaded: in Eq. (4) it denotes an interpolation index for output frames, while in Section 3.1 and Eq. (7) it denotes the diffusion timestep. Please use distinct symbols.","section":"Eq. (4) and Section 3.1"},{"comment":"The first-frame generation step is described only as 'we recommend using A(·), representing (Mou et al., 2024),' but A(·) is elsewhere the sketch adapter, not a first-frame image generator; the actual procedure for producing x1,0 from a sketch is therefore unclear.","section":"Section 3.1"},{"comment":"The caption says 'we input a sketch sequence containing only two hand-drawn sketches ... three times to guide the generation of three video animations,' which is ambiguous about whether the same two sketches are reused or different sketch pairs are meant.","section":"Figure 7 caption"},{"comment":"The novelty claim of being 'first' should be sharpened against the cited works on sketch animation (Gal et al., 2024; Bandyopadhyay & Song, 2024), clarifying whether those methods generate video from sketches or animate static drawings and what exactly distinguishes VidSketch from them.","section":"Related work, Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting and practical problem, and the proposed architecture is plausible. In its current form, however, the central LBSC mechanism has an internally inconsistent formula and missing parameters, and the quantitative evaluation is not reproducible or directly comparable across ablations. These issues are load-bearing for the paper's central claims. I would encourage the authors to correct the continuity formula, specify all weights and thresholds, release the user-study data, and rerun matched ablations with significance testing; with those changes the paper could be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is genuinely new: taking any number of hand-drawn sketches plus a text prompt and generating a video with an automatically adjusted guidance strength is not something the cited prior work (Breathing Life, FlipSketch) does. The TempSpatial Attention module is a reasonable extension of standard attention for video consistency, and the qualitative figures suggest the system works in practice. That deserves credit.\n\nBut the paper as written is not solid enough to support its own claims. The most serious issue is Equation (8). The continuity score AC = S·Pmax/(Smax·P) is an area-to-perimeter ratio, so a filled shape gets a higher score than a sparse line. The paper defines higher AC as more abstraction, which contradicts ordinary sketch intuition and its own statement that lower AC means less abstraction. Depending on the unknown weights wC, wL, wT, this can mis-order the abstraction levels and apply the wrong control strength. The mapping table in Appendix A.4 uses thresholds on SC, but those thresholds are not justified, and the weights are never given. None of this is reproducible from the text.\n\nThe evaluation is also shaky. Table 1 lists multiple \"OURS\" rows with different metric values and does not make clear which configuration each row refers to; the caption mentions \"different samples,\" but the text does not explain the protocol. The user study reports aggregate scores from 42 participants but withholds the raw scores, the video examples, and any significance tests. The closest sketch-animation baselines (FlipSketch, Breathing Life into Sketches) are cited but never compared, which weakens the \"first\" claim. No code is released.\n\nOn the positive side, the training and inference setup is clearly described, the loss modification to keep the first frame fixed is sensible, and the idea of controlling adapter strength via a sketch-abstraction estimate is worth exploring. The missing details (weights, code, consistent ablation table) look fixable rather than fatal.\n\nWho is this for? Practitioners in creative media who want a working sketch-to-video pipeline, and researchers working on adapters and control in video diffusion. It is not a scientific breakthrough, but it is a legitimate system contribution.\n\nRecommendation: Yes, send it to review, but with the expectation of major revision. The reviewer should verify the direction of Eq. (8), request the missing weights and code, and ask for a proper comparison against sketch-animation baselines. Without those, the central claim is not supported.","headline":"A plausible but sloppy system paper: the multi-sketch video workflow is new and the qualitative results look promising, but the abstraction-control score appears internally inconsistent and the evaluation is too under-specified to verify the central claim.","tokens_in":13540,"tokens_out":4294,"would_cite":false,"duration_ms":49540,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VidSketch generates video animations directly from any number of hand-drawn sketches plus a text prompt, adapting guidance strength to the sketch's abstraction level and keeping frames consistent over time.","keywords":["sketch-driven video generation","video diffusion models","hand-drawn sketch animation","sketch abstraction level","spatiotemporal consistency","T2I-Adapter","level-based sketch control","text-to-video generation"],"falsifier":"Take a fixed prompt and a corpus of sketches of the same scene drawn at systematically varied abstraction levels (rough scribble, loose gesture, clean line art, shaded). For each sketch, compute the paper's scores $A_C$, $A_L$, $A_T$, generate the video with the Level-Based control active, and have human raters score sketch-to-video fidelity and overall quality. The claim is falsified if fidelity ratings do not track the mapping — for instance, if a sketch the strategy labels abstract (low $s$, late $\\tau$) yields visibly weaker sketch adherence than the same sketch with stronger control, or if two sketches humans rank equally abstract land in different mapping brackets — and any such test currently requires guessing the unpublished weights $w_C$, $w_L$, $w_T$.","tokens_in":12641,"feed_emoji":"✏️","tokens_out":20864,"duration_ms":168882,"temperature":0.7,"pith_summary":"VidSketch sets out to close the gap the authors see between sketch-to-image and sketch-to-video generation: they claim their method is the first to turn hand-drawn sketch sequences — any number of them, drawn by hands of any skill level — together with a short text prompt into a high-quality, temporally consistent animated video. The stakes are practical: sketching is the most intuitive way for a non-expert to specify what a video should contain, and previous sketch-driven diffusion models stop at static images. The paper argues that making this work requires solving two problems that image generation does not face: how strongly the sketch should steer generation, which must depend on how abstract the drawing is, and how to keep frames coherent across time. VidSketch answers the first with its Level-Based Sketch Control Strategy, which scores abstraction from a sketch's continuity, connectivity, and texture detail and converts that score into an adapter scale and an injection threshold; it answers the second with a TempSpatial Attention mechanism that ties each frame to the first, second, and preceding frames during denoising.","feed_headline":"Turn hand-drawn sketches into video with a text prompt","feed_subtitle":"A diffusion model adapts sketch control to the drawing's abstraction level and keeps frames consistent over time.","key_machinery":"Three named mechanisms carry the argument. The Level-Based Sketch Control Strategy (LBSC) is the auto-tuning core: it derives an abstraction score from three normalized component scores — continuity $A_C$ (a contour area-to-perimeter ratio computed with connected-component analysis and the Suzuki border-following algorithm, normalized by image area and perimeter), connectivity $A_L$ (the count of 8-connected components found by depth-first search), and texture detail $A_T$ (averaged contrast, dissimilarity, and homogeneity from a gray-level co-occurrence matrix) — combined as $S_C = w_C A_C + w_L A_L + w_T A_T$. That score is mapped through a hand-picked piecewise rule in Appendix A.4 to the adapter scale $s$ and the diffusion-stage threshold $\\tau$, which decide how strongly the inflated T2I-Adapter's residual $R = s \\cdot A_{\\mathrm{Inflated}}(F_{\\mathrm{final}})$ is added to the hidden states and from which timestep onward. The TempSpatial Attention mechanism (TSA) replaces each frame $i$'s attention key and value set with the concatenated keys and values of frame 1, frame 2, and frame $i-1$, so every frame is continually re-anchored to the video's opening and its immediate past. The first-frame-preserving diffusion schedule — a Dirac delta on $x_{1,t}$ so the user-derived first frame never receives noise, with the noise-prediction loss applied only to $f > 1$ — supplies the long-range anchor that makes the other two mechanisms effective.","core_discovery":"On its own terms, the paper's discovery is a complete recipe for sketch-conditioned video generation. From a text-to-video diffusion backbone, VidSketch inflates the T2I-Adapter conditioning model to the temporal dimension and injects its scaled output $R = s \\cdot A_{\\mathrm{Inflated}}(F_{\\mathrm{final}})$ into the denoiser's upsampling layers, where $F_{\\mathrm{final}}$ is the user's sketch sequence with linearly interpolated intermediate frames. The first frame is held frozen through every diffusion timestep — noise is added only to frames $f > 1$, and the training loss scores only those later frames — which gives the generation a fixed reference. The Level-Based Sketch Control Strategy computes an abstraction score $S_C = w_C A_C + w_L A_L + w_T A_T$ from contour continuity, connected-component count, and gray-level co-occurrence texture features, then maps that score through a fixed three-step rule to the adapter scale $s$ and the injection threshold $\\tau$, so that rougher, more abstract doodles steer the generation less than detailed line art. The TempSpatial Attention mechanism computes each frame's attention with key and value tokens drawn from the first frame, the second frame, and the immediately preceding frame, a design the ablations credit with removing screen tearing and preserving inter-frame consistency. The paper reports that with these components it outperforms IP-Adapter, ControlNet, and T2I-Adapter on VBench metrics and in a 42-participant user study across aesthetics, consistency, smoothness, stability, and detail richness.","pith_inferences":["Editorial inference: the three-step mapping from abstraction score to control strength appears hand-chosen, so fitting a continuous mapping to human-ranked sketches would likely generalize to drawing styles the fixed thresholds miss.","Editorial inference: the continuity-connectivity-texture scoring could serve as a general quality filter for other conditioning signals whose reliability varies from input to input, such as depth maps, edge maps, or pose sequences.","Editorial inference: because training used only 8–12 videos per action category, the reported consistency gains may be tied to those small, homogeneous datasets; training the same modules on larger, more varied motion corpora is a direct test of whether the gains scale.","Editorial inference: the 'any number of sketches' claim depends on linear interpolation between user frames, so very sparse keyframe sequences probably expose the method's practical limit; a comparison with two versus eight keyframes per prompt would map it."],"forward_implications":["Anyone who can sketch a few keyframes and type a short prompt can produce an animated video; animation creation no longer requires rigging, keyframe software, or professional editing skills.","Because guidance strength adjusts automatically, one method serves users of very different drawing abilities — rough doodles steer weakly and detailed line art steers strongly — rather than requiring a manually tuned control knob per user.","A sketch sequence of any length, not just a single first frame, can carry the structure of an animation, with linear interpolation bridging between user-supplied keyframes.","Keeping the first frame noiseless through the diffusion process and scoring only later frames in the training loss is what lets an existing video diffusion model be adapted for this task on very small per-category datasets."],"supporting_citations":[{"why":"Supplies the T2I-Adapter that VidSketch inflates to the temporal dimension; its output carries the sketch residual $R$ injected into the denoiser.","marker":"(Mou et al., 2024)"},{"why":"Provides the pre-trained latent-diffusion weights from which the Temporal Attention and TempSpatial Attention modules are fine-tuned.","marker":"(Rombach et al., 2022)"},{"why":"Defines the gray-level co-occurrence matrix features used for the texture-detail component $A_T$ of the abstraction score.","marker":"(Haralick et al., 1973)"},{"why":"Supplies the VBench benchmark whose metrics (subject consistency, motion smoothness, aesthetic quality, imaging quality) anchor the quantitative comparisons.","marker":"(Huang et al., 2024)"},{"why":"ControlNet is one of the three baselines the method must beat in the quantitative and user studies.","marker":"(Zhang et al., 2023)"},{"why":"IP-Adapter is a baseline for comparison and represents the image-conditioning adapter approach the method extends beyond.","marker":"(Ye et al., 2023)"},{"why":"The closest prior art on animating sketches with text-to-video priors; VidSketch's novelty claim of guiding video with multiple sketches rather than a single first frame is positioned against it.","marker":"(Gal et al., 2024)"},{"why":"Supplies the border-following algorithm used to extract hierarchical contours for the continuity score $A_C$.","marker":"(Suzuki et al., 1985)"}],"fun_headline_variants":["Sketch-to-video AI adapts to your drawing skill","From doodles to videos: diffusion adjusts to sketch quality","VidSketch turns sketches into video with text prompts","Adaptive sketch control for consistent video generation","Draw frames, add text, watch animation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a sketch's abstraction level — and therefore the right strength of guidance — is faithfully captured by the weighted sum of continuity, connectivity, and texture scores in Eq. 11, where the paper never states the weights, and that the fixed piecewise mapping from that score to the adapter scale $s$ and threshold $\\tau$ holds across every drawing style.","fun_headline_variants_meta":{"raw":{"variants":["Sketch-to-video AI adapts to your drawing skill","From doodles to videos: diffusion adjusts to sketch quality","VidSketch turns sketches into video with text prompts","Adaptive sketch control for consistent video generation","Draw frames, add text, watch animation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1446,"prompt_tokens":1019,"completion_tokens":427,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":351}},"tokens_in":635,"tokens_out":427,"duration_ms":4678,"temperature":1.0,"reasoning_tokens":351,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T16:34:50.773556+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed prompt and a corpus of sketches of the same scene drawn at systematically varied abstraction levels (rough scribble, loose gesture, clean line art, shaded). For each sketch, compute the paper's scores $A_C$, $A_L$, $A_T$, generate the video with the Level-Based control active, and have human raters score sketch-to-video fidelity and overall quality. The claim is falsified if fidelity ratings do not track the mapping — for instance, if a sketch the strategy labels abstract (low $s$, late $\\tau$) yields visibly weaker sketch adherence than the same sketch with stronger control, or if two sketches humans rank equally abstract land in different mapping brackets — and any such test currently requires guessing the unpublished weights $w_C$, $w_L$, $w_T$.","supporting_citations":[{"cited_title":"T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the T2I-Adapter that VidSketch inflates to the temporal dimension; its output carries the sketch residual $R$ injected into the denoiser."},{"cited_title":"M., Shanmugam, K., and Dinstein, I","cited_arxiv_id":null,"evidence_quote":"Defines the gray-level co-occurrence matrix features used for the texture-detail component $A_T$ of the abstraction score."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the border-following algorithm used to extract hierarchical contours for the continuity score $A_C$."}],"review_version":1}