{"id":"da7c7cd2-baa6-445c-a3cd-0ae84487bab8","arxiv_id":"2412.11755","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"FCVG generates stable inbetween frames by injecting linearly interpolated line-match and pose conditions into every denoising step of a pretrained video diffusion model.","lead":"This paper proposes FCVG, a method that adds per-frame control conditions derived from matched lines and body poses to a video diffusion model, improving temporal stability in generative inbetweening. It offers a practical way to reduce interpolation ambiguity when the motion between two key frames is large.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that frame-wise conditions align bidirectional paths is load-bearing but is least secure where line matches are sparse; Sec. 5 concedes artifacts under large input differences. A stratified test by match density is needed before accepting the large-motion advantage.","rationale":"The paper is a plausible and lightweight applied contribution: it builds on a pre-trained SVD, injects conditions through a small adapter, reports consistent aggregate improvements, and provides code and video examples. Reading charitably, the mechanism is that explicit per-frame control reduces the stochastic gap between forward and backward sampling paths, so simple averaging suffices. The reader's conditional verdict is appropriate, and the weakest assumption identified is indeed the reliability of the frame-wise conditions. My stress-test agrees with that assessment and sharpens it: the load-bearing failure mode is not merely hypothetical, because the authors themselves document in Sec. 5 that sparse matching under large frame differences produces artifacts that gamma adjustment cannot solve. That is exactly the setting where the method claims its largest advantage. The main missing evidence is a quantitative stratification showing that the method's benefit is robust when the conditioning signal degrades. Without such a test, the central claim about large-motion temporal stability rests on selected visual examples, which the paper itself acknowledges are needed because standard metrics are not precise for temporal stability. The requested revision is therefore a targeted robustness study with match-density stratification and error bars, plus the missing comparison to VIBIDSampler. This does not change the reader's conditional verdict; it reinforces it.","tokens_in":12292,"tokens_out":3799,"duration_ms":39021,"concrete_test":"Construct a large-motion and occlusion test subset from the existing evaluation data. For each clip, compute the GlueStick match count and optionally verify correspondences with a flow-based or manual check. Stratify clips into high-match-density and low-match-density groups, then compare FCVG against GI and VIBIDSampler on FVMD, FVD, and a direct temporal-consistency measure (e.g., warped-frame error or CLIP-based temporal coherence), running each method with multiple seeds and reporting error bars. If FCVG's advantage over GI is concentrated in the high-density stratum and disappears or reverses in the low-density stratum, the Sec. 3.2 alignment mechanism is not robust for large motion, and the headline claim should be weakened accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central assertion in Sec. 3.2 is that frame-wise conditions keep the forward and backward denoising paths aligned, so that the simple linear fusion in Eq. 5 stays close to the manifold. For this to hold, the color-visualized GlueStick line matches and DWPose skeletons interpolated frame-by-frame must provide a faithful, unambiguous control path for every intermediate frame, especially under large motion and occlusion. This is precisely the regime the paper highlights, and it is where the premise is least secure. GlueStick matches become sparse or incorrect when the two keyframes differ substantially; linearly interpolating color-coded lines does not guarantee topologically consistent correspondence, since lines can cross, endpoints can slip, and occluded segments can vanish. The control injection in Eq. 6 has no mechanism to detect or down-weight unreliable conditions, and the paper's own Sec. 5 states that when input frames differ significantly, sparse matches produce artifacts that adjusting gamma cannot fix. Thus the failure mode occurs in the regime the headline claim targets. The quantitative evidence does not isolate this mechanism either: Table 1 reports aggregate FVMD/FVD without error bars or stratification, and the authors explicitly caution that these metrics are not precise for temporal stability. The central advantage is therefore supported mainly by selected visuals rather than a controlled demonstration that the alignment mechanism is robust exactly where it is needed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Frame-wise Conditions-driven Video Generation (FCVG) for generative inbetweening. Given two keyframes, FCVG extracts GlueStick line matches and DWPose pose skeletons, visualizes them as colored images, interpolates these visual conditions frame-by-frame (linearly or with user-specified non-linear paths), and injects them into Stable Video Diffusion via a ControlNeXt-style lightweight adapter. Inference uses bidirectional time-reversal denoising with a simple weighted average fusion of the two paths. The authors report quantitative comparisons on a collected dataset (524 clips from DAVIS, RealEstate10K, and Pexels) against FILM, DynamiCrafter, TRF, and GI, plus ablations, computational-efficiency measurements, and qualitative generalization to animation and line art. The central claim is that per-frame explicit conditions align the forward and backward denoising paths, thereby improving temporal stability especially for large motion gaps, while preserving frame quality.","tokens_in":12567,"tokens_out":3939,"duration_ms":38741,"significance":"If validated, the method would be a simple and practical addition to generative inbetweening: it requires only a small fine-tuned adapter, avoids noise reinjection, halves the number of denoising steps relative to prior generative inbetweening methods, and offers user control over the interpolation path. The paper is honest about its limitations and provides a project page and code link, which supports reproducibility. The main weakness is that the headline claim—temporal stability under large motion—is supported primarily by selected qualitative videos and by aggregate metrics that the authors themselves state cannot precisely measure temporal stability. The mechanism also depends on the reliability of line matching in exactly the large-motion regime where the paper acknowledges sparse matches cause artifacts. These gaps prevent the current evidence from fully establishing the central advantage.","major_comments":[{"comment":"The paper's central claim is 'significantly enhanced temporal stability', but Table 1 reports no confidence intervals, no multiple-seed results, and no significance tests, and the text in §4.1 explicitly states that all used metrics 'are not capable of precisely evaluating temporal stability'. As a result, the quantitative evidence does not substantiate the central claim. I ask the authors to either add a dedicated temporal-coherence metric or human evaluation, or at minimum report means and variances over multiple inference seeds for LPIPS, FID, FVMD, and FVD so readers can judge whether the observed differences (e.g., FVMD 5607.2 vs. 5682.6 at gap 23) are meaningful.","section":"§4.1, Table 1"},{"comment":"The method's advantage is claimed to be largest for large motion gaps, but §5 states that when input frames differ significantly, matched lines may be sparse and artifacts cannot be fixed by adjusting gamma. This is precisely the regime the headline claim targets. The paper should include a stratified evaluation by match density or motion magnitude (e.g., bins based on number of GlueStick matches or optical-flow magnitude) and show that FCVG retains its advantage in the sparse-match regime. Without such a test, the core mechanism is only demonstrated where conditions are reliable, not where the paper claims it matters most.","section":"§5, §3.2"},{"comment":"The assertion that frame-wise conditions 'ensure that the forward and backward paths are relatively aligned during the denoising process' is presented without direct evidence. The final-frame metrics in Table 1 are an indirect test. I suggest a direct measurement of path alignment, such as the average latent distance between z_t from the forward path and flip(z_t) from the backward path during sampling, with and without frame-wise conditions. This would test the proposed mechanism more convincingly and help explain why simple weighted fusion in Eq. (5) suffices.","section":"§3.2, §3.3, Algorithm 1"},{"comment":"The ablation results lack error bars, and some differences are small or inconsistent with the narrative. For example, in Table 3, removing the pose condition improves FVMD (5520.9 vs. 5607.2 for the full model), and in Table 4, gamma=2.0 gives FID 24.66 and FVMD 5726.9 while gamma=0.5 gives FID 23.80 and FVMD 5920.0, so no clear monotonic trend in the metrics supports the claim that the method is insensitive to gamma. Reporting multiple seeds with confidence intervals would make the component analysis more reliable.","section":"§4.5, Tables 3 and 4"}],"minor_comments":[{"comment":"The phrase 'We conduct ablations to to discuss' contains a duplicated word 'to'.","section":"§4.5"},{"comment":"The word 'outdor' in 'indoor/outdor scenes' should be 'outdoor'.","section":"§4.1"},{"comment":"Equation (3) writes the denoiser input as z_{t+1}, while Algorithm 1 at loop step t uses z_t. Please make the time indexing consistent between the equation, the algorithm, and Figure 4.","section":"Eq. (3) vs. Algorithm 1"},{"comment":"The notation c_{1→N} is described as a concatenation of frame-wise conditions along the time dimension, but Algorithm 1 writes 'c_{1→N} = interpolate(c_1, c_N)' without showing the concatenation. Please clarify the exact tensor shape and how interpolation and concatenation compose.","section":"§3.3.1, Algorithm 1"},{"comment":"The test set is only one fifth of 524 clips, which is roughly 105 clips, and the split appears to be random. Please report the number of test clips and consider providing the dataset or split metadata so that the results can be reproduced.","section":"§4.1"},{"comment":"Fine-tuning uses 512×320 crops while inference is performed at 1024×576. A brief discussion of this resolution mismatch and its potential effect on the adapter would be helpful.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the approach is interesting, but the evaluation does not currently support the strength of the central temporal-stability claim. The authors' own admission that the metrics cannot capture temporal stability, combined with the failure case for large motion in §5, means that the paper needs either a stronger quantitative evaluation or a carefully scoped claim. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a useful applied paper. The new idea is to give the diffusion model an explicit per-frame condition for inbetweening, built by interpolating GlueStick line matches and DWPose skeletons between the two keyframes, and injecting these through a ControlNeXt-style adapter into SVD while fusing forward and backward denoising paths. It's not a conceptual breakthrough, but it's a clean, lightweight synthesis that shows consistent gains over the generative baselines it compares against, and the ablations show both the line and pose conditions earn their keep. I also give them credit for the honest limitations section.\n\nThe soft spots are real but proportionate. The central claim about temporal stability is supported mostly by selected videos; the paper itself says its metrics can't precisely evaluate temporal stability. There are no error bars, the test set is about a hundred clips, and there is no comparison with VIBIDSampler, which they cite. The stress-test concern about sparse matches is on the mark: the paper concedes in Sec. 5 that when the input frames differ significantly, the line matches get sparse and artifacts can't be fixed by adjusting the control weight. That is precisely the large-motion regime the paper opens with, so the headline advantage is not demonstrated exactly where it is supposed to bite. It doesn't sink the paper: the average improvements are consistent across gaps and metrics, and the weakness is disclosed rather than hidden. But it does mean the evidence for the mechanism is not as strong as the framing suggests.\n\nWho should read this? Anyone working on generative inbetweening or controllable video generation, especially if they need a practical way to stabilize interpolation. It deserves a serious referee; it is not desk-reject material. I would send it to review with a request for a proper temporal-stability evaluation, uncertainty quantification, and a head-to-head with VIBIDSampler, plus a stratified analysis by match density if they have the data.","headline":"A practical, honestly-limited method for stabilizing generative inbetweening via per-frame control conditions; deserves review but needs stronger temporal-stability evidence.","tokens_in":13067,"tokens_out":2426,"would_cite":true,"duration_ms":24869,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that injecting per-frame interpolated line and pose conditions into every denoising step removes the interpolation-path ambiguity destabilizing generative inbetweening, producing temporally stable videos even for large…","keywords":["generative inbetweening","video frame interpolation","frame-wise conditions","diffusion models","temporal stability","line matching","pose-guided video generation","image-to-video generation"],"falsifier":"Feed the method a keyframe pair with large motion and repetitive texture, such as a patterned curtain or a crowd scene; if the line matcher yields few or wrong correspondences, intermediate frames should show abrupt jumps or ghosting. If no setting of the control weight removes those artifacts, the claim that frame-wise conditions confine simple fusion close to the manifold fails in exactly the large-gap regime the paper targets.","tokens_in":1591,"feed_emoji":"🎬","tokens_out":2533,"duration_ms":91291,"temperature":0.7,"pith_summary":"The paper argues that generative inbetweening — synthesizing intermediate frames between two key frames with a video diffusion model — is unstable because the interpolation path is ambiguous: the start and end frames condition only the first and last frames. To remove that ambiguity, the authors propose FCVG, which gives every intermediate frame its own explicit condition by linearly interpolating color-coded matched-line maps, optionally augmented with pose skeletons, between the two key frames. These frame-wise conditions are injected at every denoising step of a pretrained image-to-video model, aligning the forward and backward sampling paths so that a simple weighted fusion stays close to the video manifold. The paper reports that this yields temporally stable inbetweening under large motion gaps, with fewer sampling steps, no noise re-injection, and support for user-specified non-linear interpolation paths.","feed_headline":"Frame-wise guides stabilize AI inbetweening on large gaps","feed_subtitle":"Interpolated line and pose maps pin down each intermediate frame, preventing forward and backward paths from drifting apart.","key_machinery":"The load-bearing object is the frame-wise condition sequence, a 'control path' of color-coded line-match images, optionally with pose skeletons, interpolated frame by frame between the two key frames and fed to the denoiser at every denoising step. The forward and backward condition sequences are time-flips of each other, so both sampling directions see the same intermediate guidance; the fusion is simply $z_t = \\lambda \\tilde z_t + (1-\\lambda)\\,\\mathrm{flip}(\\tilde z'_t)$ with linear weights $\\lambda_i = 1 - (i-1)/(N-1)$. The conditions enter through a lightweight control branch that uses cross-normalization, and only the value and output projections of attention layers plus small residual blocks are fine-tuned, keeping the pretrained model largely frozen.","core_discovery":"The central claim is that a frame-wise 'control path' aligns the forward and backward denoising paths so closely that a plain weighted average of the two predictions stays close to the video manifold. The authors extract matched lines between start and end frames, render them as color-coded images, optionally add pose skeletons, and linearly interpolate these maps frame by frame to form the condition sequence. Each denoising step then conditions on the frame it is producing, not just on the start or end frame, and the backward path uses the same condition sequence flipped in time. The paper reports that this substantially reduces incoherent transitions and artifacts under large motion while keeping frame quality, and that the method works across natural scenes, human poses, camera motion, and animation and line art.","pith_inferences":["Editorial inference: The control-path idea is a general recipe: any per-frame signal that can be interpolated between keyframes, such as depth, edges, optical flow, or text prompts, could replace lines and poses in the same injection mechanism.","Editorial inference: If the line-condition bottleneck is real, replacing the hand-crafted matcher with a learned semantic correspondence module, or with user-specified correspondences, should extend the same stabilization to scenes with occlusion and repetitive texture.","Editorial inference: Because the paper's own metrics do not precisely measure temporal stability, a dedicated stability metric that checks path consistency under non-linear trajectories would make the claimed advantage directly measurable and comparable across methods."],"forward_implications":["Generative inbetweening becomes stable at large motion gaps where optical-flow methods and prior diffusion-based fusion approaches produce incoherent transitions.","Noise re-injection is unnecessary and the number of denoising steps can be halved relative to prior time-reversal approaches, substantially reducing inference cost.","Users can specify non-linear interpolation paths, such as ease-in and ease-out motion, and the same frame-wise condition mechanism follows them.","The approach generalizes to animation and line art without fine-tuning on those data types, because the control path itself encodes the motion.","A single control weight setting works across diverse scenes, removing per-input tuning of motion-related hyper-parameters."],"supporting_citations":[{"why":"Supplies the pre-trained image-to-video diffusion model that FCVG controls and lightly fine-tunes.","marker":"[3]"},{"why":"Provides the line-matching model used to establish correspondences between the two key frames and build the initial conditions.","marker":"[29]"},{"why":"Provides the lightweight control-injection design with cross-normalization that FCVG adapts for frame-wise conditions.","marker":"[30]"},{"why":"Introduces the generative inbetweening baseline that fine-tunes temporal attention; FCVG compares against it and avoids its noise re-injection.","marker":"[44]"},{"why":"Introduces the time-reversal bidirectional sampling strategy that FCVG extends with explicit frame-wise conditions.","marker":"[7]"},{"why":"Proposes a multi-channel bidirectional sampling alternative that FCVG competes with, showing where simple fusion had previously failed.","marker":"[50]"},{"why":"Supplies pose estimation used to augment the frame-wise conditions for scenarios with human motion.","marker":"[51]"},{"why":"Motivates the use of sparse global matching for large motions, the basis for choosing line matching in FCVG.","marker":"[22]"}],"fun_headline_variants":["Stable AI inbetweening via frame-wise guides","Per-frame guides prevent video inbetweening drift","Line-based guides stabilize video inbetweening","Frame-wise conditioning pins each intermediate frame","Matched-line guides make AI inbetweening stable"],"cache_read_input_tokens":15232,"weakest_assumption_plain":"The load-bearing premise is that lines and poses extracted from the two key frames, rendered as images and linearly interpolated, provide a faithful description of the motion path for every intermediate frame; when matches are incorrect or too sparse, the control path itself misleads the denoiser and artifacts persist.","fun_headline_variants_meta":{"raw":{"variants":["Stable AI inbetweening via frame-wise guides","Per-frame guides prevent video inbetweening drift","Line-based guides stabilize video inbetweening","Frame-wise conditioning pins each intermediate frame","Matched-line guides make AI inbetweening stable"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000972,"raw_usage":{"total_tokens":4117,"prompt_tokens":917,"completion_tokens":3200,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":3130}},"tokens_in":533,"tokens_out":3200,"duration_ms":22937,"temperature":1.0,"reasoning_tokens":3130,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:36:20.510078+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed the method a keyframe pair with large motion and repetitive texture, such as a patterned curtain or a crowd scene; if the line matcher yields few or wrong correspondences, intermediate frames should show abrupt jumps or ghosting. If no setting of the control weight removes those artifacts, the claim that frame-wise conditions confine simple fusion close to the manifold fails in exactly the large-gap regime the paper targets.","supporting_citations":[{"cited_title":"Gluestick: Robust image matching by stick- ing points and lines together","cited_arxiv_id":null,"evidence_quote":"Provides the line-matching model used to establish correspondences between the two key frames and build the initial conditions."},{"cited_title":"Ex- plorative inbetweening of time and space","cited_arxiv_id":null,"evidence_quote":"Introduces the time-reversal bidirectional sampling strategy that FCVG extends with explicit frame-wise conditions."},{"cited_title":"Effec- tive whole-body pose estimation with two-stages distillation","cited_arxiv_id":null,"evidence_quote":"Supplies pose estimation used to augment the frame-wise conditions for scenarios with human motion."},{"cited_title":"Sparse global matching for video frame interpolation with large motion","cited_arxiv_id":null,"evidence_quote":"Motivates the use of sparse global matching for large motions, the basis for choosing line matching in FCVG."}],"review_version":1}