{"id":"8a4c6791-98f9-4fc5-8b51-fc939ffa217f","arxiv_id":"2502.08189","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"AnyCharV is a two-stage diffusion method that places a reference character into a target video scene using pose and mask guidance, with a self-boosting stage that improves identity preservation.","lead":"AnyCharV is a video-generation method that takes one image of a character and a separate driving video, then produces a new video where that character performs the driving video's motions in its scene. It uses a two-stage training trick: first compose the character and scene with precise masks, then retrain using the model's own outputs with looser boxes to preserve identity.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantitative superiority claim rests on a task-mismatched FVD reference set, and the user-study margins over the closest baseline are within the reported error bars.","rationale":"I read the paper in good faith and found the proposed two-stage fine-to-coarse mechanism coherent and the qualitative examples suggestive. However, the central claim of state-of-the-art performance is only as strong as the evaluation that backs it. The FVD metric as specified in Appendix A.3 is not aligned with the task: comparing generated videos to a random set of 1,000 real training videos measures generic video quality, not whether the source character and target scene were combined correctly. This is a load-bearing flaw because it is the primary quantitative justification in Table 1. The user study, which is more task-aligned, shows that AnyCharV's advantage over Viggle lies within the reported uncertainty intervals, further weakening the superiority claim. The reader centered on the reliability of self-boosting on generated pairs; that is a valid concern about the training mechanism, but the evaluation flaw is more immediate: even if self-boosting works perfectly, the paper has not demonstrated the claimed margins with a valid protocol. My proposed check, recomputing FVD against the actual target videos with confidence intervals, would settle whether the quantitative superiority holds. The manuscript should be conditional on releasing the evaluation protocol, code, and results with proper statistical treatment.","tokens_in":13521,"tokens_out":8172,"duration_ms":67394,"concrete_test":"Recompute the headline FVD comparison using the 10 target driving videos as the reference set for each method, and report bootstrap confidence intervals over the 200 generated videos; if AnyCharV no longer ranks first or the margin over Viggle collapses, the quantitative superiority claim in Table 1 is not supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that AnyCharV outperforms prior methods is supported mainly by Table 1, but the FVD protocol in Appendix A.3 compares the 200 generated videos to 1,000 unrelated real character videos from the training dataset, not to the target driving videos that define the task. FVD measures distributional realism, not whether the generated video preserves the target scene, motion, and identity; a model that ignores both inputs and produces generic realistic videos could score well. The paper provides no details on how many videos per method were used, at what resolution, or whether the 1,000 reference videos overlap with training data. This makes the reported 3.88-5.55% FVD improvements over Viggle and MIMO uninterpretable as evidence of task performance. Furthermore, the user study in Table 2 reports average rankings with error intervals that overlap between AnyCharV and Viggle on all three criteria (Identity 1.89±0.24 vs 2.04±0.24; Motion 2.02±0.37 vs 2.20±0.33; Scene 1.99±0.27 vs 2.04±0.31), so even the perceptual superiority over the closest closed-source method is not statistically robust. The reader's concern about self-boosting reliability is plausible but secondary; the immediate load-bearing issue is that the headline quantitative evidence does not actually measure the claimed capability.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AnyCharV, a two-stage framework for controllable character video generation. Given a reference character image and a target driving video, the method synthesizes a new video in which the reference character performs the target video's motion within the target scene, guided by 2D pose and masks. Stage 1 trains a base model in a self-supervised manner with a fine segmentation mask. Stage 2 constructs 64,000 generated video pairs using the stage-1 model and fine-tunes the UNet with a coarse bounding-box mask, a 'self-boosting' step intended to preserve identity details. Experiments compare against Make-A-Protagonist, Viggle, MIMO, and VACE using FVD, DOVER++, CLIP-I, and a user study.","tokens_in":13779,"tokens_out":6558,"duration_ms":50870,"significance":"The task is timely and practically useful, and the framework is elegant in avoiding explicit 3D reconstruction. The self-supervised composition strategy and the fine-to-coarse self-boosting design are interesting contributions, and the paper includes thorough ablations and a substantial user study. However, the quantitative evidence for the central superiority claim is not currently convincing: the FVD protocol is mismatched to the task, the user-study error intervals overlap with the closest baseline, and the self-boosting mechanism's reliability is not quantitatively characterized. If the evaluation is strengthened, the method could be a valuable practical pipeline.","major_comments":[{"comment":"The FVD values in Table 1 are computed between the 200 generated videos and 1,000 real character videos from the CharVG training set (Appendix A.3), not against the 10 target driving videos that define the evaluated task. FVD is a distribution-level realism metric; a method that ignores both conditioning inputs and emits generic plausible character videos can achieve a low FVD. Consequently, the reported FVD improvements (e.g., 5.55% over VACE, 3.88% over MIMO) are uninterpretable as evidence of identity, motion, or scene preservation, and the reference set's overlap with training data is not addressed. The paper also does not report standard errors or confidence intervals for any of the three metrics in Table 1, although the evaluation consists of only 200 generated videos. To support the central claim, the authors should either compute FVD against the target driving videos (or a task-matched paired reference set) or drop FVD as a headline metric and rely on metrics that directly measure conditional fidelity.","section":"Appendix A.3, Table 1"},{"comment":"The user study's average rankings show overlapping error intervals between AnyCharV and the strongest closed-source baseline, Viggle, on all three criteria (Identity: 1.89±0.24 vs. 2.04±0.24; Motion: 2.02±0.37 vs. 2.20±0.33; Scene: 1.99±0.27 vs. 2.04±0.31). The paper does not report how the intervals were computed, per-rater statistics, or a significance test; the Section 4.2 claim that AnyCharV 'significantly outperforms' Viggle is therefore not supported. Given that the user study is the only evidence directly aligned with the claimed identity/motion/scene preservation, the authors should add a paired significance test and either report the full distribution or acknowledge the lack of statistical separation.","section":"Table 2, Section 4.2"},{"comment":"The self-boosting stage trains on 64,000 video pairs generated by the stage-1 model. Because the conditioning input in Eq. (3) is the generated reference-driving video x_r^d, any systematic artifact in the base model's outputs (e.g., shape distortion at mask borders, blurred hands, motion jitter) can be reinforced during fine-tuning. The regression target is the original target video, so this is not classic circularity, but the reliability of the synthesized pairs is nonetheless a load-bearing assumption. The paper shows selected examples in Figure 7 and an ablation in Table 3, but it does not quantify artifact frequency, apply any filtering, or compare the distribution of generated pairs to real data. The authors should validate this assumption, for example by measuring pose/identity error on the pairs or by ablating with a quality-filtered subset.","section":"Section 3.3, Eq. (3)"}],"minor_comments":[{"comment":"In the user-study paragraph, the text refers to 'the open-source Make-A-Protagonist and MACE'; 'MACE' is undefined and appears to be a typo for 'VACE'.","section":"Section 4.2"},{"comment":"In the user-study metric definitions, 'Character identity preservation (Identity in Table 1)' should reference Table 2 rather than Table 1.","section":"Appendix A.3"},{"comment":"The first paragraph says 'The based model developed in Section 3.2' and should read 'The base model developed in Section 3.2'.","section":"Section 3.3"},{"comment":"The sentence 'driving by a target video generated by text-to-video (T2V) models' should be 'driven by a target video...' for grammatical correctness.","section":"Section 4.4"},{"comment":"The notation 'md = J - md' for the complementary mask is confusing because the same symbol 'md' denotes both the mask and its complement; consider using a different symbol such as 'm_d' for the complement. The figure caption also contains unresolved LaTeX artifacts for the scene symbol.","section":"Section 3.2, Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The core method is plausible and likely of interest to the journal's audience, but the evaluation must be improved before the central superiority claim can be accepted. The FVD protocol is mismatched to the task, and the user study appears underpowered for the claimed separation from Viggle. I would encourage the editor to ask the authors to clarify whether all comparison methods were run on the identical evaluation set at the same resolution, and to make the evaluation videos or a reproducible protocol available. The paper currently does not release code or data, which limits reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I'll cut to it: the fine-to-coarse two-stage training idea is the real thing. The first stage does self-supervised composition with a fine segmentation mask; the second stage generates 64,000 source-target pairs and fine-tunes the UNet with coarse bounding-box masks, using the generated video as the scene input. That is not in the cited prior work, and the ablations in Tables 3 and 4 make the case that each piece helps. The qualitative results look plausible, and the method description is clear enough to reproduce with effort.\n\nBut the quantitative evidence for the headline claim doesn't hold up. The FVD protocol in Appendix A.3 compares the 200 generated videos to 1,000 real character videos from the training dataset, not to the 10 target driving videos that define the task. FVD measures distributional realism. A model that ignores both inputs and outputs generic realistic videos could score well. So the reported 3.88–5.55% FVD improvements over Viggle and MIMO are not evidence of preserving target scene, motion, or identity. The user study in Table 2 has overlapping error bars with Viggle on all three criteria (identity, motion, scene), so 'significantly outperforms' is not supported. Missing error bars on FVD/DOVER++/CLIP-I and a 200-video evaluation set add to the problem. The self-boosting assumption—that first-stage outputs are reliable training data—is plausible, and the paper does not quantify failure modes across the 64k pairs, but that is secondary to the metric issue.\n\nThe paper does acknowledge the back-view limitation in the appendix, which is honest. The authors clearly put thought into the training strategy and ablations.\n\nWho's this for? Researchers working on character animation and controllable video generation, and anyone building on ReferenceNet-style pipelines. If the claims are recalibrated, the method is a solid incremental advance.\n\nRecommendation: send it to peer review, but with major revision required. A serious referee should demand a matched FVD (or per-pair evaluation against the driving video), error bars, code/data release, and a tempered comparison with Viggle. The core idea deserves the attention; the current numbers don't justify the stated superiority.","headline":"New training strategy, weak evaluation: the FVD reference set is mismatched to the task and the user-study margins over Viggle are within error bars.","tokens_in":14307,"tokens_out":2669,"would_cite":false,"duration_ms":24045,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage, fine-to-coarse guidance schedule lets a diffusion model put any reference character into any target video, beating four prior systems on identity, motion, and scene preservation.","keywords":["character video generation","controllable video generation","diffusion model","fine-to-coarse guidance","self-boosting training","pose guidance","identity preservation","video composition"],"falsifier":"Measure identity preservation, such as CLIP-I or face distance, of the final model while tracking the visual fidelity of the 64,000 synthesized training pairs; if identity gains appear only when those pairs are nearly artifact-free and disappear when pairs contain visible mask-shaped artifacts, the self-boosting assumption is the active ingredient. A more direct test is to corrupt a subset of the generated pairs, for instance by blurring or misaligning the character, and check whether the second stage amplifies those corruptions.","tokens_in":13319,"feed_emoji":"🎬","tokens_out":6696,"duration_ms":48431,"temperature":0.7,"pith_summary":"This paper aims to establish that character video generation can be made fully flexible: given any reference character image and any target driving video, a diffusion model can put the character into the video, following the target motion and preserving the target scene. The proposed two-stage scheme first trains a base model under a fine segmentation mask to compose character and scene, then uses the base model's own outputs to build 64,000 character-replacement pairs and fine-tunes with only a coarse bounding-box mask. The authors report that this fine-to-coarse guidance outperforms open-source and closed-source state-of-the-art systems in quantitative metrics and user studies, and extends to characters and scenes synthesized by text-to-image and text-to-video models. If correct, the result is a practical character-replacement tool for film and creative production that requires no 3D reconstruction and no per-video tuning.","feed_headline":"Fine-to-coarse masks put any character in any video","feed_subtitle":"Self-boosted diffusion swaps a reference character into a target scene, beating open- and closed-source rivals.","key_machinery":"The load-bearing mechanism is the fine-to-coarse guidance schedule applied in two stages. Stage 1 trains the denoising UNet with a ReferenceNet and a pose guider, conditioning generation on the reference image via CLIP cross-attention and ReferenceNet spatial features, and on the target video via a fine segmentation mask concatenated to the noisy latent and augmented at its borders, together with 2D pose skeletons. Stage 2 freezes ReferenceNet and pose guider and fine-tunes only the denoising UNet on self-generated (reference-driving video, target-driving video) pairs, replacing the fine mask with a coarse bounding-box mask and feeding the unmasked generated video as the scene input. This self-boosting loop turns mask shape from a hard constraint into a soft location cue, which is what the paper credits for preserving character details.","core_discovery":"The central discovery is that mask shape granularity controls the trade-off between compositional accuracy and identity fidelity. A fine segmentation mask in the first stage teaches the model where and how to place the reference character into the target scene, but its exact boundary leaks into the generated character and produces blur and artifacts. The fix is a self-boosting stage: the first-stage model generates 64,000 pairs of videos that share scene and motion but differ in character, and the model is then fine-tuned on these pairs using a coarse bounding-box mask on the unmasked video. This looser constraint lets the model recover the reference character's details while still knowing where to put it. The paper reports consistent gains over using fine masks throughout, coarse masks throughout, or no masks, and superior identity, motion, and scene scores against Make-A-Protagonist, Viggle, MIMO, and VACE.","pith_inferences":["The self-boosting stage is effectively self-distillation: the model trains on its own outputs to relax a conditioning signal. The same fine-to-coarse relaxation might transfer to other guidance modalities such as depth, skeleton, or normal maps, where fine annotations can leak shape artifacts.","Because the pipeline depends on SAM2 and DWPose for masks and poses, its robustness is bounded by those detectors; occluded or unusual poses may fail before the diffusion model gets a chance to correct them.","The paper does not measure how the quality of the 64,000 synthesized pairs varies; if a subset contains systematic errors, the second stage could be silently reinforcing them. A straightforward diagnostic would be to plot identity metrics against per-pair confidence scores."],"forward_implications":["Character replacement in arbitrary footage becomes a single-image-plus-video operation, with no 3D reconstruction, no per-video fine-tuning, and no manual compositing.","The method composes with text-to-image and text-to-video generators, so a user can describe a character and a scene in text and get that character animated in that scene.","Because only the UNet is fine-tuned in stage 2, the self-boosting stage is comparatively cheap and could be re-run on new data to adapt to new character types.","The reported 5-minute inference for a 5-second 576x1024 clip places the method within reach of interactive editing tools."],"supporting_citations":[{"why":"Supplies the ReferenceNet architecture and pose-guided character animation approach that stage 1 builds on.","marker":"[9]"},{"why":"MIMO is a closed-source comparison baseline that the paper claims to outperform on all metrics.","marker":"[14]"},{"why":"Make-A-Protagonist is an open-source comparison baseline that the paper claims to beat.","marker":"[16]"},{"why":"VACE is an open-source comparison baseline that the paper claims to surpass in identity and quality.","marker":"[18]"},{"why":"SAM2 provides the fine segmentation masks used for stage-1 supervision and mask augmentation.","marker":"[44]"},{"why":"CLIP supplies the image encoder for cross-attention conditioning and the CLIP-I metric for identity preservation.","marker":"[45]"},{"why":"DWPose extracts the 2D pose sequences that guide motion in both training stages.","marker":"[46]"},{"why":"Stable Diffusion 1.5 initializes both the ReferenceNet and the denoising UNet.","marker":"[48]"},{"why":"AnimateDiff initializes the motion module in the denoising UNet, providing temporal consistency.","marker":"[49]"},{"why":"ControlNet initializes the pose guider used to embed pose skeletons into the noisy latent.","marker":"[50]"}],"fun_headline_variants":["Self-boosted coarse masks swap any character into any scene","Fine-to-coarse guidance: put any character in any video","AnyCharV self-boosts fine masks to coarse for better swaps","Coarse mask self-boost improves character video identity","Two-stage self-boosting enables arbitrary character video generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The second stage assumes that videos synthesized by the first stage are faithful enough to train on; if those synthesized pairs contain systematic shape, texture, or motion artifacts, the self-boosting retraining will reinforce them rather than correct them, and the paper does not quantify failure rates across the 64,000 pairs.","fun_headline_variants_meta":{"raw":{"variants":["Self-boosted coarse masks swap any character into any scene","Fine-to-coarse guidance: put any character in any video","AnyCharV self-boosts fine masks to coarse for better swaps","Coarse mask self-boost improves character video identity","Two-stage self-boosting enables arbitrary character video generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000862,"raw_usage":{"total_tokens":3713,"prompt_tokens":891,"completion_tokens":2822,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":2739}},"tokens_in":507,"tokens_out":2822,"duration_ms":19045,"temperature":1.0,"reasoning_tokens":2739,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T10:04:45.063731+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure identity preservation, such as CLIP-I or face distance, of the final model while tracking the visual fidelity of the 64,000 synthesized training pairs; if identity gains appear only when those pairs are nearly artifact-free and disappear when pairs contain visible mask-shaped artifacts, the self-boosting assumption is the active ingredient. A more direct test is to corrupt a subset of the generated pairs, for instance by blurring or misaligning the character, and check whether the second stage amplifies those corruptions.","supporting_citations":[{"cited_title":"Animate anyone: Consistent and controllable image-to-video synthesis for character animation","cited_arxiv_id":null,"evidence_quote":"Supplies the ReferenceNet architecture and pose-guided character animation approach that stage 1 builds on."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"CLIP supplies the image encoder for cross-attention conditioning and the CLIP-I metric for identity preservation."},{"cited_title":"Effective whole-body pose estimation with two- stages distillation","cited_arxiv_id":null,"evidence_quote":"DWPose extracts the 2D pose sequences that guide motion in both training stages."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Stable Diffusion 1.5 initializes both the ReferenceNet and the denoising UNet."},{"cited_title":"Animatediff: Animate your personalized text-to-image diffusion models without specific tuning","cited_arxiv_id":null,"evidence_quote":"AnimateDiff initializes the motion module in the denoising UNet, providing temporal consistency."}],"review_version":1}