{"id":"e4748565-4411-47b7-b52b-833e0aab0b61","arxiv_id":"2411.17048","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A training strategy that uses reward feedback on generated videos to put a specific face into text-to-video outputs while preserving motion and prompt following.","lead":"PersonalVideo customizes text-to-video generation to a specific person's face by training with reward signals on fully generated videos instead of reconstructing reference photos, and it adds a semantic-consistency loss to protect the model's original motion and prompt-following abilities.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SCR loss in Eq. 4 is invariant to additive shifts in HPSv2 scores, so it cannot detect uniform semantic degradation; the paper never compares to the original T2V model, leaving 'no degradation' unsubstantiated.","rationale":"The reader correctly identified that the semantic-consistency proxy is unverified and that original-model baselines are missing. I sharpen this into a concrete, load-bearing technical flaw: the softmax-based KL divergence in Eq. 4 is invariant to additive constants over frames, so it cannot penalize a uniform drop in text-image alignment. This is not merely a missing comparison; the loss as designed is mathematically incapable of enforcing absolute semantic preservation. The central claim of the paper is precisely that the method avoids semantic and dynamic degradation, and this claim relies on SCR. The absence of an original-model baseline for CLIP-T, HPSv2, and dynamic degree means the reader cannot tell whether even the relative improvement over baselines corresponds to true preservation. The Isolated Identity Adapter may be the actual cause of dynamic preservation, and SCR may only help balance frame-level quality without recovering uniform degradation. The paper does contain ablations, a user study, and honest limitations, but the core claim needs either a modified loss that tracks absolute semantic quality or an explicit empirical demonstration against the frozen T2V model. My proposed test would settle whether the concern lands; the verdict of CONDITIONAL remains appropriate while that evidence is missing.","tokens_in":56,"tokens_out":3880,"duration_ms":86814,"concrete_test":"Compute the gradient of L_SCR with respect to adding a constant to all target frame scores: it is identically zero. Then, on the same 50 prompts used in Table 1, run the frozen HunyuanVideo and the tuned PersonalVideo with matched seeds; measure mean HPSv2/CLIP-T and VBench dynamic degree. If the tuned model's mean scores are significantly below the frozen model's, the 'no semantic/dynamic degradation' claim fails. Also report per-frame semantic metrics to see if degradation is uniform.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 defines the semantic consistency reward as D_KL(V_c^T || V_c^S) where V_c = Softmax({R_sem(frame_i)}). Because softmax is invariant to adding a constant to all logits, L_SCR = 0 whenever the customized model's per-frame HPSv2 scores differ from the original model's by a uniform additive constant. That means the loss provides no gradient to raise absolute text-alignment; it only aligns the relative shape of score distributions across frames. The paper's central claim is that PersonalVideo preserves the original T2V model's dynamic and semantic capabilities. But the only component designed for semantic preservation is structurally blind to a uniform degradation in semantic following. Table 3 reports CLIP-T for w/ SCR vs w/o SCR, but never reports the original model's CLIP-T or HPSv2. Similarly, 'dynamic degree' is never compared to the frozen base model. Thus the headline claim 'without dynamic and semantic degradation' is not established either empirically (missing baseline) or theoretically (the loss cannot distinguish a uniform drop).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes PersonalVideo, a text-to-video customization framework that replaces reconstruction-based tuning on static reference images with reward supervision on videos sampled from the target T2V model. The training objective (Eq. 5) combines an identity consistency reward (ICR) on a randomly selected generated frame and a semantic consistency reward (SCR) that matches the softmax-normalized HPSv2 score distribution of sampled frames against that of the frozen original generator. The method also uses simulated prompt augmentation and an isolated identity adapter applied only in later denoising steps. Experiments on HunyuanVideo and AnimateDiff compare with DreamBooth-LoRA, MagicMe, ID-Animator, and ConsisID using face similarity, dynamic degree, FVD, temporal consistency, CLIP-T/CLIP-I, ablations, and a user study.","tokens_in":14179,"tokens_out":7326,"duration_ms":66725,"significance":"The core idea—applying differentiable reward feedback directly to generated videos rather than reconstructing identity images—is a sensible way to address the tuning-inference gap and is potentially useful for the customization community. The paper has several strengths: the training rewards and the evaluation metrics use different models (ResNet-100/Glint360K for identity training vs Antelopev2 for evaluation; HPSv2 for semantic training vs CLIP-T and VBench), which mitigates reward-hacking concerns; the ablations isolate the contributions of non-reconstructive tuning, prompt augmentation, SCR, and the isolated adapter; and the single-reference and style-LoRA results indicate practical robustness. However, the headline claim of \"without dynamic and semantic degradation\" is not yet established because the quantitative evaluation omits the original unmodified T2V model as a baseline and because the SCR loss is structurally insensitive to uniform reductions in semantic alignment.","major_comments":[{"comment":"Section 5.3 reports quantitative results in Table 1 and Table 3, but neither table includes the frozen original T2V model (HunyuanVideo) on the same prompts. The paper's central claim is that PersonalVideo preserves the original model's dynamic and semantic capabilities; without the original model's CLIP-T, dynamic degree, and FVD as a reference row, a uniform drop in these quantities relative to the base model is invisible. For instance, PersonalVideo's CLIP-T (26.30) is only marginally above DreamBooth (26.26), so the reported numbers do not by themselves show that no semantic degradation occurred. Please add the base-model metrics and, if the placeholder token cannot be used by the base model, run it with the placeholder replaced by a generic subject phrase on the same prompts and seeds.","section":"§5.3, Table 1 and Table 3"},{"comment":"The SCR uses V_c = Softmax({R_sem(frame_i)}), and softmax is invariant to adding the same constant to every logit. Therefore L_SCR = 0 whenever the customized model's per-frame HPSv2 scores are the original model's scores plus a constant, no matter how large the absolute drop in text alignment is. The loss thus cannot detect or penalize uniform semantic degradation, and it provides no gradient to raise absolute HPSv2 scores; it only shapes the relative distribution across frames. Since the claim of preserved semantic following rests on this term, please either add absolute-score matching (e.g., a term on mean HPSv2/CLIP score) or demonstrate empirically with the base-model comparison that no absolute drop occurs.","section":"§4.2, Eq. (3)–(4)"},{"comment":"Even if the additive-shift issue were fixed, HPSv2 is a per-frame image-text preference model and the SCR treats the sampled frames as an unordered set. The KL divergence over frame scores contains no temporal information, so it is not evident that aligning this distribution preserves motion dynamics. Figure 10 and Table 3 show that removing SCR changes the dynamic degree (13.22 vs 17.85), but without a video-level dynamic regularizer or a comparison against the original model's dynamic degree, the mechanism by which SCR protects motion remains unexplained. Please provide evidence that the SCR, rather than prompt augmentation or the adapter design, is responsible for the dynamic-preservation effect.","section":"§4.2 and §5.6"},{"comment":"None of the quantitative tables reports error bars, confidence intervals, or significance tests, although the protocol states that 1000 videos for 20 identities were evaluated. Differences such as CLIP-T 26.30 vs 26.26 (Table 1) are within plausible run-to-run noise, and the ablation deltas in Tables 2–4 may also be within noise. Please report means and standard deviations over at least three seeds or bootstrap intervals, and state the number of videos per cell.","section":"§5.3, Tables 1–4"}],"minor_comments":[{"comment":"The text says \"minimize the cosine similarity of them,\" but the displayed objective is L_ICR = CosSim(...); minimizing CosSim would push the embeddings apart. Please change to \"maximize the cosine similarity\" or \"minimize the cosine distance.\"","section":"§4.1, Eq. (2)"},{"comment":"There are typographical errors in the captions (\"anda\" in Fig. 1; \"PreviousmethodsReference\" and the run-on \"T2I Reconstructive Tuning\" in Fig. 2). Please copyedit.","section":"Figures 1 and 2"},{"comment":"The appendix gives learning rates and iteration counts but not the values of M (number of sampled frames for SCR), the weights of L_ICR and L_SCR in Eq. (5), the adapter rank r in Eq. (6), or the precise denoising-stage injection range for the \"1/4 steps\" variant. These are needed for reproducibility.","section":"Section 7 (Implementation Details)"},{"comment":"The user study reports preferences from 15 participants over 50 sets but does not report inter-rater agreement or a statistical test; add at least a pairwise significance test to support the preference claims.","section":"§5.4, User Study"}],"recommendation":"major_revision","confidential_remarks":"The paper's claim of \"no degradation\" is stronger than the evidence. The missing baseline and the SCR softmax invariance are fixable, so I recommend major revision rather than rejection. The authors should also be asked to state whether they will release code and weights as promised, since the reproducibility statement currently only promises future release."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PersonalVideo is a serious entry in the video-personalization space. The core idea—replace image-reconstruction tuning with direct reward supervision on generated videos—is genuinely useful and addresses a real tuning-inference gap. The semantic consistency reward that aligns score distributions with a frozen T2V model is a clever regularizer, and the simulated-prompt augmentation plus later-step-only adapter injection are sensible, well-ablated design choices. On the empirical side, the paper shows clear gains over DreamBooth, MagicMe, ID-Animator, and ConsisID on face similarity, dynamic degree, and CLIP scores, and the ablations are informative.\n\nThat said, the headline claim—\"without dynamic and semantic degradation\"—is not backed up by the numbers. The paper never reports dynamic degree, CLIP-T, or any metric for the original unmodified T2V model. Without that baseline, you cannot conclude the customized model preserves the original capabilities; you can only say it degrades less than the compared baselines. This is the largest soft spot.\n\nThere is also a structural issue with the semantic consistency reward that the authors do not address. Eq. 4 applies softmax to per-frame HPSv2 scores before taking the KL divergence. Softmax is invariant to additive shifts in the logits, so if the customized model drops every frame's semantic quality by the same constant amount, the loss is exactly zero. The SCR therefore only encourages the relative shape of the score distribution across frames to match the original. It cannot detect or penalize a uniform degradation in text-image alignment. Since the authors present SCR as the component that preserves semantic following, this is a real gap between the stated mechanism and what the loss actually does. It is not fatal—the ablations show SCR helps on CLIP-T—but the explanation is overstated.\n\nMinor issues: the identity reward supervises a single random frame, which leaves temporal ID consistency under-explored; there are no error bars or significance tests on the main table; and the code and weights are promised but not yet available.\n\nBottom line: a promising method with sound engineering, but the central preservation claim is unsubstantiated until an original-model baseline is added and the SCR's invariance is acknowledged or fixed. I would send it to peer review—there is a real contribution here—but the revision needs to address those points. Worth a reading-group slot.","headline":"A promising non-reconstructive reward method for video personalization, but the no-degradation claim is unsupported without an original-model baseline and is theoretically shaky because the SCR loss is shift-invariant.","tokens_in":14761,"tokens_out":2529,"would_cite":true,"duration_ms":22476,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PersonalVideo applies reward supervision directly to generated videos, combining identity and semantic consistency rewards to avoid dynamic and semantic degradation in identity-specific video customization.","keywords":["text-to-video generation","identity customization","reward-based fine-tuning","diffusion models","tuning-inference gap","semantic consistency","video personalization"],"falsifier":"Run the tuned model and the frozen original model on the same set of prompts, measure motion amount (for example with a video-dynamics benchmark) and per-frame text alignment for both, and compare the two sets of scores. If the tuned model shows a substantial drop in motion or prompt following even though its HPSv2 score distribution matches the original model's, the claim that the semantic consistency reward preserves dynamics and semantics would be refuted.","tokens_in":13762,"feed_emoji":"🎬","tokens_out":4795,"duration_ms":42027,"temperature":0.7,"pith_summary":"The paper is trying to establish that identity-specific video customization fails when tuning is done by reconstructing static reference images on a text-to-image model, because that model's distribution differs from the text-to-video model's. The proposed fix, PersonalVideo, supervises the actual generated videos with a mixture of rewards: an identity consistency reward and a semantic consistency reward. The method also injects identity only in later denoising steps and uses simulated prompt augmentation. If correct, it would let users personalize video generators from a single photo without losing motion dynamics or prompt-following ability.","feed_headline":"Reward tuned videos, not images - identity stays, motion survives","feed_subtitle":"PersonalVideo closes the tuning-inference gap by rewarding the video itself, keeping both face fidelity and prompt-following.","key_machinery":"The central object is a mixture of two differentiable rewards applied to videos sampled from the model being tuned. The identity consistency reward extracts face-recognition embeddings from a reference image and one randomly selected generated frame and minimizes their cosine similarity. The semantic consistency reward softmaxes HPSv2 image-text scores over sampled frames for the original and target models and minimizes the KL divergence between the two distributions. The Isolated Identity Adapter is a LoRA-style residual path, $\\tilde{W} = W + A_{\\mathrm{down}} A_{\\mathrm{up}}$, added to query, key, and value layers but active only during the later denoising steps, based on the observation that motion forms early and appearance forms late. These pieces carry the argument: reward supervision replaces reconstruction, the adapter placement protects motion, and the semantic reward protects prompt following.","core_discovery":"The paper claims that applying reward supervision directly on generated videos—rather than reconstruction loss on images—bridges the tuning-inference gap and achieves high identity fidelity without dynamic and semantic degradation. The full objective is $\\mathcal{L}_{\\mathrm{train}} = \\mathcal{L}_{\\mathrm{ICR}} + \\mathcal{L}_{\\mathrm{SCR}}$ (Eq. 5). The identity consistency reward uses a pre-trained face-recognition model to minimize cosine similarity between reference identity embeddings and a randomly selected frame of the generated video. The semantic consistency reward samples frames from both the frozen original video model and the tuned model, computes their HPSv2 image-text alignment scores, and minimizes the KL divergence between the two score distributions. This distribution alignment is what the paper identifies as preserving the original model's motion dynamics and semantic following capability.","pith_inferences":["A natural extension: the semantic consistency reward idea could apply to any customization that risks shifting the video prior—style, object, or action customization—by pinning the target model's semantic-score distribution to the frozen model's distribution.","A direct test of the no-degradation claim would compare motion amount and per-frame text alignment of the customized model against the frozen original model on identical prompts; the paper's tables compare against other customization methods, not against the original text-to-video model.","Because the identity reward relies on a face-recognition embedding, the framework as described is face-centric; swapping the reward model for a generic object-similarity model would be needed for non-human subjects."],"forward_implications":["Single-reference identity customization no longer requires image-reconstruction tuning, so the tuning-inference gap disappears for any text-to-video backbone that can be reward-supervised.","The semantic consistency reward offers a general recipe for injecting a new condition into a video generator without destroying its prior: align reward-model score distributions with the frozen original model.","Injecting identity only in later denoising steps means motion is already formed before identity is applied, so dynamic degree stays close to the original model's level.","Simulated prompt augmentation makes the adapter robust to unseen prompts and backgrounds even when only one reference image is available."],"supporting_citations":[{"why":"Supplies the face-recognition embeddings used in the identity consistency reward.","marker":"[11]"},{"why":"Provides the image-text semantic scores used to align distributions in the semantic consistency reward.","marker":"[44]"},{"why":"MagicMe is the reconstruction-based video customization baseline whose tuning-inference gap motivates the method.","marker":"[29]"},{"why":"DreamBooth is the reconstruction-based text-to-image baseline compared and shown to degrade in video settings.","marker":"[33]"},{"why":"HunyuanVideo is the DiT-based text-to-video backbone used for the main experiments.","marker":"[24]"},{"why":"AnimateDiff is the UNet-based text-to-video backbone used to demonstrate generalization.","marker":"[14]"},{"why":"VBench provides the dynamic-degree metric used to measure motion preservation.","marker":"[23]"}],"fun_headline_variants":["Reward the video, not the image – ID stays, motion survives","Video reward closes tuning gap – identity and motion both kept","Direct video reward keeps face fidelity and motion intact","PersonalVideo: video reward slashes tuning-inference gap"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that making the distribution of image-text preference scores of sampled frames match the original model is enough to keep the original model's motion dynamics and prompt following, but the paper does not directly measure the original model's dynamic degree or per-frame semantic quality.","fun_headline_variants_meta":{"raw":{"variants":["Reward the video, not the image – ID stays, motion survives","Video reward closes tuning gap – identity and motion both kept","Direct video reward keeps face fidelity and motion intact","PersonalVideo: video reward slashes tuning-inference gap"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2598,"prompt_tokens":963,"completion_tokens":1635,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":1576}},"tokens_in":579,"tokens_out":1635,"duration_ms":10209,"temperature":1.0,"reasoning_tokens":1576,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:34:29.214342+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the tuned model and the frozen original model on the same set of prompts, measure motion amount (for example with a video-dynamics benchmark) and per-frame text alignment for both, and compare the two sets of scores. If the tuned model shows a substantial drop in motion or prompt following even though its HPSv2 score distribution matches the original model's, the claim that the semantic consistency reward preserves dynamics and semantics would be refuted.","supporting_citations":[{"cited_title":"Arcface: Additive angular margin loss for deep face recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the face-recognition embeddings used in the identity consistency reward."}],"review_version":1}