Pith. sign in

REVIEW 4 major objections 4 minor 5 cited by

PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PersonalVideo applies reward supervision directly to generated videos, combining identity and semantic consistency rewards to avoid dynamic and semantic degradation in identity-specific video customization.

desk verdict A promising non-reconstructive reward method for video personalization, but the no-degradation claim is unsupported without an original-model baseline and is theoretically shaky because the SCR loss is shift-invariant. read the letter →

arxiv 2411.17048 v2 pith:TVVIF6ZW submitted 2024-11-26 cs.CV

Hengjia Li , Haonan Qiu , Shiwei Zhang , Xiang Wang , Yujie Wei , Zekun Li , Yingya Zhang , Boxi Wu
show 1 more author
Deng Cai
This is my paper · ORCID
classification cs.CV
keywords text-to-videogenerationidentitycustomizationreward-basedfine-tuningdiffusionmodelstuning-inferencegapsemanticconsistencyvideopersonalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that identity-specific video customization fails when tuning is done by reconstructing static reference images on a text-to-image model, because that model's distribution differs from the text-to-video model's. The proposed fix, PersonalVideo, supervises the actual generated videos with a mixture of rewards: an identity consistency reward and a semantic consistency reward. The method also injects identity only in later denoising steps and uses simulated prompt augmentation. If correct, it would let users personalize video generators from a single photo without losing motion dynamics or prompt-following ability.

What carries the argument

The central object is a mixture of two differentiable rewards applied to videos sampled from the model being tuned. The identity consistency reward extracts face-recognition embeddings from a reference image and one randomly selected generated frame and minimizes their cosine similarity. The semantic consistency reward softmaxes HPSv2 image-text scores over sampled frames for the original and target models and minimizes the KL divergence between the two distributions. The Isolated Identity Adapter is a LoRA-style residual path, $\tilde{W} = W + A_{\mathrm{down}} A_{\mathrm{up}}$, added to query, key, and value layers but active only during the later denoising steps, based on the observation that motion forms early and appearance forms late. These pieces carry the argument: reward supervision replaces reconstruction, the adapter placement protects motion, and the semantic reward protects prompt following.

What would settle it

Run the tuned model and the frozen original model on the same set of prompts, measure motion amount (for example with a video-dynamics benchmark) and per-frame text alignment for both, and compare the two sets of scores. If the tuned model shows a substantial drop in motion or prompt following even though its HPSv2 score distribution matches the original model's, the claim that the semantic consistency reward preserves dynamics and semantics would be refuted.

Watch

Extended reading notes

Core claim

The paper claims that applying reward supervision directly on generated videos—rather than reconstruction loss on images—bridges the tuning-inference gap and achieves high identity fidelity without dynamic and semantic degradation. The full objective is $\mathcal{L}_{\mathrm{train}} = \mathcal{L}_{\mathrm{ICR}} + \mathcal{L}_{\mathrm{SCR}}$ (Eq. 5). The identity consistency reward uses a pre-trained face-recognition model to minimize cosine similarity between reference identity embeddings and a randomly selected frame of the generated video. The semantic consistency reward samples frames from both the frozen original video model and the tuned model, computes their HPSv2 image-text alignment scores, and minimizes the KL divergence between the two score distributions. This distribution alignment is what the paper identifies as preserving the original model's motion dynamics and semantic following capability.

Load-bearing premise

The method assumes that making the distribution of image-text preference scores of sampled frames match the original model is enough to keep the original model's motion dynamics and prompt following, but the paper does not directly measure the original model's dynamic degree or per-frame semantic quality.

Editorial extensions

If this is right

  • Single-reference identity customization no longer requires image-reconstruction tuning, so the tuning-inference gap disappears for any text-to-video backbone that can be reward-supervised.
  • The semantic consistency reward offers a general recipe for injecting a new condition into a video generator without destroying its prior: align reward-model score distributions with the frozen original model.
  • Injecting identity only in later denoising steps means motion is already formed before identity is applied, so dynamic degree stays close to the original model's level.
  • Simulated prompt augmentation makes the adapter robust to unseen prompts and backgrounds even when only one reference image is available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension: the semantic consistency reward idea could apply to any customization that risks shifting the video prior—style, object, or action customization—by pinning the target model's semantic-score distribution to the frozen model's distribution.
  • A direct test of the no-degradation claim would compare motion amount and per-frame text alignment of the customized model against the frozen original model on identical prompts; the paper's tables compare against other customization methods, not against the original text-to-video model.
  • Because the identity reward relies on a face-recognition embedding, the framework as described is face-centric; swapping the reward model for a generic object-similarity model would be needed for non-human subjects.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes PersonalVideo, a text-to-video customization framework that replaces reconstruction-based tuning on static reference images with reward supervision on videos sampled from the target T2V model. The training objective (Eq. 5) combines an identity consistency reward (ICR) on a randomly selected generated frame and a semantic consistency reward (SCR) that matches the softmax-normalized HPSv2 score distribution of sampled frames against that of the frozen original generator. The method also uses simulated prompt augmentation and an isolated identity adapter applied only in later denoising steps. Experiments on HunyuanVideo and AnimateDiff compare with DreamBooth-LoRA, MagicMe, ID-Animator, and ConsisID using face similarity, dynamic degree, FVD, temporal consistency, CLIP-T/CLIP-I, ablations, and a user study.

Significance. The core idea—applying differentiable reward feedback directly to generated videos rather than reconstructing identity images—is a sensible way to address the tuning-inference gap and is potentially useful for the customization community. The paper has several strengths: the training rewards and the evaluation metrics use different models (ResNet-100/Glint360K for identity training vs Antelopev2 for evaluation; HPSv2 for semantic training vs CLIP-T and VBench), which mitigates reward-hacking concerns; the ablations isolate the contributions of non-reconstructive tuning, prompt augmentation, SCR, and the isolated adapter; and the single-reference and style-LoRA results indicate practical robustness. However, the headline claim of "without dynamic and semantic degradation" is not yet established because the quantitative evaluation omits the original unmodified T2V model as a baseline and because the SCR loss is structurally insensitive to uniform reductions in semantic alignment.

major comments (4)
  1. [§5.3, Table 1 and Table 3] Section 5.3 reports quantitative results in Table 1 and Table 3, but neither table includes the frozen original T2V model (HunyuanVideo) on the same prompts. The paper's central claim is that PersonalVideo preserves the original model's dynamic and semantic capabilities; without the original model's CLIP-T, dynamic degree, and FVD as a reference row, a uniform drop in these quantities relative to the base model is invisible. For instance, PersonalVideo's CLIP-T (26.30) is only marginally above DreamBooth (26.26), so the reported numbers do not by themselves show that no semantic degradation occurred. Please add the base-model metrics and, if the placeholder token cannot be used by the base model, run it with the placeholder replaced by a generic subject phrase on the same prompts and seeds.
  2. [§4.2, Eq. (3)–(4)] The SCR uses V_c = Softmax({R_sem(frame_i)}), and softmax is invariant to adding the same constant to every logit. Therefore L_SCR = 0 whenever the customized model's per-frame HPSv2 scores are the original model's scores plus a constant, no matter how large the absolute drop in text alignment is. The loss thus cannot detect or penalize uniform semantic degradation, and it provides no gradient to raise absolute HPSv2 scores; it only shapes the relative distribution across frames. Since the claim of preserved semantic following rests on this term, please either add absolute-score matching (e.g., a term on mean HPSv2/CLIP score) or demonstrate empirically with the base-model comparison that no absolute drop occurs.
  3. [§4.2 and §5.6] Even if the additive-shift issue were fixed, HPSv2 is a per-frame image-text preference model and the SCR treats the sampled frames as an unordered set. The KL divergence over frame scores contains no temporal information, so it is not evident that aligning this distribution preserves motion dynamics. Figure 10 and Table 3 show that removing SCR changes the dynamic degree (13.22 vs 17.85), but without a video-level dynamic regularizer or a comparison against the original model's dynamic degree, the mechanism by which SCR protects motion remains unexplained. Please provide evidence that the SCR, rather than prompt augmentation or the adapter design, is responsible for the dynamic-preservation effect.
  4. [§5.3, Tables 1–4] None of the quantitative tables reports error bars, confidence intervals, or significance tests, although the protocol states that 1000 videos for 20 identities were evaluated. Differences such as CLIP-T 26.30 vs 26.26 (Table 1) are within plausible run-to-run noise, and the ablation deltas in Tables 2–4 may also be within noise. Please report means and standard deviations over at least three seeds or bootstrap intervals, and state the number of videos per cell.
minor comments (4)
  1. [§4.1, Eq. (2)] The text says "minimize the cosine similarity of them," but the displayed objective is L_ICR = CosSim(...); minimizing CosSim would push the embeddings apart. Please change to "maximize the cosine similarity" or "minimize the cosine distance."
  2. [Figures 1 and 2] There are typographical errors in the captions ("anda" in Fig. 1; "PreviousmethodsReference" and the run-on "T2I Reconstructive Tuning" in Fig. 2). Please copyedit.
  3. [Section 7 (Implementation Details)] The appendix gives learning rates and iteration counts but not the values of M (number of sampled frames for SCR), the weights of L_ICR and L_SCR in Eq. (5), the adapter rank r in Eq. (6), or the precise denoising-stage injection range for the "1/4 steps" variant. These are needed for reproducibility.
  4. [§5.4, User Study] The user study reports preferences from 15 participants over 50 sets but does not report inter-rater agreement or a statistical test; add at least a pairwise significance test to support the preference claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: training rewards and evaluation metrics are distinct, and the semantic consistency loss is a regularizer toward the frozen original model rather than a fitted prediction.

full rationale

The paper's derivation chain is self-contained. The identity reward (Eq. 2) is a cosine-similarity loss computed with a pre-trained face recognition model on generated frames, while the reported ID fidelity metric uses Antelopev2, a different face recognition model; the training objective therefore does not directly optimize the evaluation metric. The semantic consistency reward (Eqs. 3-4) aligns the HPSv2 score distribution of the target model with that of the frozen original T2V model; this is a distillation-style regularizer, not a quantity fitted to the paper's own headline outcome. The paper's evidence for preserved dynamics and semantics is independent of the SCR objective: Table 1 reports VBench Dynamic Degree and CLIP-T, and Table 3 compares w/SCR vs w/o SCR on those same metrics rather than reporting the training loss itself. Self-citations (e.g., Modelscope T2V, DreamVideo, DreamVideo-2) appear only as related-work context and are not load-bearing premises. The skeptic's observation that the softmax-based SCR is invariant to uniform additive shifts in HPSv2 scores is a validity/coverage limitation of the proxy, not circularity: it does not make the reported CLIP-T or Dynamic Degree numbers equal to the training objective by construction. No equation or fitted parameter is renamed as a prediction, so no circular step is identified.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method is an empirical training recipe with several hand-set hyperparameters but no free parameters fitted to a scientific constant. The main assumptions are about the reliability of proxy reward models and the causal claim about denoising steps. No new physical or conceptual entities are introduced.

free parameters (5)
  • Loss weights for ICR and SCR = 1 (default)
    Eq. 5 sums the losses without learned or tuned coefficients; no sensitivity analysis is reported.
  • Number of simulated prompts = 50
    Section 4.3 fixes 50 LLM-generated prompts; the choice is not ablated.
  • M frames sampled for SCR = not specified
    Eq. 3 samples M frames, but M is never stated; this controls the softmax distribution alignment.
  • Adapter rank r = not specified
    Eq. 6 uses low-rank matrices Adown and Aup, but the rank r is not given in the appendix.
  • Denoising injection range = last quarter of steps
    Section 4.4 chooses later-step injection based on observation; ablation shows different trade-offs for all steps, 1/2, and 1/4.
assumptions (5)
  • domain assumption Pre-trained face recognition model ResNet-100 on Glint360K provides a valid and sufficient identity similarity signal for generated faces in arbitrary scenes.
    Section 5.1 uses this model for the identity reward; if it fails on stylized or novel poses, identity injection could be biased.
  • domain assumption HPSv2 semantic reward scores are a reliable proxy for semantic quality and prompt following in videos; aligning softmax score distributions preserves motion and semantics.
    Section 4.2 builds the semantic consistency loss on this premise and does not validate it independently of CLIP-T and VBench.
  • domain assumption Motion dynamics are primarily formed in early denoising steps, so injecting identity only in later steps does not harm motion.
    Section 4.4 and Fig. 4 make this observation-based assumption; it is plausible but not rigorously proven.
  • domain assumption The frozen original T2V model is an appropriate reference; matching its semantic score distribution does not reduce diversity or introduce identity bleed.
    Used in Eq. 3; if the original model has biases or frame sampling is unrepresentative, the alignment may be misleading.
  • standard math Diffusion model noise-prediction and DDIM sampling assumptions hold for both base models.
    The training relies on the standard diffusion objective (Eq. 1) and 50-step DDIM sampling.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation." pith.science (2026). https://pith.science/paper/TVVIF6ZW

@misc{pith2026241117048,
  author       = {Pith},
  title        = {Pith review of: PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TVVIF6ZW}},
  note         = {Machine review of arXiv:2411.17048}
}
abstract

The current text-to-video (T2V) generation has made significant progress in synthesizing realistic general videos, but it is still under-explored in identity-specific human video generation with customized ID images. The key challenge lies in maintaining high ID fidelity consistently while preserving the original motion dynamic and semantic following after the identity injection. Current video identity customization methods mainly rely on reconstructing given identity images on text-to-image models, which have a divergent distribution with the T2V model. This process introduces a tuning-inference gap, leading to dynamic and semantic degradation. To tackle this problem, we propose a novel framework, dubbed $\textbf{PersonalVideo}$, that applies a mixture of reward supervision on synthesized videos instead of the simple reconstruction objective on images. Specifically, we first incorporate identity consistency reward to effectively inject the reference's identity without the tuning-inference gap. Then we propose a novel semantic consistency reward to align the semantic distribution of the generated videos with the original T2V model, which preserves its dynamic and semantic following capability during the identity injection. With the non-reconstructive reward training, we further employ simulated prompt augmentation to reduce overfitting by supervising generated results in more semantic scenarios, gaining good robustness even with only a single reference image. Extensive experiments demonstrate our method's superiority in delivering high identity faithfulness while preserving the inherent video generation qualities of the original T2V model, outshining prior methods.

Figures

Figures reproduced from arXiv: 2411.17048 by the authors.

Figure 1
Figure 1. Results of PersonalVideo. Given the reference images of a specific identity, PersonalVideo can generate high ID-fidelity videos with promising motion dynamics and prompt following. Abstract The current text-to-video (T2V) generation has made signif￾icant progress in synthesizing realistic general videos, but it is still under-explored in identity-specific human video gen￾eration with customized ID images. The key ch… view at source ↗
Figure 2
Figure 2. Analysis of the tuning-inference gap. Previous T2V customization supervises the tuning process via reconstructing images on T2I models, suffering from a tuning-inference gap. Differently, we aim to directly apply the supervision on generated videos, which aligns with inference and bridges the gap. pervision on synthesized videos instead of the simple recon￾struction objective on images. Specifically, we first incor￾… view at source ↗
Figure 3
Figure 3. Overview of the framework of PersonalVideo. To bridge the tuning-inference gap, we directly apply reward supervision on generated videos starting from pure noises, including identity consistency reward with the reference and semantic consistency reward with the original video. During the optimization, we adopt simulated prompt sampled from the Large Language Model to supervise generated results in more semantic scen… view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: Visualization of the video denoising steps. The mo￾tion of the person, e.g., his hand, is formed in early stages of the denoising process. the later steps focus on the recovering of the detailed appearance. original model’s semantic distribution, effectively alleviat￾i…
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison for a single reference. As observed, the baseline methods not only demonstrate suboptimal identity fidelity but also suffer from compromised dynamic motions and insufficient video quality. In comparison, our PersonalVideo maintains higher ID fide…
Figure 7
Figure 7. Figure 7: User study. Our PersonalVideo achieves the best human preference compared with DreamBooth and MagicMe. 5.4. User Study To further assess the effectiveness of our approach, we per￾form a human evaluation comparing our method with ex￾isting T2V identity customization tec…
Figure 8
Figure 8. Figure 8: Compatibility with customized style LoRAs, i.e., RCNZ Cartoon 3D, GuoFengRealMix, and GuoFeng. Prompt Tuning on T2I w Prompt Aug. Tuning on T2V w/o Prompt Aug. Tuning on T2V w Prompt Aug. Reference A <V> woman holding a bottle of red wine, besides the wine rack [PITH_…
Figure 9
Figure 9. Figure 9: Ablation study for the non-reconstructive training and simulated prompt augmentation. As observed, tuning on the T2I model suffers from inferior ID fidelity and blurred back￾ground. Besides, tuning without prompt augmentation degrades the semantic following, i.e., the …
Figure 10
Figure 10. Figure 10: Ablation study for the Semantic Consistency Re￾ward. As observed, tuning without SCR suffers from the dynamic degradation and the inferior semantic following, i.e., by the lake. ground. In contrast, tuning on the T2V model bridges the gap to achieve better ID fidelity…
Figure 12
Figure 12. Figure 12 [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: More comparisons on HunyuanVideo. As observed, Dreambooth suffers from inferior ID fidelity, while our PersonalVideo maintains higher ID fidelity without dynamic and semantic degradation, which is consistent with [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: More comparison for a single reference. While ConsisID and Dreambooth suffer from the inferior ID fidelity, as well as severe degradation of motion dynamics and semantic following, e.g. the stool by the lake, our PersonalVideo achieves robust customization with high I…
Figure 15
Figure 15. Figure 15: Ablation for different layers to inject the identity. As observed, injecting it on the cross-attention layer disrupts the ability of semantic following, e.g., the losing of exquisite armor [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]
Figure 16
Figure 16. Figure 16: More results of PersonalVideo. “A <V> musician man sitting on a stool, strumming an acoustic guitar with a serene expression. He wears a casual button-down shirt and jeans, lost in the melody.” Reference [PITH_FULL_IMAGE:figures/full_fig_p014_16.png]
Figure 17
Figure 17. Figure 17: More results of PersonalVideo [PITH_FULL_IMAGE:figures/full_fig_p014_17.png]
Figure 18
Figure 18. Figure 18: More results of PersonalVideo [PITH_FULL_IMAGE:figures/full_fig_p015_18.png]
Figure 19
Figure 19. Figure 19: More results of PersonalVideo [PITH_FULL_IMAGE:figures/full_fig_p016_19.png]
Figure 20
Figure 20. Figure 20: More results of PersonalVideo. A <V> doctor man in a white lab coat, smiling warmly while holding a clipboard. His stethoscope hangs around his neck as he stands confidently in a bright hospital hallway. A <V> musician man sitting on a stool, strumming an acoustic gui…
Figure 21
Figure 21. Figure 21: More results of PersonalVideo [PITH_FULL_IMAGE:figures/full_fig_p017_21.png]
Figure 22
Figure 22. Figure 22: More results of PersonalVideo. Single Reference A <V> woman playing basketball A <V> woman reading a book in the classroom A <V> woman expertly pours a glass of wine, savoring the aroma A <V> woman playing the guitar A <V> woman, cherry blossoms sway in the breeze A <…
Figure 23
Figure 23. Figure 23: More results of PersonalVideo with only just one image [PITH_FULL_IMAGE:figures/full_fig_p018_23.png]
Figure 24
Figure 24. Figure 24: More results of PersonalVideo with few images [PITH_FULL_IMAGE:figures/full_fig_p019_24.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LaVieID: Local Autoregressive Diffusion Transformers for Identity-Preserving Video Creation

    cs.CV 2025-08 conditional novelty 6.0 of 10

    LaVieID improves identity-preserving text-to-video by routing local facial parts into early DiT blocks and autoregressively refining denoised video tokens in temporal chunks.

  2. BridgeIV: Bridging Customized Image and Video Generation through Test-Time Autoregressive Identity Propagation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    BridgeIV improves subject consistency in customized text-to-video generation by warping attention maps and self-attention values across frames, then refining latents with a CLIP-based reward.

  3. GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generators

    cs.CV 2024-12 conditional novelty 6.0 of 10

    GCA-3D adapts 3D generators to text or one-shot image domains without dataset synthesis, using depth-aware score distillation and hierarchical spatial consistency losses.

  4. RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control

    cs.CV 2025-02 conditional novelty 5.0 of 10

    Metric-scale depth alignment plus scene-constrained noise shaping improves camera controllability and video quality for image-to-video generation on RealEstate10K.

  5. A Summer Meridional Subsurface Temperature Dipole Mode in the South China Sea

    physics.ao-ph 2025-08 unverdicted novelty 4.0 of 10

    The manuscript body does not match the abstract, so the ocean dipole claim is unsupported by any presented evidence.

Reference graph

Works this paper leans on

55 extracted references · 24 canonical work pages · cited by 5 Pith papers

  1. [1]

    https : / / civitai

    Guofeng v3. https : / / civitai . com / models / 10415/3-guofeng3, 2023. 7

    work page 2023

  2. [2]

    https : / / civitai

    Rcnz cartoon 3d v1.0. https : / / civitai . com / models/66347?modelVersionId=71009, 2023. 7

    work page 2023

  3. [3]

    https : / / civitai

    Realistic vision v5.1. https : / / civitai . com / models/4201/realistic-vision-v51, 2023. 11

    work page 2023

  4. [4]

    https://civitai.com/models/ 77650/guofengrealmix, 2023

    Guofeng realmix. https://civitai.com/models/ 77650/guofengrealmix, 2023. 7

    work page 2023

  5. [5]

    Partial fc: Training 10 million identities on a single ma- chine

    Xiang An, Xuhan Zhu, Yuan Gao, Yang Xiao, Yongle Zhao, Ziyong Feng, Lan Wu, Bin Qin, Ming Zhang, Debing Zhang, et al. Partial fc: Training 10 million identities on a single ma- chine. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 1445–1449, 2021. 5

    work page 2021

  6. [6]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In CVPR, 2023. 3, 4

    work page 2023

  7. [7]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators

  8. [8]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In CVPR, pages 22560–22570, 2023. 5

    work page 2023

Show all 55 references
  1. [9]

    Still-moving: Customized video generation without customized video data

    Hila Chefer, Shiran Zada, Roni Paiss, Ariel Ephrat, Omer Tov, Michael Rubinstein, Lior Wolf, Tali Dekel, Tomer Michaeli, and Inbar Mosseri. Still-moving: Customized video generation without customized video data. arXiv preprint arXiv:2407.08674, 2024. 2

    2024 arXiv

  2. [10]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In CVPR, pages 7310–7320, 2024. 2

    2024

  3. [11]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 4690–4699, 2019. 4, 6

    2019

  4. [12]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 2, 3

    2022 arXiv

  5. [13]

    Lcm- lookahead for encoder-based text-to-image personalization

    Rinon Gal, Or Lichter, Elad Richardson, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Lcm- lookahead for encoder-based text-to-image personalization. arXiv preprint arXiv:2404.03620, 2024. 3, 4

    2024 arXiv

  6. [14]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 2, 4

    2023 arXiv

  7. [15]

    Pulid: Pure and lightning id customization via con- trastive alignment

    Zinan Guo, Yanze Wu, Zhuowei Chen, Lang Chen, and Qian He. Pulid: Pure and lightning id customization via con- trastive alignment. arXiv preprint arXiv:2404.16022, 2024. 2, 3, 4

    2024 arXiv

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

  9. [17]

    Id-animator: Zero-shot identity-preserving human video generation.arXiv preprint arXiv:2404.15275, 2024

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, Man Zhou, and Jie Zhang. Id-animator: Zero-shot identity-preserving human video generation.arXiv preprint arXiv:2404.15275, 2024. 3, 5

    2024 arXiv

  10. [18]

    Latent video diffusion models for high-fidelity video generation with arbitrary lengths

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity video generation with arbitrary lengths. arXiv preprint arXiv:2211.13221, 2022. 3

    2022 arXiv

  11. [19]

    Animate-a-story: Storytelling with retrieval-augmented video generation

    Yingqing He, Menghan Xia, Haoxin Chen, Xiaodong Cun, Yuan Gong, Jinbo Xing, Yong Zhang, Xintao Wang, Chao Weng, Ying Shan, et al. Animate-a-story: Storytelling with retrieval-augmented video generation. arXiv preprint arXiv:2307.06940, 2023. 2

    2023 arXiv

  12. [20]

    Imagen video: High definition video generation with diffusion mod- els

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022. 3

    arXiv

  13. [21]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. In NeurIPS, 2022. 3

    2022

  14. [22]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3, 5

    2021 arXiv

  15. [23]

    Vbench: Comprehensive bench- mark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2024

  16. [24]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 4, 5

    2024 arXiv

  17. [25]

    Few-shot hybrid domain adaptation of im- age generator

    Hengjia Li, Yang Liu, Linxuan Xia, Yuqi Lin, Wenxiao Wang, Tu Zheng, Zheng Yang, Xiaohui Zhong, Xiaobo Ren, and Xiaofei He. Few-shot hybrid domain adaptation of im- age generator. In The Twelfth International Conference on Learning Representations, 2023. 3

    2023

  18. [26]

    Unihda: Towards universal hybrid domain adapta- tion of image generators

    Hengjia Li, Yang Liu, Yuqi Lin, Zhanwei Zhang, Yibo Zhao, Tu Zheng, Zheng Yang, Yuchun Jiang, Boxi Wu, Deng Cai, et al. Unihda: Towards universal hybrid domain adapta- tion of image generators. arXiv preprint arXiv:2401.12596, 2024

    2024 arXiv

  19. [27]

    Photomaker: Customizing re- alistic human photos via stacked id embedding

    Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming- Ming Cheng, and Ying Shan. Photomaker: Customizing re- alistic human photos via stacked id embedding. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8640–8650, 2024. 2

    2024

  20. [28]

    Subject- diffusion: Open domain personalized text-to-image genera- tion without test-time fine-tuning

    Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject- diffusion: Open domain personalized text-to-image genera- tion without test-time fine-tuning. In ACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024. 3

    2024

  21. [29]

    Magic-me: Identity-specific video customized diffu- sion

    Ze Ma, Daquan Zhou, Chun-Hsiao Yeh, Xue-She Wang, Xi- uyu Li, Huanrui Yang, Zhen Dong, Kurt Keutzer, and Jiashi Feng. Magic-me: Identity-specific video customized diffu- sion. arXiv preprint arXiv:2402.09368, 2024. 2, 3, 5

    2024 arXiv

  22. [30]

    Localizing object-level shape variations with text-to-image diffusion models

    Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch- Elor, and Daniel Cohen-Or. Localizing object-level shape variations with text-to-image diffusion models. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision (ICCV), 2023. 5

    2023

  23. [31]

    Portraitbooth: A versatile portrait model for fast identity-preserved personalization

    Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, and Rongrong Ji. Portraitbooth: A versatile portrait model for fast identity-preserved personalization. InProceedings of the IEEE/CVF Conference on Computer Vision a...

    2024

  24. [32]

    Encoding in style: a stylegan encoder for image-to-image translation

    Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-Or. Encoding in style: a stylegan encoder for image-to-image translation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 2287–22...

    2021

  25. [33]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pages 22500–22510, 2023. 2, 3, 5

    2023

  26. [34]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. In ICLR, 2023. 3

    2023

  27. [35]

    Face0: Instantaneously conditioning a text-to- image model on a face

    Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan. Face0: Instantaneously conditioning a text-to- image model on a face. InSIGGRAPH Asia 2023 Conference Papers, pages 1–10, 2023. 3

    2023

  28. [36]

    Animatelcm: Accelerating the animation of personalized diffusion mod- els and adapters with decoupled consistency learning

    Fu-Yun Wang, Zhaoyang Huang, Xiaoyu Shi, Weikang Bian, Guanglu Song, Yu Liu, and Hongsheng Li. Animatelcm: Accelerating the animation of personalized diffusion mod- els and adapters with decoupled consistency learning. arXiv preprint arXiv:2402.00769, 2024. 4

    2024 arXiv

  29. [37]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 2, 3, 4

    2023 arXiv

  30. [38]

    Instantid: Zero-shot identity-preserving gener- ation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and An- thony Chen. Instantid: Zero-shot identity-preserving gener- ation in seconds. arXiv preprint arXiv:2401.07519, 2024. 2, 3

    2024 arXiv

  31. [39]

    To- wards real-world blind face restoration with generative fa- cial prior

    Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. To- wards real-world blind face restoration with generative fa- cial prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9168–9178,

  32. [40]

    Lavie: High-quality video gener- ation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video gener- ation with cascaded latent diffusion models. arXiv preprint arXiv:2309.15103, 2023. 3

    2023 arXiv

  33. [41]

    Dreamvideo: Composing your dream videos with customized subject and motion

    Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhi- heng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hong- ming Shan. Dreamvideo: Composing your dream videos with customized subject and motion. In CVPR, pages 6537– 6549, 2024. 2, 3

    2024

  34. [42]

    Dreamvideo-2: Zero-shot subject- driven video customization with precise motion control

    Yujie Wei, Shiwei Zhang, Hangjie Yuan, Xiang Wang, Hao- nan Qiu, Rui Zhao, Yutong Feng, Feng Liu, Zhizhong Huang, Jiaxin Ye, et al. Dreamvideo-2: Zero-shot subject- driven video customization with precise motion control. arXiv preprint arXiv:2410.13830, 2024. 3

    arXiv

  35. [43]

    Cus- tomcrafter: Customized video generation with preserving motion and concept composition abilities

    Tao Wu, Yong Zhang, Xintao Wang, Xianpan Zhou, Guang- cong Zheng, Zhongang Qi, Ying Shan, and Xi Li. Cus- tomcrafter: Customized video generation with preserving motion and concept composition abilities. arXiv preprint arXiv:2408.13239, 2024. 3

    2024 arXiv

  36. [44]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  37. [45]

    Fastcomposer: Tuning-free multi- subject image generation with localized attention

    Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention. Interna- tional Journal of Computer Vision, pages 1–20, 2024. 3

    2024

  38. [46]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  39. [47]

    Identity- preserving text-to-video generation by frequency decompo- sition

    Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge, Yu- jun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan. Identity- preserving text-to-video generation by frequency decompo- sition. arXiv preprint arXiv:2411.17440, 2024. 3, 5

    2024 arXiv

  40. [48]

    Magicvideo: Efficient video generation with latent diffusion models

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022. 3 PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradati...

    2022 arXiv

  41. [49]

    For AnimateDiff, we use Stable Diffu- sion 1.5 with Realistic Vision [3] during training and in- ference

    Implementation Details Training Details. For AnimateDiff, we use Stable Diffu- sion 1.5 with Realistic Vision [3] during training and in- ference. During training, we learn the Isolated Identity Adapter for 800 iterations with a learning rate of 1e-4 with the batch size 1. We ...

  42. [50]

    During infer- ence, we use 50 steps of DDIM sampler and classifier-free guidance with a scale of 7.5 for all baselines

    For Hunyuanvideo, we employ the AdamW optimizer configured with a learning rate of 2e-5 and a weight decay parameter of 1e-4 for 4000 training steps. During infer- ence, we use 50 steps of DDIM sampler and classifier-free guidance with a scale of 7.5 for all baselines. We gene...

  43. [51]

    More Comparison We provide more comparison including more base models and different number of the references in Fig. 12, Fig. 13, and Fig. 14. As shown, both Dreambooth and MagicMe suf- fer from inferior ID fidelity. Besides, MagicMe has a severe misalignment of the prompt, e....

  44. [52]

    15 and Tab

    More ablation study Fig. 15 and Tab. 4b verify the improvement in semantic fol- lowing of our Isolated Identity Adapter to inject the identity only on the spatial self-attention layer. As observed, inject- ing only on the cross-attention layer gets inferior ID fidelity with th...

  45. [53]

    More Results As shown in Fig. 20, Fig. 21, Fig. 22, Fig. 23, and Fig. 24, we present more customization results of PersonalVideo, in- cluding few or just one reference image. They showcase it achieves high ID fidelity and preserves original motion dynamics and semantic followi...

  46. [54]

    (2) We provide training details in the appendix (Sec

    Reproducibility Statement We make the following efforts to ensure the reproducibil- ity of PersonalVideo: (1) Our training and inference codes together with the trained model weights will be publicly available. (2) We provide training details in the appendix (Sec. 7), which is...

  47. [55]

    A <V> musician man sitting on a stool, strumming an acoustic guitar with a serene expression. He wears a casual button-down shirt and jeans, lost in the melody

    Impact Statement Our main objective in this work is to empower novice users to generate visual content creatively and flexibly. However, we acknowledge the potential for misuse in creating fake or harmful content with our method. Thus, we believe it’s es- sential to develop an...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.