Pith. sign in

REVIEW 4 major objections 5 minor 83 references

GroupVideo uses multimodal identity alignment and ID localization to generate videos with multiple consistent characters and natural motion.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 08:39 UTC pith:FOI5CDOK

load-bearing objection A promising architecture and a potentially valuable dataset, but the paper's central quantitative claim is undercut by an unexplained discrepancy between its own tables. the 4 major comments →

arxiv 2607.21027 v1 pith:FOI5CDOK submitted 2026-07-23 cs.CV

GroupVideo: Multi-Identity Customized Text-to-Video Generation

Xinyang Song , Libin Wang , Jianxin Sun , Qi Li , Dandan Zheng , JingDong Chen , Zhenan Sun This is my paper
classification cs.CV
keywords multi-identity video generationidentity preservationtext-to-video generationvideo diffusion transformeridentity localizationmultimodal alignmentcopy-paste artifactmulti-person video dataset
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that multi-identity text-to-video generation fails when identity conditions are simply stacked, and the fix is to split identity injection into visual alignment, semantic alignment, and an ID localization module that routes each identity to its own spatiotemporal region. If correct, the framework yields videos where each face stays faithful, characters move naturally instead of appearing copy-pasted, and the method extends to an arbitrary number of reference faces without per-scene masks. The authors also contribute an open 20,000-video dataset of 1080x1920 two-to-three-person clips with face masks and captions.

Core claim

GroupVideo claims that identity blending and the 'copy-paste' artifact in multi-ID video generation can be eliminated by treating each reference face as an independent conditioning signal in both visual and semantic spaces, then routing the resulting identity embeddings through learned localization masks. The visual branch concatenates per-face VAE tokens with the noise latent along channels, while the semantic branch fuses face embeddings into text embeddings via a semantic perceiver. The ID localization module computes a soft mask over spatiotemporal positions for each identity and gates cross-attention, preventing identities from overlapping. The paper reports the best text alignment and

What carries the argument

Three mechanisms: (1) visual identity alignment, which concatenates each reference face's VAE tokens with noise latents along the channel axis; (2) semantic identity alignment, which derives per-face identity embeddings and fuses them into text embeddings via a semantic perceiver; (3) the ID localization module, which computes a soft mask for each identity from DiT hidden states and identity embeddings and uses it to gate masked cross-attention. A progressive two-stage training, with a bounding-box loss in the first stage and a mask-regularization loss in the second, stabilizes optimization and speeds convergence.

Load-bearing premise

The method relies on face masks and identity embeddings that are correct and separable, and on a training set of 2-3-person clips that represents arbitrary multi-person scenes; if those are biased, the localization module may fail for dynamic or atypical prompts.

What would settle it

Generate prompts where two identities swap positions or one turns away for several frames and inspect the predicted localization masks frame by frame; if the masks blend or swap identities, the claimed routing fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Multi-identity video generation can be done without per-prompt fine-tuning or predefined spatial masks, using an offline-trained diffusion transformer.
  • The ID localization module supports an arbitrary number of identities, including three or more, by assigning each to its own spatiotemporal region.
  • The curated 20,000-video dataset with face masks and captions provides a resource for training and evaluating multi-identity video generation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the method scales, identity-conditioned generation could move from single-character avatars to controllable multi-character scenes; a testable extension is to combine learned masks with user-provided boxes for explicit layout control.
  • The dataset's filter for 2-3 clearly visible faces may be a hidden factor; an extension is to test on out-of-distribution prompts with crowded or partially occluded scenes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes GroupVideo, a multi-identity customized text-to-video generation framework built on a Video Diffusion Transformer (CogVideoX-5B). It introduces two main components: multimodal identity alignment (visual identity alignment concatenating face-derived latent tokens with noise latents, and semantic alignment via a Q-Former + semantic perceiver that fuses identity embeddings with text embeddings) and an ID localization module that predicts spatial masks to route identity embeddings through masked cross-attention. Training is progressive: stage I optimizes alignment modules with a bounding-box loss, and stage II fine-tunes the localization module with a mask-regularization loss. The authors also curate a 20,000-video multi-person dataset and report quantitative, qualitative, user-study, and efficiency comparisons against existing single-ID and multi-ID methods. The central claim is that GroupVideo outperforms prior methods in multi-character identity consistency and motion naturalness.

Significance. If the empirical claims were stable and reproducible, GroupVideo would be a meaningful contribution to multi-identity video customization: it addresses a real limitation of single-ID frameworks, introduces a scalable identity-alignment design, and provides a curated multi-person video dataset that could support future work. The two-stage training with explicit face-region losses and the lightweight ID localization module are reasonable engineering choices, and the computational overhead analysis (Tab. VI) is a useful addition. However, the paper's own numbers are internally inconsistent, and the text misreads its own table, so the central 'outperforms' claim is not currently supported by a reliable evaluation basis.

major comments (4)
  1. [Section IV-C, Tables II and IV] The full GroupVideo model is reported with inconsistent numbers in the main comparison and the ablation table. In Tab. II, GroupVideo has FaceSim 0.636, CLIPScore 32.035, FID 147.228; in Tab. IV, the same 'GroupVideo' row reports FaceSim 0.733, CLIPScore 31.95, FID 136.94. These are large discrepancies (~0.1 FaceSim, ~10 FID). No explanation is given (different test subsets, different seeds, or different checkpoints). Since the paper's central claim is that GroupVideo outperforms existing methods, the evaluation must be based on a single, clearly defined protocol. As written, a reader cannot tell which numbers represent the official comparison, and the 'outperforms' claim has no stable evidentiary basis.
  2. [Section IV-C, text near Table II] The text states: 'Although Ingredients and ConsisID achieve higher scores in face similarity due to their copy-paste phenomena...' Tab. II shows ConsisID FaceSim = 0.634, which is below GroupVideo's 0.636; only Ingredients (0.644) is higher. This is a direct misreading of the paper's own results. This is not a minor wording issue: it affects the interpretation of whether GroupVideo actually leads on identity fidelity among DiT-based baselines. The comparison sentence should be corrected and the quantitative claims aligned with the table.
  3. [Section IV-A and IV-B (dataset, test set, and evaluation protocol)] The evaluation protocol has several unaddressed confounds. First, the test set is only 50 face images and 40 prompts (500 cases), with no error bars or significance tests; FaceSim differences of 0.002 (e.g., 0.636 vs 0.634) are within likely noise. Second, baselines use different backbone models: ID-Animator uses AnimateDiff, ConsisID/Ingredients/Concat-ID use CogVideoX, and MAGREF uses Wan-2.1. Differences may therefore reflect base-model capacity rather than the proposed modules. Third, the curated 20k dataset is filtered to exactly 2–3 discernible characters with high face visibility (steps 4–6), but no distributional analysis is provided, and despite Tab. I claiming 'Open Accessibility', no dataset release link is given. If the dataset is biased toward static, interaction-light indoor scenes, the claimed generalization to dynamic or atypical prompts (Fig. 7) may be limited. Please pr
  4. [Section IV-C, Table III (user study)] The text says GroupVideo 'achieves the highest ratings across nearly all evaluation dimensions,' but Tab. III shows GroupVideo's Identity Similarity is 6.54, lower than Ingredients' 6.75. The following sentence ('Although Ingredients achieves a higher user score in FaceSim') also misnames the dimension: Tab. III has no FaceSim column; the dimension is 'Identity Similarity.' Since identity consistency is one of the two headline claims, a user-study result where the main multi-ID competitor scores higher on identity is important and should be discussed transparently, not obscured by wording.
minor comments (5)
  1. [Author block] The author block contains a typo: 'Graduate Student Membder' should be 'Member'.
  2. [Fig. 2 and Fig. 5 captions] Fig. 2 caption spells 'Ingradients' instead of 'Ingredients'; Fig. 5 caption spells 'ConisisID' instead of 'ConsisID'.
  3. [Section II-B and Tab. VI] Typographical issues: 'adavancements' should be 'advancements' in Section II-B; 'semantic perciever' appears in Tab. VI and should be 'semantic perceiver'; 'Consis-ID' should be 'ConsisID' for consistency in Section IV-C.
  4. [Table I and Table II formatting] Tab. I row for GroupVideo is missing spaces ('GroupVideoFace&Body1080×1920'); Tab. II has a missing space in the Ingredients row ('0.64430.220'). Please format tables consistently.
  5. [Conclusion] The conclusion describes GroupVideo as a 'zero-shot framework,' but the method requires offline training on a large curated dataset. If the intended meaning is 'no test-time fine-tuning,' please define it explicitly to avoid confusion with zero-shot generation in the strict sense.

Circularity Check

0 steps flagged

No significant circularity: external losses and metrics anchor the claims; self-citations are background-only. Table inconsistencies are correctness risks, not circularity.

full rationale

All training objectives are externally anchored. Stage I uses the standard diffusion MSE (Eq. 2) and a bounding-box-weighted variant (Eq. 6), where boxes are supplied by external YOLO face detection; Stage II uses a mask-regularization loss (Eq. 7) where the ground-truth masks come from external SAM-2 segmentation, described as stored ground truth for the bounding box constraint and the mask regularization loss. No equation defines a predicted quantity in terms of a fitted constant that is later called a prediction. The localization masks (Eqs. 4-5) are supervised by, not defined by, those annotations. The central empirical claim is assessed with external metrics (FaceSim, CLIPScore, FID, VBench, user study), none of which coincide with the ArcFace-based conditioning. Self-citations by the first author appear only in literature enumeration and are not load-bearing. The noted numerical mismatches between Tab. II and Tab. IV (FaceSim 0.636 vs 0.733, FID 147.228 vs 136.94) and the text misreading of ConsisID's FaceSim are evidentiary/reporting concerns, not constructional circularity. The curated test set is the same for all baselines, so the comparison is not statistically forced by construction. The derivation chain is therefore self-contained relative to its external data and metrics.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The system is built on standard pretrained components (CogVideoX-5B, ArcFace, CLIP, Q-Former, YOLO, SAM-2) plus the authors' curated data; two hand-set loss weights (alpha, lambda) are free parameters. No new physical entities are introduced; the semantic perceiver and ID localization module are learned components with in-paper ablation evidence, not postulated entities.

free parameters (2)
  • alpha (bounding-box loss probability) = 0.5
    Controls the probability of applying L_box instead of L_mse in Eq. (6); chosen by hand with no sensitivity analysis (Section IV-B).
  • lambda (mask regularization weight) = 0.1
    Weight on L_mask in Eq. (8); chosen by hand with no sensitivity analysis (Section IV-B).
axioms (4)
  • domain assumption CogVideoX-5B provides a sufficiently strong text-to-video DiT prior that can absorb additional identity conditioning without catastrophic forgetting.
    The entire framework is built on this pretrained backbone, and first-stage training optimizes all backbone parameters jointly (Section III-D).
  • domain assumption ArcFace identity embeddings remain discriminative across pose, expression, lighting, and video appearance.
    Identity embeddings are extracted via a face recognition backbone (Section III-B), and identity fidelity is measured by FaceNet-style similarity; if the embeddings fail to generalize, identity preservation fails.
  • domain assumption YOLO face boxes and SAM-2 face masks are accurate enough to serve as ground truth for the bounding-box and mask losses.
    Dataset annotations are used directly as B in Eq. (6) and M_gt in Eq. (7) (Section IV-A).
  • ad hoc to paper The curated 20k-video dataset plus 400k internal samples covers the target distribution of multi-identity interactions.
    No distributional analysis or external validation is provided; the dataset is collected and filtered by the authors and is not released (Section IV-A).

pith-pipeline@v1.3.0-alltime-deepseek · 17421 in / 13370 out tokens · 147082 ms · 2026-08-01T08:39:35.086534+00:00 · methodology

0 comments
read the original abstract

Current identity customized video generation methodologies are predominantly limited to single-identity scenarios, as the lack of explicit identity separation mechanisms often leads to identity confusion in multi-identity settings. Existing multi-identity approaches, which directly extend single-identity frameworks by concatenating face images as input conditions, frequently result in unnatural facial expressions and motions, manifesting as the "copy-paste" phenomenon. To overcome these limitations, we introduce GroupVideo, a novel framework that leverages multiple individual photographs to generate identitycustomized video. Built upon Video Diffusion Transformers, GroupVideo incorporates multimodal identity alignment: visual alignment jointly encodes multiple face images to provide robust identity references, while semantic alignment introduces a semantic perceiver to enhance the naturalness of motions. An ID localization module with spatial guidance is introduced to address identity blending and enhance identity fidelity, along with bounding box constraints and mask regularization loss, to focus on facial regions and improve training efficiency. In response to the shortage of multi-ID video datasets, we have curated a comprehensive high-quality dataset of 20,000 videos, thereby establishing a crucial resource to advance future research in multi-ID video generation. Extensive experiments demonstrate that GroupVideo outperforms existing methods in generating multi-character videos with consistent identities and natural motions.

Figures

Figures reproduced from arXiv: 2607.21027 by Dandan Zheng, Jianxin Sun, Jingdong Chen, Libin Wang, Qi Li, Xinyang Song, Zhenan Sun.

Figure 2
Figure 2. Figure 2: These limitations suggest that multi-ID customized [PITH_FULL_IMAGE:figures/full_fig_p001_2.png] view at source ↗
Figure 1
Figure 1. Figure 1: Sample results of the proposed GroupVideo. Given facial reference images, GroupVideo generates [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Existing multi-character generation methods, [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of GroupVideo. GroupVideo introduces multimodal identity alignment based on (c) Video Diffusion Transformer. The (a) visual identity alignment and (b) semantic identity alignment fuse multiple identity features with visual latents and semantic embeddings respectively. This is followed by an (d) ID localization module to route different identities, which is illustrated on the lower right. By optimi… view at source ↗
Figure 4
Figure 4. Figure 4: Examples from the multi-character video dataset. The dataset comprises a total of 20,000 videos, encompassing a diverse range of multi￾angle, high-resolution portraits. feature of the DiT block) and multiple identity embeddings F n id ∈ R 1×lid×cid , n = 1, ..., N: Mn = Softmax(ϕ(H) ∗ θ(F n id)), (4) where ϕ and θ are two linear projection layers, used for feature-dimension alignment and sequence-length co… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison with state-of-the-art ID-preserved video generation methods. The identity references are displayed on the top-left, the attributes in the textual instructions are highlighted in red [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Single-ID customization comparison with the state-of-the-art methods ConsisID [3], HunyuanVideo [28] and Wan2.1 [81]. (a) A man and a woman engage in (b) a playful snowball fight in a snowy outdoor setting, their laughter echoing in the crisp winter air. They are dressed in warm winter clothing, with scarves, gloves, and jackets adding to the cozy atmosphere. The man, wearing a dark beanie and a wide grin,… view at source ↗
Figure 7
Figure 7. Figure 7: More visual results with (a) dynamic motion, (b) atypical backgrounds, (c) more individuals, (d) stylization, (e) multi-shots. Degree improved by 12.0% and FID improved by 8.25%, indi￾cating our enhanced video fidelity and naturalness. Even com￾pared with MAGREF built on Wan-2.1, GroupVideo achieves better performance on all metrics except dynamic degree. Al￾though Ingredients and ConsisID achieve higher s… view at source ↗
Figure 8
Figure 8. Figure 8: Examples for ablation studies. Left: Ablation on structural modules including visual alignment, semantic perceiver and ID localization module. Right: Ablation on loss functions and training strategies. Zoom in for more details. TABLE III USER STUDY RESULTS. THE TABLE PRESENTS THE AVERAGE USER RATINGS ACROSS FOUR DIMENSIONS. Method Identity Similarity Text Alignment Visual Quality Motion Degree ID-Animator … view at source ↗
Figure 9
Figure 9. Figure 9: Ablation study of the ID localization module with three individuals. TABLE V COMPARISON OF COMPUTATIONAL RESOURCES FOR ID CUSTOMIZED VIDEO GENERATION. Method Base Model Training Speed(s/step) Inference Time(s) Inference Memory(GB) ConsisID CogVideoX-5B 24.6 531 9.7 Ingredients CogVideoX-5B - 552 31.0 Concat-ID CogVideoX-5B 27.7 535 17.8 GroupVideo CogVideoX-5B 26.2 550 12.5 MAGREF Wan-2.1-14B - 1400 72.0 d… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

83 extracted references · 30 linked inside Pith

  1. [1]

    N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein and K. Aberman, ”DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject- Driven Generation,” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 2023, pp. 22500- 22510

    2023

  2. [2]

    An image is worth one word: Per- sonalizing text-to-image generation using textual inversion, 2022, arXiv preprint arXiv:2208.01618

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel CohenOr. An image is worth one word: Per- sonalizing text-to-image generation using textual inversion, 2022, arXiv preprint arXiv:2208.01618

    Pith/arXiv arXiv 2022

  3. [3]

    S. Yuan et al., ”Identity-Preserving Text-To-Video Generation by Fre- quency Decomposition,” 2025 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), Nashville, TN, USA, 2025, pp. 12978-12988

    2025

  4. [4]

    He, X.; Liu, Q.; Qian, S.; Wang, X.; Hu, T.; Cao, K.; Yan, K.; and Zhang, J., Id-animator: Zero-shot identity-preserving human video generation, 2024, arXiv preprint arXiv:2404.15275

    Pith/arXiv arXiv 2024

  5. [5]

    Wei, J.; Yan, S.; Lin, W.; Liu, B.; Chen, R.; and Guo, M, EchoVideo: Identity-Preserving Human Video Generation by Multimodal Feature Fusion, 2025, arXiv preprint arXiv:2501.13452

    Pith/arXiv arXiv 2025

  6. [6]

    Zhang, Y .; Liu, Y .; Xia, B.; Peng, B.; Yan, Z.; Lo, E.; and Jia, J., Magic Mirror: ID-Preserved Video Generation in Video Diffusion Transformers, 2025, arXiv preprint arXiv:2501.03931

    arXiv 2025

  7. [7]

    Ingredients: Blending Custom Photos with Video Diffusion Transformers, 2025, arXiv preprint arXiv:2501.01790

    Fei, Z.; Li, D.; Qiu, D.; Yu, C.; and Fan, M. Ingredients: Blending Custom Photos with Video Diffusion Transformers, 2025, arXiv preprint arXiv:2501.01790

    Pith/arXiv arXiv 2025

  8. [8]

    Peebles and S

    W. Peebles and S. Xie, ”Scalable Diffusion Models with Transformers,” 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 2023, pp. 4172-4182

    2023

  9. [9]

    Generative adversarial networks

    Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y . Generative adversarial networks. Communications of the ACM, 63(11), 2020, pp. 139–144

    2020

  10. [10]

    Karras, S

    T. Karras, S. Laine and T. Aila, ”A Style-Based Generator Architecture for Generative Adversarial Networks,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 12, 2021, pp. 4217-4228

    2021

  11. [11]

    J. Liu, W. Wang, S. Chen, X. Zhu and J. Liu, ”Sounding Video Genera- tor: A Unified Framework for Text-Guided Sounding Video Generation,” in IEEE Transactions on Multimedia, vol. 26, pp. 141-153, 2024

    2024

  12. [12]

    K ¨oksal, K

    A. K ¨oksal, K. E. Ak, Y . Sun, D. Rajan and J. H. Lim, ”Controllable Video Generation With Text-Based Instructions,” in IEEE Transactions on Multimedia, vol. 26, pp. 190-201, 2024

    2024

  13. [13]

    Taming transformers for high- resolution image synthesis

    Esser, P.; Rombach, R.; and Ommer, B. Taming transformers for high- resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 12873–12883

    2021

  14. [14]

    Y .; Luong, T.; Baid, G.; Wang, Z.; Vasude- van, V .; Ku, A.; Yang, Y .; Ayan, B

    Yu, J.; Xu, Y .; Koh, J. Y .; Luong, T.; Baid, G.; Wang, Z.; Vasude- van, V .; Ku, A.; Yang, Y .; Ayan, B. K.; et al. Scaling autoregressive models for content-rich text-to-image generation, 2022, arXiv preprint arXiv:2206.10789, 2(3): 5

    Pith/arXiv arXiv 2022

  15. [15]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser and B. Ommer, ”High-Resolution Image Synthesis with Latent Diffusion Models,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 10674-10685

    2022

  16. [16]

    L.; Ghasemipour, K.; Gontijo Lopes, R.; Karagol Ayan, B.; Salimans, T.; et al

    Saharia, C.; Chan, W.; Saxena, S.; Li, L.; Whang, J.; Denton, E. L.; Ghasemipour, K.; Gontijo Lopes, R.; Karagol Ayan, B.; Salimans, T.; et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 2022, pp. 36479–36494

    2022

  17. [17]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023, arXiv preprint arXiv:2307.01952

    Podell, D.; English, Z.; Lacey, K.; Blattmann, A.; Dockhorn, T.; M ¨uller, J.; Penna, J.; and Rombach, R. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023, arXiv preprint arXiv:2307.01952

    Pith/arXiv arXiv 2023

  18. [18]

    Unialignment: Semantic alignment for unified image generation, understanding, manipulation and perception

    Song X, Wang L, Wang W, et al. Unialignment: Semantic alignment for unified image generation, understanding, manipulation and perception. Proceedings of the AAAI Conference on Artificial Intelligence, 2026, pp. 9116-9126

    2026

  19. [19]

    3SGen: Unified Subject, Style, and Structure-Driven Image Generation with Adaptive Task-specific Memory, 2025, arXiv preprint arXiv:2512.19271

    Song X, Wang L, Wang W, et al. 3SGen: Unified Subject, Style, and Structure-Driven Image Generation with Adaptive Task-specific Memory, 2025, arXiv preprint arXiv:2512.19271

    arXiv 2025

  20. [20]

    Fine-Grained Text-to-Image Synthesis with Semantic Refinement

    Song X, Sun J, Zhang Y , et al. Fine-Grained Text-to-Image Synthesis with Semantic Refinement. ICASSP 2026-2026 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), 2026, pp. 8637-8641

    2026

  21. [21]

    Y . Hu, C. Luo and Z. Chen, ”A Benchmark for Controllable Text-Image- to-Video Generation,” in IEEE Transactions on Multimedia, vol. 26, pp. 1706-1719, 2024. 12

    2024

  22. [22]

    M. Zhao, W. Wang, T. Chen, R. Zhang and R. Li, ”TA2V: Text-Audio Guided Video Generation,” in IEEE Transactions on Multimedia, vol. 26, pp. 7250-7264, 2024

    2024

  23. [23]

    Video generation models as world simulators

    Brooks, T.; Peebles, B.; Holmes, C.; DePue, W.; Guo, Y .; Jing, L.; Schnurr, D.; Taylor, J.; Luhman, T.; Luhman, E.; Ng, C.; Wang, R.; and Ramesh, A. Video generation models as world simulators. 2024

    2024

  24. [24]

    Latte: Latent diffusion transformer for video generation, 2024, arXiv preprint arXiv:2401.03048

    Ma, X.; Wang, Y .; Jia, G.; Chen, X.; Liu, Z.; Li, Y .-F.; Chen, C.; and Qiao, Y . Latte: Latent diffusion transformer for video generation, 2024, arXiv preprint arXiv:2401.03048

    Pith/arXiv arXiv 2024

  25. [25]

    Open-sora plan: Open-source large video generation model

    Lin, B.; Ge, Y .; Cheng, X.; Li, Z.; Zhu, B.; Wang, S.; He, X.; Ye, Y .; Yuan, S.; Chen, L.; et al. Open-sora plan: Open-source large video generation model. 2024, arXiv preprint arXiv:2412.00131

    Pith/arXiv arXiv 2024

  26. [26]

    Open-sora: Democratizing efficient video production for all

    Zheng, Z.; Peng, X.; Yang, T.; Shen, C.; Li, S.; Liu, H.; Zhou, Y .; Li, T.; and You, Y . Open-sora: Democratizing efficient video production for all. 2024, arXiv preprint arXiv:2412.20404

    Pith/arXiv arXiv 2024

  27. [27]

    Cogvideox: Text-to-video diffusion models with an expert transformer, 2024, arXiv preprint arXiv:2408.06072

    Yang, Z.; Teng, J.; Zheng, W.; Ding, M.; Huang, S.; Xu, J.; Yang, Y .; Hong, W.; Zhang, X.; Feng, G.; et al. Cogvideox: Text-to-video diffusion models with an expert transformer, 2024, arXiv preprint arXiv:2408.06072

    Pith/arXiv arXiv 2024

  28. [28]

    Hunyuanvideo: A systematic framework for large video generative models

    Kong, W.; Tian, Q.; Zhang, Z.; Min, R.; Dai, Z.; Zhou, J.; Xiong, J.; Li, X.; Wu, B.; Zhang, J.; et al. Hunyuanvideo: A systematic framework for large video generative models. 2024, arXiv preprint arXiv:2412.03603

    Pith/arXiv arXiv 2024

  29. [29]

    Toklip: Marry visual tokens to clip for multimodal comprehension and generation, 2026, arXiv preprint arXiv:2505.05422

    Lin H, Wang T, Ge Y , et al. Toklip: Marry visual tokens to clip for multimodal comprehension and generation, 2026, arXiv preprint arXiv:2505.05422

    Pith/arXiv arXiv 2026

  30. [30]

    IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation, 2026, arXiv preprint arXiv:2606.24849

    Li, Z., Lin, H., Xiao, Y ., Li, Z., Song, X, et al. IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation, 2026, arXiv preprint arXiv:2606.24849

    Pith/arXiv arXiv 2026

  31. [31]

    Concept-Guided Tokenization: Closing the Gap Between Reconstruction and Generation, 2026

    Yunqiao Yang and Haokun Lin and Guanzhong Wu and Ying Wei. Concept-Guided Tokenization: Closing the Gap Between Reconstruction and Generation, 2026

    2026

  32. [32]

    Shaping a Stabilized Video by Mitigating Unintended Changes for Concept-Augmented Video Editing

    Guo, Mingce and He, Jingxuan and Tang, Shengeng and Wang, Zhangye and Cheng, Lechao. Shaping a Stabilized Video by Mitigating Unintended Changes for Concept-Augmented Video Editing. 2024. arXiv preprint arXiv:2410.12526

    Pith/arXiv arXiv 2024

  33. [33]

    N. Ruiz et al., ”HyperDreamBooth: HyperNetworks for Fast Personaliza- tion of Text-to-Image Models,” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, W A, USA, 2024, pp. 6527-6536

    2024

  34. [34]

    J.; Shen, Y .; Wallis, P.; Allen-Zhu, Z.; Li, Y .; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; Shen, Y .; Wallis, P.; Allen-Zhu, Z.; Li, Y .; Wang, S.; Wang, L.; and Chen, W. Lora: Low-rank adaptation of large language models. 2021, arXiv preprint arXiv:2106.09685

    Pith/arXiv arXiv 2021

  35. [35]

    Zhang, T

    H. Zhang, T. Wu and Y . Wei, ”Multi-View User Preference Modeling for Personalized Text-to-Image Generation,” in IEEE Transactions on Multimedia, vol. 27, pp. 3082-3091, 2025

    2025

  36. [36]

    Y . Xu, X. Xu, H. Gao and F. Xiao, ”SGDM: An Adaptive Style-Guided Diffusion Model for Personalized Text to Image Generation,” in IEEE Transactions on Multimedia, vol. 26, pp. 9804-9813, 2024

    2024

  37. [37]

    Instantid: Zero-shot identity-preserving generation in seconds

    Wang, Q.; Bai, X.; Wang, H.; Qin, Z.; Chen, A.; Li, H.; Tang, X.; and Hu, Y . Instantid: Zero-shot identity-preserving generation in seconds. 2024, arXiv preprint arXiv:2401.07519

    Pith/arXiv arXiv 2024

  38. [38]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models

    Ye, H.; Zhang, J.; Liu, S.; Han, X.; and Yang, W. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. 2023, arXiv preprint arXiv:2308.06721

    Pith/arXiv arXiv 2023

  39. [39]

    PuLID: Pure and Lightning ID Customization via Contrastive Alignment

    Guo, Z.; Wu, Y .; Chen, Z.; Chen, L.; and He, Q. PuLID: Pure and Lightning ID Customization via Contrastive Alignment. 2024, arXiv preprint arXiv:2404.16022

    Pith/arXiv arXiv 2024

  40. [40]

    Blip-diffusion: Pre-trained subject represen- tation for controllable text-to-image generation and editing

    Li, D.; Li, J.; and Hoi, S. Blip-diffusion: Pre-trained subject represen- tation for controllable text-to-image generation and editing. Advances in Neural Information Processing Systems, 2024, pp. 36

    2024

  41. [41]

    Z. Li, M. Cao, X. Wang, Z. Qi, M. -M. Cheng and Y . Shan, ”Pho- toMaker: Customizing Realistic Human Photos via Stacked ID Embed- ding,” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, W A, USA, 2024, pp. 8640-8650

    2024

  42. [42]

    Dreamidentity: Enhanced editability for efficient face-identity preserved image generation

    Chen, Z.; Fang, S.; Liu, W.; He, Q.; Huang, M.; and Mao, Z. Dreamidentity: Enhanced editability for efficient face-identity preserved image generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 2024, pp. 1281–1289

    2024

  43. [43]

    H.; Chechik, G.; and Cohen- Or, D

    Gal, R.; Arar, M.; Atzmon, Y .; Bermano, A. H.; Chechik, G.; and Cohen- Or, D. 2023. Encoder-based domain tuning for fast personalization of text- to-image models. ACM Transactions on Graphics (TOG), 42(4): 1–13

    2023

  44. [44]

    X-portrait: Expressive portrait animation with hierarchical motion attention

    Xie, Y .; Xu, H.; Song, G.; Wang, C.; Shi, Y .; and Luo, L. X-portrait: Expressive portrait animation with hierarchical motion attention. In ACM SIGGRAPH 2024 Conference Papers, 2024, pp. 1–11

    2024

  45. [45]

    H. Li, ”Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation,” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, W A, USA, 2024, pp. 8153-8163

    2024

  46. [46]

    PoseGen: In-Context LoRA Finetuning for Pose-Controllable Long Human Video Generation

    He, Jingxuan and Su, Busheng and Wong, Finn. PoseGen: In-Context LoRA Finetuning for Pose-Controllable Long Human Video Generation

  47. [47]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Guo, Y .; Yang, C.; Rao, A.; Liang, Z.; Wang, Y .; Qiao, Y .; Agrawala, M.; Lin, D.; and Dai, B. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. 2023, arXiv preprint arXiv:2307.04725

    Pith/arXiv arXiv 2023

  48. [48]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Zhang, S.; Wang, J.; Zhang, Y .; Zhao, K.; Yuan, H.; Qin, Z.; Wang, X.; Zhao, D.; and Zhou, J. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. 2023, arXiv preprint arXiv:2311.04145

    Pith/arXiv arXiv 2023

  49. [49]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Xing, J.; Xia, M.; Zhang, Y .; Chen, H.; Yu, W.; Liu, H.; Liu, G.; Wang, X.; Shan, Y .; and Wong, T.-T. Dynamicrafter: Animating open-domain images with video diffusion priors. In European Conference on Computer Vision, 2024, pp. 399–417

    2024

  50. [50]

    Ma, Z.; Zhou, D.; Yeh, C.-H.; Wang, X.-S.; Li, X.; Yang, H.; Dong, Z.; Keutzer, K.; and Feng, J. 2024b. Magic-me: Identity-specific video customized diffusion. arXiv preprint arXiv:2402.09368

    Pith/arXiv arXiv

  51. [51]

    Chefer, H.; Zada, S.; Paiss, R.; Ephrat, A.; Tov, O.; Rubinstein, M.; Wolf, L.; Dekel, T.; Michaeli, T.; and Mosseri, I. 2024. Still-moving: Customized video generation without customized video data. ACM Transactions on Graphics (TOG), 43(6): 1–11

    2024

  52. [52]

    Y . Wei et al., ”Dream Video: Composing Your Dream Videos with Cus- tomized Subject and Motion,” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, W A, USA, 2024, pp. 6537-6549

    2024

  53. [53]

    Jiang et al., ”VideoBooth: Diffusion-based Video Generation with Image Prompts,” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, W A, USA, pp

    Y . Jiang et al., ”VideoBooth: Diffusion-based Video Generation with Image Prompts,” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, W A, USA, pp. 6689-6700, 2024

    2024

  54. [54]

    Multi- concept customization of text-to-image diffusion

    Kumari, N.; Zhang, B.; Zhang, R.; Shechtman, E.; and Zhu, J.-Y . Multi- concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1931–1941, 2023

    1931

  55. [55]

    Cones 2: Customizable image synthesis with multiple subjects

    Liu, Z.; Zhang, Y .; Shen, Y .; Zheng, K.; Zhu, K.; Feng, R.; Liu, Y .; Zhao, D.; Zhou, J.; and Cao, Y . Cones 2: Customizable image synthesis with multiple subjects. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pp. 57500–57519, 2023

    2023

  56. [56]

    H. Chen et al., ”VideoDreamer: Customized Multi-Subject Text-to-Video Generation With Disen-Mix Finetuning on Language-Video Foundation Models,” in IEEE Transactions on Multimedia, vol. 27, pp. 2875-2885, 2025

    2025

  57. [57]

    Jiang, Q

    Y . Jiang, Q. Liu, D. Chen, L. Yuan and Y . Fu, ”AnimeDiff: Customized Image Generation of Anime Characters Using Diffusion Model,” in IEEE Transactions on Multimedia, vol. 26, pp. 10559-10572, 2024

    2024

  58. [58]

    InstantFam- ily: Masked Attention for Zero-shot Multi-ID Image Generation

    Kim, C.; Lee, J.; Joung, S.; Kim, B.; and Baek, Y .-M. InstantFam- ily: Masked Attention for Zero-shot Multi-ID Image Generation. arXiv preprint arXiv:2404.19427, 2024

    Pith/arXiv arXiv 2024

  59. [59]

    Z.; Shi, Y .; Chen, Y .; Fan, Z.; Xiao, W.; Zhao, R.; Chang, S.; Wu, W.; et al

    Gu, Y .; Wang, X.; Wu, J. Z.; Shi, Y .; Chen, Y .; Fan, Z.; Xiao, W.; Zhao, R.; Chang, S.; Wu, W.; et al. Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models. Advances in Neural Information Processing Systems, pp. 36, 2024

    2024

  60. [60]

    T.; Durand, F.; and Han, S

    Xiao, G.; Yin, T.; Freeman, W. T.; Durand, F.; and Han, S. Fastcomposer: Tuning-free multi-subject image generation with localized attention. In- ternational Journal of Computer Vision, pp. 1–20, 2024

    2024

  61. [61]

    Wang et al., ”StableIdentity: Inserting Anybody into Anywhere at First Sight,” in IEEE Transactions on Multimedia, 2025

    Q. Wang et al., ”StableIdentity: Inserting Anybody into Anywhere at First Sight,” in IEEE Transactions on Multimedia, 2025

    2025

  62. [62]

    Moa: Mixture-of-attention for subject-context disentanglement in person- alized image generation

    Wang, K.-C.; Ostashev, D.; Fang, Y .; Tulyakov, S.; and Aberman, K. Moa: Mixture-of-attention for subject-context disentanglement in person- alized image generation. In SIGGRAPH Asia 2024 Conference Papers, 2024, pp. 1–12

    2024

  63. [63]

    Uniportrait: A unified framework for identity- preserving single-and multi-human image personalization[J]

    He J, Geng Y , Bo L. Uniportrait: A unified framework for identity- preserving single-and multi-human image personalization[J]. 2024, arXiv preprint arXiv:2408.05939

    Pith/arXiv arXiv 2024

  64. [64]

    Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y .; Li, W.; and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 2019, 21(140): 1–67

    2019

  65. [65]

    Prompt disentanglement via language guidance and representation alignment for domain generalization

    Cheng, De; Xu, Zhipeng; Jiang, Xinyang; Li, Dongsheng; Wang, Nannan and Gao, Xinbo. Prompt disentanglement via language guidance and representation alignment for domain generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026, pp. 1-18

    2026

  66. [66]

    Isolating Interference Factors for Robust Cloth-Changing Person Re-Identification

    Cheng, De; Li, Yubo; Fang, Chaowei; Zhang, Shizhou; Wang, Nannan and Gao, Xinbo. Isolating Interference Factors for Robust Cloth-Changing Person Re-Identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026, pp. 1-18. 13

    2026

  67. [67]

    Semantic-Aligned Learning with Collaborative Refinement for Unsupervised VI-ReID

    Cheng, De; He, Lingfeng; Wang, Nannan; Zhang, Dingwen and Gao, Xinbo. Semantic-Aligned Learning with Collaborative Refinement for Unsupervised VI-ReID. International Journal of Computer Vision, 2025, pp.5992-6014

    2025

  68. [68]

    Arcface: Additive an- gular margin loss for deep face recognition

    Deng, J.; Guo, J.; Xue, N.; and Zafeiriou, S. Arcface: Additive an- gular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4690–4699

    2019

  69. [69]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. In International conference on machine learning, 2023, pp. 19730–19742

    2023

  70. [70]

    C.; Cai, W.; and Wu, W

    Yu, J.; Zhu, H.; Jiang, L.; Loy, C. C.; Cai, W.; and Wu, W. CelebV- Text: A Large-Scale Facial Text-Video Dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023

    2023

  71. [71]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv:2308.12966

    Pith/arXiv arXiv 2023

  72. [72]

    You only look once: Unified, real-time object detection

    Redmon, Joseph and Divvala, Santosh and Girshick, Ross and Farhadi, Ali. You only look once: Unified, real-time object detection. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, 2016, pp: 779-788

    2016

  73. [73]

    Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Rong- hang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R¨adle, Roman and Rolland, Chloe and Gustafson, Laura and others

  74. [74]

    Wang, W.; Lv, Q.; Yu, W.; Hong, W.; Qi, J.; Wang, Y .; Ji, J.; Yang, Z.; Zhao, L.; Song, X.; et al. 2023. Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079

    Pith/arXiv arXiv 2023

  75. [75]

    Zhong, Yong and Yang, Zhuoyi and Teng, Jiayan and Gu, Xiaotao and Li, Chongxuan. 2025. Concat-ID: Towards Universal Identity-Preserving Video Synthesis. arXiv preprint arXiv:2503.14151

    Pith/arXiv arXiv 2025

  76. [76]

    Deng, Yufan and Guo, Xun and Yin, Yuanyang and Fang, Jacob Zhiyuan and Yang, Yiding and Wang, Yizhi and Yuan, Shenghai and Wang, Angtian and Liu, Bo and Huang, Haibin and others. 2025. MAGREF: Masked Guidance for Any-Reference Video Generation. arXiv preprint arXiv:2505.23742

    arXiv 2025

  77. [77]

    Facenet: A unified embedding for face recognition and clustering

    Schroff, F.; Kalenichenko, D.; and Philbin, J. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 815–823

    2015

  78. [78]

    L.; and Choi, Y

    Hessel, J.; Holtzman, A.; Forbes, M.; Bras, R. L.; and Choi, Y . 2021. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718

    Pith/arXiv arXiv 2021

  79. [79]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 2017, pp. 30

    2017

  80. [80]

    Vbench: Comprehensive benchmark suite for video generative models

    Huang, Z.; He, Y .; Yu, J.; Zhang, F.; Si, C.; Jiang, Y .; Zhang, Y .; Wu, T.; Jin, Q.; Chanpaisit, N.; et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 21807–21818

    2024

Showing first 80 references.