Pith. sign in

REVIEW 3 major objections 3 minor 66 references

FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that audio-driven portrait animation can be aligned simultaneously with human preferences for motion naturalness, lip-sync accuracy, and visual quality by decomposing the preference objective into per-dimension experts and

desk verdict Plausible three-part contribution that deserves a referee to check whether the reward model is curating its own preferred outputs. read the letter →

arxiv 2508.11255 v1 pith:U7AAAVJE submitted 2025-08-15 cs.CV

MengChao Wang , Qiang Wang , Fan Jiang , Mu Xu This is my paper
classification cs.CV
keywords audio-drivenportraitanimationpreferenceoptimizationrewardmodeldiffusionlip-syncaccuracymotionnaturalnessvisualqualitymulti-expertLoRA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that audio-driven portrait animation can be aligned with three human preferences at the same time—lip-sync accuracy, motion naturalness, and visual quality—even though those preferences often fight each other. It builds this on a learned reward model, Talking-Critic, which scores generated videos along the three dimensions, and on Talking-NSQ, a 410K-pair preference dataset produced with that model. The proposed training scheme, TLPO, splits the reward into per-dimension expert modules and fuses them adaptively at each diffusion timestep and network layer. The authors report that the resulting model beats baselines on all three axes in both qualitative and quantitative evaluations. A sympathetic reader would take the paper's core bet to be that decomposing preferences and recombining them at many small decision points is a better route to human-aligned generation than a single aggregated objective.

What carries the argument

TLPO's timestep-layer adaptive collaborative fusion. The method maintains separate LoRA expert modules for motion naturalness, lip-sync, and visual quality. At each diffusion timestep and network layer, a small gating network computes a weighted combination of the experts' output adjustments, with weights produced by a low-rank projection of the current timestep embedding and layer index, followed by softmax. A preference-optimization loss with a winner-loser contrast is then applied to the fused output, so the gate learns both which preference matters and where in the network it should act. Talking-Critic provides the reward signal that labels winners and losers in the data.

What would settle it

A head-to-head human study on held-out videos: if human raters prefer the baseline over TLPO at chance frequency, or if Talking-Critic's win predictions disagree with majority human judgment on a fresh test set, the claimed preference gains fail. A simpler check: remove the timestep-layer gate and train with a plain sum of per-dimension losses; if performance does not drop, the adaptive fusion is not the active ingredient.

Watch

Extended reading notes

Core claim

The central claim is that TLPO simultaneously improves lip-sync accuracy, motion naturalness, and visual quality in diffusion-based portrait animation by decoupling the preference signal into three specialized LoRA experts and fusing them with a timestep- and layer-adaptive gate. The gate learns, for each denoising step and each network layer, how much weight to give each dimension's expert, so that the optimization pushes the generated frame to satisfy the right preference at the right moment. This is enabled by Talking-Critic, a multimodal reward model trained on binary question-answer judgments about whether a video's motion is natural, its lips are synchronized, and its visual quality is

Load-bearing premise

The load-bearing premise is that Talking-Critic's learned scores are faithful proxies for human preferences on unseen videos and that the 410K preference pairs in Talking-NSQ are correctly labeled and representative enough to train that critic.

Editorial extensions

If this is right

  • If TLPO works as reported, portrait-animation systems can be aligned on multiple preference dimensions without the usual trade-off, because each dimension gets its own expert and a learned gate decides where to apply it.
  • Learned reward models like Talking-Critic can replace expensive human annotators at scale, curating large preference datasets and enabling iterative self-improvement loops.
  • Timestep- and layer-adaptive fusion suggests that quality dimensions are best corrected at different phases of the diffusion process, which could inform future architecture design.
  • The same decouple-and-fuse pattern could be applied to any diffusion model facing conflicting objectives, not only talking-head generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The gating weights learned by TLPO could be inspected to identify which network layers and denoising steps dominate each preference dimension; the paper does not analyze this, but the mechanism makes it directly measurable.
  • Editorial inference: The Talking-NSQ dataset, if released, could become a shared benchmark for reward-model evaluation in portrait animation, extending beyond the paper's own training use.
  • Editorial inference: A stress test on out-of-distribution content (different languages, speaking styles, or non-human faces) would reveal whether the preference alignment generalizes or overfits to the data distribution used to train the critic.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The manuscript proposes a three-component system for audio-driven portrait animation: (i) Talking-Critic, a multimodal reward model trained to score generated videos along motion naturalness, lip-sync, and visual quality; (ii) Talking-NSQ, a claimed 410K-pair multidimensional preference dataset curated using Talking-Critic; and (iii) TLPO, a timestep-layer adaptive multi-expert preference optimization method that uses DPO-style losses to align a diffusion-based talking-head model with these preferences. The abstract claims substantial improvements over baselines in all three dimensions, with both qualitative and quantitative support. The supplied full text is largely corrupted and unreadable, so only the abstract, a few equations, figure fragments, and table shells could be inspected. The central claim of human-preference-aligned gains is therefore not currently verifiable from the manuscript as submitted.

Significance. If the results hold, the paper would make a useful contribution: a 410K-pair multidimensional preference dataset for talking-head generation is a potentially valuable community asset, and the idea of decoupling preference dimensions into expert LoRAs with timestep/layer-dependent fusion is technically interesting and goes beyond simple scalar-reward fine-tuning. The paper also makes a plausible case that multi-dimensional preference conflicts are a real bottleneck in this application. However, the significance is conditional on evidence that the reward model and the optimization actually track human preferences rather than artifacts of the reward model itself. The current manuscript does not supply that evidence in a readable or verifiable form.

major comments (3)
  1. [Abstract; Dataset curation and reward model sections] There is a load-bearing circularity risk. The abstract states 'leveraging this model, we curate Talking-NSQ' and describes TLPO as driven by reward signals from Talking-Critic. Thus Talking-Critic supplies both the preference labels for the 410K pairs and the optimization objective. If the final evaluation also uses Talking-Critic (or if no independent human evaluation is reported), the claimed improvements in 'lip-sync accuracy, motion naturalness, and visual quality' may reflect reward over-optimization rather than genuine human preference gains. The manuscript must clarify (i) whether the preference pairs are human-annotated or critic-generated, (ii) whether any held-out human evaluation was performed, and (iii) whether the final reported numbers come from humans or from Talking-Critic. This is not a minor omission; it is central to the paper's claim of human-preference alignment.
  2. [Experiments; Tables in the experimental section] The quantitative results are not legible in the supplied manuscript. The table shells contain only arrows and unreadable entries; no metric values, baseline names, or error bars can be seen. The abstract's assertion of 'substantial improvements' and 'superior performance' is therefore unsupported by any extractable numerical evidence. The authors need to provide a fully readable experimental section with concrete numbers, standard deviations or confidence intervals, baseline descriptions, and statistical significance tests if applicable. As submitted, the central empirical claim cannot be checked.
  3. [Eq. (1) in the TLPO method] The displayed loss L = -E[log σ(-β/2(L(x_w_t,t)-L(x_l_t,t)))] is a DPO-style reward-maximizing objective. Such objectives are known to over-optimize the reward proxy when the reward model is also used to curate the preference data. The manuscript gives no safeguard (e.g., KL regularization, reward-model validation against held-out human judgments, or early-stopping based on an independent metric). The paper should either provide a theoretical or empirical justification that the multi-expert decomposition prevents proxy over-optimization, or add an independent human evaluation that confirms the optimized videos are preferred on all three dimensions. Without that, the claim that TLPO 'does not interfere' across dimensions is not established.
minor comments (3)
  1. [Table headers and notation in experiments] The table column headers use arrows (↑/↓) without defining which direction is better or what each acronym (e.g., 'LP', 'SS', 'FID') stands for. Please add a caption that explains all metrics and arrows.
  2. [Figures and captions] Several figure captions and text blocks are corrupted or contain placeholders. For example, the qualitative comparison figure labels are unreadable. The authors should ensure the final PDF renders all text correctly, since this is necessary for review and for reproducibility.
  3. [Free parameters and hyperparameters] The preference-loss temperature β, the fusion gate weights W_fuse and bias b_fuse, and the low-rank dimension k are introduced without a sensitivity analysis. At minimum, report the chosen values and provide an ablation or reference to justify them.

Circularity Check

1 steps flagged · score 6.0 of 10

Talking-Critic both curates the 410K preference pairs and supplies the TLPO reward signal, so the claimed human-preference gains may reduce to optimization against the critic's own scores.

  1. fitted input called prediction [Abstract and TLPO loss (unnumbered equation in Sec. 3.3)]
    "we first introduce Talking-Critic, a multimodal reward model that learns human-aligned reward functions... Leveraging this model, we curate Talking-NSQ, a large-scale multidimensional human preference dataset containing 410K preference pairs... L = −E[log σ(−β/2(L(xw_t,t) − L(xl_t,t)))]"

    The preference pairs in Talking-NSQ are curated by Talking-Critic, the same model that provides the reward signal in the TLPO objective. The loss directly maximizes Talking-Critic's reward difference between winning and losing samples, so the 'human preference' labels are not independent of the optimized objective. Improving TLPO against this reward is therefore close to a construction of the training objective rather than a prediction about human preference, unless the final evaluation is separately based on independent human raters, which the abstract does not establish.

full rationale

The architectural components (multi-expert LoRA, timestep-layer gating, and the fusion mechanism) have independent content and are not inherently circular. However, the central alignment claim reduces the preference pipeline to a single model: Talking-Critic is used to curate Talking-NSQ, and the same Talking-Critic reward scores drive the DPO-style TLPO loss. The abstract states that Talking-Critic 'learns human-aligned reward functions' and then 'leveraging this model, we curate Talking-NSQ', so the dataset's preference labels appear to be generated by the model rather than by independent human raters. Consequently, the reported improvements in lip-sync, motion naturalness, and visual quality are at least partially improvements on the critic's own scoring function, and cannot be treated as independent evidence of human-preference gains unless human evaluations are clearly separated. The claim that Talking-Critic outperforms existing methods in aligning with human preference ratings could provide external support if those ratings are real, but it does not by itself validate the circular data-curation/optimization loop. Score 6 reflects partial circularity: one key prediction is forced by construction, while the architectural contributions are not.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three domain assumptions about decomposability, reward-model reliability, and dataset representativeness, all asserted in the abstract rather than demonstrated. No standard-math axioms beyond diffusion-model background could be checked because the full text is unreadable.

free parameters (2)
  • Preference loss temperature beta = 1000 (reported in partially readable implementation section)
    Hand-chosen scalar controlling the strength of the preference loss; not derived from first principles.
  • Expert fusion gate weights W_fuse, b_fuse = not reported in abstract
    Learned per-layer, per-timestep fusion weights that combine expert LoRA outputs; values are trained, not derived.
assumptions (3)
  • domain assumption Human preference decomposes into independent dimensions (motion naturalness, lip-sync accuracy, visual quality).
    The abstract treats these as separable and assumes expert modules can be optimized separately and then fused without mutual interference; this is a modeling premise, not derived.
  • domain assumption Talking-Critic reward scores faithfully approximate human preferences and generalize beyond training data.
    TLPO's reward signal and the claimed preference alignment depend on this; the abstract offers no agreement statistics or out-of-distribution evidence.
  • domain assumption Talking-NSQ's 410K preference pairs are correctly labeled and representative.
    The dataset is the training substrate for the reward model; the abstract does not state annotation protocol, inter-annotator agreement, or identity and audio coverage.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation." pith.science (2026). https://pith.science/paper/U7AAAVJE

@misc{pith2026250811255,
  author       = {Pith},
  title        = {Pith review of: FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U7AAAVJE}},
  note         = {Machine review of arXiv:2508.11255}
}
read the original abstract

Recent advances in audio-driven portrait animation have demonstrated impressive capabilities. However, existing methods struggle to align with fine-grained human preferences across multiple dimensions, such as motion naturalness, lip-sync accuracy, and visual quality. This is due to the difficulty of optimizing among competing preference objectives, which often conflict with one another, and the scarcity of large-scale, high-quality datasets with multidimensional preference annotations. To address these, we first introduce Talking-Critic, a multimodal reward model that learns human-aligned reward functions to quantify how well generated videos satisfy multidimensional expectations. Leveraging this model, we curate Talking-NSQ, a large-scale multidimensional human preference dataset containing 410K preference pairs. Finally, we propose Timestep-Layer adaptive multi-expert Preference Optimization (TLPO), a novel framework for aligning diffusion-based portrait animation models with fine-grained, multidimensional preferences. TLPO decouples preferences into specialized expert modules, which are then fused across timesteps and network layers, enabling comprehensive, fine-grained enhancement across all dimensions without mutual interference. Experiments demonstrate that Talking-Critic significantly outperforms existing methods in aligning with human preference ratings. Meanwhile, TLPO achieves substantial improvements over baseline models in lip-sync accuracy, motion naturalness, and visual quality, exhibiting superior performance in both qualitative and quantitative evaluations. Ours project page: https://fantasy-amap.github.io/fantasy-talking2/

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 18 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Avrahami, O.; Patashnik, O.; Fried, O.; Nemchinov, E.; Aberman, K.; Lischinski, D.; and Cohen-Or, D. 2025. Stable flow: Vital layers for training-free image editing. In Proceedings of the Computer Vision and Pattern Recognition Conference, 7877--7888

    work page 2025

  4. [4]

    G.; Guo, Z

    Azar, M. G.; Guo, Z. D.; Piot, B.; Munos, R.; Rowland, M.; Valko, M.; and Calandriello, D. 2024. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, 4447--4455. PMLR

    work page 2024

  5. [5]

    Baevski, A.; Zhou, Y.; Mohamed, A.; and Auli, M. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33: 12449--12460

    2020

  6. [6]

    Chen, G.; Zhao, X.; Zhou, Y.; Qu, X.; Chen, T.; and Cheng, Y. 2024. Towards Stabilized and Efficient Diffusion Transformers through Long-Skip-Connections with Spectral Constraints. arXiv preprint arXiv:2411.17616

    arXiv 2024

  7. [7]

    Chen, Y.; Liang, S.; Zhou, Z.; Huang, Z.; Ma, Y.; Tang, J.; Lin, Q.; Zhou, Y.; and Lu, Q. 2025 a . HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characters. arXiv preprint arXiv:2505.20156

    arXiv 2025

  8. [8]

    Chen, Z.; Cao, J.; Chen, Z.; Li, Y.; and Ma, C. 2025 b . Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 2403--2410

    work page 2025

Show all 66 references
  1. [9]

    S.; and Zisserman, A

    Chung, J. S.; and Zisserman, A. 2016. Out of time: automated lip sync in the wild. In Asian conference on computer vision, 251--263. Springer

    2016

  2. [10]

    Cui, J.; Chen, Y.; Xu, M.; Shang, H.; Chen, Y.; Zhan, Y.; Dong, Z.; Yao, Y.; Wang, J.; and Zhu, S. 2025. Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimization and Temporal Motion Modulation. arXiv preprint arXiv:2505.23525

    2025

  3. [11]

    Cui, J.; Li, H.; Zhan, Y.; Shang, H.; Cheng, K.; Ma, Y.; Mu, S.; Zhou, H.; Wang, J.; and Zhu, S. 2024. Hallo3: Highly dynamic and realistic portrait image animation with diffusion transformer networks. arXiv e-prints, arXiv--2412

    2024

  4. [12]

    Deb, K. 2011. Multi-objective optimisation using evolutionary algorithms: an introduction. In Multi-objective evolutionary optimisation for product design and manufacturing, 3--34. Springer

    2011

  5. [13]

    Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; et al. 2024. The llama 3 herd of models. arXiv e-prints, arXiv--2407

    2024

  6. [14]

    A.; Tewari, A.; Wuhrer, S.; Zollhoefer, M.; Beeler, T.; Bernard, F.; Bolkart, T.; Kortylewski, A.; Romdhani, S.; et al

    Egger, B.; Smith, W. A.; Tewari, A.; Wuhrer, S.; Zollhoefer, M.; Beeler, T.; Bernard, F.; Bolkart, T.; Kortylewski, A.; Romdhani, S.; et al. 2020. 3d morphable face models—past, present, and future. ACM Transactions on Graphics (ToG), 39(5): 1--38

    2020

  7. [15]

    Gan, Q.; Yang, R.; Zhu, J.; Xue, S.; and Hoi, S. 2025. OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animation. arXiv preprint arXiv:2506.18866

    2025 arXiv

  8. [16]

    Guo, J.; Zhang, D.; Liu, X.; Zhong, Z.; Zhang, Y.; Wan, P.; and Zhang, D. 2024. Liveportrait: Efficient portrait animation with stitching and retargeting control. arXiv preprint arXiv:2407.03168

    2024 arXiv

  9. [17]

    He, X.; Jiang, D.; Zhang, G.; Ku, M.; Soni, A.; Siu, S.; Chen, H.; Chandra, A.; Jiang, Z.; Arulraj, A.; et al. 2024. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. arXiv preprint arXiv:2406.15252

    2024 arXiv

  10. [18]

    Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; and Hochreiter, S. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30

    2017

  11. [19]

    Ho, J.; and Salimans, T. 2022. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598

    2022 arXiv

  12. [20]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2): 3

    2022

  13. [21]

    Hu, L. 2024. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8153--8163

    2024

  14. [22]

    Huang, Y.; Wang, W.; Zhao, S.; Xu, T.; Liu, L.; and Chen, E. 2025. Bind-Your-Avatar: Multi-Talking-Character Video Generation with Dynamic 3D-mask-based Embedding Router. arXiv preprint arXiv:2506.19833

    2025 arXiv

  15. [23]

    Ji, X.; Hu, X.; Xu, Z.; Zhu, J.; Lin, C.; He, Q.; Zhang, J.; Luo, D.; Chen, Y.; Lin, Q.; et al. 2025. Sonic: Shifting focus to global audio perception in portrait animation. In Proceedings of the Computer Vision and Pattern Recognition Conference, 193--203

    2025

  16. [24]

    P.; Welling, M.; et al

    Kingma, D. P.; Welling, M.; et al. 2013. Auto-encoding variational bayes

    2013

  17. [25]

    C.; Lo, W.-Y.; et al

    Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; et al. 2023. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, 4015--4026

    2023

  18. [26]

    Kong, W.; Tian, Q.; Zhang, Z.; Min, R.; Dai, Z.; Zhou, J.; Xiong, J.; Li, X.; Wu, B.; Zhang, J.; et al. 2024. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603

    2024 arXiv

  19. [27]

    Kong, Z.; Gao, F.; Zhang, Y.; Kang, Z.; Wei, X.; Cai, X.; Chen, G.; and Luo, W. 2025. Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation. arXiv preprint arXiv:2505.22647

    2025 arXiv

  20. [28]

    Li, H.; Xu, M.; Zhan, Y.; Mu, S.; Li, J.; Cheng, K.; Chen, Y.; Chen, T.; Ye, M.; Wang, J.; et al. 2025. Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, 7752--7762

    2025

  21. [29]

    Li, K.; Zhang, T.; and Wang, R. 2020. Deep reinforcement learning for multiobjective optimization. IEEE transactions on cybernetics, 51(6): 3103--3114

    2020

  22. [30]

    Liang, C.; Jiang, J.; Liao, W.; Yang, J.; Zeng, W.; Liang, H.; et al. 2025. AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation. arXiv preprint arXiv:2506.11144

    2025 arXiv

  23. [31]

    Liang, Z.; Yuan, Y.; Gu, S.; Chen, B.; Hang, T.; Li, J.; and Zheng, L. 2024. Step-aware preference optimization: Aligning preference with denoising performance at each step. arXiv preprint arXiv:2406.04314, 2(5): 7

    2024 arXiv

  24. [32]

    Lin, G.; Jiang, J.; Liang, C.; Zhong, T.; Yang, J.; Zheng, Z.; and Zheng, Y. 2025 a . Cyberhost: A one-stage diffusion framework for audio-driven talking body generation. In The Thirteenth International Conference on Learning Representations

    2025

  25. [33]

    Lin, G.; Jiang, J.; Yang, J.; Zheng, Z.; and Liang, C. 2025 b . Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models. arXiv preprint arXiv:2502.01061

    2025 arXiv

  26. [34]

    T.; Ben-Hamu, H.; Nickel, M.; and Le, M

    Lipman, Y.; Chen, R. T.; Ben-Hamu, H.; Nickel, M.; and Le, M. 2022. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747

    2022 arXiv

  27. [35]

    Liu, J.; Liu, G.; Liang, J.; Yuan, Z.; Liu, X.; Zheng, M.; Wu, X.; Wang, Q.; Qin, W.; Xia, M.; et al. 2025 a . Improving video generation with human feedback. arXiv preprint arXiv:2501.13918

    2025 arXiv

  28. [36]

    Liu, L.; Ma, T.; Li, B.; Chen, Z.; Liu, J.; Li, G.; Zhou, S.; He, Q.; and Wu, X. 2025 b . Phantom: Subject-consistent video generation via cross-modal alignment. arXiv preprint arXiv:2502.11079

    2025 arXiv

  29. [37]

    Liu, R.; Wu, H.; Zheng, Z.; Wei, C.; He, Y.; Pi, R.; and Chen, Q. 2025 c . Videodpo: Omni-preference alignment for video diffusion generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, 8009--8019

    2025

  30. [38]

    Liu, Z.; Zang, Y.; Dong, X.; Zhang, P.; Cao, Y.; Duan, H.; He, C.; Xiong, Y.; Lin, D.; and Wang, J. 2024. Mia-dpo: Multi-image augmented direct preference optimization for large vision-language models. arXiv preprint arXiv:2410.17637

    2024 arXiv

  31. [39]

    G.; Lee, J.; et al

    Lugaresi, C.; Tang, J.; Nash, H.; McClanahan, C.; Uboweja, E.; Hays, M.; Zhang, F.; Chang, C.-L.; Yong, M. G.; Lee, J.; et al. 2019. Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172

    2019 arXiv

  32. [40]

    Ma, Y.; Zhang, S.; Wang, J.; Wang, X.; Zhang, Y.; and Deng, Z. 2023. Dreamtalk: When expressive talking head generation meets diffusion probabilistic models. arXiv preprint arXiv:2312.09767, 2(3)

    2023 arXiv

  33. [41]

    H.; Cao, Q.; Horton, M.; Jin, Y.; Sun, C.; Mirzadeh, I.; Najibi, M.; Belenko, D.; Zatloukal, P.; et al

    Mehta, S.; Sekhavat, M. H.; Cao, Q.; Horton, M.; Jin, Y.; Sun, C.; Mirzadeh, I.; Najibi, M.; Belenko, D.; Zatloukal, P.; et al. 2024. Openelm: An efficient language model family with open training and inference framework. arXiv preprint arXiv:2404.14619

    2024 arXiv

  34. [42]

    Meng, Y.; Xia, M.; and Chen, D. 2024. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37: 124198--124235

    2024

  35. [43]

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 27730--27744

    2022

  36. [44]

    Peebles, W.; and Xie, S. 2023. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, 4195--4205

    2023

  37. [45]

    D.; Ermon, S.; and Finn, C

    Rafailov, R.; Sharma, A.; Mitchell, E.; Manning, C. D.; Ermon, S.; and Finn, C. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36: 53728--53741

    2023

  38. [46]

    Shen, S.; Zhao, W.; Meng, Z.; Li, W.; Zhu, Z.; Zhou, J.; and Lu, J. 2023. Difftalk: Crafting diffusion models for generalized audio-driven portraits animation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 1982--1991

    2023

  39. [47]

    Stiennon, N.; Ouyang, L.; Wu, J.; Ziegler, D.; Lowe, R.; Voss, C.; Radford, A.; Amodei, D.; and Christiano, P. F. 2020. Learning to summarize with human feedback. Advances in neural information processing systems, 33: 3008--3021

    2020

  40. [48]

    Teed, Z.; and Deng, J. 2020. Raft: Recurrent all-pairs field transforms for optical flow. In European conference on computer vision, 402--419. Springer

    2020

  41. [49]

    Unterthiner, T.; Van Steenkiste, S.; Kurach, K.; Marinier, R.; Michalski, M.; and Gelly, S. 2019. FVD: A new metric for video generation

    2019

  42. [50]

    Wallace, B.; Dang, M.; Rafailov, R.; Zhou, L.; Lou, A.; Purushwalkam, S.; Ermon, S.; Xiong, C.; Joty, S.; and Naik, N. 2024. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8228--8238

    2024

  43. [51]

    Wan, T.; Wang, A.; Ai, B.; Wen, B.; Mao, C.; Xie, C.-W.; Chen, D.; Yu, F.; Zhao, H.; Yang, J.; et al. 2025. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314

    2025 arXiv

  44. [52]

    Wang, M.; Wang, Q.; Jiang, F.; Fan, Y.; Zhang, Y.; Qi, Y.; Zhao, K.; and Xu, M. 2025 a . Fantasytalking: Realistic talking portrait generation via coherent motion synthesis. arXiv preprint arXiv:2504.04842

    2025 arXiv

  45. [53]

    Wang, Q.; Liu, M.; Hu, J.; Jiang, F.; and Xu, M. 2024 a . Controllable Longer Image Animation with Diffusion Models. arXiv preprint arXiv:2405.17306

    2024 arXiv

  46. [54]

    Wang, Q.; Wang, M.; Jiang, F.; Fan, Y.; Qi, Y.; and Xu, M. 2025 b . FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformers. arXiv preprint arXiv:2507.12956

    2025 arXiv

  47. [55]

    Wang, Y.; Tan, Z.; Wang, J.; Yang, X.; Jin, C.; and Li, H. 2024 b . Lift: Leveraging human feedback for text-to-video model alignment. arXiv preprint arXiv:2412.04814

    2024 arXiv

  48. [56]

    Wei, H.; Yang, Z.; and Wang, Z. 2024. Aniportrait: Audio-driven synthesis of photorealistic portrait animation. arXiv preprint arXiv:2403.17694

    2024 arXiv

  49. [57]

    Wu, H.; Zhang, Z.; Zhang, W.; Chen, C.; Liao, L.; Li, C.; Gao, Y.; Wang, A.; Zhang, E.; Sun, W.; et al. 2023. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090

    2023 arXiv

  50. [58]

    Wu, Z.; Kag, A.; Skorokhodov, I.; Menapace, W.; Mirzaei, A.; Gilitschenski, I.; Tulyakov, S.; and Siarohin, A. 2025. DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models. arXiv preprint arXiv:2506.03517

    2025

  51. [59]

    Xu, J.; Guo, Z.; He, J.; Hu, H.; He, T.; Bai, S.; Chen, K.; Wang, J.; Fan, Y.; Dang, K.; et al. 2025. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215

    2025 arXiv

  52. [60]

    Xu, J.; Huang, Y.; Cheng, J.; Yang, Y.; Xu, J.; Wang, Y.; Duan, W.; Yang, S.; Jin, Q.; Li, S.; et al. 2024. Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation. arXiv preprint arXiv:2412.21059

    2024 arXiv

  53. [61]

    Xu, J.; Liu, X.; Wu, Y.; Tong, Y.; Li, Q.; Ding, M.; Tang, J.; and Dong, Y. 2023. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36: 15903--15935

    2023

  54. [62]

    Yuan, S.; Huang, J.; He, X.; Ge, Y.; Shi, Y.; Chen, L.; Luo, J.; and Yuan, L. 2025. Identity-preserving text-to-video generation by frequency decomposition. In Proceedings of the Computer Vision and Pattern Recognition Conference, 12978--12988

    2025

  55. [63]

    Zhang, W.; Cun, X.; Wang, X.; Zhang, Y.; Shen, X.; Guo, Y.; Shan, Y.; and Wang, F. 2023. Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation. In Proceedings of the IEEE/CVF conference on computer vision and pattern...

    2023

  56. [64]

    Zhang, Y.; Wang, Q.; Jiang, F.; Fan, Y.; Xu, M.; and Qi, Y. 2025. Fantasyid: Face knowledge enhanced id-preserving video generation. arXiv preprint arXiv:2502.13995

    2025 arXiv

  57. [65]

    Zhou, Z.; Liu, J.; Shao, J.; Yue, X.; Yang, C.; Ouyang, W.; and Qiao, Y. 2023. Beyond one-preference-fits-all alignment: Multi-objective direct preference optimization. arXiv preprint arXiv:2310.03708

    2023 arXiv

  58. [66]

    L.; Dai, Z.; Dong, Z.; Xu, Y.; Cao, X.; Yao, Y.; Zhu, H.; and Zhu, S

    Zhu, S.; Chen, J. L.; Dai, Z.; Dong, Z.; Xu, Y.; Cao, X.; Yao, Y.; Zhu, H.; and Zhu, S. 2024. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision, 145--162. Springer

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.