Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FADA distills a 75-step diffusion avatar model into a 6-step one with comparable video quality.

desk verdict Solid first distillation framework for talking avatars with a genuinely new CFG-token mechanism, but the HDTF quality parity claim is weakened by test-set tuning of the CFG scales. read the letter →

arxiv 2412.16915 v2 pith:HDS4CAJN submitted 2024-12-22 cs.CV cs.AIcs.GRcs.SDeess.AS

Tianyun Zhong , Chao Liang , Jianwen Jiang , Gaojie Lin , Jiaqi Yang , Zhou Zhao This is my paper · ORCID
classification cs.CVcs.AIcs.GRcs.SDeess.AS
keywords audio-driventalkingavatardiffusiondistillationclassifier-freeguidancelearnabletokenspiecewiserectifiedflowmixed-supervisedfew-stepinferenceportraitanimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a diffusion-based audio-driven talking-avatar model can be distilled into a much faster student without the usual collapse in robustness or audio-video correlation. Naive distillation fails, the authors argue, because talking avatars depend on two weak conditions, audio and reference image, and the standard remedy of running classifier-free guidance three times per step becomes prohibitively slow. FADA solves both problems at once: an adaptive mixed-supervised loss lets the student learn from large amounts of moderate-quality data while still following a high-quality teacher, and a learnable-token multi-CFG distillation folds the three-pass guidance computation into a single network pass. On the HDTF and CelebV-HQ test sets, the 6-step student stays close to the 75-step teacher on video fidelity and lip-sync metrics while reducing denoising-network evaluations from 75 to 6, a reported 4.17 to 12.5 times speedup.

What carries the argument

The load-bearing object is the learnable CFG token embedding Emb_cfg = cfga*(γa − γr) + cfgr*(γr − γb) + γb, a linear combination of three learnable tokens with coefficients cfga, cfgr, and 1 that exactly mirrors the teacher's multi-CFG combination of noise predictions. A CFG control layer after each audio attention layer injects this embedding through cross-attention, so the student receives the guidance scales as an input condition and can emulate all three teacher runs in a single forward pass. The other supporting mechanism is the adaptive mixed-supervised loss, where the ground-truth weight W is ramped up with the ratio R = L_gt / L_teacher, capped, and then zeroed beyond a dead threshold, so the student selectively learns from moderate-quality data without being led astray by badly synchronized samples.

What would settle it

Run the trained 6-step student on out-of-distribution inputs, such as singing audio, non-English speech, or portraits with heavy head motion, at fixed CFG scales, and measure Sync-D and FVD against the teacher; if lip-sync degrades substantially or the video quality gap widens far beyond the HDTF numbers, the claim of robust one-pass multi-CFG emulation fails. Alternatively, vary the audio CFG outside the tested range and check whether the student's FVD curve still tracks the teacher's multi-CFG curve.

Watch

Extended reading notes

Core claim

The paper's central claim is that the composite three-pass classifier-free guidance used by current avatar diffusion models can be taught to a distilled student through a CFG embedding that mirrors the teacher's noise-space algebra. The student is conditioned on Emb_cfg = cfga*(γa − γr) + cfgr*(γr − γb) + γb, a linear combination of learnable tokens with the same coefficient structure as the teacher's multi-CFG formula, and this embedding is injected through cross-attention layers after each audio layer. Trained on teacher outputs that already include full multi-CFG reasoning, the student learns to reproduce the effect of all three teacher runs in one forward pass. A second claim is that the adaptive mixed-supervised loss, whose ground-truth weight W is a function of R = L_gt / L_teacher, lets the student exploit large moderate-quality datasets without inheriting their defects: samples where the teacher and ground truth diverge too much are automatically down-weighted. Together with piecewise rectified-flow distillation as the base, this yields 6-step inference with 6 denoising-network evaluations per frame instead of 75.

Load-bearing premise

The load-bearing premise is that the linear token embedding Emb_cfg = cfga*(γa − γr) + cfgr*(γr − γb) + γb contains enough information for the student to reproduce the teacher's three-pass multi-CFG behavior on inputs beyond the specific test sets and guidance scales used to validate it.

Editorial extensions

If this is right

  • The 6-step student runs the denoising network once per step instead of three times, cutting total NFE-D from 75 to 6, a reported up to 12.5x speedup over the teacher.
  • Moderate-quality, less-curated data becomes usable at distillation time: the mixed-supervised loss turns a larger dataset into better robustness and lower FVD rather than the degradation seen when the same data is added to teacher training.
  • The learned CFG embedding responds smoothly to audio and reference guidance scales, so users can adjust condition strength at inference without paying for multiple classifier-free guidance runs.
  • The framework is positioned as the first diffusion-distillation method for audio-driven talking avatars and, the authors argue, can be extended to other multi-condition generation tasks such as text-to-image and text-to-video.
  • Reported HDTF results put the 6-step student at FVD-R 16.67, FID 18.51, and E-FID 1.635, close to the teacher's 18.27, 18.49, and 1.365, while using about 8% of the teacher's denoising-network evaluations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the token embedding has the same coefficient structure as the multi-CFG formula, the method suggests a general recipe: any composite classifier-free guidance with several conditions can be collapsed by conditioning the student on a token combination that mirrors the guidance algebra, not just on scalar guidance values.
  • The reported quality is tied to the CFG scales chosen by tuning on the HDTF test set (audio 6.5, reference 2.0), so the 12.5x speedup could silently trade quality at other scales or on out-of-distribution inputs; a deployment would need its own validation protocol.
  • The ratio R = L_gt / L_teacher could itself be reused as an automatic data-quality filter: samples above the dead threshold are effectively flagged as low-quality, so the same signal could refine future training sets without manual filtering.
  • The adaptive mixed-supervised loss should transfer to other weakly conditioned generation tasks, but the specific thresholds in this paper would likely need re-tuning rather than carrying over directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FADA, a distillation framework for audio-driven diffusion-based talking avatar synthesis. A dual-UNet teacher trained on a strictly filtered high-quality dataset is distilled with a piecewise-rectified-flow (PeRFlow) objective into a student network that also trains on a larger, less filtered dataset through an adaptive mixture of teacher-supervised and ground-truth losses. To eliminate the three network evaluations required by multi-CFG inference, the student is conditioned on audio and reference CFG scales through a learnable-token embedding of the form cfga*(γa-γr) + cfgr*(γr-γb) + γb, injected via additional CFG control layers. The paper reports 18 NFE-D for Ours-Balanced and 6 NFE-D for Ours-Fast, versus 75 NFE-D for the teacher, and claims quality comparable to recent diffusion methods on HDTF and CelebV-HQ.

Significance. If the reported numbers hold up under a clean evaluation protocol, the contribution is practically significant: the 4.17-12.5x NFE speedup is large for a task in which multi-CFG triples the inference cost, and the ablations provide a clear decomposition of the proposed components. The learnable-token CFG embedding is a simple and transparent way to expose the multi-CFG arithmetic to the student, and the design is not circular, since the student is trained against teacher outputs rather than fitted to evaluation metrics. The main quantitative support for 'comparable' quality, however, is weakened by test-set selection of CFG hyperparameters and by the absence of error bars on the central video-level metrics; the paper also provides no code or model release, which limits independent verification.

major comments (3)
  1. [§4.3 (Figures 3 and 4), §4.1 (Implementation Details), Table 1] The CFG scales used for Ours-Fast are selected from FVD and Sync-D curves computed on the same HDTF test set that is later used for the headline comparison in Table 1. Specifically, Figure 3 fixes reference CFG to 2.0 and sweeps audio CFG on HDTF, Figure 4 fixes audio CFG to 6.5 and sweeps reference CFG on HDTF, and the text then states that the authors 'ultimately chose 6.5' and observe 'optimal performance achieved around the 2.0 position.' Table 1 evaluates Ours-Fast on that same HDTF set with these selected values, and the resulting FVD-R of 16.67 is better than the teacher's 18.27. Since the baseline methods' CFG settings were not tuned on this HDTF subset, the comparison is not equitably calibrated, and the 'comparable' claim in the abstract rests on numbers that may reflect test-set selection rather than genuine generation quality. The authors should select hyperparameters on a held-out validation split, or report results for a pre-specified/default CFG setting, and then report the test-set results separately.
  2. [§3.3 (Eqs. 9-10), §4.3 (Table 4), §4.1 (Implementation Details)] The central mechanistic claim is that the learnable-token embedding of Eq. (10) allows the student to mimic the three-pass multi-CFG calculation of Eq. (9) in a single pass, but the paper does not test this mimicry directly. Table 4 compares student variants at 6 NFE-D with a teacher that uses CFG scales (2.5 reference, 6.5 audio), while Ours-Fast with the token-based CFG layer uses the test-set-selected scales (2.0 reference, 6.5 audio). The ablation therefore does not isolate whether the student reproduces the teacher's multi-CFG output at a fixed CFG setting. A direct test should hold the CFG scales fixed, compare the one-pass student output with the three-pass teacher output on a held-out set, and ideally sweep several CFG values to show that the learned CFG control behaves like the true multi-CFG operator.
  3. [§4.2 (Metrics and Baselines), Table 1] Only IQA is reported with 95% confidence intervals; the FVD-R, FID, and E-FID metrics are single numbers computed on a 100-sample subset. The main claim of 'comparable quality' relies on small differences, such as Ours-Fast FVD-R 54.69 versus Hallo 53.99 on CelebV-HQ, and FID 18.51 versus 18.49 on HDTF, so without variance estimates it is not possible to tell whether these differences are meaningful. Please provide confidence intervals or bootstrap estimates for the video-level metrics as well, and specify how the 100 test samples were drawn.
minor comments (5)
  1. [§3.3, around Eq. (10)] The text says the model introduces 'learnable tokens γb, γr, and γr'; the third token should presumably be γa to match Eq. (10).
  2. [§3.2, around Eqs. (6)-(7)] The notation is inconsistent: the text refers to 'Lteacher' in the discussion of the ratio R, but Eq. (7) defines the first term as Ldistill. Please align the notation or define Lteacher explicitly.
  3. [§5 (Conclusion)] The conclusion states that limitations, future work, and ethical concerns are discussed in Appendices B, H, and I, but those appendices are not present in the submitted full text. Please include the appendix material or remove the references.
  4. [§4.3, Figures 3 and 4] The curves in Figures 3 and 4 are used to select CFG values, but no measure of variability is given; adding error bars or multiple seeds would make the selection procedure more robust and transparent.
  5. [§4.2 and project page] No code or model is released, and the paper reports NFE-D rather than wall-clock runtime. Since the abstract emphasizes practical inference speed, please provide actual GPU runtime measurements for the teacher, Ours-Balanced, and Ours-Fast, and consider releasing the student model for independent verification.

Circularity Check

1 steps flagged · score 4.0 of 10

HDTF CFG scales are tuned on the same test set later reported in Table 1, so the headline HDTF quality parity is partly a fitted result; the rest of the derivation is self-contained.

  1. fitted input called prediction [Section 4.3 'Analysis of Multi-CFG Distillation' (Figures 3 and 4); Section 4.1 'Implementation Details'; reported in Table 1]
    "we plotted line charts of FVD and Sync-D metrics with variations in audio CFG and reference CFG, as shown in Figures 3 and 4. First, we observed improvements in both FVD and Sync-D as audio CFG increased, reaching a stable maximum value after 6.5. Hence, we ultimately chose 6.5 as the audio CFG for inference. Second, the influence of reference CFG on FVD and Sync-D metrics displayed an initially positive, then negative trend, with optimal performance achieved around the 2.0 position."

    The audio and reference CFG scales (6.5 and 2.0) are selected by minimizing FVD/Sync-D on the HDTF test set, and the same HDTF test set is then used in Table 1 to report Ours-Fast's FVD-R 16.67, FID 18.51, and E-FID 1.635. The headline HDTF numbers are therefore the values at the optimum of the tuning curve on the same 100-video set, so the claim of parity with the teacher (FVD-R 18.27) is in part an artifact of test-set fitting rather than an independent prediction. The CelebV-HQ results and the NFE speedup arithmetic are independent, so the circularity is partial.

full rationale

The core derivation chain is not circular: the student is trained with PeRFlow-style distillation (Eqs. 2-5) against a frozen teacher, the mixed-supervised loss (Eqs. 6-8) is an auxiliary supervised objective with data-driven weighting, and the multi-CFG token embedding (Eq. 10) is an architectural control signal that mirrors the teacher's three-pass CFG arithmetic (Eq. 9) but does not algebraically determine the student output; the reported FVD/IQA/Sync-D values are measured, not derived from the embedding formula. The self-citations to Loopy [10] and CyberHost [14] supply the teacher architecture and the standard multi-CFG formula, but they are used as a starting point rather than as an unexamined uniqueness argument, so they do not constitute load-bearing circularity. The one concrete circular step is the selection of CFG scales on the HDTF test set followed by evaluation on the same set, which inflates the quantitative parity claim in Table 1. The method itself, the CelebV-HQ numbers, and the speedup calculation remain independent, so the overall circularity score is moderate rather than severe.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method depends on a handful of hand-set loss weights and CFG scales, several of which are tuned on the test set. No new physical entities are introduced; the learned tokens are parameters of the network, not external postulates. The main domain assumptions are the weak-audio premise and the sufficiency of the linear CFG token embedding.

free parameters (3)
  • Adaptive loss hyperparameters = s=0.25, W0=0.2, Rp=30, Rd=100
    Hand-chosen based on qualitative observation of R values and small ablations (Table 3), not derived from first principles.
  • CFG guidance scales (audio, reference) = cfga=6.5, cfgr=2.0 for CFG-distilled; 2.5 and 6.5 for non-distilled
    Selected from HDTF test-set line charts (Figures 3 and 4) to minimize FVD and Sync-D, then used for the reported HDTF and CelebV-HQ numbers.
  • PeRFlow time windows and inference schedule = K=4 windows; 6-step inference with two steps near T=1
    Design choice that sets the 6 NFE-D for Ours-Fast; other schedules are not systematically explored.
assumptions (5)
  • standard math PeRFlow piecewise rectified flow correctly models the ODE trajectory for an SD1.5-based teacher with a DDIM solver.
    The distillation target is taken from PeRFlow [37]; the teacher is trained with epsilon-prediction DDPM loss, and PeRFlow is claimed to work with epsilon-prediction (Section 3.1).
  • domain assumption Audio has weaker influence on video than the reference image, motivating multi-CFG and the token design.
    Stated in the Introduction and Section 3.3; if audio were as strong as the reference, the three-pass CFG and its distillation would be less necessary.
  • domain assumption A well-trained teacher model on high-quality data provides reliable audio-lip sync guidance for moderate-quality data.
    Section 3.2; this assumption justifies using teacher targets for samples the teacher never saw.
  • ad hoc to paper The linear token embedding of Eq. (10) can represent the multi-CFG combination of Eq. (9) sufficiently for a single-pass student.
    Section 3.3; this is the core architecture premise, validated only empirically on two test sets.
  • standard math SD1.5 latent space and VAE decode preserve talking-head fidelity.
    The backbone is pre-trained SD1.5, as in several cited methods; no re-training of the VAE is done.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation." pith.science (2026). https://pith.science/paper/HDS4CAJN

@misc{pith2026241216915,
  author       = {Pith},
  title        = {Pith review of: FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HDS4CAJN}},
  note         = {Machine review of arXiv:2412.16915}
}
read the original abstract

Diffusion-based audio-driven talking avatar methods have recently gained attention for their high-fidelity, vivid, and expressive results. However, their slow inference speed limits practical applications. Despite the development of various distillation techniques for diffusion models, we found that naive diffusion distillation methods do not yield satisfactory results. Distilled models exhibit reduced robustness with open-set input images and a decreased correlation between audio and video compared to teacher models, undermining the advantages of diffusion models. To address this, we propose FADA (Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation). We first designed a mixed-supervised loss to leverage data of varying quality and enhance the overall model capability as well as robustness. Additionally, we propose a multi-CFG distillation with learnable tokens to utilize the correlation between audio and reference image conditions, reducing the threefold inference runs caused by multi-CFG with acceptable quality degradation. Extensive experiments across multiple datasets show that FADA generates vivid videos comparable to recent diffusion model-based methods while achieving an NFE speedup of 4.17-12.5 times. Demos are available at our webpage http://fadavatar.github.io.

Figures

Figures reproduced from arXiv: 2412.16915 by the authors.

Figure 1
Figure 1. Overall distillation framework of FADA. The teacher model is trained only with high-quality data, which is omitted in the figure. The student model is trained by a mixed loss of ground-truth and teacher-supervised loss to leverage data of varying quality. Learnable token-based CFG conditions enable the student model to mimic the multi-CFG process, further reducing inference times. For simplicity, we have omitted som… view at source ↗
Figure 2
Figure 2. Qualitative comparisons between FADA and baselines across different portraits and pronunciations in openset. the addition of dataset B under different parameters. Com￾paring the effects of different Fixed-x values, it is evident that the ground-truth loss improves model performance, and a smaller loss weight value of 0.2 yields better results. We further investigated the effectiveness of the adaptive loss weight str… view at source ↗
Figure 3
Figure 3. Line charts showing the variations of FVD and Sync￾D metrics with different audio CFG on HDTF test set. Refer￾ence CFG is set to 2.0 in this figure. 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 Refenrece CFG 27 54 81 108 135 FVD↓ 7.8 8.1 8.4 8.7 9.0 9.3 Sync-D↓ [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Line charts showing the variations of FVD and Sync￾D metrics with different ref CFG on HDTF test set. Audio CFG is set to 6.5 in this figure. tested the results of conditional reasoning without CFG dis￾tillation, which significantly deteriorated the model perfor￾mance,…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models

    cs.CV 2025-02 conditional novelty 6.0 of 10

    A mixed-condition training recipe, combining text, audio, and pose signals, lets a single diffusion transformer animate photos into realistic talking or singing videos at unprecedented data scale.

Reference graph

Works this paper leans on

49 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [1]

    Echomimic: Lifelike audio-driven por- trait animations through editable landmark conditions

    Zhiyuan Chen, Jiajiong Cao, Zhiquan Chen, Yuming Li, and Chenguang Ma. Echomimic: Lifelike audio-driven por- trait animations through editable landmark conditions. arXiv preprint arXiv:2407.08136, 2024. 1, 2, 3, 6, 7

    arXiv 2024

  2. [2]

    Hallo2: Long-duration and high-resolution audio-driven portrait im- age animation

    Jiahao Cui, Hui Li, Yao Yao, Hao Zhu, Hanlin Shang, Kaihui Cheng, Hang Zhou, Siyu Zhu, and Jingdong Wang. Hallo2: Long-duration and high-resolution audio-driven portrait im- age animation. arXiv preprint arXiv:2410.07718, 2024. 1, 2, 3

    arXiv

  3. [3]

    Swiftbrush v2: Make your one-step diffusion model better than its teacher

    Trung Dao, Thuan Hoang Nguyen, Thanh Le, Duc Vu, Khoi Nguyen, Cuong Pham, and Anh Tran. Swiftbrush v2: Make your one-step diffusion model better than its teacher. In European Conference on Computer Vision, pages 176–192. Springer, 2025. 1, 3

    work page 2025

  4. [4]

    Livepor- trait: Efficient portrait animation with stitching and retarget- ing control

    Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang. Livepor- trait: Efficient portrait animation with stitching and retarget- ing control. arXiv preprint arXiv:2407.03168, 2024. 1, 2

    arXiv 2024

  5. [5]

    Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis

    Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, and Juyong Zhang. Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 5784–5794, 2021. 1, 2

    work page 2021

  6. [6]

    Ptqd: Accurate post-training quantization for diffusion models

    Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Ptqd: Accurate post-training quantization for diffusion models. Advances in Neural Information Pro- cessing Systems, 36, 2024. 2

    2024

  7. [7]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  8. [8]

    Depth-aware generative adversarial network for talking head video generation

    Fa-Ting Hong, Longhao Zhang, Li Shen, and Dan Xu. Depth-aware generative adversarial network for talking head video generation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 3397–3406, 2022. 1, 2

    work page 2022

Show all 49 references
  1. [9]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 3

    2024

  2. [10]

    Loopy: Taming audio-driven portrait avatar with long-term motion dependency

    Jianwen Jiang, Chao Liang, Jiaqi Yang, Gaojie Lin, Tianyun Zhong, and Yanbo Zheng. Loopy: Taming audio-driven portrait avatar with long-term motion dependency. arXiv preprint arXiv:2409.02634, 2024. 1, 2, 3, 5, 6

    2024 arXiv

  3. [11]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3

    2013 arXiv

  4. [12]

    Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds

    Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Snap- fusion: Text-to-image diffusion model on mobile devices within two seconds. Advances in Neural Information Pro- cessing Systems, 36, 2024. 1, 2

    2024

  5. [13]

    Q-dm: An efficient low-bit quantized dif- fusion model

    Yanjing Li, Sheng Xu, Xianbin Cao, Xiao Sun, and Baochang Zhang. Q-dm: An efficient low-bit quantized dif- fusion model. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  6. [14]

    Cyberhost: Taming audio- driven avatar diffusion model with region codebook atten- tion

    Gaojie Lin, Jianwen Jiang, Chao Liang, Tianyun Zhong, Ji- aqi Yang, and Yanbo Zheng. Cyberhost: Taming audio- driven avatar diffusion model with region codebook atten- tion. arXiv preprint arXiv:2409.01876, 2024. 1, 2, 3, 5

    2024 arXiv

  7. [15]

    Sdxl- lightning: Progressive adversarial diffusion distillation

    Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl- lightning: Progressive adversarial diffusion distillation. arXiv preprint arXiv:2402.13929, 2024. 1, 3

    2024 arXiv

  8. [16]

    Rectified flow: A marginal preserving approach to optimal transport

    Qiang Liu. Rectified flow: A marginal preserving approach to optimal transport. arXiv preprint arXiv:2209.14577, 2022. 6

    2022 arXiv

  9. [17]

    Latent consistency models: Synthesizing high- resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high- resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023. 1, 3

    2023 arXiv

  10. [18]

    Fastercache: Training-free video diffusion model acceleration with high quality

    Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan-Yee K Wong. Fastercache: Training-free video diffusion model acceleration with high quality. arXiv preprint arXiv:2410.19355, 2024. 1

    2024 arXiv

  11. [19]

    Styletalk: One-shot talking head generation with controllable speaking styles

    Yifeng Ma, Suzhen Wang, Zhipeng Hu, Changjie Fan, Tangjie Lv, Yu Ding, Zhidong Deng, and Xin Yu. Styletalk: One-shot talking head generation with controllable speaking styles. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1896–1904, 2023. 1, 2

    1904

  12. [20]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 14297–14306, 2023. 5, 8

    2023

  13. [21]

    Swiftbrush: One-step text-to-image diffusion model with variational score distilla- tion

    Thuan Hoang Nguyen and Anh Tran. Swiftbrush: One-step text-to-image diffusion model with variational score distilla- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 7807–7816,

  14. [22]

    A lip sync expert is all you need for speech to lip generation in the wild

    KR Prajwal, Rudrabha Mukhopadhyay, Vinay P Nambood- iri, and CV Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. In Proceedings of the 28th ACM international conference on multimedia , pages 484–492, 2020. 6

    2020

  15. [23]

    High-resolution image 9 Preprint synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image 9 Preprint synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3

    2022

  16. [24]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprint arXiv:2311.17042, 2023. 1, 2

    2023 arXiv

  17. [25]

    Post-training quantization on diffusion models

    Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 1972–1981, 2023. 2

    1972

  18. [26]

    Temporal dynamic quantization for dif- fusion models

    Junhyuk So, Jungwon Lee, Daehyun Ahn, Hyungjun Kim, and Eunhyeok Park. Temporal dynamic quantization for dif- fusion models. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  19. [27]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 1, 3

    2010 arXiv

  20. [28]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 2023. 1, 3

    2023 arXiv

  21. [29]

    Bitsfusion: 1.99 bits weight quantization of diffusion model

    Yang Sui, Yanyu Li, Anil Kag, Yerlan Idelbayev, Junli Cao, Ju Hu, Dhritiman Sagar, Bo Yuan, Sergey Tulyakov, and Jian Ren. Bitsfusion: 1.99 bits weight quantization of diffusion model. arXiv preprint arXiv:2406.04333, 2024. 2

    2024 arXiv

  22. [30]

    Real-time neural radiance talking portrait synthesis via audio-spatial decomposition

    Jiaxiang Tang, Kaisiyuan Wang, Hang Zhou, Xiaokang Chen, Dongliang He, Tianshu Hu, Jingtuo Liu, Gang Zeng, and Jingdong Wang. Real-time neural radiance talking portrait synthesis via audio-spatial decomposition. arXiv preprint arXiv:2211.12368, 2022. 1, 2

    2022 arXiv

  23. [31]

    Emo: Emote portrait alive-generating expressive portrait videos with audio2video diffusion model under weak conditions

    Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive-generating expressive portrait videos with audio2video diffusion model under weak conditions. arXiv preprint arXiv:2402.17485, 2024. 1, 2, 3, 6

    2024 arXiv

  24. [32]

    Fvd: A new metric for video generation

    Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Rapha¨el Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. OpenReview, 2019. 6

    2019

  25. [33]

    V-express: Conditional dropout for progres- sive training of portrait video generation

    Cong Wang, Kuan Tian, Jun Zhang, Yonghang Guan, Feng Luo, Fei Shen, Zhiwei Jiang, Qing Gu, Xiao Han, and Wei Yang. V-express: Conditional dropout for progres- sive training of portrait video generation. arXiv preprint arXiv:2406.02511, 2024. 1, 2, 3, 6, 7

    2024 arXiv

  26. [34]

    One-shot free-view neural talking-head synthesis for video conferenc- ing

    Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 10039–10049,

  27. [35]

    Q-align: Teaching lmms for visual scoring via discrete text-defined levels

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023. 6

    2023 arXiv

  28. [36]

    Hallo: Hierarchical audio-driven vi- sual synthesis for portrait image animation

    Mingwang Xu, Hui Li, Qingkun Su, Hanlin Shang, Li- wei Zhang, Ce Liu, Jingdong Wang, Luc Van Gool, Yao Yao, and Siyu Zhu. Hallo: Hierarchical audio-driven vi- sual synthesis for portrait image animation. arXiv preprint arXiv:2406.08801, 2024. 1, 2, 3, 6, 7

    2024 arXiv

  29. [37]

    Perflow: Piecewise rectified flow as universal plug-and-play accelerator

    Hanshu Yan, Xingchao Liu, Jiachun Pan, Jun Hao Liew, Qiang Liu, and Jiashi Feng. Perflow: Piecewise rectified flow as universal plug-and-play accelerator. arXiv preprint arXiv:2405.07510, 2024. 3, 6

    2024 arXiv

  30. [38]

    Geneface++: Generalized and stable real- time audio-driven 3d talking face generation

    Zhenhui Ye, Jinzheng He, Ziyue Jiang, Rongjie Huang, Ji- awei Huang, Jinglin Liu, Yi Ren, Xiang Yin, Zejun Ma, and Zhou Zhao. Geneface++: Generalized and stable real- time audio-driven 3d talking face generation. arXiv preprint arXiv:2305.00787, 2023. 1, 2

    2023 arXiv

  31. [39]

    Geneface: Generalized and high- fidelity audio-driven 3d talking face synthesis.arXiv preprint arXiv:2301.13430, 2023

    Zhenhui Ye, Ziyue Jiang, Yi Ren, Jinglin Liu, JinZheng He, and Zhou Zhao. Geneface: Generalized and high- fidelity audio-driven 3d talking face synthesis.arXiv preprint arXiv:2301.13430, 2023

    2023 arXiv

  32. [40]

    Mimictalk: Mimicking a per- sonalized and expressive 3d talking face in minutes

    Zhenhui Ye, Tianyun Zhong, Yi Ren, Ziyue Jiang, Jiawei Huang, Rongjie Huang, Jinglin Liu, Jinzheng He, Chen Zhang, Zehan Wang, et al. Mimictalk: Mimicking a per- sonalized and expressive 3d talking face in minutes. arXiv preprint arXiv:2410.06734, 2024. 2

    arXiv

  33. [41]

    Real3d-portrait: One-shot realistic 3d talking portrait synthesis

    Zhenhui Ye, Tianyun Zhong, Yi Ren, Jiaqi Yang, Weichuang Li, Jiawei Huang, Ziyue Jiang, Jinzheng He, Rongjie Huang, Jinglin Liu, et al. Real3d-portrait: One-shot realistic 3d talking portrait synthesis. arXiv preprint arXiv:2401.08503,

    arXiv

  34. [42]

    Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation

    Wenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang, Xi Shen, Yu Guo, Ying Shan, and Fei Wang. Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2023

  35. [43]

    Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset

    Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3661–3670, 2021. 6

    2021

  36. [44]

    Thin-plate spline motion model for image animation

    Jian Zhao and Hui Zhang. Thin-plate spline motion model for image animation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3657–3666, 2022. 1, 2

    2022

  37. [45]

    Vidit-q: Efficient and accurate quantization of diffusion transformers for im- age and video generation

    Tianchen Zhao, Tongcheng Fang, Enshu Liu, Wan Rui, Widyadewi Soedarmadji, Shiyao Li, Zinan Lin, Guohao Dai, Shengen Yan, Huazhong Yang, et al. Vidit-q: Efficient and accurate quantization of diffusion transformers for im- age and video generation. arXiv preprint arXiv:2406.02540,

    arXiv

  38. [46]

    Trajectory consistency distillation

    Jianbin Zheng, Minghui Hu, Zhongyi Fan, Chaoyue Wang, Changxing Ding, Dacheng Tao, and Tat-Jen Cham. Trajectory consistency distillation. arXiv preprint arXiv:2402.19159, 2024. 1, 3, 6

    2024 arXiv

  39. [47]

    Pose-controllable talking face generation by implicitly modularized audio-visual rep- resentation

    Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, and Ziwei Liu. Pose-controllable talking face generation by implicitly modularized audio-visual rep- resentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4176–4186,

  40. [48]

    Makelttalk: speaker-aware talking-head animation

    Yang Zhou, Xintong Han, Eli Shechtman, Jose Echevar- ria, Evangelos Kalogerakis, and Dingzeyu Li. Makelttalk: speaker-aware talking-head animation. ACM Transactions On Graphics (TOG), 39(6):1–15, 2020. 1, 2 10 Preprint

    2020

  41. [49]

    Celebv- hq: A large-scale video facial attributes dataset

    Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. Celebv- hq: A large-scale video facial attributes dataset. InEuropean conference on computer vision , pages 650–667. Springer,

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.