Pith. sign in

REVIEW 3 major objections 2 minor 4 cited by

CEM-Net: Cross-Emotion Memory Network for Emotional Talking Face Generation

T0 review · 3 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A cross-emotion memory makes talking faces follow audio emotion even when the reference photo is emotionally conflicting.

desk verdict A plausible fix for a real emotion-conflict problem in talking faces, but the unreadable full text means the evidence is still a promise. read the letter →

arxiv 2508.12368 v1 pith:JSFIFTMF submitted 2025-08-17 cs.MM cs.SD

Kangyi Wu , Pengna Li , Jingwen Fu , Yang Wu , Yuhan Liu , Sanping Zhou , Jinjun Wang This is my paper
classification cs.MMcs.SD
keywords emotionaltalkingfacegenerationcross-emotionmemorynetworkexpressiondisplacementaudioemotionenhancementlipsyncanimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that existing emotional talking face generation methods fail when the reference image shows a strong emotion that contradicts the emotion in the driving audio, producing inaccurate expressions and distorted frames. To fix this, CEM-Net first strengthens the audio emotion signal with an Audio Emotion Enhancement module, then uses an Emotion Bridging Memory that stores expression displacements from the reference emotion to the audio emotion. At inference, a cross-emotion feature query retrieves the stored displacement, supplying the facial motion information the reference image cannot provide. If the claim holds, the method produces expressive, natural, and lip-synced talking faces even for mismatched reference-emotion and audio-emotion pairs.

What carries the argument

The central mechanism is the Emotion Bridging Memory (EBM), a memory module that stores expression displacement vectors from the reference image emotion to the audio emotion; at inference, a cross-emotion feature query retrieves the matching displacement. The companion Audio Emotion Enhancement (AEE) module, trained with a cross-reconstruction strategy, strengthens the audio emotion signal so the reference image's conflicting emotion does not dominate. The EBM carries the argument by supplying the facial motion information that the reference image cannot provide under a different audio emotion.

What would settle it

Build a test set of identities never seen in training, each with a reference image showing a strong emotion opposite to the audio emotion, and compare emotion accuracy and identity preservation against seen identities; a sharp drop would indicate the memory lookup does not generalize. Alternatively, compare the memory-retrieved displacement to the ground-truth displacement measured from a paired video of the same person expressing both emotions; a large retrieval error would invalidate the mechanism.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that expression displacement between emotions can be learned, stored, and retrieved as a memory entry, so that the driving audio's emotion can control the face even when the reference image carries a conflicting emotion. The Audio Emotion Enhancement module uses a cross-reconstruction training strategy to isolate the audio's emotional content from the reference image's emotional interference. The Emotion Bridging Memory module stores the displacement from the reference emotion to the audio emotion, and a cross-emotion feature query retrieves the matching displacement at inference time. Together these modules compensate for the missing facial

Load-bearing premise

The expression displacement stored in the memory generalizes across identities and speech content, so that retrieving it for a new face and new audio produces a valid, compatible motion rather than identity-specific artifacts.

Editorial extensions

If this is right

  • If correct, talking face generation becomes practical for arbitrary combinations of reference image emotion and audio emotion, such as animating a neutral or sad photo with happy or angry speech.
  • Emotion accuracy can be improved without requiring paired video of the target emotion for the same identity, because the needed expression change is retrieved from memory rather than synthesized from the reference alone.
  • The memory-based displacement retrieval suggests that emotion transfer can be treated as a lookup problem, which may generalize to emotion intensity control by interpolating retrieved displacements.
  • Cross-emotion training pairs become a useful resource instead of a problem to be avoided, potentially enlarging the effective training distribution for emotional talking face models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit is that the stored expression displacements could support fine-grained emotion intensity adjustment, since interpolation between stored displacements might yield intermediate emotional states, but this is not tested here.
  • The approach implicitly assumes expression displacement is largely identity-independent and content-independent; if that holds, the same memory could animate diverse or even stylized faces, an extension the paper does not explore.
  • A testable extension would stress the memory with held-out identities and extreme emotion pairs; if emotion accuracy degrades sharply relative to seen identities, the memory is partly memorizing training identities rather than generalizing expression transfer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The paper proposes CEM-Net, a cross-emotion memory network for emotional talking face generation. The stated problem is that reference images may carry an emotion conflicting with the driving audio, causing emotion inaccuracy and distortion. The proposed method has two modules: an Audio Emotion Enhancement module (AEE) with a cross-reconstruction training strategy, and an Emotion Bridging Memory module (EBM) that stores expression displacements from the reference-image emotion to the audio emotion and retrieves them at inference. The abstract claims extensive experiments demonstrate improved emotion accuracy, naturalness, and lip sync over existing methods. However, the available full text is corrupted mojibake, so no experimental details, results, datasets, baselines, or ablations can be inspected.

Significance. If the claimed improvements hold, CEM-Net would address a real and under-explored issue: reference/audio emotion conflict in emotional talking face generation. The AEE/EBM architecture is a plausible design: AEE enhances audio emotion while EBM supplies facial motion information missing from the reference. The paper's framing is useful and the proposed mechanism is internally coherent. However, the significance cannot currently be assessed because the only support offered is the abstract's assertion of 'extensive experiments'; no quantitative evidence, code, or verifiable protocol is provided. The paper also does not offer machine-checked proofs or reproducible artifacts that could substitute for empirical detail.

major comments (3)
  1. [Abstract / Full Text] The central empirical claim—'Extensive experiments have demonstrated that our CEM-Net can synthesize expressive, natural and lip-synced talking face videos with better emotion accuracy'—is unsupported in the available manuscript. The full text is corrupted (mojibake) and contains no datasets, baselines, evaluation metrics, ablations, or numerical results. A reader cannot verify the superiority claim or even assess whether the method was tested in the cross-emotion regime. This is a load-bearing gap; the paper must provide a complete, readable experimental section with quantitative comparisons and ablations.
  2. [Abstract, EBM module description] The EBM's inference-time retrieval is the load-bearing step: 'Given a cross-emotion feature as a query, the matching displacement can be retrieved at inference time.' The abstract provides no evidence that the learned displacement generalizes across unseen identities and speech content. Expression displacement is identity- and content-dependent; a category-level memory could collapse distinct emotions or stitch incompatible motion onto a face. The manuscript needs an identity-disjoint evaluation and a comparison to a simple per-emotion-pair average displacement to rule out trivial memorization and to show that the memory preserves identity-specific expressiveness.
  3. [Abstract, emotion accuracy metric] The claimed 'better emotion accuracy' is undefined. If the evaluation metric is an emotion classifier trained with labels that also supervise AEE or EBM, the improvement could be circular. The paper must specify the emotion-accuracy metric, the classifier or annotation protocol, and demonstrate that the metric is independent of the training supervision. Without this, the headline claim is not falsifiable.
minor comments (2)
  1. [Abstract] Typographical issues: 'network(CEM-Net)' should have a space before the parenthesis; 'audio emotion.Given' is missing a space after the period.
  2. [General] The abstract introduces AEE and EBM but does not describe the cross-reconstruction training strategy; a one-sentence explanation or a pointer to a section would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; CEM-Net's claims are empirical and do not reduce to their inputs by construction.

full rationale

The available text (the abstract and a garbled/incomplete body) does not exhibit any derivation step that reduces by construction to its own inputs. The claimed contributions are (1) an Audio Emotion Enhancement module trained with a cross-reconstruction strategy and (2) an Emotion Bridging Memory module that stores and retrieves expression displacements between reference-image emotion and audio emotion. These are architectural and training-time mechanisms, not fitted parameters subsequently renamed as predictions. The abstract does not cite the authors' own prior results as the load-bearing justification, nor does it invoke a self-authored uniqueness theorem. The concern that EBM retrieval may not generalize across identities or audio content is a plausibility/empirical-validity risk, not a circularity: a method can be non-circular and still fail to generalize. Likewise, the possibility that the emotion metric might be confounded with training supervision is not demonstrated from the manuscript text, and no equations or evaluation details are available to exhibit such a reduction. Therefore, under the hard rule that circularity must be shown by quoting a specific reduction, no circular step can be identified. The appropriate finding is a non-finding: the paper's central claims are ordinary empirical ML claims whose validation depends on datasets and experiments, which is not circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 2 invented entities

Provisional ledger from abstract-only review, because the supplied full text is corrupted. No numerical hyperparameters, loss weights, or fitted constants are visible. The network weights and the EBM memory bank are learned from data; a complete ledger would require the intact manuscript.

assumptions (4)
  • domain assumption Expression displacement from reference emotion to audio emotion is learnable and transfers across identities and audio samples.
    The EBM stores such displacement ('expression displacement from the reference image emotion to the audio emotion') and retrieves it at inference; if displacement does not generalize across speakers, the memory retrieval fails. This assumption carries the method's main bet.
  • domain assumption Cross-reconstruction training enhances the audio emotion representation without degrading identity or lip synchronization.
    The AEE module's 'cross-reconstruction training strategy' is asserted to 'enhance audio emotion, overcoming the disruption from reference image emotion'; the abstract states this, the full text would need to demonstrate it.
  • domain assumption Driving audio contains sufficient emotion information to condition facial expressions.
    The entire pipeline generates expressions from audio emotion; this is the standard assumption of the talking face literature and is implicit throughout the abstract.
  • standard math Standard end-to-end deep learning (backpropagation, reconstruction or adversarial objectives) behaves as expected for this architecture.
    The method relies on routine differentiable training of neural networks; no formal guarantees are claimed.
invented entities (2)
  • Emotion Bridging Memory (EBM) bank of cross-emotion expression displacements
    purpose: Stores the expression displacement from reference emotion to audio emotion and, at inference, retrieves the matching displacement from a cross-emotion feature query to compensate for missing facial motion information.
    A learned, paper-specific construct. Its utility rests only on this paper's own experimental claims, which are not quantified in the abstract; no external falsifiable handle, such as a publicly released artifact or independent benchmark prediction, is visible.
  • Audio Emotion Enhancement module (AEE) with cross-reconstruction training
    purpose: Enhances emotion extracted from the audio so the reference image's emotion does not dominate the generated expression.
    An architectural component introduced by this paper. Its effectiveness is supported only by the authors' stated experiments, which cannot be examined here; hence no independent evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CEM-Net: Cross-Emotion Memory Network for Emotional Talking Face Generation." pith.science (2026). https://pith.science/paper/JSFIFTMF

@misc{pith2026250812368,
  author       = {Pith},
  title        = {Pith review of: CEM-Net: Cross-Emotion Memory Network for Emotional Talking Face Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JSFIFTMF}},
  note         = {Machine review of arXiv:2508.12368}
}
read the original abstract

Emotional talking face generation aims to animate a human face in given reference images and generate a talking video that matches the content and emotion of driving audio. However, existing methods neglect that reference images may have a strong emotion that conflicts with the audio emotion, leading to severe emotion inaccuracy and distorted generated results. To tackle the issue, we introduce a cross-emotion memory network(CEM-Net), designed to generate emotional talking faces aligned with the driving audio when reference images exhibit strong emotion. Specifically, an Audio Emotion Enhancement module(AEE) is first devised with the cross-reconstruction training strategy to enhance audio emotion, overcoming the disruption from reference image emotion. Secondly, since reference images cannot provide sufficient facial motion information of the speaker under audio emotion, an Emotion Bridging Memory module(EBM) is utilized to compensate for the lacked information. It brings in expression displacement from the reference image emotion to the audio emotion and stores it in the memory.Given a cross-emotion feature as a query, the matching displacement can be retrieved at inference time. Extensive experiments have demonstrated that our CEM-Net can synthesize expressive, natural and lip-synced talking face videos with better emotion accuracy.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    Dual-Anchoring Framework mitigates progress drift via structured instruction tokens and memory drift via landmark-centric retrospective prediction, yielding 15.2% success rate gain and 24.7% on long trajectories.

  2. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 conditional novelty 6.0 of 10

    Privatar partitions VR avatar reconstruction via frequency-domain decomposition, keeping sensitive components local and offloading the rest with distribution-aware minimal perturbation noise, achieving 2.37x throughpu...

  3. MORN: Metacognitive Object-Goal Regulation for Resource-Rational Long-Horizon Navigation

    cs.RO 2026-05 unverdicted novelty 5.0 of 10

    MORN augments frozen VLM-based object navigation agents with a System 2 meta-controller using Potentiality Index, Persistence Gating, and Evidence Accumulation to improve goal completion rate from 0.23 to 0.30 and red...

  4. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    Dual-Anchoring adds explicit progress tokens and retrospective landmark verification to VLN agents, cutting state drift and lifting success rate 15.2% overall with 24.7% gains on long trajectories.

Reference graph

Works this paper leans on

68 extracted references · 60 canonical work pages · cited by 2 Pith papers

  1. [1]

    !1A Qa

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

    work page

  2. [2]

    L. Yu, J. Yu, M. Li, and Q. Ling, ``Multimodal inputs driven talking face generation with spatial--temporal dependency,'' IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 1, pp. 203--216, 2020

    work page 2020

  3. [3]

    B. Chen, Z. Wang, B. Li, S. Wang, and Y. Ye, ``Compact temporal trajectory representation for talking face video compression,'' IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 11, pp. 7009--7023, 2023

    work page 2023

  4. [4]

    Z. Ye, M. Xia, R. Yi, J. Zhang, Y.-K. Lai, X. Huang, G. Zhang, and Y.-j. Liu, ``Audio-driven talking face video generation with dynamic convolution kernels,'' IEEE Transactions on Multimedia, vol. 25, pp. 2033--2046, 2022

    work page

  5. [5]

    Z. Liu, X. Liu, S. Chen, J. Liu, L. Wang, and C. Bi, ``Multimodal fusion for talking face generation utilizing speech-related facial action units,'' ACM Transactions on Multimedia Computing, Communications and Applications, vol. 20, no. 9, pp. 1--24, 2024

    work page 2024

  6. [6]

    J. Liu, X. Wang, X. Fu, Y. Chai, C. Yu, J. Dai, and J. Han, ``Osm-net: One-to-many one-shot talking head generation with spontaneous head motions,'' IEEE Transactions on Circuits and Systems for Video Technology, 2024

    work page 2024

  7. [7]

    Zhang, C

    J. Zhang, C. Liu, K. Xian, and Z. Cao, ``Hierarchical feature warping and blending for talking head animation,'' IEEE Transactions on Circuits and Systems for Video Technology, 2024

    work page 2024

  8. [8]

    M. Liu, D. Li, Y. Li, X. Song, and L. Nie, ``Audio-semantic enhanced pose-driven talking head generation,'' IEEE Transactions on Circuits and Systems for Video Technology, 2024

    work page 2024

Show all 68 references
  1. [9]

    Thies, M

    J. Thies, M. Elgharib, A. Tewari, C. Theobalt, and M. Nie ner, ``Neural voice puppetry: Audio-driven facial reenactment,'' in Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XVI 16. 1em plus 0.5em minus 0.4em Springer, ...

    2020

  2. [10]

    Zhang, Z

    Z. Zhang, Z. Hu, W. Deng, C. Fan, T. Lv, and Y. Ding, ``Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video,'' in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 3, 2023, pp. 3543--3551

    2023

  3. [11]

    H. Kim, M. Elgharib, M. Zollh \"o fer, H.-P. Seidel, T. Beeler, C. Richardt, and C. Theobalt, ``Neural style-preserving visual dubbing,'' ACM Transactions on Graphics (TOG), vol. 38, no. 6, pp. 1--13, 2019

    2019

  4. [12]

    N. M. Szajnberg, ``What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system (facs),'' 2022

    2022

  5. [13]

    Goyal, S

    S. Goyal, S. Bhagat, S. Uppal, H. Jangra, Y. Yu, Y. Yin, and R. R. Shah, ``Emotionally enhanced talking face generation,'' in Proceedings of the 1st International Workshop on Multimedia Content Generation and Evaluation: New Methods and Practice, 2023, pp. 81--90

    2023

  6. [14]

    C. Xu, S. Zhu, J. Zhu, T. Huang, J. Zhang, Y. Tai, and Y. Liu, ``Multimodal-driven talking face generation via a unified diffusion-based generator,'' arXiv preprint arXiv:2305.02594, 2023

    2023 arXiv

  7. [15]

    G. Feng, H. Cheng, Y. Li, Z. Ma, C. Li, Z. Qian, Q. Miao, and C.-M. Pun, ``Emospeaker: One-shot fine-grained emotion-controlled talking face generation,'' 2024

    2024

  8. [16]

    Gururani, A

    S. Gururani, A. Mallya, T.-C. Wang, R. Valle, and M.-Y. Liu, ``Space: Speech-driven portrait animation with controllable expression,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 20\,914--20\,923

    2023

  9. [17]

    Sheng, L

    Z. Sheng, L. Nie, M. Zhang, X. Chang, and Y. Yan, ``Stochastic latent talking face generation toward emotional expressions and head poses,'' IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 4, pp. 2734--2748, 2023

    2023

  10. [18]

    Agarwal, R

    M. Agarwal, R. Mukhopadhyay, V. P. Namboodiri, and C. Jawahar, ``Audio-visual face reenactment,'' in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 5178--5187

    2023

  11. [19]

    X. Ji, H. Zhou, K. Wang, W. Wu, C. C. Loy, X. Cao, and F. Xu, ``Audio-driven emotional video portraits,'' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 14\,080--14\,089

    2021

  12. [20]

    S. Tan, B. Ji, and Y. Pan, ``Emmn: Emotional motion memory network for audio-driven emotional talking face generation,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 22\,146--22\,156

    2023

  13. [21]

    Y. Sun, W. Chu, H. Zhou, K. Wang, and H. Koike, ``Avi-talking: Learning audio-visual instructions for expressive 3d talking face generation,'' arXiv preprint arXiv:2402.16124, 2024

    2024 arXiv

  14. [22]

    S. E. Eskimez, Y. Zhang, and Z. Duan, ``Speech driven talking face generation from a single image and an emotion condition,'' IEEE Transactions on Multimedia, vol. 24, pp. 3480--3490, 2021

    2021

  15. [23]

    Liang, Y

    B. Liang, Y. Pan, Z. Guo, H. Zhou, Z. Hong, X. Han, J. Han, J. Liu, E. Ding, and J. Wang, ``Expressive talking head generation with granular audio-visual control,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3387--3396

    2022

  16. [24]

    Y. Ma, S. Wang, Y. Ding, B. Ma, T. Lv, C. Fan, Z. Hu, Z. Deng, and X. Yu, ``Talkclip: Talking head generation with text-guided expressive speaking styles,'' arXiv preprint arXiv:2304.00334, 2023

    2023 arXiv

  17. [25]

    S. I. Serengil and A. Ozpinar, ``Hyperextended lightface: A facial attribute analysis framework,'' in 2021 International Conference on Engineering and Emerging Technologies (ICEET). 1em plus 0.5em minus 0.4em IEEE, 2021, pp. 1--4. [Online]. Available: https://doi.org/10.1109/I...

    2021

  18. [26]

    K. Wang, Q. Wu, L. Song, Z. Yang, W. Wu, C. Qian, R. He, Y. Qiao, and C. C. Loy, ``Mead: A large-scale audio-visual dataset for emotional talking-face generation,'' in European Conference on Computer Vision. 1em plus 0.5em minus 0.4em Springer, 2020, pp. 700--717

    2020

  19. [27]

    Son Chung, A

    J. Son Chung, A. Senior, O. Vinyals, and A. Zisserman, ``Lip reading sentences in the wild,'' in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 6447--6456

    2017

  20. [28]

    R. P. Gabriel, ``Worse is better,'' URL http://www. dreamsongs. com/WorseIsBetter. html, 1990

    1990

  21. [29]

    Y. Shen, J. Gu, X. Tang, and B. Zhou, ``Interpreting the latent space of gans for semantic face editing,'' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9243--9252

    2020

  22. [30]

    Azari and A

    B. Azari and A. Lim, ``Emostyle: One-shot facial expression editing using continuous emotion parameters,'' in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 6385--6394

    2024

  23. [31]

    Nitzan, K

    Y. Nitzan, K. Aberman, Q. He, O. Liba, M. Yarom, Y. Gandelsman, I. Mosseri, Y. Pritch, and D. Cohen-Or, ``Mystyle: A personalized generative prior,'' ACM Transactions on Graphics (TOG), vol. 41, no. 6, pp. 1--10, 2022

    2022

  24. [32]

    J. Deng, J. Guo, N. Xue, and S. Zafeiriou, ``Arcface: Additive angular margin loss for deep face recognition,'' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4690--4699

    2019

  25. [33]

    Siarohin, S

    A. Siarohin, S. Lathuili \`e re, S. Tulyakov, E. Ricci, and N. Sebe, ``Animating arbitrary objects via deep motion transfer,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2377--2386

    2019

  26. [34]

    Zhang, L

    W. Zhang, L. Yang, S. Geng, and S. Hong, ``Self-supervised time series representation learning via cross reconstruction transformer,'' arXiv preprint arXiv:2205.09928, 2022

    2022 arXiv

  27. [35]

    Weston, S

    J. Weston, S. Chopra, and A. Bordes, ``Memory networks,'' arXiv preprint arXiv:1410.3916, 2014

    2014 arXiv

  28. [36]

    Y. Guo, K. Chen, S. Liang, Y.-J. Liu, H. Bao, and J. Zhang, ``Ad-nerf: Audio driven neural radiance fields for talking head synthesis,'' in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 5784--5794

    2021

  29. [37]

    Zhang, Y

    C. Zhang, Y. Zhao, Y. Huang, M. Zeng, S. Ni, M. Budagavi, and X. Guo, ``Facial: Synthesizing dynamic talking face with implicit attribute learning,'' in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 3867--3876

    2021

  30. [38]

    Hong and D

    F.-T. Hong and D. Xu, ``Implicit identity representation conditioned memory compensation network for talking head video generation,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 23\,062--23\,072

    2023

  31. [39]

    Zhong, C

    W. Zhong, C. Fang, Y. Cai, P. Wei, G. Zhao, L. Lin, and G. Li, ``Identity-preserving talking face generation with landmark and appearance priors,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9729--9738

    2023

  32. [40]

    Y. Liu, L. Lin, F. Yu, C. Zhou, and Y. Li, ``Moda: Mapping-once audio-driven portrait animation with dual attentions,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 23\,020--23\,029

    2023

  33. [41]

    X. Ji, H. Zhou, K. Wang, Q. Wu, W. Wu, F. Xu, and X. Cao, ``Eamm: One-shot emotional talking face via audio-based emotion-aware motion model,'' in ACM SIGGRAPH 2022 Conference Proceedings, 2022, pp. 1--10

    2022

  34. [42]

    D. Wang, Y. Deng, Z. Yin, H.-Y. Shum, and B. Wang, ``Progressive disentangled representation learning for fine-grained controllable talking head synthesis,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17\,979--17\,989

    2023

  35. [43]

    Zhang, X

    W. Zhang, X. Cun, X. Wang, Y. Zhang, X. Shen, Y. Guo, Y. Shan, and F. Wang, ``Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2023

  36. [44]

    S. Yoo, H. Bahng, S. Chung, J. Lee, J. Chang, and J. Choo, ``Coloring with limited data: Few-shot colorization via memory augmented networks,'' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 11\,283--11\,292

    2019

  37. [45]

    Huang, A

    H. Huang, A. Yu, and R. He, ``Memory oriented transfer learning for semi-supervised image deraining,'' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 7732--7741

    2021

  38. [46]

    G. Sun, Y. Hua, G. Hu, and N. Robertson, ``Mamba: Multi-level aggregation via memory bank for video object detection,'' in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 3, 2021, pp. 2620--2627

    2021

  39. [47]

    Fei, ``Memory-augmented image captioning,'' in Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Z. Fei, ``Memory-augmented image captioning,'' in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 2, 2021, pp. 1317--1324

    2021

  40. [48]

    S. J. Park, M. Kim, J. Hong, J. Choi, and Y. M. Ro, ``Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory,'' in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 2, 2022, pp. 2062--2070

    2022

  41. [49]

    D. Yi, Z. Lei, S. Liao, and S. Z. Li, ``Learning face representation from scratch,'' arXiv preprint arXiv:1411.7923, 2014

    2014 arXiv

  42. [50]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, . Kaiser, and I. Polosukhin, ``Attention is all you need,'' Advances in neural information processing systems, vol. 30, 2017

    2017

  43. [51]

    Aberman, R

    K. Aberman, R. Wu, D. Lischinski, B. Chen, and D. Cohen-Or, ``Learning character-agnostic motion for motion retargeting in 2d,'' arXiv preprint arXiv:1905.01680, 2019

    1905 arXiv

  44. [52]

    D. J. Berndt and J. Clifford, ``Using dynamic time warping to find patterns in time series,'' in Proceedings of the 3rd international conference on knowledge discovery and data mining, 1994, pp. 359--370

    1994

  45. [53]

    Johnson, A

    J. Johnson, A. Alahi, and L. Fei-Fei, ``Perceptual losses for real-time style transfer and super-resolution,'' in Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. 1em plus 0.5em minus 0.4em Springer...

    2016

  46. [54]

    Wang, M.-Y

    T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, ``High-resolution image synthesis and semantic manipulation with conditional gans,'' in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8798--8807

    2018

  47. [55]

    Prajwal, R

    K. Prajwal, R. Mukhopadhyay, V. P. Namboodiri, and C. Jawahar, ``A lip sync expert is all you need for speech to lip generation in the wild,'' in Proceedings of the 28th ACM international conference on multimedia, 2020, pp. 484--492

    2020

  48. [56]

    Y. Gan, Z. Yang, X. Yue, L. Sun, and Y. Yang, ``Efficient emotional adaptation for audio-driven talking-head generation,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 22\,634--22\,645

    2023

  49. [57]

    Afouras, J

    T. Afouras, J. S. Chung, A. Senior, O. Vinyals, and A. Zisserman, ``Deep audio-visual speech recognition,'' IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 12, pp. 8717--8727, 2018

    2018

  50. [58]

    Lugaresi, J

    C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee et al., ``Mediapipe: A framework for building perception pipelines,'' arXiv preprint arXiv:1906.08172, 2019

    1906 arXiv

  51. [59]

    Guo, Z.-N

    M.-H. Guo, Z.-N. Liu, T.-J. Mu, and S.-M. Hu, ``Beyond self-attention: External attention using two linear layers for visual tasks,'' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 5, pp. 5436--5447, 2022

    2022

  52. [60]

    D. P. Kingma and J. Ba, ``Adam: A method for stochastic optimization,'' arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  53. [61]

    L. Chen, R. K. Maddox, Z. Duan, and C. Xu, ``Hierarchical cross-modal talking face generation with dynamic pixel-wise loss,'' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7832--7841

    2019

  54. [62]

    J. S. Chung and A. Zisserman, ``Out of time: automated lip sync in the wild,'' in Computer Vision--ACCV 2016 Workshops: ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13. 1em plus 0.5em minus 0.4em Springer, 2017, pp. 251--263

    2016

  55. [63]

    Suwajanakorn, S

    S. Suwajanakorn, S. M. Seitz, and I. Kemelmacher-Shlizerman, ``Synthesizing obama: learning lip sync from audio,'' ACM Transactions on Graphics (ToG), vol. 36, no. 4, pp. 1--13, 2017

    2017

  56. [64]

    G. Kim, T. Kwon, and J. C. Ye, ``Diffusionclip: Text-guided diffusion models for robust image manipulation,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 2426--2435

    2022

  57. [65]

    Alaluf, O

    Y. Alaluf, O. Tov, R. Mokady, R. Gal, and A. Bermano, ``Hyperstyle: Stylegan inversion with hypernetworks for real image editing,'' in Proceedings of the IEEE/CVF conference on computer Vision and pattern recognition, 2022, pp. 18\,511--18\,521

    2022

  58. [66]

    Roich, R

    D. Roich, R. Mokady, A. H. Bermano, and D. Cohen-Or, ``Pivotal tuning for latent-based editing of real images,'' ACM Transactions on graphics (TOG), vol. 42, no. 1, pp. 1--13, 2022

    2022

  59. [67]

    Y. Deng, J. Yang, J. Xiang, and X. Tong, ``Gram: Generative radiance manifolds for 3d-aware image generation,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10\,673--10\,683

    2022

  60. [68]

    T. Wang, Y. Zhang, Y. Fan, J. Wang, and Q. Chen, ``High-fidelity gan inversion for image attribute editing,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 11\,379--11\,388

    2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.