A conversation avatar that maps a speaker's facial actions and head pose to a listener's reactions, maps text to speaking expressions, and renders the results as realistic face images using Pix2PixHD.
A Neural Virtual Anchor Synthesizer based on Seq2Seq and GAN Models
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
This paper presents a novel framework to generate realistic face video of an anchor, who is reading certain news. This task is also known as Virtual Anchor. Given some paragraphs of words, we first utilize a pretrained Word2Vec model to embed each word into a vector; then we utilize a Seq2Seq-based model to translate these word embeddings into action units and head poses of the target anchor; these action units and head poses will be concatenated with facial landmarks as well as the former $n$ synthesized frames, and the concatenation serves as input of a Pix2PixHD-based model to synthesize realistic facial images for the virtual anchor. The experimental results demonstrate our framework is feasible for the synthesis of virtual anchor.
fields
cs.CV 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
A Realistic Face-to-Face Conversation System based on Deep Neural Networks
A conversation avatar that maps a speaker's facial actions and head pose to a listener's reactions, maps text to speaking expressions, and renders the results as realistic face images using Pix2PixHD.