REVIEW 3 major objections 4 minor 42 references
One-shot Face Reenactment
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper claims that one target photo per person is enough to train a single model that transfers pose and expression while preserving identity.
desk verdict A useful one-shot reenactment architecture whose headline training claim is not actually backed by the data description. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the disentangle-and-compose architecture: a frozen shape encoder $E$ turns the source image into colored face-parsing maps plus gaze channels, an appearance auto-encoder $F$ turns the target image into multi-scale appearance feature maps, and the semantically adaptive decoder $D$, built from SPADE residual blocks, composes the two. SPADE, spatially-adaptive normalization, predicts per-pixel normalization parameters from the face-parsing map instead of using uniform normalization. Multi-scale appearance features from $F$ are concatenated into $D$ to retain texture and identity, and the FusionNet learns a mask to fuse the decoder's output with a classical warping result. The machinery's role is to let a single generic model handle a new identity at test time without per-person optimization.
What would settle it
Look up the identity labels of the 20k CelebA-HQ subset used for training: if the number of unique identities is clearly smaller than 20,000, the one-shot training claim is violated. A direct check is to retrain the same architecture on a verified one-image-per-identity subset and compare identity-preservation accuracy, AU consistency, and pose error with the paper's Table 1; a sizeable drop would show the reported results depended on multiple images per identity.
Extended reading notes
Core claim
The paper's central claim is that one-shot face reenactment is achievable through disentangle-and-compose: a pretrained shape encoder maps the source face into face-parsing and gaze heatmaps, an appearance auto-encoder maps the target face into multi-scale appearance features, and a semantically adaptive decoder composed of SPADE residual blocks fuses the two to generate the reenacted face. Training couples an appearance-reconstruction branch with the reenactment branch, and an identity-feature loss supervises preservation of the target identity. In experiments on CelebA-HQ, FFHQ, and RAF-DB targets, the same model transfers expressions and head poses under large changes, and with one target image it matches or approaches the quality of ReenactGAN, which requires abundant target-specific training data.
Load-bearing premise
The central comparison assumes that the 20,000-image CelebA-HQ training subset contains exactly one image per identity, so the model is genuinely trained one-shot rather than on repeated images of the same people.
Editorial extensions
If this is right
- With one photo of a user, a reenacted video sequence can be generated without collecting a target video or training a per-person model.
- The same trained model generalizes to target faces from different sources, including in-the-wild images, with only a small drop in AU and pose consistency.
- When more than one image per target is available, performance improves: identity preservation rises from 97.2 percent one-shot to 99.3 percent at 3-shot and 99.4 percent at 5-shot.
- The multi-scale appearance-feature concatenation is a key component: without it, identity preservation drops from 89.1 percent to 77.7 percent at the chosen loss weight.
- FusionNet's warping blend pushes identity preservation from about 90 percent to about 98 percent, at the cost of a small decrease in action-unit consistency.
Reading between the lines
- If the one-shot training assumption holds, the practical bottleneck of face reenactment moves from collecting per-person data and training to a single offline training run, which is what makes the competitive comparison with ReenactGAN meaningful for real deployments.
- The disentangle-and-compose recipe is not inherently face-specific; the same appearance-encoder plus shape-conditioned decoder could be applied to one-shot pose transfer for bodies or animals, given a suitable shape representation.
- Training on a verified identity-unique subset and comparing the resulting AU, pose, and identity numbers with Table 1 would quantify how much of the reported performance depends on the one-shot data assumption.
- The FusionNet mask idea suggests a general hybrid: use learned synthesis for large pose changes and classical warping for small changes, a combination that could transfer to other image-synthesis tasks.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a one-shot face reenactment method. Given a source image that provides pose and expression and a target image that provides appearance, the method encodes the target with an appearance auto-encoder F and the source with a frozen shape encoder E, then a SPADE-based decoder D composes the two codes and multi-scale appearance features to generate the reenacted face. A FusionNet blends the decoder output with a classical warping result to improve texture and mustache. Experiments compare against single-image generators (GANimation, PG2, gauGAN, VU-Net) and against the target-specific ReenactGAN, with quantitative AU, pose, and identity metrics on same-source, cross-source, and in-the-wild test sets. The paper claims that training with one target image per subject yields results competitive with target-specific methods.
Significance. The contribution addresses a real practical gap, and the disentangle-and-compose formulation with SPADE blocks and FusionNet is reasonable. The public release of code and models is a strength, and the evaluation spans three test distributions with multiple metrics. If the one-shot training claim is verified, this is a useful advance. However, the current manuscript does not establish the central one-shot training claim because the training-data description is inconsistent with a one-image-per-identity setup; the hyperparameter selection also biases the reported quantitative gains. These issues are fixable within the manuscript's scope by documenting the exact identity split and re-running ablations on a validation set.
major comments (3)
- [Section 4, Training Set; Section 1; Abstract] The paper claims that 'only a single shot is available for each person no matter during training or testing' (Section 1) and that the model is 'trained with only one target image per subject' (Abstract). The only training-set description, however, is 'a subset of the CelebA-HQ with 20k images' (Section 4), with no statement that identities are unique. This is not a cosmetic omission: CelebA-HQ is derived from CelebA, whose identity-annotated set contains about 10,177 identities, so any 20k-image subset necessarily contains multiple images for many identities unless external identity labels are used to restrict sampling. If the model saw multiple images of the same identity during training, it is not trained one-shot per identity and can learn identity-specific appearance variation across expressions and poses, which would explain part of the reported advantage over target-specific methods in Figure 6. The authors must state the exact identity split, or retrain with exactly one image per identity and re-run Table 1 and Figure 6; the released code makes this verifiable.
- [Section 4.2, Table 1] The central comparison against target-specific methods is only qualitative. Section 4.2 presents Figure 6 as evidence that the one-shot method is 'competitive' with ReenactGAN, but Table 1 contains no ReenactGAN row, and no AU/pose/identity numbers are reported for ReenactGAN under full or limited data. Without a quantitative head-to-head under the same metrics and test protocol, the claim that one-shot training is competitive with a set of target images is not established. Please add the missing quantitative comparison or explicitly restrict the claim.
- [Section 4.3, Ablation Study and Eq. (1)] The loss weights in Eq. (1) are selected through the ablation in Table 2, which reports identity-preserving accuracy on the same-source test data. This is model selection on the evaluation set, so the reported identity numbers in Table 1 are optimistically biased. A validation split should be used for choosing lambda and the alpha weights, and the final numbers should be reported on held-out test sets only. The identity-preservation loss also uses a verification network trained on CelebA (Section 3.3), the same source as the training data, which the authors should discuss as a possible source of bias for the CelebA-HQ test partition.
minor comments (4)
- [Section 2] In the first paragraph, 'face difficulty in handling' should be 'have difficulty in handling'.
- [Figure 6 caption] The phrase 'Contrast to that' should be 'In contrast'.
- [Section 4, Training Set] The preprocessing step applies Faster R-CNN detection on CelebA-HQ, which is already aligned and cropped; the authors should clarify why detection is needed and what alignment the detected boxes provide.
- [Table 1] The rows 'Ours w/o Fusion' and 'Ours' would be clearer if the caption explained that the latter includes FusionNet, and if the FusionNet contribution to each metric were discussed in the text.
Circularity Check
No derivation-level circularity, but the reported identity-preservation score is partly a fitted test-set criterion: λ and the concatenation design are selected by ablating the same identity metric on the same test data that Table 1 reports.
-
fitted input called prediction
[Section 4, Ablation Study (Table 2); Section 3.3 Eq. (1)]
"We investigate the contribution of the concatenation between the appearance decoder and spatially adaptive decoder, and the influence of λ, the appearance reconstruction loss weight. Table 2 provides the ablation study results in terms of identity preserving metric values."
The final configuration fixes λ=25 in Eq. (1), and Table 2 identifies λ=25 (together with concatenation) as the best setting on 'the same source test data'. Table 1 then reports identity preservation on that same test data as evidence for the method. Thus the headline identity-preserving 'prediction' is not an independent evaluation: the hyperparameter and architecture choice were selected by maximizing that same metric on that same split. The pose and AU metrics are not selected this way, so the circularity is partial and confined to the identity-preservation claim.
full rationale
This is an empirical system paper rather than a derivation chain: the reenactment output D(F(xt), E(xs)) is generated by a learned decoder and is not definitionally equal to any input, and the losses in Eq. (1) are standard reconstruction, perceptual, GAN, and identity-matching objectives rather than hidden restatements of the evaluation claims. Pretrained components (boundary encoder from ReenactGAN/WFLW, gaze model from EOTT, SPADE blocks from prior work) are external or independently trained; the self-citation to ReenactGAN supplies a component and a comparison protocol, not an unverified uniqueness theorem, so it is not load-bearing. The one real circular-adjacent step is test-set hyperparameter selection: Section 4's ablation is run on 'the same source test data' and is used to choose λ=25 and the concatenation design, after which Table 1 reports identity preservation on that same data. This inflates the identity metric and makes that particular 'prediction' partly fitted, but the central qualitative reenactment capability and the AU/pose comparisons retain independent content. The separate concern that a 20k-image CelebA-HQ subset cannot contain 20k unique identities is a data-validity and claim-support issue, not a by-construction circularity; it would affect whether the model was truly trained one-shot, but it does not make any result equivalent to its input by definition.
Assumptions & free parameters
free parameters (3)
- lambda =
25
- alpha_r =
25
- alpha_p, alpha_g, alpha_i =
1 each
assumptions (4)
- domain assumption Face parsing maps from the pretrained boundary encoder (WFLW) plus gaze channels (EOTT) provide a sufficient shape representation for reenactment.
- ad hoc to paper The training subset of CelebA-HQ (20k images) contains one image per identity, realizing the one-shot training assumption.
- domain assumption The identity verification model trained on CelebA provides a valid supervision signal for identity preservation.
- domain assumption SPADE normalization preserves spatial information as claimed in the cited work [23].
Cite this review
Pith. "Pith review of One-shot Face Reenactment." pith.science (2026). https://pith.science/paper/23GW4MT3
@misc{pith2026190803251,
author = {Pith},
title = {Pith review of: One-shot Face Reenactment},
year = {2026},
howpublished = {\url{https://pith.science/paper/23GW4MT3}},
note = {Machine review of arXiv:1908.03251}
}
read the original abstract
To enable realistic shape (e.g. pose and expression) transfer, existing face reenactment methods rely on a set of target faces for learning subject-specific traits. However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable. In this work, we bridge this gap by proposing a novel one-shot face reenactment learning framework. Our key insight is that the one-shot learner should be able to disentangle and compose appearance and shape information for effective modeling. Specifically, the target face appearance and the source face shape are first projected into latent spaces with their corresponding encoders. Then these two latent spaces are associated by learning a shared decoder that aggregates multi-level features to produce the final reenactment results. To further improve the synthesizing quality on mustache and hair regions, we additionally propose FusionNet which combines the strengths of our learned decoder and the traditional warping method. Extensive experiments show that our one-shot face reenactment system achieves superior transfer fidelity as well as identity preserving capability than alternatives. More remarkably, our approach trained with only one target image per subject achieves competitive results to those using a set of target images, demonstrating the practical merit of this work. Code, models and an additional set of reenacted faces have been publicly released at the project page.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
https://github.com/alyssaq/face_morpher
-
[2]
Syn- thesizing images of humans in unseen poses
Guha Balakrishnan, Amy Zhao, Adrian V Dalca, Fredo Durand, and John Guttag. Syn- thesizing images of humans in unseen poses. In CVPR, 2018
work page 2018
-
[3]
Caroline Chan, Shiry Ginosar, Tinghui Zhou, and Alexei A Efros. Everybody dance now. arXiv preprint arXiv:1808.07371, 2018
arXiv 2018
-
[4]
Pairedcyclegan: Asym- metric style transfer for applying and removing makeup
Huiwen Chang, Jingwan Lu, Fisher Yu, and Adam Finkelstein. Pairedcyclegan: Asym- metric style transfer for applying and removing makeup. In CVPR, 2018
work page 2018
-
[5]
Facelet-bank for fast portrait manipulation
Ying-Cong Chen, Huaijia Lin, Michelle Shu, Ruiyu Li, Xin Tao, Xiaoyong Shen, Yan- gang Ye, and Jiaya Jia. Facelet-bank for fast portrait manipulation. In CVPR, 2018
work page 2018
-
[6]
3d-model-based face replacement in video
Yi-Ting Cheng, Virginia Tzeng, Yu Liang, Chuan-Chang Wang, Bing-Yu Chen, Yung- Yu Chuang, and Ming Ouhyoung. 3d-model-based face replacement in video. In SIG- GRAPH’09: Posters. ACM, 2009
work page 2009
-
[7]
Stargan: Unified generative adversarial networks for multi-domain image-to- image translation
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. Stargan: Unified generative adversarial networks for multi-domain image-to- image translation. In CVPR, 2018
work page 2018
-
[8]
Johnson, Daniel Vlasic, Wojciech Matusik, and Hanspeter Pfister
Kevin Dale, Kalyan Sunkavalli, Micah K. Johnson, Daniel Vlasic, Wojciech Matusik, and Hanspeter Pfister. Video face replacement. ACM Transactions on Graphics (TOG), 30(6):1–10, 2011
work page 2011
Show all 42 references
-
[9]
A variational u-net for conditional appearance and shape generation
Patrick Esser, Ekaterina Sutter, and Björn Ommer. A variational u-net for conditional appearance and shape generation. In CVPR, 2018
2018
-
[10]
Vdub: Modifying face video of actors for plausible visual alignment to a dubbed audio track
Pablo Garrido, Levi Valgaerts, Hamid Sarmadi, Ingmar Steiner, Kiran Varanasi, Patrick Perez, and Christian Theobalt. Vdub: Modifying face video of actors for plausible visual alignment to a dubbed audio track. In Computer graphics forum , volume 34, pages 193–204. Wiley Online...
2015
-
[11]
Cyclegan face-off
Xiaohan Jin, Ye Qi, and Shangxuan Wu. Cyclegan face-off. arXiv preprint arXiv:1712.03451, 2017
2017 arXiv
-
[12]
Progressive growing of gans for improved quality, stability, and variation
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 , 2017
2017 arXiv
-
[13]
A style-based generator architecture for generative adversarial networks
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR, 2019
2019
-
[14]
Deep video portraits
Hyeongwoo Kim, Pablo Carrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Niessner, Patrick Pérez, Christian Richardt, Michael Zollhöfer, and Christian Theobalt. Deep video portraits. ACM Transactions on Graphics (TOG), 37(4):163, 2018
2018
-
[15]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 12 ZHANG ET AL.: ONE-SHOT FACE REENACTMENT
2014 arXiv
-
[16]
Reliable crowdsourcing and deep locality- preserving learning for expression recognition in the wild
Shan Li, Weihong Deng, and JunPing Du. Reliable crowdsourcing and deep locality- preserving learning for expression recognition in the wild. In CVPR, 2017
2017
-
[17]
Dense intrinsic appearance flow for human pose transfer
Yining Li, Chen Huang, and Chen Change Loy. Dense intrinsic appearance flow for human pose transfer. In CVPR, 2019
2019
-
[18]
Deep learning face attributes in the wild
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015
2015
-
[19]
Yeh, Xiaoou Tang, Yiming Liu, and Aseem Agarwala
Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, and Aseem Agarwala. Video frame synthesis using deep voxel flow. In ICCV, 2017
2017
-
[20]
Pose guided person image generation
Liqian Ma, Xu Jia, Qianru Sun, Bernt Schiele, Tinne Tuytelaars, and Luc Van Gool. Pose guided person image generation. In NIPS, 2017
2017
-
[21]
Disentangled person image generation
Liqian Ma, Qianru Sun, Stamatios Georgoulis, Luc Van Gool, Bernt Schiele, and Mario Fritz. Disentangled person image generation. In CVPR, 2018
2018
-
[22]
The eye of the typer: a benchmark and analysis of gaze behavior during typing
Alexandra Papoutsaki, Aaron Gokaslan, James Tompkin, Yuze He, and Jeff Huang. The eye of the typer: a benchmark and analysis of gaze behavior during typing. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications , page 16. ACM, 2018
2018
-
[23]
Semantic image synthesis with spatially-adaptive normalization
Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. Semantic image synthesis with spatially-adaptive normalization. In CVPR, 2019
2019
-
[24]
Ganimation: Anatomically-aware facial animation from a single im- age
Albert Pumarola, Antonio Agudo, Aleix M Martinez, Alberto Sanfeliu, and Francesc Moreno-Noguer. Ganimation: Anatomically-aware facial animation from a single im- age. In ECCV, 2018
2018
-
[25]
Faster r-cnn: Towards real- time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real- time object detection with region proposal networks. In NIPS, 2015
2015
-
[26]
Real-time expression transfer for facial reenactment
Justus Thies, Michael Zollhöfer, Matthias Nießner, Levi Valgaerts, Marc Stamminger, and Christian Theobalt. Real-time expression transfer for facial reenactment. ACM Transactions on Graphics (TOG), 34(6):183–1, 2015
2015
-
[27]
Face2face: Real-time face capture and reenactment of rgb videos
Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. Face2face: Real-time face capture and reenactment of rgb videos. In CVPR, 2016
2016
-
[28]
Disentangled representation learning gan for pose-invariant face recognition
Luan Tran, Xi Yin, and Xiaoming Liu. Disentangled representation learning gan for pose-invariant face recognition. In CVPR, 2017
2017
-
[29]
Face transfer with multilinear models
Daniel Vlasic, Matthew Brand, Hanspeter Pfister, and Jovan Popovi ´c. Face transfer with multilinear models. ACM Transactions on Graphics (TOG), 24(3):426–433, 2005
2005
-
[30]
Residual attention network for image classification
Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xi- aogang Wang, and Xiaoou Tang. Residual attention network for image classification. In CVPR, 2017
2017
-
[31]
High-resolution image synthesis and semantic manipulation with condi- tional gans
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image synthesis and semantic manipulation with condi- tional gans. In CVPR, 2018. ZHANG ET AL.: ONE-SHOT FACE REENACTMENT 13
2018
-
[32]
Recovering realistic texture in image super-resolution by deep spatial feature transform
Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Recovering realistic texture in image super-resolution by deep spatial feature transform. In CVPR, 2018
2018
-
[33]
X2face: A network for con- trolling face generation using images, audio, and pose codes
Olivia Wiles, A Sophia Koepke, and Andrew Zisserman. X2face: A network for con- trolling face generation using images, audio, and pose codes. In ECCV, 2018
2018
-
[34]
Look at boundary: A boundary-aware face alignment algorithm
Wayne Wu, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, and Qiang Zhou. Look at boundary: A boundary-aware face alignment algorithm. In CVPR, 2018
2018
-
[35]
Reenactgan: Learning to reenact faces via boundary transfer
Wayne Wu, Yunxuan Zhang, Cheng Li, Chen Qian, and Chen Change Loy. Reenactgan: Learning to reenact faces via boundary transfer. In ECCV, 2018
2018
-
[36]
Face transfer with generative adversarial network
Runze Xu, Zhiming Zhou, Weinan Zhang, and Yong Yu. Face transfer with generative adversarial network. arXiv preprint arXiv:1710.06090, 2017
2017 arXiv
-
[37]
Learning face age progres- sion: A pyramid architecture of gans
Hongyu Yang, Di Huang, Yunhong Wang, and Anil K Jain. Learning face age progres- sion: A pyramid architecture of gans. In CVPR, 2018
2018
-
[38]
Semantic facial expression editing using autoencoded flow
Raymond Yeh, Ziwei Liu, Dan B Goldman, and Aseem Agarwala. Semantic facial expression editing using autoencoded flow. arXiv preprint arXiv:1611.09961, 2016
2016 arXiv
-
[39]
Instance level facial attributes transfer with geometry-aware flow
Weidong Yin, Ziwei Liu, and Chen Change Loy. Instance level facial attributes transfer with geometry-aware flow. In AAAI, 2019
2019
-
[40]
Joint pose and ex- pression modeling for facial expression recognition
Feifei Zhang, Tianzhu Zhang, Qirong Mao, and Changsheng Xu. Joint pose and ex- pression modeling for facial expression recognition. In CVPR, 2018
2018
-
[41]
Talking face generation by adversarially disentangled audio-visual representation
Hang Zhou, Yu Liu, Ziwei Liu, Ping Luo, and Xiaogang Wang. Talking face generation by adversarially disentangled audio-visual representation. In AAAI, 2019
2019
-
[42]
Unpaired image-to- image translation using cycle-consistent adversarial networks
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to- image translation using cycle-consistent adversarial networks. In ICCV, 2017
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.