REVIEW 4 major objections 5 minor 35 references
SliderGAN: Synthesizing Expressive Face Images by Sliding 3D Blendshape Parameters
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read SliderGAN is a generative model that edits a face photo by sliding continuous 3D blendshape parameters for expression and speech.
desk verdict A genuinely new conditioning signal for face editing, but the evaluation leans heavily on the same 3DMM fitting used to create the pseudo-labels; still worth refereeing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the linear 3D blendshape model $S(\mathbf{p}) = \bar{s} + U\mathbf{p}$, whose sparse localized deformation components were learned from 4DFAB scans for expression and from LRW-3D audio alignment for speech. SliderGAN uses pseudo-annotations from 3DMM fitting of this model to label 250,000 Emotionet images, and renders 200,000 synthetic image pairs with known parameter values to supervise the generator. The mechanism that carries the argument is the continuous, interpretable parameter vector itself: it is the conditioning signal that drives the attention-based generator, the regression target for the relativistic discriminator, and the interpolation basis for expression transfer.
What would settle it
Render a set of faces from 4D scans with known blendshape coefficients, slide one coefficient of SliderGAN from -1 to 1 on those input images, then refit the output meshes with an independent 3D face reconstruction system that was not used to generate the pseudo-annotations; if the recovered coefficients do not move monotonically with the slider or if identity parameters shift substantially, the claimed continuous slider-to-expression mapping is not established.
Extended reading notes
Core claim
SliderGAN establishes that a GAN generator, conditioned on a normalized vector of 3D blendshape parameters, can translate an input face image into an output that exhibits the facial motion instantiated by that parameter vector in the blendshape model $S_{\text{exp}}(\mathbf{p}) = \bar{s} + U_{\text{exp}}\mathbf{p}$. The generator outputs a deformation mask and a deformation image that are combined with the original image, and it is trained with adversarial, expression-regression, reconstruction, image-pair, identity, and attention-sparsity losses. A relativistic average discriminator both judges realism and regresses the expression parameters, which doubles as a by-product network for recovering blendshape parameters from images. The paper shows this works for 30 expression components from 4DFAB and 10 speech components from LRW-3D, enabling single- and multi-parameter editing, expression transfer and interpolation, neutralization, and speech animation, and reports that SliderGAN trained with the relativistic discriminator yields lower Image Euclidean Distance than GANimation on 4DFAB and LRW benchmarks.
Load-bearing premise
The load-bearing premise is that the 3DMM fitting that produces the pseudo-annotations, and the audio alignment that produces speech parameters, recover the expression separably from identity, texture, pose, and camera settings, so that the training labels actually encode the facial motion the sliders claim to control.
Editorial extensions
If this is right
- Expression editing becomes continuous: sliding any single blendshape parameter between -1 and 1 yields smooth changes in the image, with the zero vector producing a neutral expression.
- Expression transfer and interpolation between two images can be done without 3DMM fitting at test time, because the discriminator regresses the blendshape parameters of any target image.
- The same architecture handles speech: conditioning on LRW-3D speech blendshape parameters animates lip and mouth motion from a single image, with lower IED than AU-based GANimation.
- The by-product regression network recovers blendshape parameters from images with higher consistency than OpenFace AU detection, making blendshape coding a more reliable control signal than action units for this task.
- Relativistic GAN training improves texture detail over WGAN-GP in this image-to-image translation setting.
Reading between the lines
- Because the control signal is a physical face model rather than categorical labels, the same architecture could in principle be steered by any 3DMM parameterization (age, weight, identity) or by other dense physical models, such as body pose or hand articulation, replacing discrete domain labels with continuous physics-driven sliders.
- The reliance on a single fitting method for pseudo-annotations suggests a direct test: re-annotate the same training images with an independent 3D face fitter or with manual 4D scan alignment; if SliderGAN's image quality and expression accuracy degrade, the method's success is partly an artifact of the annotation procedure rather than a general property of blendshape conditioning.
- The reported identity and speech results imply a practical recipe for talking-head animation from a single image; combining SliderGAN with audio-to-blendshape predictors would produce video avatars whose speech motion is controlled by a compact, interpretable parameter stream rather than by per-frame image regression.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SliderGAN, an image-to-image translation method that edits a face image according to continuous values of a 3D statistical blendshape model of facial expression and speech. The generator is conditioned on a vector of blendshape parameters, and a relativistic discriminator with an auxiliary regression head provides a parameter-consistency signal. Training is semi-supervised: it uses unpaired images whose blendshape parameters are obtained by 3DMM fitting, plus synthetic image pairs rendered from fitted identity/camera models with known random expression values. The authors report qualitative results for single- and multi-parameter editing, expression transfer/interpolation, discrete expression synthesis, neutralization, and combined expression-speech animation, along with quantitative comparisons to GANimation using Image Euclidean Distance on a rendered 4DFAB dataset and on LRW frames, expression recognition accuracy, and parameter-regression consistency.
Significance. If the quantitative claims hold, SliderGAN is a meaningful advance over AU- and discrete-expression-driven face editing: it provides a continuous, physically interpretable control space shared with 3D face models, extends naturally to speech-driven animation, and avoids manual annotation. The introduction of a rendered 4DFAB benchmark with ground-truth blendshape parameters is a useful resource for the community. The paper also demonstrates a concrete benefit of the relativistic discriminator in this task, and the ablation study gives some evidence about which losses matter. However, the load-bearing quantitative evidence currently has gaps in statistical rigor and independence, which the major comments detail.
major comments (4)
- [Eq. (15), Tables 1 and 3] The Image Euclidean Distance formula as printed is identically zero: the second norm inside the double sum is written as (||xj−xj||^2), which vanishes for every j, so the reported IED values in Tables 1 and 3 cannot be reproduced from the given definition. This appears to be a typo for (||xj−yj||^2), but as published the metric is undefined and the central quantitative comparison is not reproducible.
- [Tables 1 and 3] The IED comparisons report only point estimates computed over 10,000 pairs, with no standard deviations, confidence intervals, or significance tests. The differences between GANimation (1.04e−02), SliderGAN-WGP (7.93e−03), and SliderGAN-RaD (6.84e−03) in Table 1, and similarly in Table 3, may or may not be meaningful given the variance of the metric. The claim that SliderGAN outperforms GANimation therefore needs error bars or a statistical test.
- [Section 4.1] The manuscript acknowledges that 3DMM fitting 'can be mistakenly explained by the identity, texture or camera model of the 3DMM,' yet the quantitative evaluation on the rendered 4DFAB dataset derives ground-truth parameters by the same projection p = U^T(S−mean) from the same blendshape model. A systematic fitting bias is therefore invisible in Table 1, and the synthetic-pair mitigation in Section 4.1 reuses the same identity/camera fits for the input images, so it does not independently validate the slider-to-expression mapping.
- [Section 4.6, Table 4] The second row of Table 4 measures consistency between the target parameters ptrg and D(G(Iorg, ptrg)) on images generated by SliderGAN. Because the generator is explicitly trained with Lexp,G (Eq. 8) to make this regression match ptrg, this is a measure of how well the model satisfies its own training objective, not an independent estimate of expression accuracy. It should be validated against 3DMM fitting of the generated images or against manual/landmark-based measures before claiming that D 'extracts expression parameters with high accuracy.'
minor comments (5)
- [Eq. (7)] There is an extra closing parenthesis in Lexp,D = 1/N ||D(Iorg) − porg)||^2; the formula should read ||D(Iorg) − porg||^2.
- [Section 4.7] The ablation text says 'c) LG does not include bothLid andLid' and later 'both Lid andLid are omitted'; the intended meaning is clearly 'both Lid and Lgen,' and the figure caption 'without Lid+Lgen' should be harmonized with the text.
- [Abstract] The phrase 'and particular using the learning paradigm' should read 'and in particular using.'
- [Section 4.1] The description of the synthetic data states that 200,000 images are rendered from 10,000 identities with 20 expressions each; providing details on how the rendered images are aligned and whether any post-processing is applied would help reproducibility.
- [Section 3.1] The paper calls the method 'annotation-free,' but it uses pseudo-annotations from 3DMM fitting; the distinction between manual annotation and automatic pseudo-labels should be made explicit in the introduction to avoid overstatement.
Circularity Check
The discriminator-consistency evaluation in Table 4 restates the training objective Lexp,G; the central editing claim retains independent IED and recognition benchmarks.
-
fitted input called prediction
[Section 4.6, Eq. (8), Table 4 row 2]
"Lexp,G = 1/N||D(G(Iorg, ptrg))-ptrg||^2. ... b) we test the consistency of our model and calculate the error between some target parameters ptrg and those regressed by D on a manipulated image which was translated to expression ptrg by SliderGAN-RaD."
This row-2 error is exactly the quantity minimized by Eq. (8) during training: D is trained to output ptrg when shown G(Iorg, ptrg). Reporting a low value of this residual as a consistency result is therefore measuring the training objective itself, not an independent check that the synthesized image carries the expression ptrg. The only external validity of ptrg comes from the 3DMM fitting or rendering pipeline; the Table 4 row-2 number adds no evidence beyond the fit.
-
fitted input called prediction
[Section 4.6, Eq. (7), Table 4 row 1]
"Lexp,D = 1/N||D(Iorg)-porg)||^2. ... a) we calculate the error between parameters recovered by 3DMM fitting and those regressed by D on the same image (Table 4 row 1)."
The regression head of D is trained under Eq. (7) to match porg produced by the same 3DMM fitting pipeline that later supplies the reference values in Table 4 row 1. A low row-1 error on held-out images shows that D generalizes the fitting function to new images, not that the fitted parameters are physically accurate or that SliderGAN generates correct expressions. The benchmark is self-referential with respect to the pseudo-label source.
full rationale
SliderGAN's central editing claim is not fully circular: the IED experiments on rendered 4DFAB images and the discrete-expression recognition results use external benchmarks, and the synthetic-pair training with Lgen is a genuine supervised signal from rendered images. However, the '3D Expression Reconstruction' evaluation in Section 4.6 is partly self-referential. Table 4 row 2 measures ||ptrg - D(G(Iorg, ptrg))||, which is exactly the Lexp,G term minimized in Eq. (8), so the reported consistency is a restatement of the training fit rather than independent evidence. Table 4 row 1 similarly compares D to the same 3DMM fitting procedure that generated D's pseudo-labels via Eq. (7). The paper itself admits in Section 4.1 that 'parameter values are not always consistent as small variations in expression can be mistakenly explained by the identity, texture or camera model of the 3DMM,' which means these internal agreements cannot validate physical expression accuracy. The self-citations to [9], [10], [11], and [30] are data and tools with external provenance, not an unverified uniqueness argument, so they do not by themselves raise the score. Because external benchmarks exist, the circularity is partial rather than total.
Assumptions & free parameters
free parameters (4)
- Loss weights lambda_adv, lambda_exp, lambda_rec, lambda_gen, lambda_id, lambda_att =
30, 1000, 10, 10, 4, 0.3
- Number of blendshape components =
30 expression (4DFAB), 10 speech (LRW-3D)
- Image resolution =
128x128
- Two-stage training schedule =
20 epochs paired + 40 epochs unpaired
assumptions (6)
- domain assumption 3DMM fitting of Booth et al. [9] recovers expression parameters from images accurately enough to serve as pseudo-labels for training.
- domain assumption The 4DFAB blendshape model spans the facial motion space relevant for expression editing and rendering.
- domain assumption LRW-3D speech blendshape parameters accurately encode lip and jaw motion during speech.
- domain assumption ArcFace embeddings are a valid proxy for identity preservation.
- domain assumption The linear blendshape model Sexp(p) = mean + U_exp p adequately represents facial deformations in images.
- domain assumption Rendered 3D faces from 4DFAB are realistic enough to serve as ground truth for IED and as synthetic training pairs.
Cite this review
Pith. "Pith review of SliderGAN: Synthesizing Expressive Face Images by Sliding 3D Blendshape Parameters." pith.science (2026). https://pith.science/paper/RLGN55XN
@misc{pith2026190809638,
author = {Pith},
title = {Pith review of: SliderGAN: Synthesizing Expressive Face Images by Sliding 3D Blendshape Parameters},
year = {2026},
howpublished = {\url{https://pith.science/paper/RLGN55XN}},
note = {Machine review of arXiv:1908.09638}
}
read the original abstract
Image-to-image (i2i) translation is the dense regression problem of learning how to transform an input image into an output using aligned image pairs. Remarkable progress has been made in i2i translation with the advent of Deep Convolutional Neural Networks (DCNNs) and particular using the learning paradigm of Generative Adversarial Networks (GANs). In the absence of paired images, i2i translation is tackled with one or multiple domain transformations (i.e., CycleGAN, StarGAN etc.). In this paper, we study a new problem, that of image-to-image translation, under a set of continuous parameters that correspond to a model describing a physical process. In particular, we propose the SliderGAN which transforms an input face image into a new one according to the continuous values of a statistical blendshape model of facial motion. We show that it is possible to edit a facial image according to expression and speech blendshapes, using sliders that control the continuous values of the blendshape model. This provides much more flexibility in various tasks, including but not limited to face editing, expression transfer and face neutralisation, comparing to models based on discrete expressions or action units.
Reference graph
Works this paper leans on
-
[1]
Alami Mejjati, Y., Richardt, C., Tompkin, J., Cosker, D., Kim, K.I.: Unsupervised attention- guided image-to-image translation pp. 3693–3703 (2018). URL http://papers.nips.cc/paper/ 7627-unsupervised-attention-guided-image-to-image-translation. pdf
work page 2018
-
[2]
Amos, B., Ludwiczuk, B., Satyanarayanan, M.: Openface: A general-purpose face recognition library with mobile applications. Tech. rep., CMU-CS-16-118, CMU School of Computer Science (2016)
work page 2016
-
[3]
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein gen- erative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, pp. 214–223 (2017). URL http://proceedings.mlr.press/ v70/arjovsky17a.html
work page 2017
-
[4]
Foundations and Trends in Machine Learning 4(1), 1–106 (2012)
Bach, F., Jenatton, R., Mairal, J., Obozinski, G.: Opti- mization with sparsity-inducing penalties. Foundations and Trends in Machine Learning 4(1), 1–106 (2012). DOI 10.1561/2200000015. URL http://dx.doi.org/10. 1561/2200000015
-
[5]
In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp
Benitez-Quiroz, C.F., Srinivasan, R., Martinez, A.M.: Emotionet: An accurate, real-time algorithm for the au- tomatic annotation of a million facial expressions in the wild. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5562–5570 (2016). DOI 10.1109/CVPR.2016.600
-
[6]
Benitez-Quiroz, C.F., Wang, Y., Martinez, A.M.: Recog- nition of action units in the wild with deep nets and a new global-local loss. In: ICCV, pp. 3990–3999 (2017)
work page 2017
-
[7]
IEEE transactions on pattern analysis and machine in- telligence (2018)
Benitez-Quiroz, F., Srinivasan, R., Martinez, A.M.: Dis- criminant functional learning of color features for the recognition of facial action units and their intensities. IEEE transactions on pattern analysis and machine in- telligence (2018)
work page 2018
-
[8]
In: Proceedings of the IEEE Confer- ence on ComputerVision and Pattern Recognition (2017)
Booth, J., Antonakos, E., Ploumpis, S., Trigeorgis, G., Panagakis, Y., Zafeiriou, S., et al.: 3d face morphable models in-the-wild. In: Proceedings of the IEEE Confer- ence on ComputerVision and Pattern Recognition (2017)
work page 2017
Show all 35 references
-
[9]
IEEE Transactions on Pattern Analysis and Machine In- telligence (2018)
Booth, J., Roussos, A., Ververas, E., Antonakos, E., Poumpis, S., Panagakis, Y., Zafeiriou, S.P.: 3d recon- struction of” in-the-wild” faces in images and videos. IEEE Transactions on Pattern Analysis and Machine In- telligence (2018)
2018
-
[10]
In: 2016 IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), pp
Booth, J., Roussos, A., Zafeiriou, S., Ponniahy, A., Dun- away, D.: A 3d morphable model learnt from 10,000 faces. In: 2016 IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), pp. 5543–5552 (2016). DOI 10.1109/CVPR.2016.598
2016 doi
-
[11]
In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018)
Cheng, S., Kotsia, I., Pantic, M., Zafeiriou, S.: 4dfab: A large scale 4d database for facial expression analysis and biometric applications. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018). Salt Lake City, Utah, US (2018)
2018
-
[12]
In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
2018
-
[13]
In: Asian Conference on Computer Vision (2016)
Chung, J.S., Zisserman, A.: Lip reading in the wild. In: Asian Conference on Computer Vision (2016)
2016
-
[14]
arXiv:1801.07698 (2018)
Deng, J., Guo, J., Zafeiriou, S.: Arcface: Additive angular margin loss for deep face recognition. arXiv:1801.07698 (2018)
2018 arXiv
-
[15]
A human face (2002)
Ekman, P.: Facial action coding system (facs). A human face (2002)
2002
-
[16]
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Ben- gio, Y.: Generative adversarial nets. In: Z. Ghahra- mani, M. Welling, C. Cortes, N.D. Lawrence, K.Q. Weinberger (eds.) Advances in Neural Information Processing Systems 27, ...
2014
-
[17]
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasser- stein gans. In: I. Guyon, U.V. Luxburg, S. Ben- gio, H. Wallach, R. Fergus, S. Vishwanathan, R. Gar- nett (eds.) Advances in Neural Information Process- ing Systems 30, pp. 5767...
2017
-
[18]
CVPR (2017)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to- image translation with conditional adversarial networks. CVPR (2017)
2017
-
[19]
In: Interna- tional Conference on Learning Representations (2019)
Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. In: Interna- tional Conference on Learning Representations (2019). URL https://openreview.net/forum?id=S1erHoR5t7
2019
-
[20]
CoRR abs/1412.6980 (2014)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. CoRR abs/1412.6980 (2014). URL http: //arxiv.org/abs/1412.6980
2014 arXiv
-
[21]
CoRR abs/1610.05586 (2016)
Li, M., Zuo, W., Zhang, D.: Deep identity-aware trans- fer of facial attributes. CoRR abs/1610.05586 (2016). URL http://arxiv.org/abs/1610.05586
2016 arXiv
-
[22]
In: Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pp
Li, S., Deng, W., Du, J.: Reliable crowdsourcing and deep locality-preserving learning for expression recognition in 18 Evangelos Ververas ⋆, Stefanos Zafeiriou † the wild. In: Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pp. 2584–2593. IEEE (2017)
2017
-
[23]
ACM Transactions on Graphics (TOG) 32(6), 179 (2013)
Neumann, T., Varanasi, K., Wenger, S., Wacker, M., Magnor, M., Theobalt, C.: Sparse localized deformation components. ACM Transactions on Graphics (TOG) 32(6), 179 (2013)
2013
-
[24]
CoRR abs/1611.06355 (2016)
Perarnau, G., van de Weijer, J., Raducanu, B., ´Alvarez, J.M.: Invertible conditional gans for image editing. CoRR abs/1611.06355 (2016). URL http://arxiv.org/abs/ 1611.06355
2016 arXiv
-
[25]
In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)
Pumarola, A., Agudo, A., Martinez, A., Sanfeliu, A., Moreno-Noguer, F.: Ganimation: Anatomically-aware fa- cial animation from a single image. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)
2018
-
[26]
In: Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pp
Richardson, E., Sela, M., Or-El, R., Kimmel, R.: Learn- ing detailed face reconstruction from a single image. In: Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pp. 5553–5562. IEEE (2017)
2017
-
[27]
ACM Transactions on Graphics (TOG) 36(4), 95 (2017)
Suwajanakorn, S., Seitz, S.M., Kemelmacher-Shlizerman, I.: Synthesizing obama: learning lip sync from audio. ACM Transactions on Graphics (TOG) 36(4), 95 (2017)
2017
-
[28]
arXiv preprint arXiv:1712.02859 2 (2017)
Tewari, A., Zollh¨ ofer, M., Garrido, P., Bernard, F., Kim, H., P´ erez, P., Theobalt, C.: Self-supervised multi-level face model learning for monocular reconstruction at over 250 hz. arXiv preprint arXiv:1712.02859 2 (2017)
2017 arXiv
-
[29]
arXiv preprint arXiv:1804.03786 (2018)
Tran, L., Liu, X.: Nonlinear 3d face morphable model. arXiv preprint arXiv:1804.03786 (2018)
2018 arXiv
-
[30]
Tzirakis, P., Papaioannou, A., Lattas, A., Tarasiou, M., Schuller, B., Zafeiriou, S.: Synthesising 3d facial motion from”in-the-wild”speech (2019)
2019
-
[31]
IEEE Trans
Wang, L., Zhang, Y., Feng, J.: On the euclidean distance of images. IEEE Trans. Pattern Anal. Mach. Intell.27(8), 1334–1339 (2005). DOI 10.1109/TPAMI.2005.165. URL https://doi.org/10.1109/TPAMI.2005.165
2005 doi
-
[32]
In: The European Conference on Computer Vision Workshops (ECCVW) (2018)
Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., Loy, C.C.: Esrgan: Enhanced super-resolution gener- ative adversarial networks. In: The European Conference on Computer Vision Workshops (ECCVW) (2018)
2018
-
[33]
In: Proc
Wiles, O., Koepke, A.S., Zisserman, A.: X2face: A net- work for controlling face generation using images, audio, and pose codes. In: Proc. ECCV (2018)
2018
-
[34]
IEEE Trans- actions on Signal Processing 57(7), 2479–2493 (2009)
Wright, S.J., Nowak, R.D., Figueiredo, M.A.T.: Sparse reconstruction by separable approximation. IEEE Trans- actions on Signal Processing 57(7), 2479–2493 (2009). DOI 10.1109/TSP.2009.2016892
2009
-
[35]
In: Computer Vision (ICCV), 2017 IEEE International Conference on (2017)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adver- sarial networkss. In: Computer Vision (ICCV), 2017 IEEE International Conference on (2017)
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.