Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation

Chen Qian; Jiaya Jia; Keqiang Sun; Shengju Qian; Wayne Wu

REVIEW 4 major objections 5 minor 88 references

Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Style-augmented semi-supervised training beats fully supervised facial landmark detection by re-rendering each face geometry with other images' styles.

desk verdict Useful augmentation idea and benchmark, but the missing equal-data control means the reported gains may just be from 9x more training samples. read the letter →

arxiv 1908.06440 v1 pith:3A5C3W6J submitted 2019-08-18 cs.CV

Shengju Qian , Keqiang Sun , Wayne Wu , Chen Qian , Jiaya Jia This is my paper · ORCID

classification cs.CV

keywords faciallandmarkdetectionfacealignmentstyletranslationdisentangledrepresentationlearningsemi-superviseddataaugmentationconditionalvariationalauto-encoder

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the main obstacle to accurate facial landmark detection is not only pose, expression, or occlusion, but the 'style' of an image: lighting, texture, background, image quality, glasses, and similar environmental factors. It argues that a face can be factored into a geometry part (landmark locations) and a style part, and that once the two are truly separated, any face's geometry can be re-rendered with any other face's style. These re-rendered images, labeled with the original landmarks, act as free training data: a model trained on the augmented set is reported to beat the same model trained only on real, fully supervised data. On WFLW the style-augmented SAN model reaches 4.39% NME, and consistent gains are reported on 300W, COFW, and AFLW.

What carries the argument

The load-bearing engine is a two-branch conditional variational auto-encoder: $E_{\mathrm{style}}$ maps an image and its landmark heatmap to a compact style code $z$, $E_{\mathrm{struct}}$ maps landmark heatmaps to a structure representation, and $D_{\mathrm{render}}$ re-renders the image from both. The KL divergence term against a unit Gaussian prior pushes $z$ toward carrying no structure information, the perceptual loss on deep convolutional features improves visual quality, and skip connections from $E_{\mathrm{struct}}$ to $D_{\mathrm{render}}$ keep geometry explicit. The central identity is the style-translation equation $x_{ij} = D_{\mathrm{render}}(E_{\mathrm{style}}(x_j, y_i), E_{\mathrm{struct}}(y_i))$, which converts the dataset into $n^2$ potential training pairs, of which the paper samples $k$ per geometry.

What would settle it

Measure structure leakage directly: render many style-translated images $x_{ij}$ and run a landmark detector on them; if predicted landmarks drift systematically away from $y_i$ and toward the geometry of the style-source image $x_j$, then the style code carries geometry and the augmentation is partly injecting wrong labels.

Watch

Extended reading notes

Core claim

The paper's central claim is that every face image $x$ can be modeled as a sample from a conditional distribution $p(x|y,z)$, where $y$ is the facial geometry encoded as landmark heatmaps and $z$ is a style code capturing all non-geometric factors. A conditional variational auto-encoder with a structure encoder, a style encoder, and a renderer is trained with KL regularization and perceptual reconstruction loss so that $z$ is encouraged to carry no structure information. Style translation then combines the structure code of image $i$ with the style code of image $j$ via $x_{ij} = D_{\mathrm{render}}(E_{\mathrm{style}}(x_j, y_i), E_{\mathrm{struct}}(y_i))$. The paper argues that these re-rendered images preserve $y_i$ faithfully, so they can be added to the training set with the original landmarks as labels. With $k=8$ styles sampled per landmark, the semi-supervised detectors outperform their fully supervised counterparts by large margins, including 4.39% NME on WFLW with a SAN backbone and 3.86% on 300W.

Load-bearing premise

The entire method depends on the learned style code containing no facial geometry at all, so that swapping styles never moves the landmarks; the paper supports this only with visual examples, not with a numerical disentanglement check.

Editorial extensions

If this is right

Any landmark detector, from a light baseline network to stronger SAN and LAB baselines, reports gains when style-augmented synthetic samples are added to its training set.
With only half the labeled training data, style augmentation lets a strong baseline reach results comparable to or better than previous fully supervised state-of-the-art methods on 300W and WFLW.
The number of sampled styles per geometry has a sweet spot near $k=8$; larger $k$ makes the model overfit to synthetic texture and hurts performance.
The gains are not confined to one benchmark: the paper reports improvements on occlusion-heavy COFW and large-pose AFLW as well as on WFLW and 300W.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the disentanglement is as clean as claimed, the same augmentation recipe should transfer to other keypoint-labeling tasks such as human pose estimation or hand keypoints, wherever 'content' can be defined by geometry and 'style' by appearance.
A direct numerical check is missing: measure whether predicted landmarks drift when one geometry is rendered with many different styles. Such a test would separate genuine geometry-preserving augmentation from label noise caused by structure leakage.
The large gains at low data fractions point to a practical use for expensive annotation regimes, but they also mean the disentangler's quality under data scarcity is the bottleneck that future work should target.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read the paper. The core idea is genuinely new: a conditional VAE that factors face images into style and structure, then uses landmark-preserving style translation to augment training. They show consistent gains on WFLW, 300W, COFW, and AFLW, across a ResNet-18 baseline, LAB, and SAN, and they release code and a re-annotated AFLW-68 benchmark. That is real work and the benchmark alone has value.

The soft spot is load-bearing: no control for data volume or compute. With k=8, the augmented model sees 9x more training images than the baseline, so the reported margins could come from extra data rather than style translation. The paper never trains a baseline on 9x epochs, with repeated originals, or with equivalent compute, so the causal claim in the abstract is not established. The limited-data experiments have the same confound, since 'Ours' at a given PCT sees k synthetic versions of each real image while the baseline sees only the real ones. A second, smaller problem: Table 6 tunes k directly on the WFLW test set, which inflates the headline number. I'd also drop the 'semi-supervised' framing—no unlabeled data is used; it's data augmentation.

The disentanglement concern—that the style code might leak structure—is real but secondary, because the data-volume confound alone is enough to undermine the attribution. The ablations for KL and perceptual loss are a point in the paper's favor: they suggest the generation mechanism matters, not just the raw count. But they don't replace the missing equal-data control.

Bottom line: the paper is worth refereeing and the benchmark/code are useful, but the central claim needs one clean control experiment before I'd believe the mechanism. I would not cite it for the causal claim as is.

Referee Report

4 major / 5 minor

Summary. The paper proposes a semi-supervised framework for facial landmark detection based on disentangling each face image into a structure code (derived from landmark heatmaps) and a style code (lighting, texture, occlusion, etc.), then re-rendering the same structure under styles sampled from other training images. The augmented synthetic images, together with the original labels, are used to train landmark detectors. The authors report consistent gains over baselines on WFLW, 300W, COFW, and AFLW, including a new state-of-the-art result on WFLW (4.39% NME), and they provide ablations showing that the KL divergence and perceptual losses contribute to the performance. They also release the code and a re-annotated 68-point AFLW dataset.

Significance. If the central mechanism is validated, the paper offers a novel and potentially general augmentation perspective for landmark detection and similar structured prediction tasks. The strengths are explicit: the approach is evaluated on four benchmarks with multiple backbones; the gains are consistent; the code is public; and the new AFLW-68 annotation is a useful resource. The ablations in the appendix (Table 7) indicate that both the KL term and the perceptual loss matter. However, the headline causal claim—that style-translated synthetic samples, rather than merely additional training data, are responsible for the gains—is not yet established because the experiments lack a control for data volume and compute, and the hyperparameter k is selected on a test set. These issues are fixable with additional experiments, so the work is promising but needs revision before the central claim is supported.

major comments (4)

[Section 3.2 and Section 4.2, Tables 1–5] The experiments do not isolate the effect of style translation from the effect of simply training on more data. With k=8, every 'Ours' model is trained on 9x as many images as the baseline (one original plus eight style-translated versions), while the baselines in Tables 1–5 are trained only on the original set. The reported gains could therefore be an artifact of increased data volume or compute rather than of style translation. The manuscript needs an equal-data or equal-compute control, such as training the baseline on the same number of effective samples by repeating each real image k times, training for 9x epochs on the original data, or matching the total number of gradient steps while holding the data distribution fixed. Without such a control, the abstract's statement that 'these augmented synthetic samples' outperform fully-supervised training is not a causal claim supported by the evidence.
[Section 4.4.2, Table 6] The augmentation factor k is selected directly on the WFLW test set. The headline NME of 4.39% in Table 1 uses k=8, and Table 6 reports the test-set NME for k=0,2,4,8,16,32, with k=8 chosen as the best value. This is test-set tuning and inflates the reported improvement over baselines, which are not tuned in the same way. The authors should select k using a validation split or cross-validation and report the corresponding test-set numbers; they should also state explicitly which k was used for each reported result.
[Section 3.1 and Eq. (4)] The disentanglement of the style code z is not verified quantitatively. The paper argues that the KL divergence and skip connections prevent Estyle from encoding structure information, and it provides visual examples in Figures 3 and 7, but there is no numerical measure of structure preservation or style-code leakage. This matters because Eq. (4) assumes that x_ij = D_render(E_style(x_j, y_i), E_struct(y_i)) preserves the landmark geometry y_i exactly. If z leaks structure information, the synthetic images carry incorrect labels, and the augmentation would inject label noise. The manuscript should include a quantitative disentanglement metric (e.g., landmark distance between the input and re-rendered image, or a consistency check by running a landmark detector on synthetic images) or otherwise demonstrate that the synthetic labels are reliable.
[Section 4.4.1, Tables 4 and 5] The limited-data study has the same data-volume asymmetry as the main experiments. At each PCT, 'Ours' trains on the real images plus k synthetic versions of them, while the baseline sees only the real images; the relative gains are therefore partly explained by the baseline being undertrained at small PCT. The large improvements at low data percentages (e.g., +42.71% on 300W at 10% in Table 4) may reflect this asymmetry rather than a property of style-based augmentation. The equal-data and equal-compute controls requested above should also be applied to the limited-data setting before drawing conclusions about the method's behavior under scarce supervision.

minor comments (5)

[Section 4.2] The text says the baseline models are trained 'with the same affine augmentation' as the augmented models, but it does not specify whether the total number of training iterations, learning-rate schedule, and batch composition are also matched. Please clarify these settings so that the comparison is unambiguous.
[Appendix, Table 7] The baseline NME reported in Table 7 (8.49) appears inconsistent with the Res-18 baseline NME in Table 1 (6.09). If these numbers come from different settings (e.g., a different backbone or a different training configuration), this should be stated explicitly so the ablation is interpretable.
[Eq. (2)] The loss expression has mismatched parentheses: '-KL(q_φ(z|x,y))||p_θ(z|y))' is not well-formed. Please rewrite the KL term in conventional notation, e.g., D_KL(q_φ(z|x,y) || p_θ(z|y)).
[References] Several references lack years or venues (e.g., [14], [52], [57]), and the reference list would benefit from a consistency pass.
[Table 6] The table header 'Number' should be labeled as the style augmentation factor k, and the row '0' should be identified as the baseline without style augmentation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the style-translation augmentation pipeline is an empirical recipe whose gains are measured, not derived from its own inputs.

full rationale

The paper's central claim is empirical: synthetic images produced by re-rendering structure y_i with style code from another face improve downstream landmark detection. The generation equation x_ij = D_render(E_style(x_j, y_i), E_struct(y_i)) uses the original landmarks y_i as the training label for the synthetic image, but this is a data-generation choice, not a derivation that assumes the detector's output. The disentanglement of style and structure is learned through a reconstruction objective and KL regularization, and the facial landmark detector is an independent network trained on the resulting augmented set. No fitted parameter is renamed as a prediction: Table 6 varies k and reports test NME, but choosing k from the test-set curve is test-set selection, not an equivalence-by-construction reduction. The cited baselines LAB [69] and SAN [15] are used as external comparison frameworks and backbones; although LAB includes coauthors of this paper, no load-bearing argument depends on that citation. Concerns about missing data-volume controls, possible style-code leakage of structure, and test-set hyperparameter selection are external-validity and evaluation-protocol issues, not circular derivation. The paper is self-contained as an empirical study, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The paper does not derive its improvement from first principles; it relies on a factored representation of faces. The most important input is k, the number of style transfers per landmark, chosen on the test set. The claimed separation of style and structure is an assumption validated only qualitatively and through an indirect ablation.

free parameters (2)

k, number of random styles sampled per landmark during augmentation = 8
Section 4.2 sets k=8; Section 4.4.2 chooses it from a sweep on the WFLW test set for k in {0,2,4,8,16,32}. This is a hyperparameter tuned on the test split.
KL divergence loss weight relative to perceptual loss = not stated (implicitly 1)
Eq. (2) sums the KL term and the reconstruction term with no weighting coefficient, and no ablation over the weight is reported. If the balance is not intended to be 1, it is an undisclosed hand-chosen constant.

assumptions (4)

domain assumption Face images factor into an independent style latent space and a structure space, whose Cartesian product captures all image variation (p(x|y,z)).
Section 3.1, Eq. (1): 'if the latent space of style and shape is mostly uncorrelated, using Cartesian product of z and y latent space should capture all variation included in a face image.' This is the load-bearing factorization and is not proven.
domain assumption KL divergence to a unit Gaussian plus skip connections prevents the style encoder from encoding structure-related information.
Section 3.1, KL-Divergence Loss paragraph. The argument is heuristic; no quantitative disentanglement metric is provided.
domain assumption Perceptual loss with VGG-19 (Eq. 3) is a suitable reconstruction objective for preserving style while reconstructing faces.
Section 3.1. The paper relies on this standard but not otherwise justified objective for the quality of the synthetic images.
standard math The standard VAE ELBO derivation applies to the conditional model p(x|y,z).
Used in Eq. (1) to define the training objective; this is a standard derivation, but its applicability depends on the conditional factorization being correct.

invented entities (1)

Style latent code z
purpose: Captures lighting, texture, image quality, glasses, blur, and other environment factors that are not facial geometry; used to re-render synthetic training images via D_render.
The code is a learned latent variable. The paper provides qualitative style translation and ablations, but no external falsifiable handle such as a style-labeled benchmark or a downstream task that isolates z.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation." pith.science (2026). https://pith.science/paper/3A5C3W6J

@misc{pith2026190806440,
  author       = {Pith},
  title        = {Pith review of: Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3A5C3W6J}},
  note         = {Machine review of arXiv:1908.06440}
}

read the original abstract

Facial landmark detection, or face alignment, is a fundamental task that has been extensively studied. In this paper, we investigate a new perspective of facial landmark detection and demonstrate it leads to further notable improvement. Given that any face images can be factored into space of style that captures lighting, texture and image environment, and a style-invariant structure space, our key idea is to leverage disentangled style and shape space of each individual to augment existing structures via style translation. With these augmented synthetic samples, our semi-supervised model surprisingly outperforms the fully-supervised one by a large margin. Extensive experiments verify the effectiveness of our idea with state-of-the-art results on WFLW, 300W, COFW, and AFLW datasets. Our proposed structure is general and could be assembled into any face alignment frameworks. The code is made publicly available at https://github.com/thesouthfrog/stylealign.

Figures

Figures reproduced from arXiv: 1908.06440 by the authors.

**Figure 2.** Our framework. It consists of two stages. The first stage is to train the network to disentangle face images to style [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Visualization of style translation. Given the input images in red [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Cumulative error distribution curve on COFW 68- [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Visual comparison on WFLW test set between the original baseline model and the boosted framework via style translation. Method NME(%) AUC@ 0.1 FR(%) Frontal Full Full Full LAB [69] 2.23 7.15 0.39 11.28 SAN [15] 2.01 6.94 0.44 10.43 Res-18 2.30 7.23 0.37 11.89 Ours w. R…

**Figure 6.** Figure 6: Results of style translation using different num [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Qualitative analysis of each component in our framework. Given input images in red, 3 different styles are provided to perform translation towards input structure. 3 incomplete variants of our framework are used to show the functionality of each component. S2.1 Compa…

**Figure 9.** Figure 9 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

**Figure 8.** Figure 8: Visual comparison with style transfer approach. For the style transfer algorithm, we use [21]. Our results are more realistic than stylized images, with better structure coherence. Database Environment Number Multi-PIE [23] Controlled 750000 XM2VTS [49] 2360 LFPW [4] 1…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

88 extracted references · 68 canonical work pages

[1]

Data augmentation generative adversarial networks, 2018

Anthreas Antoniou, Amos Storkey, and Harrison Edwards. Data augmentation generative adversarial networks, 2018. 12

2018
[2]

Synthesizing images of humans in unseen poses

Guha Balakrishnan, Amy Zhao, Adrian V Dalca, Fredo Du- rand, and John Guttag. Synthesizing images of humans in unseen poses. In CVPR, 2018. 2

2018
[3]

Mutual information neural estimation

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajesh- war, Sherjil Ozair, Yoshua Bengio, Devon Hjelm, and Aaron Courville. Mutual information neural estimation. In ICML,
[4]

Localizing parts of faces using a consensus of exemplars

Peter N Belhumeur, David W Jacobs, David J Kriegman, and Neeraj Kumar. Localizing parts of faces using a consensus of exemplars. IEEE transactions on pattern analysis and machine intelligence, 35(12):2930–2940, 2013. 5, 13

2013
[5]

Convolutional aggregation of local evidence for large pose face alignment

Adrian Bulat and Georgios Tzimiropoulos. Convolutional aggregation of local evidence for large pose face alignment
[6]

How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks)

Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In ICCV, 2017. 1

2017
[7]

Robust face landmark estimation under occlusion

Xavier P Burgos-Artizzu, Pietro Perona, and Piotr Doll ´ar. Robust face landmark estimation under occlusion. In CVPR,
[8]

Face alignment by explicit shape regression

Xudong Cao, Yichen Wei, Fang Wen, and Jian Sun. Face alignment by explicit shape regression. IJCV, 2014. 2, 6

2014

Show all 88 references

[9]

Joint cascade face detection and alignment

Dong Chen, Shaoqing Ren, Yichen Wei, Xudong Cao, and Jian Sun. Joint cascade face detection and alignment. In ECCV, 2014. 2

2014
[10]

Infogan: Interpretable rep- resentation learning by information maximizing generative adversarial nets

Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable rep- resentation learning by information maximizing generative adversarial nets. In NIPS, 2016. 1, 2

2016
[11]

Active appearance models

Timothy F Cootes, Gareth J Edwards, and Christopher J Tay- lor. Active appearance models. IEEE Transactions on Pat- tern Analysis & Machine Intelligence , 2001. 2

2001
[12]

Feature detection and tracking with constrained local models

David Cristinacce and Timothy F Cootes. Feature detection and tracking with constrained local models. In Bmvc, 2006. 2

2006
[13]

Joint multi-view face alignment in the wild

Jiankang Deng, George Trigeorgis, Yuxiang Zhou, and Ste- fanos Zafeiriou. Joint multi-view face alignment in the wild. arXiv preprint arXiv:1708.06023, 2017. 2

2017 arXiv
[14]

Unsuper- vised visual representation learning by context prediction

Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsuper- vised visual representation learning by context prediction. In ICCV. 2
[15]

Style aggregated network for facial landmark detection

Xuanyi Dong, Yan Yan, Wanli Ouyang, and Yi Yang. Style aggregated network for facial landmark detection. In CVPR,
[16]

A varia- tional u-net for conditional appearance and shape generation

Patrick Esser, Ekaterina Sutter, and Bj ¨orn Ommer. A varia- tional u-net for conditional appearance and shape generation. In CVPR, 2018. 1, 2

2018
[17]

Joint 3d face reconstruction and dense alignment with position map regression network

Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, and Xi Zhou. Joint 3d face reconstruction and dense alignment with position map regression network. In ECCV, 2018. 1

2018
[18]

Cascaded collaborative regression for robust facial landmark detection trained using a mix- ture of synthetic and real images with dynamic weighting

Zhen-Hua Feng, Guosheng Hu, Josef Kittler, William Christ- mas, and Xiao-Jun Wu. Cascaded collaborative regression for robust facial landmark detection trained using a mix- ture of synthetic and real images with dynamic weighting. IEEE Transactions on Image Processing, 24(11):...
[19]

Wing loss for robust facial landmark localisation with convolutional neural networks

Zhen-Hua Feng, Josef Kittler, Muhammad Awais, Patrik Hu- ber, and Xiao-Jun Wu. Wing loss for robust facial landmark localisation with convolutional neural networks. In CVPR,
[20]

Learning invariance from transformation se- quences

Peter F ¨oldi´ak. Learning invariance from transformation se- quences. Neural Computation, 3(2):194–200, 1991. 2

1991
[21]

Im- age style transfer using convolutional neural networks

Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Im- age style transfer using convolutional neural networks. In CVPR, 2016. 1, 2, 13

2016
[22]

Occlusion co- herence: Localizing occluded faces with a hierarchical de- formable part model

Golnaz Ghiasi and Charless C Fowlkes. Occlusion co- herence: Localizing occluded faces with a hierarchical de- formable part model. In CVPR, 2014. 5, 13

2014
[23]

Multi-pie

Ralph Gross, Iain Matthews, Jeffrey Cohn, Takeo Kanade, and Simon Baker. Multi-pie. Image and Vision Computing , 28(5):807–813, 2010. 13

2010
[24]

Identity mappings in deep residual networks

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV,
[25]

beta-vae: Learning basic visual con- cepts with a constrained variational framework

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual con- cepts with a constrained variational framework. In ICLR,
[26]

Improving landmark localization with semi-supervised learning

Sina Honari, Pavlo Molchanov, Stephen Tyree, Pascal Vin- cent, Christopher Pal, and Jan Kautz. Improving landmark localization with semi-supervised learning. In CVPR, 2018. 2, 6

2018
[27]

Recombinator networks: Learning coarse-to-ﬁne feature aggregation

Sina Honari, Jason Yosinski, Pascal Vincent, and Christo- pher Pal. Recombinator networks: Learning coarse-to-ﬁne feature aggregation. In CVPR, 2016. 6

2016
[28]

Multimodal unsupervised image-to-image translation

Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-image translation. In ECCV, 2018. 2

2018
[29]

Image-to-image translation with conditional adver- sarial networks

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In CVPR, 2017. 2

2017
[30]

Perceptual losses for real-time style transfer and super-resolution

Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV, 2016. 1, 4

2016
[31]

Pose- invariant face alignment with a single cnn

Amin Jourabloo, Mao Ye, Xiaoming Liu, and Liu Ren. Pose- invariant face alignment with a single cnn. In CVPR, 2017. 1, 5, 6

2017
[32]

An active illumination and appearance (aia) model for face alignment

Fatih Kahraman, Muhittin Gokmen, Sune Darkner, and Ras- mus Larsen. An active illumination and appearance (aia) model for face alignment. In CVPR, 2007. 2

2007
[33]

Synergy between face alignment and tracking via discriminative global consensus optimization

Muhammad Haris Khan, John McDonagh, and Georgios Tz- imiropoulos. Synergy between face alignment and tracking via discriminative global consensus optimization. In ICCV,
[34]

Adam: A method for stochastic optimization

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

arXiv
[35]

Auto-encoding varia- tional bayes

Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. arXiv preprint arXiv:1312.6114, 2013. 2

2013 arXiv
[36]

Annotated facial landmarks in the wild: A large- scale, real-world database for facial landmark localization

Martin Koestinger, Paul Wohlhart, Peter M Roth, and Horst Bischof. Annotated facial landmarks in the wild: A large- scale, real-world database for facial landmark localization. In ICCV workshops. IEEE, 2011. 1, 2, 5, 7, 13

2011
[37]

Disentangling 3d pose in a dendritic cnn for unconstrained 2d face alignment

Amit Kumar and Rama Chellappa. Disentangling 3d pose in a dendritic cnn for unconstrained 2d face alignment. In CVPR, 2018. 1, 5, 6

2018
[38]

Learning representations for automatic colorization

Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Learning representations for automatic colorization. In ECCV, 2016. 2

2016
[39]

Colorization as a proxy task for visual understanding

Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Colorization as a proxy task for visual understanding. In CVPR, 2017. 2

2017
[40]

Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis

Quoc V Le, Will Y Zou, Serena Y Yeung, and Andrew Y Ng. Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis
[41]

Interactive facial feature localization

Vuong Le, Jonathan Brandt, Zhe Lin, Lubomir Bourdev, and Thomas S Huang. Interactive facial feature localization. In ECCV, 2012. 5, 13

2012
[42]

Diverse image-to-image translation via disentangled representations

Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Yang. Diverse image-to-image translation via disentangled representations. In ECCV, 2018. 1, 2

2018
[43]

Unsupervised image-to-image translation networks

Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In NIPS, 2017. 2

2017
[44]

Sphereface: Deep hypersphere embedding for face recognition

Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In ICCV, 2017. 1

2017
[45]

A deep regression architecture with two-stage re-initialization for high performance facial landmark detec- tion

Jiangjing Lv, Xiaohu Shao, Junliang Xing, Cheng Cheng, and Xi Zhou. A deep regression architecture with two-stage re-initialization for high performance facial landmark detec- tion. In CVPR, 2017. 2, 6

2017
[46]

Pose guided person image genera- tion

Liqian Ma, Xu Jia, Qianru Sun, Bernt Schiele, Tinne Tuyte- laars, and Luc Van Gool. Pose guided person image genera- tion. In NIPS, 2017. 2

2017
[47]

Active appearance models revisited

Iain Matthews and Simon Baker. Active appearance models revisited. IJCV. 2
[48]

Robust facial landmark detection via a fully-convolutional local- global context network

Daniel Merget, Matthias Rock, and Gerhard Rigoll. Robust facial landmark detection via a fully-convolutional local- global context network. In CVPR, 2018. 2

2018
[49]

Xm2vtsdb: The extended m2vts database

Kieron Messer, Jiri Matas, Josef Kittler, Juergen Luettin, and Gilbert Maitre. Xm2vtsdb: The extended m2vts database. In Second international conference on audio and video-based biometric person authentication , volume 964, pages 965– 966, 1999. 5, 13

1999
[50]

Direct shape regression net- works for end-to-end face alignment

Xin Miao, Xiantong Zhen, Xianglong Liu, Cheng Deng, Vas- silis Athitsos, and Heng Huang. Direct shape regression net- works for end-to-end face alignment. In CVPR. 2
[51]

Stacked hour- glass networks for human pose estimation

Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hour- glass networks for human pose estimation. In ECCV, 2016. 2

2016
[52]

A recurrent encoder-decoder for sequential face alignment

Xi Peng, Rogerio S Feris, Xiaoyu Wang, and DN Metaxs. A recurrent encoder-decoder for sequential face alignment. In ECCV. 2
[53]

Unsupervised person image syn- thesis in arbitrary poses

Albert Pumarola, Antonio Agudo, Alberto Sanfeliu, and Francesc Moreno-Noguer. Unsupervised person image syn- thesis in arbitrary poses. In CVPR, 2018. 2

2018
[54]

Face detection, pose es- timation, and landmark localization in the wild

Deva Ramanan and Xiangxin Zhu. Face detection, pose es- timation, and landmark localization in the wild. In CVPR,
[55]

Face alignment at 3000 fps via regressing local binary features

Shaoqing Ren, Xudong Cao, Yichen Wei, and Jian Sun. Face alignment at 3000 fps via regressing local binary features. In CVPR, 2014. 5, 6

2014
[56]

300 faces in-the-wild challenge: The ﬁrst facial landmark localization challenge

Christos Sagonas, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 faces in-the-wild challenge: The ﬁrst facial landmark localization challenge. In ICCV Workshops, 2013. 1, 2, 5, 13

2013
[57]

A nonlinear discrimina- tive approach to aam ﬁtting

Jason Saragih and Roland Goecke. A nonlinear discrimina- tive approach to aam ﬁtting. In ICCV, 2007. 2

2007
[58]

The ﬁrst facial landmark tracking in-the-wild challenge: Benchmark and results

Jie Shen, Stefanos Zafeiriou, Grigoris G Chrysos, Jean Kos- saiﬁ, Georgios Tzimiropoulos, and Maja Pantic. The ﬁrst facial landmark tracking in-the-wild challenge: Benchmark and results. In CVPR Workshops, 2015. 5

2015
[59]

Unsupervised learning of invariant features using video

David Stavens and Sebastian Thrun. Unsupervised learning of invariant features using video. In CVPR, 2010. 2

2010
[60]

Convolutional learning of spatio-temporal features

Graham W Taylor, Rob Fergus, Yann LeCun, and Christoph Bregler. Convolutional learning of spatio-temporal features. In ECCV, 2010. 2

2010
[61]

Face2face: Real-time face capture and reenactment of rgb videos

Justus Thies, Michael Zollhofer, Marc Stamminger, Chris- tian Theobalt, and Matthias Nießner. Face2face: Real-time face capture and reenactment of rgb videos. In CVPR, 2016. 1

2016
[62]

Disentangled repre- sentation learning gan for pose-invariant face recognition

Luan Tran, Xi Yin, and Xiaoming Liu. Disentangled repre- sentation learning gan for pose-invariant face recognition. In CVPR, 2017. 1

2017
[63]

Mnemonic descent method: A recurrent process applied for end-to-end face alignment

George Trigeorgis, Patrick Snape, Mihalis A Nico- laou, Epameinondas Antonakos, and Stefanos Zafeiriou. Mnemonic descent method: A recurrent process applied for end-to-end face alignment. In CVPR, 2016. 2, 5

2016
[64]

Robust face alignment using a mixture of invariant experts

Oncel Tuzel, Tim K Marks, and Salil Tambe. Robust face alignment using a mixture of invariant experts. In ECCV,
[65]

A deeply-initialized coarse-to-ﬁne ensemble of regression trees for face alignment

Roberto Valle, Jose M Buenaposada, Antonio Valdes, and Luis Baumela. A deeply-initialized coarse-to-ﬁne ensemble of regression trees for face alignment. In ECCV, 2018. 6

2018
[66]

Unsupervised learning of visual representations using videos

Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. In CVPR, 2015. 2

2015
[67]

Convolutional pose machines

Shih-En Wei, Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. Convolutional pose machines. In CVPR, 2016. 2

2016
[68]

Slow feature analysis: Unsupervised learning of invariances

Laurenz Wiskott and Terrence J Sejnowski. Slow feature analysis: Unsupervised learning of invariances. Neural com- putation, 14(4):715–770, 2002. 2

2002
[69]

Look at boundary: A boundary-aware face alignment algorithm

Wayne Wu, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, and Qiang Zhou. Look at boundary: A boundary-aware face alignment algorithm. In CVPR, 2018. 1, 2, 4, 5, 6, 7, 12

2018
[70]

Leveraging intra and inter- dataset variations for robust face alignment

Wayne Wu and Shuo Yang. Leveraging intra and inter- dataset variations for robust face alignment. In CVPR, 2017. 5

2017
[71]

Recurrent 3d- 2d dual learning for large-pose facial landmark detection

Shengtao Xiao, Jiashi Feng, Luoqi Liu, Xuecheng Nie, Wei Wang, Shuicheng Yan, and Ashraf Kassim. Recurrent 3d- 2d dual learning for large-pose facial landmark detection. In ICCV, 2017. 5, 6

2017
[72]

Robust facial land- mark detection via recurrent attentive-reﬁnement networks

Shengtao Xiao, Jiashi Feng, Junliang Xing, Hanjiang Lai, Shuicheng Yan, and Ashraf Kassim. Robust facial land- mark detection via recurrent attentive-reﬁnement networks. In ECCV, 2016. 2, 6

2016
[73]

Global supervised descent method

Xuehan Xiong and Fernando De la Torre. Global supervised descent method. In CVPR, 2015. 2, 6

2015
[74]

Wider face: A face detection benchmark

Shuo Yang, Ping Luo, Chen-Change Loy, and Xiaoou Tang. Wider face: A face detection benchmark. In CVPR, 2016. 5

2016
[75]

The menpo facial landmark localisation challenge: A step towards the solution

Stefanos Zafeiriou, George Trigeorgis, Grigorios Chrysos, Jiankang Deng, and Jie Shen. The menpo facial landmark localisation challenge: A step towards the solution. In CVPR Workshops, 2017. 2

2017
[76]

Taskonomy: Disentangling task transfer learning

Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In CVPR, 2018. 2

2018
[77]

Coarse-to-ﬁne auto-encoder networks (cfan) for real-time face alignment

Jie Zhang, Shiguang Shan, Meina Kan, and Xilin Chen. Coarse-to-ﬁne auto-encoder networks (cfan) for real-time face alignment. In ECCV, 2014. 6

2014
[78]

Colorful image colorization

Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In ECCV, 2016. 2

2016
[79]

Split-brain autoencoders: Unsupervised learning by cross-channel pre- diction

Richard Zhang, Phillip Isola, and Alexei A Efros. Split-brain autoencoders: Unsupervised learning by cross-channel pre- diction. In CVPR, 2017. 2

2017
[80]

Learning deep representation for face alignment with auxiliary attributes

Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Learning deep representation for face alignment with auxiliary attributes. IEEE transactions on pattern analysis and machine intelligence, 2016. 2, 6

2016
[81]

Unpaired image-to-image translation using cycle- consistent adversarial networks

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. In ICCV, 2017. 4

2017
[82]

Face alignment by coarse-to-ﬁne shape searching

Shizhan Zhu, Cheng Li, Chen Change Loy, and Xiaoou Tang. Face alignment by coarse-to-ﬁne shape searching. In CVPR, 2015. 2, 5, 6

2015
[83]

Unconstrained face alignment via cascaded compo- sitional learning

Shizhan Zhu, Cheng Li, Chen-Change Loy, and Xiaoou Tang. Unconstrained face alignment via cascaded compo- sitional learning. In CVPR, 2016. 2, 5

2016
[84]

Face alignment across large poses: A 3d solu- tion

Xiangyu Zhu, Zhen Lei, Xiaoming Liu, Hailin Shi, and Stan Z Li. Face alignment across large poses: A 3d solu- tion. In CVPR, 2016. 6

2016
[85]

Deep learning of invariant features via simulated ﬁxations in video

Will Zou, Shenghuo Zhu, Kai Yu, and Andrew Y Ng. Deep learning of invariant features via simulated ﬁxations in video. In NIPS, 2012. 2 Appendix The content of our supplementary material is organized as follows

2012
[86]

More ablation studies and detailed analysis of compo- nents in our framework
[87]

Additional discussion about related directions
[88]

over-smoothness

Details of our annotated AFLW-68 dataset and some representative visualized samples. S1. More Ablation Studies In this section, we provide additional analysis about each design in our framework to facilitate understanding of our structure. Two key loss terms in our framework a...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Data augmentation generative adversarial networks, 2018

Anthreas Antoniou, Amos Storkey, and Harrison Edwards. Data augmentation generative adversarial networks, 2018. 12

2018

[2] [2]

Synthesizing images of humans in unseen poses

Guha Balakrishnan, Amy Zhao, Adrian V Dalca, Fredo Du- rand, and John Guttag. Synthesizing images of humans in unseen poses. In CVPR, 2018. 2

2018

[3] [3]

Mutual information neural estimation

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajesh- war, Sherjil Ozair, Yoshua Bengio, Devon Hjelm, and Aaron Courville. Mutual information neural estimation. In ICML,

[4] [4]

Localizing parts of faces using a consensus of exemplars

Peter N Belhumeur, David W Jacobs, David J Kriegman, and Neeraj Kumar. Localizing parts of faces using a consensus of exemplars. IEEE transactions on pattern analysis and machine intelligence, 35(12):2930–2940, 2013. 5, 13

2013

[5] [5]

Convolutional aggregation of local evidence for large pose face alignment

Adrian Bulat and Georgios Tzimiropoulos. Convolutional aggregation of local evidence for large pose face alignment

[6] [6]

How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks)

Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In ICCV, 2017. 1

2017

[7] [7]

Robust face landmark estimation under occlusion

Xavier P Burgos-Artizzu, Pietro Perona, and Piotr Doll ´ar. Robust face landmark estimation under occlusion. In CVPR,

[8] [8]

Face alignment by explicit shape regression

Xudong Cao, Yichen Wei, Fang Wen, and Jian Sun. Face alignment by explicit shape regression. IJCV, 2014. 2, 6

2014