Pith. sign in

REVIEW 4 major objections 5 minor 37 references

Atypical Facial Landmark Localisation with Stacked Hourglass Networks: A Study on 3D Facial Modelling for Medical Diagnosis

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that a stacked-hourglass face alignment network predicts facial landmarks on atypical palsy faces more accurately than traditional methods, with the largest advantage at the mouth.

desk verdict Plausible, clearly-written small evaluation study; the FAN-vs-DRMF-vs-TSM comparison is new, but missing error bars and an overreaching conclusion keep the headline claim unproven. read the letter →

arxiv 1909.02157 v1 pith:BBVA6FXC submitted 2019-09-05 cs.LG cs.CVeess.IVstat.ML

Gary Storey , Ahmed Bouridane , Richard Jiang , Chang-tsun Li This is my paper · ORCID
classification cs.LGcs.CVeess.IVstat.ML
keywords faciallandmarklocalisationpalsyatypicalasymmetrystackedhourglassnetworksFaceAlignmentNetwork3Dmodellingmedicalbiometricsdeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether facial landmark localisation methods built and trained on typical faces remain accurate when applied to atypical faces affected by facial palsy. It evaluates three methods — a tree shape model, a discriminative response-map fitting method, and the deep stacked-hourglass-based Face Alignment Network — on two datasets of facial palsy images. The authors' central finding is that only the deep network accurately predicts landmarks on these atypical faces, with its clearest advantage in the mouth region, where palsy produces the largest deformation. Landmark accuracy matters here because landmark positions are the foundation for 3D facial modelling and geometric features that could assist medical diagnosis and rehabilitation.

What carries the argument

The central mechanism is the stacked hourglass architecture, a convolutional network that repeatedly downsamples an image to low resolution and then upsamples back to full resolution, branching off intermediate features so that predictions combine information from every scale. The Face Alignment Network adapts this design for faces by stacking four hourglasses, replacing bottleneck blocks with hierarchical, parallel multi-scale residual blocks, and outputting a heatmap for each of 68 landmarks, with intermediate supervision applying an MSE loss at each hourglass. A separate depth network, based on a deep residual architecture, takes the RGB image plus predicted heatmaps and outputs per-landmark depth, extending 2D alignment to 3D. This machinery is what the paper credits for accurate localisation on atypical faces, particularly at the mouth.

What would settle it

Re-annotate the same 87 palsy images with several independent human raters and measure inter-rater landmark variability; if rater disagreement in the mouth region is as large as or larger than the reported normalised mean error gap between FAN and DRMF, the claimed advantage could stem from label noise rather than method performance.

Watch

Extended reading notes

Core claim

On two facial palsy datasets (47 images with 12 landmarks and 40 images with 18 landmarks), the paper compares the Tree Shape Model, DRMF, and the stacked-hourglass Face Alignment Network using normalised mean error. The authors report that FAN shows consistently higher accuracy across both datasets; DRMF matches it on eye and nose landmarks but degrades substantially on highly asymmetric samples, and TSM performs poorly throughout. The conclusion is direct: of the methods evaluated, only the stacked-hourglass-based FAN could accurately predict facial landmarks on atypical faces, especially the mouth landmarks that show the highest degree of atypical appearance. The paper presents this as evidence that the stacked hourglass architecture is a suitable front end for 3D facial modelling of palsy patients.

Load-bearing premise

The evaluation assumes the manually annotated ground-truth landmarks in the two facial palsy datasets are accurate and that those 87 images represent atypical faces; the paper gives no annotation protocol, inter-observer variability measure, or palsy severity distribution, so noisy or biased labels would make the reported accuracy ranking unreliable.

Editorial extensions

If this is right

  • A stacked-hourglass alignment network can serve as a reliable alignment step for 3D facial modelling in facial palsy diagnosis and rehabilitation planning.
  • Automated diagnostic pipelines can extract geometric features from mouth landmarks on atypical faces even though the network was trained on typical-face data.
  • Traditional shape-model and regression baselines such as TSM and DRMF are not reliable on atypical faces, so clinical systems should not rely on them.
  • The same architecture may transfer to other atypical landmark localisation tasks, such as ear and hand biometrics, as the paper itself suggests.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The study uses only 87 images total and reports no annotation protocol or inter-observer variability, so the size of FAN's advantage should be re-measured on larger, severity-graded palsy datasets before clinical deployment.
  • The paper validates 2D landmark accuracy but does not directly validate the depth network's 3D output on atypical faces; comparing predicted 3D geometry against clinical asymmetry measurements would be a direct next test.
  • If the mouth-region advantage is stable, palsy grading systems could weight mouth landmarks most heavily when computing asymmetry scores.
  • The transfer from typical training faces to an unseen atypical group suggests heatmap-based localisers may be robust to other facial differences, such as cleft lip or craniofacial syndromes; this is a testable extension the paper does not claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript evaluates three facial landmark localisation methods—TSM, DRMF, and the stacked-hourglass-based Face Alignment Network (FAN)—on two small datasets of facial palsy images (47 and 40 images). The evaluation uses Normalised Mean Error (NME) with cumulative error distributions and per-landmark error plots. The paper reports that FAN outperforms TSM and DRMF, especially on mouth landmarks, and concludes that FAN is the only method that can accurately predict landmarks on atypical faces. The text also reviews landmark localisation history and describes the stacked hourglass architecture and the FAN method.

Significance. If the comparative result is substantiated, the paper identifies a practical off-the-shelf landmark localiser for a medically relevant population, which would be a useful contribution for 3D facial modelling and diagnostic pipelines. The study benefits from evaluating an independent, pre-trained FAN model with no parameters fitted in this paper, avoiding circularity. The focus on atypical faces is timely and underrepresented in public benchmarks. However, the evidence is thin: only 87 images in total, no uncertainty measures, no statistical significance tests, and no explicit definition of the common landmark subset. The conclusions in Section 5 go beyond what the presented quantitative support can justify.

major comments (4)
  1. [Section 4, Figs. 6-8] The central comparative claim that FAN outperforms TSM and DRMF is not supported by uncertainty quantification. With only 47 and 40 images in the two datasets, cumulative NME curves without confidence intervals, standard deviations, or paired significance tests do not establish that the observed differences are beyond sampling variability. Please report per-image NME statistics (e.g., mean/median with bootstrapped CIs) and run a paired test, such as the Wilcoxon signed-rank test, between methods on the common landmark subset.
  2. [Section 4, common landmark subset] The text states that 'a subset of facial landmarks are used which are common across all methods', but the actual landmark subset is never specified. Since TSM, DRMF, and FAN output different landmark templates, the choice of common landmarks can materially affect the comparison. Please list the landmark indices or positions used and justify that they are semantically matched across methods.
  3. [Section 5, Conclusion] The conclusion that 'only the state-of-the-art FAN method could accurately predict facial landmarks' uses 'accurately' without defining an error threshold or a clinical accuracy requirement. Figures 6-8 do not provide such a threshold, so even a genuine ranking advantage would not establish the exclusivity claim. Please define an acceptable NME threshold or soften the wording to 'most accurately among the methods evaluated'.
  4. [Section 4, dataset description] No annotation protocol, inter-observer variability measure, or palsy severity distribution is provided for the two facial palsy datasets. If the ground truth landmarks are noisy or biased, the comparative NME results become unreliable. Please include these details or explicitly acknowledge them as limitations, especially in light of the small sample sizes.
minor comments (5)
  1. [Section 4 heading] The heading 'Evalaution' is a typo and should read 'Evaluation'.
  2. [Section 3.1] The sentence beginning 'without the application of these branches rather than learn features at each scale the network would behave in a manner previously shown in Fig.2' references Fig.2, which shows asymmetrical faces, not feature-layer behaviour; the citation is incorrect and the sentence is grammatically incomplete.
  3. [Section 2.1] The term 'Procustes' should be 'Procrustes' in the description of shape alignment.
  4. [References [21] and [22]] References [21] and [22] appear to be the same paper (same title and venue); please merge them or distinguish them appropriately.
  5. [Section 3.4] The depth network for 3D landmarks is described but not used in the evaluation; please clarify whether 3D landmark predictions are evaluated at all, and if not, state why this section is included.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the evaluation compares an independently pre-trained FAN against traditional methods on new atypical-face data, with no fitted parameters or definitional reductions.

full rationale

This paper is an empirical evaluation, not a derivation. The central claim—that FAN outperforms TSM and DRMF on facial-palsy landmark localisation—is supported by cumulative NME curves and per-landmark plots measured on two previously unseen datasets (Section 4). The FAN model was developed and pre-trained by Bulat and Tzimiropoulos (references [2] and [3]), who are not authors of this paper; the paper describes that training procedure but does not fit any parameters itself. No equation in the paper defines a predictand in terms of the measured data, and no fitted input is renamed as a prediction. The only self-citations ([21], [22], and [36]) are used as background motivation or as related work, not to justify the comparative result. The comparison metric (NME with face-size normalization) is taken from [3] but is applied uniformly to all methods, so it does not privilege FAN by construction. The common-landmark subset is unspecified, which is a reproducibility or correctness weakness, not a circularity. Therefore no circular step can be exhibited with a quote, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are introduced by this paper; the FAN model is used pre-trained from prior work. The evaluation rests on the quality of the two private datasets and the choice of NME as metric.

assumptions (3)
  • domain assumption Ground truth landmark annotations in datasets A and B are accurate and consistent.
    No annotation protocol, inter-observer agreement, or quality control is described; the entire evaluation depends on these labels.
  • domain assumption The 87 images (47+40) are representative of the facial palsy population.
    Datasets are small and no severity distribution or demographic information is provided.
  • standard math Normalized mean error with face size normalization as in [3] is an appropriate accuracy metric for this comparison.
    The metric is standard in face alignment, but its choice is not justified for atypical faces.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Atypical Facial Landmark Localisation with Stacked Hourglass Networks: A Study on 3D Facial Modelling for Medical Diagnosis." pith.science (2026). https://pith.science/paper/BBVA6FXC

@misc{pith2026190902157,
  author       = {Pith},
  title        = {Pith review of: Atypical Facial Landmark Localisation with Stacked Hourglass Networks: A Study on 3D Facial Modelling for Medical Diagnosis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BBVA6FXC}},
  note         = {Machine review of arXiv:1909.02157}
}
read the original abstract

While facial biometrics has been widely used for identification purpose, it has recently been researched as medical biometrics for a range of diseases. In this chapter, we investigate the facial landmark detection for atypical 3D facial modelling in facial palsy cases, while potentially such modelling can assist the medical diagnosis using atypical facial features. In our work, a study of landmarks localisation methods such as stacked hourglass networks is conducted and evaluated to ascertain their accuracy when presented with unseen atypical faces. The evaluation highlights that the state-of-the-art stacked hourglass architecture outperforms other traditional methods.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages

  1. [1]

    Robust discriminative response map fitting with constrained local models

    Akshay Asthana, Stefanos Zafeiriou, Shiyang Cheng, and Maja Pantic. Robust discriminative response map fitting with constrained local models. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 3444–3451, 2013

    work page 2013

  2. [2]

    Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources

    Adrian Bulat and Georgios Tzimiropoulos. Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources. 2017IEEEInternational Conference on Computer Vision (ICCV), pages 3726–3734, 3 2017. Atypical Facial Landmark Localisation: A study 15

    work page 2017

  3. [3]

    How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)

    Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks). 2017 IEEE International Conference on Computer Vision (ICCV), pages 1021–1030, 10 2017

    work page 2017

  4. [4]

    Convolutional aggregation of local evidence for large pose face alignment

    Adrian Bulat and Yorgos Tzimiropoulos. Convolutional aggregation of local evidence for large pose face alignment. In Procedings of the British Machine Vision Conference 2016 , pages 1–86. British Machine Vision Association, 2016

    work page 2016

  5. [5]

    Burgos-Artizzu, Pietro Perona, and Piotr Dollar

    Xavier P. Burgos-Artizzu, Pietro Perona, and Piotr Dollar. Robust Face Landmark Estimation under Occlusion. In 2013 IEEE International Conference on Computer Vision , pages 1513–

    work page 2013

  6. [6]

    Face alignment by explicit shape regression

    Xudong Cao, Yichen Wei, Fang Wen, and Jian Sun. Face alignment by explicit shape regression. International Journal of Computer Vision, 107(2):177–190, 2014

    work page 2014

  7. [7]

    Cootes, G.J

    T.F. Cootes, G.J. Edwards, and C.J. Taylor. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):681–685, 2001

    work page 2001

  8. [8]

    An introduction to active shape models

    Tim Cootes, Er Baldock, and J Graham. An introduction to active shape models. Image Processing and Analysis, pages 223–248, 2000

    work page 2000

Show all 37 references
  1. [9]

    Automatic feature localisation with constrained l ocal models

    David Cristinacce and Tim Cootes. Automatic feature localisation with constrained l ocal models. Pattern Recognition, 41(10):3054–3067, 2008

    2008

  2. [10]

    Cascaded pose regression

    Piotr Dollár, Peter Welinder, and Pietro Perona. Cascaded pose regression. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1078–1085, 2010

    2010

  3. [11]

    Active appearance models with occlusion

    Ralph Gross, Iain Matthews, and Simon Baker. Active appearance models with occlusion. Image and Vision Computing, 24(6):593–604, 2006

    2006

  4. [12]

    Pose -Invariant Face Alignment via CNN -Based Dense 3D Model Fitting

    Amin Jourabloo and Xiaoming Liu. Pose -Invariant Face Alignment via CNN -Based Dense 3D Model Fitting. International Journal of Computer Vision, 124(2), 2017

    2017

  5. [13]

    Pintavirooj

    Aniwat Juhong and C. Pintavirooj. Face recognition based on facial landmark detection. In 2017 10th Biomedical Engineering International Conference (BMEiCON), pages 1–4. IEEE, 8 2017

    2017

  6. [14]

    Deep Recurrent Regression for Facial Landmark Detection

    Hanjiang Lai, Shengtao Xiao, Yan Pan, Zhen Cui, Jiashi Feng, Chunyan Xu, Jian Yin, and Shuicheng Yan. Deep Recurrent Regression for Facial Landmark Detection. 10 2015

    2015

  7. [15]

    Unconstrained Facial Landmark Localization with Backbone-Branches Fully-Convolutional Networks

    Zhujin Liang, Shengyong Ding, and Liang Lin. Unconstrained Facial Landmark Localization with Backbone-Branches Fully-Convolutional Networks. arXiv:1507.03409 [cs], 1, 7 2015

    2015 arXiv

  8. [16]

    Active Shape Models with SIFT Descriptors and MARS

    Stephen Milborrow and Fred Nicolls. Active Shape Models with SIFT Descriptors and MARS. Proceedings of the 9th International Conference on Computer Vision Theory and Applications, (i):380–387, 2014

    2014

  9. [17]

    Stacked Hourglass Networks for Human Pose Estimation

    Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked Hourglass Networks for Human Pose Estimation. In Computer Vision âĂŞ ECCV 2016, pages 483–499. Springer, Cham, 10 2016

    2016

  10. [18]

    Face detection, pose estimation, and landmark localization in the wild

    Deva Ramanan, Xiangxin Zhu, and Deva Ramanan. Face detection, pose estimation, and landmark localization in the wild. 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 2879–2886, 6 2012

    2012

  11. [19]

    Face Alignment at 3000 FPS via Regressing Local Binary Features

    Shaoqing Ren, Xudong Cao, Yichen Wei, and Jian Sun. Face Alignment at 3000 FPS via Regressing Local Binary Features. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 1685–1692. IEEE, 6 2014

    2014

  12. [20]

    300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge

    Christos Sagonas, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge. In 2013 IEEE International Conference on Computer Vision Workshops, pages 397–403. IEEE, 12 2013

    2013

  13. [21]

    Storey, R

    G. Storey, R. Jiang, and A. Bouridane. Role for 2D image generated 3D face models in the rehabilitation of facial palsy. Healthcare Technology Letters, 4(4), 2017

    2017

  14. [22]

    Role for 2D image generated 3D face models in the rehabilitation of facial palsy

    Gary Storey, Richard Jiang, and Ahmed Bouridane. Role for 2D image generated 3D face models in the rehabilitation of facial palsy. Healthcare Technology Letters, 4(4):145–148, 8 2017

    2017

  15. [23]

    Deep Convolutional Network Cascade for Facial Point Detection

    Yi Sun, Xiaogang Wang, and Xiaoou Tang. Deep Convolutional Network Cascade for Facial Point Detection. In 2013 IEEE Conference on Computer Vision and Pattern Recognition , pages 3476–3483. IEEE, 6 2013. 16 Gary Storey , Ahmed Bouridane , Richard Jiang and Chang-tsun Li

    2013

  16. [24]

    Facial point detection using boosted regression and graph models

    Michel Valstar, Brais Martinez, Xavier Binefa, and Maja Pantic. Facial point detection using boosted regression and graph models. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 2729–2736, 2010

    2010

  17. [25]

    Robust Facial Landmark Detection via Recurrent Attentive-Refinement Networks

    Shengtao Xiao, Jiashi Feng, Junliang Xing, Hanjiang Lai, Shuicheng Yan, and Ashraf Kassim. Robust Facial Landmark Detection via Recurrent Attentive-Refinement Networks. pages 57–

  18. [26]

    Hand biometrics

    Erdem Yörük, Helin Dutağaci, and BÃijlent Sankur. Hand biometrics. Image and Vision Computing, 24(5):483–497, 5 2006

    2006

  19. [27]

    Facial landmark detection by deep multi -task learning

    Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial landmark detection by deep multi -task learning. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , volume 8694 LNCS, pages...

    2014

  20. [28]

    Deformable Models of Ears in-the-Wild for Alignment and Recognition

    Yuxiang Zhou and Stefanos Zaferiou. Deformable Models of Ears in-the-Wild for Alignment and Recognition. In 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), pages 626–633. IEEE, 5 2017

    2017

  21. [29]

    Face alignment by coarse-tofine shape searching

    Shizhan Zhu, Cheng Li, Chen Change Loy, and Xiaoou Tang. Face alignment by coarse-tofine shape searching. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 07-12-June, pages 4998–5006, 2015

    2015

  22. [30]

    Xiangyu Zhu, Zhen Lei, Xiaoming Liu, Hailin Shi, and Stan Z. Li . Face Alignment Across Large Poses: A 3D Solution. page 11, 11 2015

    2015

  23. [31]

    Emotion recognition from scrambled facial images via many graph embedding

    R. Jiang, et al, “Emotion recognition from scrambled facial images via many graph embedding”, Pattern Recognition, 67, 2017, p.245-251

    2017

  24. [32]

    Face recognition in the scrambled domain via salience -aware ensembles of many kernels

    R. Jiang, et al, “Face recognition in the scrambled domain via salience -aware ensembles of many kernels”, IEEE Trans. Information Forensics and Security, 11(8), 2016, p.1807-1817

    2016

  25. [33]

    Privacy -protected facial biometric verification via fuzzy forest learning

    R. Jiang, et al, “Privacy -protected facial biometric verification via fuzzy forest learning”, IEEE Trans. Fuzzy Systems, 24(4), 2016, p.779-790

    2016

  26. [34]

    Multimodal Biometric Human Recognition for Perceptual Human–Computer Interaction

    R. Jiang, et al, “Multimodal Biometric Human Recognition for Perceptual Human–Computer Interaction”, IEEE Trans. Systems, Man, & Cybernetics Part C, 40(5), 2010, p.676

    2010

  27. [35]

    Face recognition in global harmonic subspace

    R. Jiang, et al, “ Face recognition in global harmonic subspace”, IEEE Trans. Information Forensics and Security, 5(3), 2010, p.416-424

    2010

  28. [36]

    3DPalsyNet: A Facial Palsy Grading and Motion Recognition Framework using Fully 3D Convolutional Neural Networks

    G. Storey, et al, “3DPalsyNet: A Facial Palsy Grading and Motion Recognition Framework using Fully 3D Convolutional Neural Networks”, IEEE Access, 2019, in press

    2019

  29. [72]

    Springer, Cham, 10 2016

    2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.