ORFormer: Occlusion-Robust Transformer for Accurate Facial Landmark Detection

Bo-Syuan Hou; Chia-Yu Tseng; Hou-Ning Hu; Jui-Che Chiang; Min-Hung Chen; Yen-Yu Lin; Yu-Lun Liu

REVIEW 3 major objections 6 minor 49 references

ORFormer: Occlusion-Robust Transformer for Accurate Facial Landmark Detection

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A transformer that detects and repairs occluded facial patches reaches state-of-the-art landmark accuracy.

desk verdict The messenger-token mechanism is a real architectural contribution, but the headline SOTA number is confounded by a loss-function change and needs a missing control before the central claim is clean. read the letter →

arxiv 2412.13174 v2 pith:PY2JWSIZ submitted 2024-12-17 cs.CV cs.AIcs.LG

Jui-Che Chiang , Hou-Ning Hu , Bo-Syuan Hou , Chia-Yu Tseng , Yu-Lun Liu , Min-Hung Chen , Yen-Yu Lin This is my paper · ORCID

classification cs.CVcs.AIcs.LG

keywords faciallandmarkdetectionocclusionrobustnesstransformermessengertokenheatmapregressionfeaturerecoveryvectorquantization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ORFormer claims that facial landmark detection under occlusion can be made robust by having the transformer explicitly identify which image patches are non-visible and then rebuild their features from the rest of the face. The method adds one learnable messenger token per image patch; each messenger attends to every patch except its own, so the gap between a patch's own embedding and its messenger embedding reveals whether that patch is corrupted. A predicted occlusion map then blends the patch embedding with the messenger embedding, producing recovered features that yield high-quality facial-edge heatmaps. These heatmaps, fed into existing landmark detectors, push normalized mean error to 3.86 on WFLW and 4.46 on COFW, outperforming prior state-of-the-art results. The appeal is a general, annotation-free way to make transformer-based detectors resilient to partial occlusion.

What carries the argument

The messenger token is the load-bearing object: for each image patch $X_i$ a learnable token $M_i$ runs cross-attention over all patch tokens except $X_i$ (attention score set to 0 at the diagonal). The difference between the resulting messenger embedding and the patch's own embedding, passed through a linear layer and sigmoid, gives the per-patch occlusion probability $\alpha$. That $\alpha$ is used twice: multiplicatively in the next layer's cross-attention to stop messengers from borrowing features from already-occluded patches, and as the interpolation weight in the final feature recovery $Z_{\mathrm{rec}} = (1-\alpha)\otimes Z_I + \alpha\otimes Z_M$, where $Z_I$ and $Z_M$ are codebook-quantized features from regular and messenger embeddings. The codebook and decoder are pre-trained on unoccluded faces via vector quantization and frozen, so the recovery step draws on a prior of what intact faces look like.

What would settle it

Feed ORFormer a set of faces with known synthetic occlusion masks and compare its predicted alpha map to the masks: if visibly covered patches do not receive high alpha, or if forcing alpha to zero (no recovery) does not raise NME on the WFLW occlusion subset, the claimed occlusion-detection and recovery mechanism is not what drives the accuracy gain.

Watch

Extended reading notes

Core claim

The central claim is that a pair of embeddings per patch — the regular self-attended token and a messenger token that deliberately excludes the patch itself — carries enough information to locate occluded regions and to repair them. ORFormer quantifies the discrepancy between the two embeddings with a squared-difference head, converts it into a patch-wise occlusion probability $\alpha$ in $[0,1]$, and uses $\alpha$ to form a convex combination $Z_{\mathrm{rec}} = (1-\alpha)\otimes Z_I + \alpha\otimes Z_M$ of the regular and messenger quantized features before decoding heatmaps. Because the messenger gathers evidence only from visible patches, the recovered representation is robust to corruption; because $\alpha$ is predicted from the input, no ground-truth occlusion labels are required. On WFLW, COFW, and the 300W challenging subset, the method reports the best normalized mean error among the compared methods, and on the WFLW occlusion subset it gains more than the full-set average over the VQVAE and CodeFormer baselines.

Load-bearing premise

The load-bearing premise is that the gap between a patch's own embedding and its messenger embedding reliably marks occluded patches, yet that occlusion map is never supervised with ground-truth labels, so a wrong alpha would make the recovery step hurt rather than help.

Editorial extensions

If this is right

Any heatmap-based facial landmark detector can adopt ORFormer's recovered heatmaps as auxiliary inputs; in the paper both ADNet and STAR improve under their own loss functions, with the largest gains on occlusion-heavy subsets.
The messenger mechanism needs no occlusion annotations at training or test time, so it transfers to datasets (like WFLW) that do not label occlusions.
Compared with a plain ViT, ORFormer adds about 10% more parameters, and the separate attention streams keep the extra compute linear in the token count.
Against a matched-parameter hourglass baseline, the full pipeline uses 20.6% fewer mult-adds and 15.9% less inference time while improving NME.
The strongest configuration comes from training the whole landmark model from scratch with simple L2 heatmap loss plus NME landmark loss, rather than fine-tuning only the merging block.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because alpha is never supervised with occlusion masks, a natural extension is to add a weak or self-supervised loss on alpha using COFW's existing per-landmark occlusion labels; the paper's own numbers suggest this could further close the gap on the occlusion subset.
The same messenger-token design could apply to other partially occluded recognition tasks, such as object keypoint detection or pose estimation, wherever a quantized prior over intact appearances can be pre-trained.
The method's robustness to occlusion types absent from training (sunglasses, masks, hands) is untested; a synthetic-occlusion stress test with varying rectangle sizes could show whether the learned alpha generalizes beyond the augmentation used.
The paper's stated limitation that deformed facial features still hurt suggests a possible boundary: the messenger prior is a prior over intact geometry, so appearance deformation may be harder to model than simple occlusion.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The messenger-token idea is the real thing here. Attaching a learnable token to each patch that attends to all patches except its own, then using the divergence between the regular and messenger embeddings to produce a per-patch occlusion map, is a genuine architectural departure from CodeFormer and HandOccNet. The paper also earns credit for the controlled ablations: with the STAR loss held fixed, ORFormer improves WFLW NME from 4.03 to 3.92 and the occlusion subset from 4.82 to 4.66 (Table 6). Those are modest but consistent gains, and the complexity comparison against a 5-stack Hourglass (Table 12) is fair. The supplementary material is unusually complete for this area, with index mappings, hyperparameters, and complexity analysis. The soft spots are real but not fatal. The main one is the headline claim. Table 1 reports WFLW NME 3.86, which beats the reproduced STAR baseline's 4.03, but Table 10 shows that 3.86 comes from L2 heatmap loss plus NME landmark loss, not STAR's AWing+STAR losses. The paper never trains the same backbone without ORFormer under L2+NME. So the state-of-the-art improvement in Table 1 cannot be attributed to the messenger-token mechanism; the loss change alone could explain part or all of it. The missing control is one training run. This is a confound, not a fatal flaw, because the same-loss Table 6 comparison already provides positive evidence for ORFormer. Second, the paper claims ORFormer detects non-visible regions but never quantifies the alpha map against ground-truth occlusion labels, even though COFW provides them. Figure 6 is qualitative only. That is a secondary issue, since a perfect alpha map would not fix the loss-function attribution problem, but it would strengthen the detection story. Minor discrepancy: Section 3.1 says the quantized heatmap generator is trained on faces without occlusions, while Section 4.1 describes adding random occlusion for the generator and ORFormer. The text should clarify whether the generator sees occluded images during pre-training. This is a solid incremental-to-substantial empirical paper with an honest evaluation scaffold. It deserves a serious referee. I would recommend major revision: add the no-ORFormer L2+NME baseline, validate alpha on COFW occlusion labels, and clarify the training-data discrepancy. With those changes, the contribution would be cleanly established.

Referee Report

3 major / 6 minor

Summary. The paper proposes ORFormer, a ViT-based module for facial landmark detection under partial occlusion. Each image patch token is paired with a learnable messenger token that attends to all patches except its own; an occlusion detection head derives a per-patch occlusion map alpha from the squared difference between the regular and messenger embeddings, and a feature-recovery step merges the two quantized feature streams with alpha. The recovered features are decoded into edge heatmaps, which are concatenated into existing FLD backbones (ADNet/STAR). Experiments on WFLW, COFW, and 300W report state-of-the-art or competitive NME/FR/AUC, with ablation studies showing consistent gains when ORFormer is added to STAR under the same loss, and efficiency gains over a parameter-matched five-stack Hourglass.

Significance. If the empirical attribution is sound, the paper makes a useful contribution: the messenger-token design is a generic mechanism for detecting and recovering corrupted or occluded patches, and the controlled comparison in Table 6 (STAR 4.03 -> 3.92 on WFLW full, 4.82 -> 4.66 on the occlusion subset, with matching loss) is direct evidence that the mechanism helps. The paper is also transparent about implementation details, reports reproduced baselines, and includes ablations for the distance function, integration strategy, and loss choices. The main significance concern is whether the headline Table 1 numbers are attributable to ORFormer or to the change of loss function, and whether the cross-attention 'exclusion' is implemented as described.

major comments (3)

[§3.2, Eqs. (5)–(6) and Eq. (8)] The claimed exclusion of the corresponding patch is not implemented by the equations as written. Setting Across(Q_M,K_X)_{i,i}=0 before softmax assigns a finite score of 0; after softmax, the own-patch token still receives a positive attention weight unless all other scores are also 0. Similarly, Eq. (8) multiplies the cross-attention score by (1-alpha_j) rather than applying a true mask, so the messenger token does not strictly aggregate 'features from all but its patch.' Please either replace the zero assignment with -inf masking (or the equivalent) or clarify that the intended operation is a score discount rather than exclusion, and confirm that the reported experiments use the described operation.
[§4.3 and §6.2.2, Tables 1 and 10] The headline WFLW NME of 3.86 in Table 1 is produced by the L2+NME loss configuration listed in Table 10, whereas the reproduced STAR baseline (marked with a dagger, 4.03) uses STAR's AWing+STAR losses. The paper never trains the same four-stack Hourglass backbone without ORFormer under the L2+NME loss, so the 3.86 number, and the state-of-the-art claim built on it, cannot be attributed to ORFormer rather than to the loss change. Please add the missing no-ORFormer L2+NME control, or report the SOTA comparison only with matched loss functions, as in Table 6, where the ORFormer gain is 4.03 to 3.92.
[§3.2, Eq. (7), and §4.4] The occlusion map alpha is load-bearing for feature recovery in Eq. (9), yet it is never quantitatively validated against ground-truth occlusion information. COFW provides per-landmark occlusion labels and WFLW provides subset labels, but the paper only visualizes alpha (Fig. 6). Because alpha is an unconstrained learned signal, please add a direct evaluation of alpha against occlusion annotations (e.g., patch-level precision/recall or AUC, or NME grouped by alpha threshold) to support the claim that ORFormer 'detects non-visible regions'; at minimum, report an ablation where alpha is replaced by a constant or by oracle-like values.

minor comments (6)

[§3.2, Eq. (10)] In Eq. (10), S_k is a code index, not a one-hot vector; the expression '-S_k log(hat S_k)' should be written as '-log(hat S_{k,S_k})' or with explicit one-hot/cross-entropy notation.
[§3.1, Eq. (1)] The subscript notation in Eq. (1) is inconsistent: 'Z i,j' and 'Z_{i,j}' appear side by side; please use a single typesetting for subscripts throughout.
[Acknowledgements] The heading 'Ackknowledgements' contains a typo and should read 'Acknowledgements'.
[§4.4 and §6.2.2, Tables 4 and 11] Tables 4 and 11 are identical duplicate ablation tables; please keep only one copy and cross-reference the other.
[Table 4 caption] The caption contains 'the he landmark NME loss'; it should read 'the landmark NME loss'.
[§6.2.3, Table 12] The term 'mult-add' is unclear; please use 'MACs' or 'multiply-accumulate operations' for precision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ORFormer is a standard two-stage supervised pipeline whose heatmap and landmark predictions are evaluated on held-out WFLW, COFW, and 300W test sets; no prediction reduces by construction to a fitted input or to a self-citation.

full rationale

ORFormer's derivation chain is an empirical two-stage training pipeline rather than a formal derivation with hidden inputs. Section 3.1 trains a quantized heatmap generator on unoccluded faces using edge heatmaps derived from landmark annotations. Section 3.2 freezes the codebook and decoder, trains ORFormer to predict code sequences SI and SM, and refines them with the occlusion map alpha (Eqs. 4-11), where the code target S is 'obtained from the pre-trained heatmap generator.' This target is a fixed teacher-student / denoising supervision signal, not an evaluation prediction: the paper's claims are tested by inserting the recovered heatmaps into existing FLD methods and measuring NME, FR, and AUC on held-out test sets (Tables 1, 5, and 6). The final metrics therefore do not reduce by construction to the training target. Eq. (7) defines alpha as a learned function of the squared difference between regular and messenger embeddings; alpha is not defined in terms of the landmark labels or the final NME, and the paper never claims alpha is supervised by ground-truth occlusion labels. The method's reliance on a well-trained quantized heatmap generator is acknowledged as a limitation in Section 6.2.4, but that is an external architectural dependency, not circular reasoning. The Table 1 comparison with STAR uses a different loss (L2 plus NME versus STAR loss), as Table 10 makes explicit; this is an experimental attribution concern, not a circularity concern, and the controlled comparison under the same STAR loss (Table 6) still shows gains. No load-bearing self-citations, imported uniqueness theorems, or ansatz-via-citation steps appear; CodeFormer and VQVAE are external baselines, not self-citations. Overall, the derivation is self-contained and the central claim is empirically falsifiable.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central claim rests on the learned occlusion map alpha and the messenger representation, both of which are trained only through the downstream heatmap loss. The VQ codebook prior is a domain assumption that may not hold if the training data includes occluded faces. No new physical entities are introduced; the messenger token is an architectural construct without external falsifiable evidence.

free parameters (5)

beta = 0.25
Loss balance hyperparameter in Eq. (2), chosen by hand.
lambda_latent = 100
Weight for latent loss in Eq. (3), chosen by hand.
lambda_img = 50
Weight for image-level loss in Eq. (11), chosen by hand.
codebook size N = 2048
Number of learned codes in the quantized heatmap generator, set empirically.
random occlusion probability = 40%
Data augmentation probability for the FLD model, set empirically.

assumptions (5)

ad hoc to paper The squared difference between regular embedding X' and messenger embedding M', after a learned linear layer and sigmoid, yields a reliable per-patch occlusion probability alpha.
Equation (7) defines alpha from this dissimilarity without direct occlusion supervision; the training loss only supervises heatmaps and code sequences.
domain assumption The messenger token, attending to all patches except its own, provides an occlusion-free estimate of the patch's features.
This is the core premise behind Eq. (5)-(6) and the feature recovery in Eq. (9); it is plausible but not independently verified.
domain assumption The pre-trained VQ codebook encodes a prior over unoccluded face heatmaps sufficient to recover occluded patches.
Relies on the claim in Section 3.1 that training uses faces without occlusions, but Section 4.1 does not specify filtering occluded WFLW images.
domain assumption Edge heatmaps defined by Wu et al. [40] capture sufficient facial structure for landmark detection.
Adopted from prior work; standard in FLD but still an assumption about the supervision signal.
domain assumption The ground-truth code sequence S from the pre-trained heatmap generator is a valid target for ORFormer even while the encoder E is fine-tuned.
The target S is fixed from the pre-trained E, while the input features come from the fine-tuned E; this self-distillation setup assumes no harmful drift.

invented entities (1)

Messenger token
purpose: A learnable token per image patch that aggregates features from all other patches via cross-attention, simulating occlusion and providing occlusion-free feature estimates for recovery.
Its effectiveness is only demonstrated through internal ablations and downstream landmark accuracy; there is no independent, externally falsifiable measurement of the messenger representation's correctness.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ORFormer: Occlusion-Robust Transformer for Accurate Facial Landmark Detection." pith.science (2026). https://pith.science/paper/PY2JWSIZ

@misc{pith2026241213174,
  author       = {Pith},
  title        = {Pith review of: ORFormer: Occlusion-Robust Transformer for Accurate Facial Landmark Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PY2JWSIZ}},
  note         = {Machine review of arXiv:2412.13174}
}

read the original abstract

Although facial landmark detection (FLD) has gained significant progress, existing FLD methods still suffer from performance drops on partially non-visible faces, such as faces with occlusions or under extreme lighting conditions or poses. To address this issue, we introduce ORFormer, a novel transformer-based method that can detect non-visible regions and recover their missing features from visible parts. Specifically, ORFormer associates each image patch token with one additional learnable token called the messenger token. The messenger token aggregates features from all but its patch. This way, the consensus between a patch and other patches can be assessed by referring to the similarity between its regular and messenger embeddings, enabling non-visible region identification. Our method then recovers occluded patches with features aggregated by the messenger tokens. Leveraging the recovered features, ORFormer compiles high-quality heatmaps for the downstream FLD task. Extensive experiments show that our method generates heatmaps resilient to partial occlusions. By integrating the resultant heatmaps into existing FLD methods, our method performs favorably against the state of the arts on challenging datasets such as WFLW and COFW.

Figures

Figures reproduced from arXiv: 2412.13174 by the authors.

**Figure 1.** Overview of our ORFormer. (a) For each patch Pi, we introduce a patch token Xi and a learnable messenger token Mi for occlusion detection and handling. (b) The messenger token computes attention with patch tokens other than its corresponding one. (c) We detect occlusion by evaluating the dissimilarity between the regular embedding X ′ i and the messenger embedding M′ i , and then recover occluded features based on t… view at source ↗

**Figure 2.** Overview of our method. (a) We first train a quantized heatmap generator, which takes an image I as input and generates its edge heatmaps H. After pre-training, the prior knowledge of unoccluded faces is encoded in the codebook C and decoder D. (b) With the frozen codebook and decoder, we introduce ORFormer to generate the occlusion map α and two code sequences SI and SM, leading to quantized features ZI and ZM. The… view at source ↗

**Figure 3.** Network architecture of ORFormer. ORFormer takes image patches P as input and generates two code sequences SI and SM via the codebook prediction head. While SI is computed by referring to the image patch tokens, SM is by the messenger tokens. The occlusion map α represents the patch-specific occlusion likelihood and is inferred by the occlusion detection head. 3.1. Quantized Heatmap Generator To enhance robustness a… view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Integration of ORFormer into an existing FLD method. ORFormer is adopted for occlusion detection and feature recovery, resulting in high-quality heatmaps. The generated heatmaps serve as an extra input to an FLD method, and offer the recovered features to make the FLD…

**Figure 5.** Figure 5: Qualitative comparison with the reproduced baseline method, STAR, on extreme cases from the test set of WFLW. The ground-truth landmarks are marked in blue, while the predicted landmarks are in red. The green lines represent the distance between the ground-truth landm…

**Figure 6.** Figure 6: Visualization of the α maps yielded by ORFormer. Red regions indicate higher values of α, suggesting heavier feature occlusion or corruption detected by ORFormer. on the various facial landmark datasets. Our method performs favorably against state-of-the-art methods i…

**Figure 7.** Figure 7: Qualitative comparison for heatmap generation on WFLW. GT stands for the ground-truth heatmap. For better visualization, we display the distance heatmap for VQVAE, Codeformer, and ORFormer by computing the pixel-wise L2 distance between their output heatmaps and the …

**Figure 8.** Figure 8: Generation flow of the ground-truth edge heatmap. WFLW 300W COFW [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

**Figure 9.** Figure 9: Visualization of the ground-truth landmarks in different datasets. Method (Publication) Backbone WFLW-Full COFW 300W (NMEio↓) NMEio↓ FR10%↓ AUC10%↑ NMEip↓ Full Comm. Chal. LAB (CVPR18) [40] Hourglass 5.27 7.56 0.532 - 3.49 2.98 5.19 Wing (CVPR18) [7] ResNet-50 4.99 6.0…

**Figure 10.** Figure 10: Qualitative comparison with the reproduced baseline method, STAR, on extreme cases from the test set of WFLW. The ground-truth landmarks are marked in blue, while the predicted landmarks are in red. The green lines represent the distance between the ground-truth landm…

**Figure 11.** Figure 11: Qualitative comparison for heatmap generation on WFLW. GT stands for the ground-truth heatmap. For better visualization, we display the distance heatmap for VQVAE, Codeformer, and ORFormer by computing the pixel-wise L2 distance between their output heatmaps and the G…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 39 canonical work pages

[1]

Training deep networks for facial ex- pression recognition with crowd-sourced label distribution

Emad Barsoum, Cha Zhang, Cristian Canton Ferrer, and Zhengyou Zhang. Training deep networks for facial ex- pression recognition with crowd-sourced label distribution. In ACM international conference on multimodal interaction,

work page
[2]

Subpixel heatmap regression for facial landmark local- ization

Adrian Bulat, Enrique Sanchez, and Georgios Tzimiropou- los. Subpixel heatmap regression for facial landmark local- ization. arXiv preprint arXiv:2111.02360, 2021. 2

arXiv 2021
[3]

Robust face landmark estimation under occlusion

Xavier P Burgos-Artizzu, Pietro Perona, and Piotr Doll ´ar. Robust face landmark estimation under occlusion. In ICCV,

work page
[4]

Supervision-by-registration: An un- supervised approach to improve the precision of facial land- mark detectors

Xuanyi Dong, Shoou-I Yu, Xinshuo Weng, Shih-En Wei, Yi Yang, and Yaser Sheikh. Supervision-by-registration: An un- supervised approach to improve the precision of facial land- mark detectors. In CVPR, 2018. 2

work page 2018
[5]

An image is worth 16x16 words: Trans- formers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 1, 3, 8

arXiv 2010
[6]

Taming transformers for high-resolution image synthesis

Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR,
[7]

Wing loss for robust facial landmark localisation with convolutional neural networks

Zhen-Hua Feng, Josef Kittler, Muhammad Awais, Patrik Hu- ber, and Xiao-Jun Wu. Wing loss for robust facial landmark localisation with convolutional neural networks. In CVPR,

work page
[8]

Pfld: A practical facial landmark detector

Xiaojie Guo, Siyuan Li, Jinke Yu, Jiawan Zhang, Jiayi Ma, Lin Ma, Wei Liu, and Haibin Ling. Pfld: A practical facial landmark detector. arXiv preprint arXiv:1902.10859, 2019. 1, 2

arXiv 1902

Show all 49 references

[9]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR,
[10]

6d rotation representation for unconstrained head pose estimation

Thorsten Hempel, Ahmed A Abdelrahman, and Ayoub Al- Hamadi. 6d rotation representation for unconstrained head pose estimation. In ICIP, 2022. 1

2022
[11]

Adnet: Leveraging error-bias towards nor- mal direction in face alignment

Yangyu Huang, Hao Yang, Chong Li, Jongyoo Kim, and Fangyun Wei. Adnet: Leveraging error-bias towards nor- mal direction in face alignment. In ICCV, 2021. 1, 2, 6, 7, 10, 12, 13

2021
[12]

Pixel-in-pixel net: Towards efficient facial landmark detection in the wild

Haibo Jin, Shengcai Liao, and Ling Shao. Pixel-in-pixel net: Towards efficient facial landmark detection in the wild. IJCV, 2021. 2, 6, 7, 10

2021
[13]

Face recognition based on facial landmark detection

Aniwat Juhong and Chuchart Pintavirooj. Face recognition based on facial landmark detection. InBiomedical Engineer- ing International Conference, 2017. 1

2017
[14]

Real-time eye tracking for bare and sunglasses-wearing faces for augmented reality 3d head- up displays

Dongwoo Kang and Lin Ma. Real-time eye tracking for bare and sunglasses-wearing faces for augmented reality 3d head- up displays. IEEE Access, 2021. 1

2021
[15]

Adam: A method for stochastic optimization

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

arXiv
[16]

Luvli face alignment: Esti- mating landmarks’ location, uncertainty, and visibility like- lihood

Abhinav Kumar, Tim K Marks, Wenxuan Mou, Ye Wang, Michael Jones, Anoop Cherian, Toshiaki Koike-Akino, Xi- aoming Liu, and Chen Feng. Luvli face alignment: Esti- mating landmarks’ location, uncertainty, and visibility like- lihood. In CVPR, 2020. 2, 6, 10

2020
[17]

Hih: Towards more accurate face alignment via heatmap in heatmap

Xing Lan, Qinghao Hu, Qiang Chen, Jian Xue, and Jian Cheng. Hih: Towards more accurate face alignment via heatmap in heatmap. arXiv preprint arXiv:2104.03100 ,

arXiv
[18]

Towards accurate facial landmark detection via cascaded transformers

Hui Li, Zidong Guo, Seon-Min Rhee, Seungju Han, and Jae- Joon Han. Towards accurate facial landmark detection via cascaded transformers. In CVPR, 2022. 2

2022
[19]

Repformer: Refinement pyramid transformer for robust facial landmark detection

Jinpeng Li, Haibo Jin, Shengcai Liao, Ling Shao, and Pheng- Ann Heng. Repformer: Refinement pyramid transformer for robust facial landmark detection. In IJCAI, 2022. 2, 6, 7, 10

2022
[20]

Cascaded iterative transformer for jointly predicting facial landmark, occlusion probability and head pose

Yaokun Li, Guang Tan, and Chao Gou. Cascaded iterative transformer for jointly predicting facial landmark, occlusion probability and head pose. IJCV, 2024. 2

2024
[21]

Sgdr: Stochas- tic gradient descent with warm restarts

Ilya Loshchilov and Frank Hutter. Sgdr: Stochas- tic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016. 9

2016 arXiv
[22]

Robust facial landmark detection via a fully-convolutional local- global context network

Daniel Merget, Matthias Rock, and Gerhard Rigoll. Robust facial landmark detection via a fully-convolutional local- global context network. In CVPR, 2018. 3

2018
[23]

Direct shape regression net- works for end-to-end face alignment

Xin Miao, Xiantong Zhen, Xianglong Liu, Cheng Deng, Vas- silis Athitsos, and Heng Huang. Direct shape regression net- works for end-to-end face alignment. In CVPR, 2018. 2

2018
[24]

Recurrence without recurrence: Stable video landmark detection with deep equilibrium models

Paul Micaelli, Arash Vahdat, Hongxu Yin, Jan Kautz, and Pavlo Molchanov. Recurrence without recurrence: Stable video landmark detection with deep equilibrium models. In CVPR, 2023. 2, 6, 7, 10

2023
[25]

Going deeper in facial expression recognition using deep neural networks

Ali Mollahosseini, David Chan, and Mohammad H Mahoor. Going deeper in facial expression recognition using deep neural networks. In WACV, 2016. 1

2016
[26]

Stacked hour- glass networks for human pose estimation

Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hour- glass networks for human pose estimation. In ECCV, 2016. 2, 6

2016
[27]

Numerical coordinate regression with convolutional neural networks

Aiden Nibali, Zhen He, Stuart Morgan, and Luke Prender- gast. Numerical coordinate regression with convolutional neural networks. arXiv preprint arXiv:1801.07372, 2018. 2

2018 arXiv
[28]

Handoccnet: Occlusion-robust 3d hand mesh estimation network

JoonKyu Park, Yeonguk Oh, Gyeongsik Moon, Hongsuk Choi, and Kyoung Mu Lee. Handoccnet: Occlusion-robust 3d hand mesh estimation network. In CVPR, 2022. 3

2022
[29]

Deep face recognition

Omkar Parkhi, Andrea Vedaldi, and Andrew Zisserman. Deep face recognition. In BMVC, 2015. 1

2015
[30]

U-net: Convolutional networks for biomedical image segmentation

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Inter- vention, 2015. 2

2015
[31]

300 faces in-the-wild challenge: The first facial landmark localization challenge

Christos Sagonas, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 faces in-the-wild challenge: The first facial landmark localization challenge. In ICCV workshops, 2013. 6, 9

2013
[32]

Deep high-resolution representation learning for human pose esti- mation

Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose esti- mation. In CVPR, 2019. 2, 6, 7, 10 14

2019
[33]

Multi-task head pose estimation in-the-wild

Roberto Valle, Jos ´e M Buenaposada, and Luis Baumela. Multi-task head pose estimation in-the-wild. PAMI, 2020. 1

2020
[34]

Neural discrete representation learning

Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. In NeurIPS, 2017. 3, 4, 8, 13

2017
[35]

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017. 2, 3

2017
[36]

Adaptive wing loss for robust face alignment via heatmap regression

Xinyao Wang, Liefeng Bo, and Li Fuxin. Adaptive wing loss for robust face alignment via heatmap regression. In ICCV,
[37]

Lotr: face land- mark localization using localization transformer

Ukrit Watchareeruetai, Benjaphan Sommana, Sanjana Jain, Pavit Noinongyao, Ankush Ganguly, Aubin Samacoits, Samuel WF Earp, and Nakarin Sritrakool. Lotr: face land- mark localization using localization transformer. IEEE Ac- cess, 2022. 2

2022
[38]

As- sessing facial symmetry and attractiveness using augmented reality

Wei Wei, Edmond SL Ho, Kevin D McCay, Robertas Damaˇseviˇcius, Rytis Maskeli ¯unas, and Anna Esposito. As- sessing facial symmetry and attractiveness using augmented reality. Pattern Analysis and Applications, 2021. 1

2021
[39]

Transmarker: a pure vision transformer for facial landmark detection

Wenyan Wu, Yici Cai, and Qiang Zhou. Transmarker: a pure vision transformer for facial landmark detection. In ICPR,
[40]

Look at boundary: A boundary-aware face alignment algorithm

Wayne Wu, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, and Qiang Zhou. Look at boundary: A boundary-aware face alignment algorithm. In CVPR, 2018. 1, 2, 4, 6, 7, 9, 10

2018
[41]

Robust facial landmark detection un- der significant head poses and occlusion

Yue Wu and Qiang Ji. Robust facial landmark detection un- der significant head poses and occlusion. In ICCV, 2015. 2

2015
[42]

Sparse local patch transformer for robust face alignment and landmarks inherent relation learning

Jiahao Xia, Weiwei Qu, Wenjian Huang, Jianguo Zhang, Xi Wang, and Min Xu. Sparse local patch transformer for robust face alignment and landmarks inherent relation learning. In CVPR, 2022. 1, 2, 6, 7, 10

2022
[43]

Learning feature recovery transformer for occluded person re-identification

Boqiang Xu, Lingxiao He, Jian Liang, and Zhenan Sun. Learning feature recovery transformer for occluded person re-identification. TIP, 2022. 3

2022
[44]

Facial landmark detection by deep multi-task learning

Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial landmark detection by deep multi-task learning. In ECCV, 2014. 2

2014
[45]

Extensive facial landmark localization with coarse- to-fine convolutional network cascade

Erjin Zhou, Haoqiang Fan, Zhimin Cao, Yuning Jiang, and Qi Yin. Extensive facial landmark localization with coarse- to-fine convolutional network cascade. In ICCV workshops,
[46]

Towards robust blind face restora- tion with codebook lookup transformer

Shangchen Zhou, Kelvin Chan, Chongyi Li, and Chen Change Loy. Towards robust blind face restora- tion with codebook lookup transformer. In NeurIPS, 2022. 3, 4, 8, 13

2022
[47]

Star loss: Reducing semantic ambigu- ity in facial landmark detection

Zhenglin Zhou, Huaxia Li, Hong Liu, Nanyang Wang, Gang Yu, and Rongrong Ji. Star loss: Reducing semantic ambigu- ity in facial landmark detection. In CVPR, 2023. 1, 2, 6, 7, 10, 12, 13

2023
[48]

Occlusion-robust face alignment using a viewpoint-invariant hierarchical network architecture

Congcong Zhu, Xintong Wan, Shaorong Xie, Xiaoqiang Li, and Yinzheng Gu. Occlusion-robust face alignment using a viewpoint-invariant hierarchical network architecture. In CVPR, 2022. 3

2022
[49]

Robust facial landmark detection via occlusion- adaptive deep networks

Meilu Zhu, Daming Shi, Mingjie Zheng, and Muhammad Sadiq. Robust facial landmark detection via occlusion- adaptive deep networks. In CVPR, 2019. 3 15

2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Training deep networks for facial ex- pression recognition with crowd-sourced label distribution

Emad Barsoum, Cha Zhang, Cristian Canton Ferrer, and Zhengyou Zhang. Training deep networks for facial ex- pression recognition with crowd-sourced label distribution. In ACM international conference on multimodal interaction,

work page

[2] [2]

Subpixel heatmap regression for facial landmark local- ization

Adrian Bulat, Enrique Sanchez, and Georgios Tzimiropou- los. Subpixel heatmap regression for facial landmark local- ization. arXiv preprint arXiv:2111.02360, 2021. 2

arXiv 2021

[3] [3]

Robust face landmark estimation under occlusion

Xavier P Burgos-Artizzu, Pietro Perona, and Piotr Doll ´ar. Robust face landmark estimation under occlusion. In ICCV,

work page

[4] [4]

Supervision-by-registration: An un- supervised approach to improve the precision of facial land- mark detectors

Xuanyi Dong, Shoou-I Yu, Xinshuo Weng, Shih-En Wei, Yi Yang, and Yaser Sheikh. Supervision-by-registration: An un- supervised approach to improve the precision of facial land- mark detectors. In CVPR, 2018. 2

work page 2018

[5] [5]

An image is worth 16x16 words: Trans- formers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 1, 3, 8

arXiv 2010

[6] [6]

Taming transformers for high-resolution image synthesis

Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR,

[7] [7]

Wing loss for robust facial landmark localisation with convolutional neural networks

Zhen-Hua Feng, Josef Kittler, Muhammad Awais, Patrik Hu- ber, and Xiao-Jun Wu. Wing loss for robust facial landmark localisation with convolutional neural networks. In CVPR,

work page

[8] [8]

Pfld: A practical facial landmark detector

Xiaojie Guo, Siyuan Li, Jinke Yu, Jiawan Zhang, Jiayi Ma, Lin Ma, Wei Liu, and Haibin Ling. Pfld: A practical facial landmark detector. arXiv preprint arXiv:1902.10859, 2019. 1, 2

arXiv 1902