Pith. sign in

REVIEW 2 major objections 5 minor 69 references

Synthetic HMC images from frontal faces close the domain gap for expression recognition inside headsets.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Synthetic HMC images generated via 3D face reconstruction plus a texture-space alignment network let FER models trained on only 7K samples beat models trained on 280K frontal images and generalize across camera layouts.

T0 review reviewed 2026-07-11 challenge →

load-bearing objection Practical synthesis pipeline that closes the HMC perspective gap: 7K unwrapped+TSAN images beat 280K frontal data across camera configs and real HMC, with the expected FLAME/sim-to-real limits already quantified. the 2 major comments →

arxiv 2607.04490 v1 pith:EY2VJ55Z submitted 2026-07-05 cs.CV

Enhancing Facial Expression Recognition in Head-Mounted Displays with Synthetic Data

Jianing Deng , Qiang Zhou , Jingtong Hu This is my paper
classification cs.CV
keywords facial expression recognitionhead-mounted camerassynthetic data3D face reconstructiontexture alignmentdomain gapmixed reality
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Head-mounted displays block most of the face, so expression recognition must rely on the few head-mounted cameras that look at the eyes and mouth. Real training data for those cameras is scarce because of privacy, cost, and hardware variety, and ordinary frontal face datasets fail once the viewpoint changes. This paper shows that the gap is mainly a camera-perspective problem and that it can be closed by reconstruction: each labeled frontal image is turned into a textured 3-D mesh, then re-rendered from any desired headset camera layout. A texture-space alignment network corrects the inevitable misalignment so that wrinkles and furrows survive the transfer. Models trained on only seven thousand of these synthetic images outperform models trained on hundreds of thousands of real frontal images and generalize across five different camera configurations on both simulated and real headset footage.

Core claim

The domain gap between frontal-view and head-mounted-camera facial expression recognition is large enough that models trained on frontal data collapse under HMC viewpoints, yet the same labels become usable once the frontal images are reconstructed into textured meshes and re-rendered from the target camera poses. Direct texture sampling plus a learned texture-space flow correction (TSAN) is sufficient to keep the expression-critical details that let a downstream classifier succeed.

What carries the argument

SynHMC: monocular FLAME/EMOCA reconstruction, direct UV texture unwrapping refined by a U-Net rectified-flow network (TSAN), followed by a configurable multi-camera renderer that merges the resulting HMC views into a single training image.

Load-bearing premise

That the 3-D mesh plus corrected texture still carries the fine expression cues a classifier actually needs, even though the model does not reconstruct the inside of the mouth and never corrects lighting or color differences.

What would settle it

Train the same classifier on SynHMC-7K versus an equal-sized set of real HMC images captured from the identical camera layout; if real data does not outperform synthetic data on a held-out real HMC test set after identical fine-tuning, the reconstruction pipeline is preserving the needed cues.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper addresses data scarcity for facial expression recognition (FER) from head-mounted cameras (HMC) in mixed-reality headsets by introducing SynHMC, a synthesis pipeline that converts abundant frontal-view FER images (e.g., AffectNet) into HMC-view images. It reconstructs expressive 3D meshes via EMOCA/FLAME geometry, unwraps textures directly from the source image (enhanced by GFP-GAN), corrects misalignment with a novel Texture-Space Alignment Network (TSAN) that predicts a rectified flow field in UV space, and renders multi-view HMC images under a configurable articulated camera system. Models trained on the resulting balanced SynHMC-7K dataset are shown to outperform counterparts trained on frontal-view (RAF-DB, AffectNet-280K), multi-view (MEAD), and alternative reconstruction methods (DECA, EMOCA, FDS, FFHQ-UV) on both simulated scan-based HMC benchmarks (Ava256-Scan, 3DRFE across five camera configurations) and the real Ava256-HMC dataset (direct test, K-NN, linear probe, and fine-tuning).

Significance. If the empirical claims hold, the work provides a practical route to reuse large-scale annotated frontal FER corpora for HMC settings, mitigating privacy, cost, and platform-diversity barriers that currently limit neural HMC FER. The configurable camera model and mixed-configuration training (Fig. 7) directly address generalization across headset designs, while the TSAN ablation and texture-sampling comparisons (Tables 3 and 5, Fig. 6) demonstrate that preserving fine expression details (wrinkles, furrows) improves downstream accuracy. The introduction of a multi-configuration simulated + real benchmark is itself a useful contribution for future HMC FER research. Strengths include systematic ablations over architectures, reconstruction methods, and camera layouts, plus explicit reporting of standard deviations over configurations.

major comments (2)
  1. [Sec. 6 / Table 4] Sec. 6 and Table 4: The paper correctly flags FLAME’s missing inner-mouth geometry and the uncorrected lighting/color domain shift as limitations, yet these are load-bearing for the central claim that texture sampling + TSAN preserves the expression cues actually used by FER classifiers. Direct-test UAR on real Ava256-HMC remains only 27.93 (vs. 53.42 after fine-tuning), and the performance gap between simulated (Table 2) and real results is large. A quantitative analysis of mouth-region contribution (e.g., occlusion or landmark-based ablation) or failure-case visualization would be needed to confirm that the synthetic data transfers the critical cues rather than merely providing better generic features.
  2. [Table 2 / Fig. 7] Table 2 and Fig. 7: The mixed-configuration model is presented as a key advantage of synthesis, yet the training protocol for the “Mixed” entry (how many samples per config, whether balanced, whether the same 7K source images are re-rendered) is not fully specified. Without this, it is difficult to isolate the contribution of multi-view diversity from simple data-volume effects when claiming superior cross-config generalization.
minor comments (5)
  1. [Sec. 4.2] Sec. 4.2, Eq. (2)–(3): The mapping M and the backward-warping operator W are introduced without explicit definitions of the barycentric interpolation or the flow-field convention; a short clarifying sentence or reference would help reproducibility.
  2. [Fig. 3 / Fig. 8] Fig. 3 and Fig. 8: The qualitative misalignment examples are useful, but the color scale / difference maps are hard to interpret without a shared color bar or quantitative landmark error before/after TSAN.
  3. [Table 4] Table 4: The “None / ImageNet” and “MEAD” rows are valuable baselines; adding the number of subjects or images used for each pre-training set would make the scale comparison more transparent.
  4. [Appendix A1] Appendix A1 / Algorithm 1: The noise scales (0.3 for shape/expression, 3° for rotation) are reasonable but presented as fixed; a brief sensitivity check or statement that they were chosen to match observed EMOCA residuals would strengthen the TSAN training claim.
  5. Throughout: minor typographical inconsistencies appear (e.g., “Ava265-HMC” vs. “Ava256-HMC” in Table 4 caption, “W AR” spacing). A final proof-read pass is recommended.

Circularity Check

0 steps flagged

No significant circularity: purely empirical synthesis-and-train pipeline whose labels, reconstruction priors, and evaluation sets are external; no derivation reduces to a fitted input by construction.

full rationale

The paper's central claim is empirical (models trained on SynHMC-7K outperform FV baselines on Ava256-Scan/3DRFE/Ava256-HMC under multiple architectures and camera configs; Tables 2-4, Fig. 7). Expression labels are taken unchanged from external annotated sets (AffectNet, RAF-DB). Geometry comes from the off-the-shelf EMOCA/FLAME regressor; texture is direct unwrapping plus a TSAN whose training pairs are synthesized from independent FFHQ-UV maps under random shape/expression/camera perturbations (Alg. 1, Sec. 4.2). Rendering uses a configurable but non-learned camera model. No equation equates a claimed prediction to a fitted parameter, no uniqueness theorem is imported from the authors, and no self-citation is load-bearing for the performance claims. Residual sim-to-real and FLAME limitations are openly stated (Sec. 6) rather than hidden circularities. The work is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central claim rests on standard 3DMM priors, the assumption that texture unwrapping plus a learned flow field preserves expression semantics, and a handful of engineering choices (balanced 7K subset, five hand-designed camera configs). No new physical entities are postulated; TSAN is an ordinary U-Net trained on synthetic pairs.

free parameters (4)
  • AffectNet-7K balanced subset size
    Authors randomly select 1K images per class (7K total) from AffectNet-280K to generate SynHMC; the exact sampling seed and class-balance decision affect all reported numbers.
  • TSAN training noise scales
    Shape/expression perturbation ~N(0,0.3²) and camera Euler ~N(0,3°) are hand-chosen to synthesize misaligned pairs (Alg. 1); different scales would change the learned flow field.
  • HMC FoV and joint parameters
    Five discrete camera configurations (FoV 66°/90°/120°, 3- or 4-view) are chosen by hand from existing headsets; results are averaged over them.
  • Rendering light intensity
    Fixed directional intensities (1.5 or 2.5) are set per dataset without photometric calibration.
axioms (4)
  • domain assumption FLAME 3DMM plus EMOCA regression supplies sufficiently accurate geometry and camera for expression-preserving texture unwrapping
    Invoked throughout Sec. 4.2; the paper notes FLAME cannot model the inner mouth yet still uses it as the sole shape prior.
  • ad hoc to paper Misalignment between projected mesh and image can be corrected by a 2-D rectified flow field learned in UV space
    Core of TSAN (Eq. 3); justified by the fixed semantic topology of the UV template but not proven to recover all expression-critical pixels.
  • domain assumption Domain gap between frontal and HMC views is dominated by camera perspective rather than lighting, color, or headset occlusion
    Stated in Sec. 3 and Sec. 6; the method deliberately ignores relighting and color transfer.
  • domain assumption Seven basic Ekman expressions are the correct label space for HMC FER
    Used for all classification experiments; inherited from AffectNet and Ava256 regrouping (Table A3).
invented entities (2)
  • Texture-Space Alignment Network (TSAN) no independent evidence
    purpose: Estimate a 2-D rectified flow that warps a misaligned unwrapped texture into the canonical UV template so that expression details survive rendering.
    Ordinary U-Net with positional encoding; trained only on synthetic pairs generated inside the paper. No external validation of the flow field itself.
  • SynHMC synthesis framework no independent evidence
    purpose: End-to-end pipeline that converts any frontal FER image into multi-view HMC images under arbitrary camera configurations.
    Composition of existing reconstruction, new TSAN, and configurable renderer; the combination is new but each piece is standard.

reviewed 2026-07-11 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Facial Expression Recognition in Head-Mounted Displays with Synthetic Data." pith.science (2026). https://pith.science/paper/EY2VJ55Z

@misc{pith2026260704490,
  author       = {Pith},
  title        = {Pith review of: Enhancing Facial Expression Recognition in Head-Mounted Displays with Synthetic Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EY2VJ55Z}},
  note         = {Machine review of arXiv:2607.04490}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Facial expression recognition (FER) is crucial for social interaction in mixed reality environments that employ head-mounted displays (HMD). However, collecting FER data from head-mounted cameras (HMC) is challenging due to privacy concerns and the diversity of HMD platforms. Moreover, existing FER datasets are not directly applicable due to the unique perspectives of HMCs. The lack of sufficient data hinders the development of neural network-based HMC FER methods. To address data scarcity, we propose a data synthesis framework that generates HMC-view images from frontal-view images, leveraging abundant existing annotated datasets. Specifically, we first reconstruct 3D textured meshes from images and then apply a configurable camera system to render images from the HMC perspective. Additionally, we introduce a texture-space alignment network (TSAN) that enables accurate texture sampling from images to preserve detailed facial expressions. To evaluate the proposed method, we conduct extensive experiments on both simulated and real HMC datasets. Experimental results demonstrate that models trained on our synthetic dataset outperform those trained on existing datasets and exhibit better generalization across different camera configurations.

Figures

Figures reproduced from arXiv: 2607.04490 by Jianing Deng, Jingtong Hu, Qiang Zhou.

Figure 1
Figure 1. Figure 1: Accuracy of AffectNet-trained models evaluated on the 3DRFE dataset with different input sources. The first row visual￾izes the input data fed into the networks. The dashed line repre￾sents the accuracy of the Swin-T model on the AffectNet test set. images, especially in practical scenarios with diverse cam￾era configurations. This highlights the need for a dedicated HMC dataset that accounts for such vari… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed SynHMC framework for synthetic HMC images generation and HMC FER training. robust face reconstruction. For appearance modeling, pre￾vious PCA-based linear methods [15, 39] and generative methods [3, 24] struggle to capture expression details and diverse appearance changes. To address this, we directly sample pixels (i.e., unwrapping) from source images, pre￾serving expression detai… view at source ↗
Figure 3
Figure 3. Figure 3: Texture unwrapped using the estimated shape and cam [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Camera configurations used in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison of 3D face reconstruction methods for HMC image synthesis. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Cross-evaluation of model performance across different [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparisons of different texture refinement methods. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 4 linked inside Pith

  1. [1]

    Pytorch 2: Faster machine learning through dynamic python bytecode trans- formation and graph compilation

    Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael V oznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, et al. Pytorch 2: Faster machine learning through dynamic python bytecode trans- formation and graph compilation. InACM ASPLOS, pages 929–947, 2024. 6

    2024

  2. [2]

    Genesis: A universal and generative physics engine for robotics and beyond, 2024

    Genesis Authors. Genesis: A universal and generative physics engine for robotics and beyond, 2024. 2

    2024

  3. [3]

    Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction

    Haoran Bai, Di Kang, Haoxian Zhang, Jinshan Pan, and Lin- chao Bao. Ffhq-uv: Normalized facial uv-texture dataset for 3d face reconstruction. InCVPR, pages 362–371, 2023. 2, 4, 5, 7, 8

    2023

  4. [4]

    High-fidelity 3d digital human head creation from rgb-d selfies.ACM TOG, 41(1):1–21, 2021

    Linchao Bao, Xiangkai Lin, Yajing Chen, Haoxian Zhang, Sheng Wang, Xuefei Zhe, Di Kang, Haozhi Huang, Xinwei Jiang, Jue Wang, et al. High-fidelity 3d digital human head creation from rgb-d selfies.ACM TOG, 41(1):1–21, 2021. 2

    2021

  5. [5]

    A morphable model for the synthesis of 3d faces

    V olker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. InACM TOG, page 187–194, 1999. 2, 3

    1999

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In ICCV, pages 9650–9660, 2021. 6

    2021

  7. [7]

    Photo-realistic facial details synthesis from single image

    Anpei Chen, Zhang Chen, Guli Zhang, Kenny Mitchell, and Jingyi Yu. Photo-realistic facial details synthesis from single image. InICCV, pages 9429–9439, 2019. 2, 7, 8

    2019

  8. [8]

    A simple framework for contrastive learn- ing of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learn- ing of visual representations. InICML, pages 1597–1607. PMLR, 2020. 6

    2020

  9. [9]

    GPAvatar: Generalizable and precise head avatar from image(s)

    Xuangeng Chu, Yu Li, Ailing Zeng, Tianyu Yang, Lijian Lin, Yunfei Liu, and Tatsuya Harada. GPAvatar: Generalizable and precise head avatar from image(s). InICLR, 2024. 4

    2024

  10. [10]

    Emoca: Emotion driven monocular face capture and animation

    Radek Dan ˇeˇcek, Michael J Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In CVPR, pages 20311–20322, 2022. 2, 4, 7, 8

    2022

  11. [11]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. InCVPR, pages 248–255, 2009. 8

    2009

  12. [12]

    Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set

    Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. InCVPRW, pages 0–0, 2019. 2, 4

    2019

  13. [13]

    An argument for basic emotions.Cognition and Emotion, 6(3-4):169–200, 1992

    Paul Ekman. An argument for basic emotions.Cognition and Emotion, 6(3-4):169–200, 1992. 2, 3

    1992

  14. [14]

    Joint 3d face reconstruction and dense alignment with position map regression network

    Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, and Xi Zhou. Joint 3d face reconstruction and dense alignment with position map regression network. InECCV, pages 534–551,

  15. [15]

    Learning an animatable detailed 3d face model from in-the- wild images.ACM TOG, 40(4):1–13, 2021

    Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in-the- wild images.ACM TOG, 40(4):1–13, 2021. 2, 4, 7, 8

    2021

  16. [16]

    Mixed Reality Market Size & Share — Industry Report, 2030, 2024

    Grand View Research. Mixed Reality Market Size & Share — Industry Report, 2030, 2024. 1

    2030

  17. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InCVPR, pages 770–778, 2016. 3, 6

    2016

  18. [18]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InCVPR, pages 16000–16009, 2022. 6

    2022

  19. [19]

    Eyemotion: Classifying facial expressions in vr using eye-tracking cameras

    Steven Hickson, Nick Dufour, Avneesh Sud, Vivek Kwatra, and Irfan Essa. Eyemotion: Classifying facial expressions in vr using eye-tracking cameras. InWACV, pages 1626–1635. IEEE, 2019. 1, 2

    2019

  20. [20]

    Multiview facial expres- sion recognition, a survey.IEEE Trans

    Mahdi Jampour and Malihe Javidi. Multiview facial expres- sion recognition, a survey.IEEE Trans. Affect. Comput., 13 (4):2086–2105, 2022. 2

    2086

  21. [21]

    Super slomo: High quality estimation of multiple intermediate frames for video interpolation

    Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz. Super slomo: High quality estimation of multiple intermediate frames for video interpolation. InCVPR, pages 9000–9008, 2018. 5

    2018

  22. [22]

    Synthetic data in human analysis: A survey.IEEE TPAMI, 2024

    Indu Joshi, Marcel Grimmer, Christian Rathgeb, Christoph Busch, Francois Bremond, and Antitza Dantcheva. Synthetic data in human analysis: A survey.IEEE TPAMI, 2024. 2

    2024

  23. [23]

    Robust egocentric photo-realistic facial expression transfer for virtual reality

    Amin Jourabloo, Fernando De la Torre, Jason Saragih, Shih- En Wei, Stephen Lombardi, Te-Li Wang, Danielle Belko, Autumn Trimble, and Hernan Badino. Robust egocentric photo-realistic facial expression transfer for virtual reality. InCVPR, pages 20323–20332, 2022. 1, 2, 6

    2022

  24. [24]

    Avatarme: Realistically ren- derable 3d facial reconstruction” in-the-wild”

    Alexandros Lattas, Stylianos Moschoglou, Baris Gecer, Stylianos Ploumpis, Vasileios Triantafyllou, Abhijeet Ghosh, and Stefanos Zafeiriou. Avatarme: Realistically ren- derable 3d facial reconstruction” in-the-wild”. InCVPR, pages 760–769, 2020. 2, 4

    2020

  25. [25]

    Fitme: Deep photorealistic 3d morphable model avatars

    Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Baris Gecer, Jiankang Deng, and Stefanos Zafeiriou. Fitme: Deep photorealistic 3d morphable model avatars. InCVPR, pages 8629–8640, 2023. 2

    2023

  26. [26]

    Latent-ofer: De- tect, mask, and reconstruct with latent vectors for occluded facial expression recognition

    Isack Lee, Eungi Lee, and Seok Bong Yoo. Latent-ofer: De- tect, mask, and reconstruct with latent vectors for occluded facial expression recognition. InICCV, pages 1536–1546,

  27. [27]

    Reliable crowd- sourcing and deep locality-preserving learning for expres- sion recognition in the wild

    Shan Li, Weihong Deng, and JunPing Du. Reliable crowd- sourcing and deep locality-preserving learning for expres- sion recognition in the wild. InCVPR, pages 2584–2593. IEEE, 2017. 1, 5, 6

    2017

  28. [28]

    Learning a model of facial shape and expression from 4d scans.ACM TOG, 36(6):194–1, 2017

    Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans.ACM TOG, 36(6):194–1, 2017. 2, 4, 5, 6

    2017

  29. [29]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pages 10012–10022, 2021. 3, 6

    2021

  30. [30]

    Deep appearance models for face rendering.ACM TOG, 37(4):1–13, 2018

    Stephen Lombardi, Jason Saragih, Tomas Simon, and Yaser Sheikh. Deep appearance models for face rendering.ACM TOG, 37(4):1–13, 2018. 1

    2018

  31. [31]

    Sgdr: Stochas- tic gradient descent with warm restarts.arXiv preprint arXiv:1608.03983, 2016

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochas- tic gradient descent with warm restarts.arXiv preprint arXiv:1608.03983, 2016. 6

    Pith/arXiv arXiv 2016

  32. [32]

    Medi- apipe: A framework for building perception pipelines.arXiv preprint arXiv:1906.08172, 2019

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris Mc- Clanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo- Ling Chang, Ming Guang Yong, Juhyun Lee, et al. Medi- apipe: A framework for building perception pipelines.arXiv preprint arXiv:1906.08172, 2019. 4

    Pith/arXiv arXiv 1906

  33. [33]

    Rapid ac- quisition of specular and diffuse normal maps from polar- ized spherical gradient illumination.Rendering Techniques, 9(10):2, 2007

    Wan-Chun Ma, Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss, Paul E Debevec, et al. Rapid ac- quisition of specular and diffuse normal maps from polar- ized spherical gradient illumination.Rendering Techniques, 9(10):2, 2007. 3

    2007

  34. [34]

    Codec avatar studio: Paired human captures for complete, drive- able, and generalizable avatars

    Julieta Martinez, Emily Kim, Javier Romero, Timur Bagaut- dinov, Shunsuke Saito, Shoou-I Yu, Stuart Anderson, Michael Zollhofer, Te-Li Wang, Shaojie Bai, et al. Codec avatar studio: Paired human captures for complete, drive- able, and generalizable avatars. InNeurIPS Datasets and Benchmarks Track, 2024. 1, 2, 5, 6

    2024

  35. [35]

    Synthetic data in healthcare.arXiv preprint arXiv:2304.03243, 2023

    Daniel McDuff, Theodore Curran, and Achuta Kadambi. Synthetic data in healthcare.arXiv preprint arXiv:2304.03243, 2023. 2

    Pith/arXiv arXiv 2023

  36. [36]

    Affectnet: A database for facial expression, valence, and arousal computing in the wild.IEEE Trans

    Ali Mollahosseini, Behzad Hasani, and Mohammad H Ma- hoor. Affectnet: A database for facial expression, valence, and arousal computing in the wild.IEEE Trans. Affect. Com- put., 10(1):18–31, 2017. 1, 3, 5, 6, 8

    2017

  37. [37]

    Springer, 2021

    Sergey I Nikolenko et al.Synthetic data for deep learning. Springer, 2021. 2

    2021

  38. [38]

    Syn- thetic data for deep learning in computer vision & medical imaging: A means to reduce data bias.ACM Computing Surveys, 56(11):1–37, 2024

    Anthony Paproki, Olivier Salvado, and Clinton Fookes. Syn- thetic data for deep learning in computer vision & medical imaging: A means to reduce data bias.ACM Computing Surveys, 56(11):1–37, 2024. 2

    2024

  39. [39]

    A 3d face model for pose and illumination invariant face recognition

    Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. A 3d face model for pose and illumination invariant face recognition. InIEEE Int. Conf. Adv. Video Signal Based Surveill., pages 296–301. IEEE,

  40. [40]

    Towards realistic generative 3d face models

    Aashish Rai, Hiresh Gupta, Ayush Pandey, Francisco Vi- cente Carrasco, Shingo Jason Takagi, Amaury Aubel, Daeil Kim, Aayush Prakash, and Fernando De la Torre. Towards realistic generative 3d face models. InWACV, pages 3738– 3748, 2024. 2

    2024

  41. [41]

    Infinite photorealistic worlds using procedural generation

    Alexander Raistrick, Lahav Lipson, Zeyu Ma, Lingjie Mei, Mingzhe Wang, Yiming Zuo, Karhan Kayan, Hongyu Wen, Beining Han, Yihan Wang, et al. Infinite photorealistic worlds using procedural generation. InCVPR, pages 12630– 12641, 2023. 2

    2023

  42. [42]

    Accelerating 3d deep learning with pytorch3d

    Nikhila Ravi, Jeremy Reizenstein, David Novotny, Tay- lor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. arXiv preprint arXiv:2007.08501, 2020. 6

    Pith/arXiv arXiv 2007

  43. [43]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMICCAI, pages 234–241. Springer, 2015. 5

    2015

  44. [44]

    As-rigid-as-possible surface modeling

    Olga Sorkine and Marc Alexa. As-rigid-as-possible surface modeling. InSymposium on Geometry Processing, pages 109–116. Citeseer, 2007. 4

    2007

  45. [45]

    What is mixed reality? InCHI, pages 1–15, 2019

    Maximilian Speicher, Brian D Hall, and Michael Nebeling. What is mixed reality? InCHI, pages 1–15, 2019. 1

    2019

  46. [46]

    Effect of illumination on automatic ex- pression recognition: a novel 3d relightable facial database

    Giota Stratou, Abhijeet Ghosh, Paul Debevec, and Louis- Philippe Morency. Effect of illumination on automatic ex- pression recognition: a novel 3d relightable facial database. InFG, pages 611–618. IEEE, 2011. 3, 6

    2011

  47. [47]

    3d face tracking from 2d video through iterative dense uv to image flow

    Felix Taubner, Prashant Raina, Mathieu Tuli, Eu Wern Teh, Chul Lee, and Jinmiao Huang. 3d face tracking from 2d video through iterative dense uv to image flow. InCVPR, pages 1227–1237, 2024. 2

    2024

  48. [48]

    An image inpainting technique based on the fast marching method.Journal of graphics tools, 9(1): 23–34, 2004

    Alexandru Telea. An image inpainting technique based on the fast marching method.Journal of graphics tools, 9(1): 23–34, 2004. 4

    2004

  49. [49]

    Facevr: Real-time gaze-aware facial reenactment in virtual reality.ACM TOG, 37(2):1–15, 2018

    Justus Thies, Michael Zollh ¨ofer, Marc Stamminger, Chris- tian Theobalt, and Matthias Nießner. Facevr: Real-time gaze-aware facial reenactment in virtual reality.ACM TOG, 37(2):1–15, 2018. 1

    2018

  50. [50]

    Recognizing action units for facial expression analysis.IEEE TPAMI, 23 (2):97–115, 2001

    Y-I Tian, Takeo Kanade, and Jeffrey F Cohn. Recognizing action units for facial expression analysis.IEEE TPAMI, 23 (2):97–115, 2001. 2

    2001

  51. [51]

    Attention is all you need.NeurIPS, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.NeurIPS, 30, 2017. 5

    2017

  52. [52]

    Rethinking the learn- ing paradigm for dynamic facial expression recognition

    Hanyang Wang, Bo Li, Shuang Wu, Siyuan Shen, Feng Liu, Shouhong Ding, and Aimin Zhou. Rethinking the learn- ing paradigm for dynamic facial expression recognition. In CVPR, pages 17958–17968, 2023. 2, 6

    2023

  53. [53]

    Suppressing uncertainties for large-scale facial expres- sion recognition

    Kai Wang, Xiaojiang Peng, Jianfei Yang, Shijian Lu, and Yu Qiao. Suppressing uncertainties for large-scale facial expres- sion recognition. InCVPR, pages 6897–6906, 2020. 2

    2020

  54. [54]

    Region attention networks for pose and occlusion robust facial expression recognition.IEEE TIP, 29:4057– 4069, 2020

    Kai Wang, Xiaojiang Peng, Jianfei Yang, Debin Meng, and Yu Qiao. Region attention networks for pose and occlusion robust facial expression recognition.IEEE TIP, 29:4057– 4069, 2020. 2

    2020

  55. [55]

    Mead: A large-scale audio-visual dataset for emotional talking-face generation

    Kaisiyuan Wang, Qianyi Wu, Linsen Song, Zhuoqian Yang, Wayne Wu, Chen Qian, Ran He, Yu Qiao, and Chen Change Loy. Mead: A large-scale audio-visual dataset for emotional talking-face generation. InECCV, pages 700–717. Springer,

  56. [56]

    To- wards real-world blind face restoration with generative facial prior

    Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. To- wards real-world blind face restoration with generative facial prior. InCVPR, pages 9168–9178, 2021. 4

    2021

  57. [57]

    Vr fa- cial animation via multiview image translation.ACM TOG, 38(4):1–16, 2019

    Shih-En Wei, Jason Saragih, Tomas Simon, Adam W Harley, Stephen Lombardi, Michal Perdoch, Alexander Hy- pes, Dawei Wang, Hernan Badino, and Yaser Sheikh. Vr fa- cial animation via multiview image translation.ACM TOG, 38(4):1–16, 2019. 1, 2, 6

    2019

  58. [58]

    Fake it till you make it: face analysis in the wild using synthetic data alone

    Erroll Wood, Tadas Baltru ˇsaitis, Charlie Hewitt, Sebastian Dziadzio, Thomas J Cashman, and Jamie Shotton. Fake it till you make it: face analysis in the wild using synthetic data alone. InICCV, pages 3681–3691, 2021. 2, 3

    2021

  59. [59]

    3d face reconstruction with dense landmarks

    Erroll Wood, Tadas Baltru ˇsaitis, Charlie Hewitt, Matthew Johnson, Jingjing Shen, Nikola Milosavljevi´c, Daniel Wilde, Stephan Garbin, Toby Sharp, Ivan Stojiljkovi´c, et al. 3d face reconstruction with dense landmarks. InECCV, pages 160–

  60. [60]

    Springer, 2022. 2, 3

    2022

  61. [61]

    Synthetic data can also teach: Synthesizing ef- fective data for unsupervised visual representation learning

    Yawen Wu, Zhepeng Wang, Dewen Zeng, Yiyu Shi, and Jingtong Hu. Synthetic data can also teach: Synthesizing ef- fective data for unsupervised visual representation learning. InAAAI, pages 2866–2874, 2023. 2

    2023

  62. [62]

    Vision transformer with attentive pooling for robust facial expression recognition.IEEE Trans

    Fanglei Xue, Qiangchang Wang, Zichang Tan, Zhongsong Ma, and Guodong Guo. Vision transformer with attentive pooling for robust facial expression recognition.IEEE Trans. Affect. Comput., 2022. 2, 6

    2022

  63. [63]

    Learn from all: Erasing attention consistency for noisy label facial expression recognition

    Yuhang Zhang, Chengrui Wang, Xu Ling, and Weihong Deng. Learn from all: Erasing attention consistency for noisy label facial expression recognition. InECCV, pages 418–434. Springer, 2022. 2

    2022

  64. [64]

    Generalizable facial expression recognition

    Yuhang Zhang, Xiuqi Zheng, Chenyi Liang, Jiani Hu, and Weihong Deng. Generalizable facial expression recognition. InECCV, pages 231–248. Springer, 2024. 2, 3, 6

    2024

  65. [65]

    Former-dfer: Dynamic fa- cial expression recognition transformer

    Zengqun Zhao and Qingshan Liu. Former-dfer: Dynamic fa- cial expression recognition transformer. InACM MM, pages 1553–1561, 2021. 2

    2021

  66. [66]

    Robust lightweight facial expression recognition network with label distribution training

    Zengqun Zhao, Qingshan Liu, and Feng Zhou. Robust lightweight facial expression recognition network with label distribution training. InAAAI, pages 3510–3519, 2021. 2, 6

    2021

  67. [67]

    Enhancing zero-shot facial expression recognition by llm knowledge transfer

    Zengqun Zhao, Yu Cao, Shaogang Gong, and Ioannis Pa- tras. Enhancing zero-shot facial expression recognition by llm knowledge transfer. InWACV, pages 1–10, 2025. 2, 6

    2025

  68. [68]

    Deep single-image portrait relighting

    Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, and David W Ja- cobs. Deep single-image portrait relighting. InICCV, pages 7194–7202, 2019. 4

    2019

  69. [69]

    face-parsing.pytorch.https://github

    zllrunning. face-parsing.pytorch.https://github. com/zllrunning/face-parsing.PyTorch, 2019. 3 Enhancing Facial Expression Recognition in Head-Mounted Displays with Synthetic Data Supplementary Material A1. Texture-Space Alignment Network Data synthesis for TSAN training.Algorithm 1 outlines the algorithm for generating a single pair of ground-truth and mi...

    2019

This paper was first reviewed by grok-4.5 on July 11, 2026.