Pith. sign in

REVIEW 3 major objections 5 minor 70 references

FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision

T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Face reconstruction trained with direct 3D expression supervision from 4D scans and an intensity-aware emotion loss reads emotions better on AffectNet while staying geometrically on par with the best reconstruction methods.

desk verdict The geometric recipe is real, but the AffectNet emotion gains are likely an artifact of training on the evaluation images. read the letter →

arxiv 2511.21245 v3 pith:ZMCUAT4U submitted 2025-11-26 cs.CV

Chen Ling , Henglin Shi , Hedvig Kjellström This is my paper
classification cs.CV
keywords 3DfacereconstructionfacialexpressionrecognitionFLAMEmodelvalence-arousalregressiondirectsupervision4Dscansparametersaffectivecomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FIELDS claims that 2D self-supervised face reconstruction leaves the learned expression codes poorly calibrated for emotion: they lack authentic 3D deformation targets and can exploit intensity exaggeration to fool affect classifiers. The paper adds two supervision signals to a standard reconstruction pipeline: an MSE loss between predicted FLAME expression parameters and parameters fitted from spontaneous 4D scans of BP4D, and an emotion head on the expression parameters trained with valence–arousal regression and discrete-class cross-entropy. On AffectNet, this raises emotion classification from 52.3% to 56.0% accuracy and valence CCC from 0.686 to 0.737, while BP4D vertex error stays effectively tied with the best geometry-centric methods (0.0998 vs 0.0989 mm). The two signals are complementary: the 3D loss calibrates expression magnitude (parameter MSE drops from 5.954 to 0.173), and the emotion loss injects affect semantics without exaggerating the mesh.

What carries the argument

The central object is the FLAME expression code: a 50-dimensional blendshape vector plus 2 eye-closure and 3 jaw-pose parameters that FLAME decodes (locally, linearly) to mesh deformation. FIELDS attaches two objectives to this code. L3D-GT is a mean-squared error between the predicted code and the code obtained from fitting FLAME to BP4D spontaneous 4D scans; it anchors deformation magnitude to real muscle motion. Lemo is an emotion head on the same code, trained with a joint cross-entropy classification and valence–arousal regression loss; it aligns the code with affect labels while the regression terms act as an amplitude regularizer that discourages the exaggeration shortcut. Alternating

What would settle it

Train FIELDS identically but replace the BP4D-fitted FLAME targets with expression parameters from a different fitting routine or a different 4D dataset; if the AffectNet gains and the parameter-MSE calibration mostly disappear or invert, the benefit is an artifact of the specific pseudo ground truth. A sharper check: on the BP4D test split, compute raw scan-to-mesh vertex displacement (neutral-to-expression, without going through FLAME) for each prediction; if FIELDS is not closer to the raw scan deformation than the geometry-only baseline, the direct supervision did not actually improve expr

Watch

Extended reading notes

Core claim

In the paper's own framing: a hybrid 2D/3D supervision recipe lets a monocular face-reconstruction encoder produce FLAME expression codes that are simultaneously faithful to real muscle deformation and informative for affect recognition. Direct supervision L3D-GT matches the predicted 50-dimensional expression and jaw parameters to FLAME fits of BP4D spontaneous 4D scans, shrinking the BP4D expression–jaw parameter MSE by about an order of magnitude compared with 2D-only training (0.159–0.173 vs 5.954). The intensity-aware emotion loss Lemo couples discrete emotion classification with valence–arousal regression so that the code must separate categories while also tracking continuous magnitud

Load-bearing premise

The load-bearing premise is that the FLAME parameters fitted to BP4D spontaneous 4D scans are faithful, unbiased expression ground truth; if that fitting under- or over-expresses, compresses rare expressions, or entangles identity with expression, the encoder learns those biases—a risk the paper itself concedes when it notes explicit 3D supervision 'inherits the biases of the pseudo ground truth.'

Editorial extensions

If this is right

  • Direct 3D expression parameter supervision improves in-the-wild emotion classification and valence–arousal regression compared with 2D self-supervision alone, with no loss of geometric fidelity (vertex error 0.0998 vs 0.0989 mm, AUC@5.0 equal at 0.982).
  • Intensity-aware emotion supervision prevents expression inflation: higher FER scores come with calibrated (non-exaggerated) FLAME parameters rather than amplified motion.
  • The two supervision signals are complementary: the 3D loss chiefly calibrates expression magnitude, the emotion loss chiefly adds affect semantics; combined they deliver the best classification while keeping near-peak V/A regression.
  • Standard mesh-level geometry metrics (vertex error, keypoint error) are insensitive to these affect improvements; FLAME parameter-space evaluation is needed to see the expression-fidelity gains.
  • The learned expression codes transfer to a 5-fold AffectNet evaluation protocol, suggesting the hybrid supervision yields representations that generalize across in-the-wild images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the complementarity generalizes, reconstruction objective design for affective applications should separate magnitude calibration from semantic alignment rather than use a single emotion-consistency loss.
  • Editorial inference: the gains likely depend on the diversity and naturalness of the BP4D fitting targets; if fitting regularization compresses rare or extreme expressions, the model may underperform on classes like Disgust/Anger, which the paper itself notes remain challenging.
  • Editorial inference: a sharper test of intensity calibration would compare predicted expression magnitude against manual Action Unit intensity ratings on held-out images; the paper's account predicts a closer, less saturated relationship than prior methods.
  • Editorial inference: the paper acknowledges a focus on per-frame affect preservation, so temporal or geometry-critical applications may require different trade-offs; extending to video with temporal cues is left open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FIELDS, a monocular 3D face reconstruction method that extends the TEASER self-supervised pipeline with (i) direct 3D expression-parameter supervision (L_3D-GT) using FLAME fits to BP4D 4D scans (Eq. 6) and (ii) an auxiliary emotion-recognition head with an intensity-aware loss (L_emo) trained on AffectNet discrete and valence-arousal labels (Eq. 7). The authors claim that this hybrid supervision improves facial expression recognition and V-A regression on AffectNet while remaining on par with geometry-centric state-of-the-art on BP4D geometric error. The paper includes ablations over the two supervision signals, a λ_emo sweep, subject-exclusive BP4D splits, and qualitative comparisons.

Significance. If the empirical claims hold, the idea of combining direct parameter-space supervision from 4D scans with an affect-aligned auxiliary loss is a useful direction for emotion-aware 3D face reconstruction. The paper provides a subject-exclusive BP4D split, detailed training schedules, and systematic ablations, which are strengths. However, the central affect-evaluation protocol appears to suffer from train/evaluation overlap on AffectNet, and the headline FLAME parameter-MSE metric (Table 3) is self-referential to the training target. These issues undermine the main empirical claims as currently presented, though they are potentially addressable with a corrected evaluation protocol.

major comments (3)
  1. [§4.1, §4.2, Eq. (7), Suppl. §6] The AffectNet evaluation appears contaminated. FIELDS trains on AffectNet images and labels (26% of each batch per the supplementary sampling schedule; L_emo in Eq. 7), then emotion classification and V-A regression are evaluated by 5-fold cross-validation on the official AffectNet training split. The paper never states that fold-specific validation images are excluded from training or that AffectNet labels are withheld for those images. As written, the encoder and emotion head have seen the exact images/labels used to train and test the probe MLP, so the reported gains in Tables 1 and 2 may reflect direct supervision rather than generalizable representation quality. The baselines are not trained on AffectNet labels, making the comparison unfair. Please clarify whether the validation folds were excluded, and if not, re-run the evaluation with proper fold-exclusion or an external test set
  2. [Table 3, Eq. (6), §3.2.1] The 'FLAME parameter MSE' on BP4D test subjects is computed against the same FLAME fitting pipeline that provides the L_3D-GT training targets. FIELDS is explicitly trained with an MSE loss to those targets, so its dramatically lower parameter error (0.173 vs 1.27-3.35) is expected by construction and is not a neutral cross-method reconstruction fidelity metric. The geometric claims should rest on vertex/keypoint errors and CED AUC (Table 6), where FIELDS is close but not superior. Reporting Table 3 as a headline result is misleading; please reposition it as a diagnostic of fitting-pipeline consistency, not as evidence of better expression fidelity.
  3. [§4.3, Table 4] The ablation F2 vs F1 shows that adding L_3D-GT reduces FLAME parameter MSE from 5.954 to 0.159. This reduction is tautological because L_3D-GT directly optimizes the same metric on the same fitted target distribution. The claim that L_3D-GT 'calibrates expression magnitude' needs independent support, e.g., evaluation on an external 3D scan benchmark with a different fitting protocol, or at least a clear acknowledgment that this metric is not independent of the training loss.
minor comments (5)
  1. [Suppl. §7, Eq. (13)] The ellipse equation is written with 'ra' in both denominators: (u/ra)^2 + (v/ra)^2 ≤ 1. The second term should presumably be (v/rb)^2. Please correct.
  2. [§4.2] The text says 'fold-specific, class-balanced validation sets' but does not specify how class balancing is performed. Please describe the procedure or clarify whether it is just the natural class distribution.
  3. [Abstract / §1] The abstract claims 'external evaluations' are improved, but the paper only evaluates on AffectNet (which is also a training set) and BP4D (used for supervision). The external-evaluation claim is not supported unless additional datasets are used.
  4. [Table 6 caption] The claim in §8.1 that FIELDS 'achieves the strongest emotion classification and V A regression' while vertex error is slightly worse than TEASER/SMIRK is internally consistent, but the wording 'strongest affect' should be conditioned on the unresolved evaluation-overlap issue.
  5. [References] Some reference page numbers are inconsistent (e.g., [7] and [63] show trailing '1'); please standardize. Also, 'Digust' in the Figure 5 description is a typo.

Circularity Check

2 steps flagged · score 6.0 of 10

FIELDS's headline gains are partly self-referential: the BP4D parameter-MSE metric is the same expression-fit MSE used as the L3D-GT training target, and the AffectNet evaluation is run on the official split that supplies 26% of training batches without any stated fold exclusion.

  1. self definitional [§3.2.1 Eq. (6); §4.2 'Reconstruction Fidelity' / Table 3; §7]
    "To directly supervise the expression encoder, we apply a mean squared error (MSE) loss between the predicted expression parameters ˆψexpr and the ground truth ψ expr ... we assess reconstruction fidelity by comparing each method's predicted FLAME expression–jaw parameters to reference parameters obtained from a unified FLAME fitting on BP4D, reporting parameter-space MSE."

    The training target in Eq. (6) is exactly the quantity reported in Table 3: MSE between the predicted FLAME expression/jaw parameters and the same BP4D FLAME fits produced by the §7 fitting pipeline (weights scan 2.0, landmark 0.1, shape 1e-4, pose 1e-3, expression 1e-4). FIELDS is explicitly optimized to minimize this loss on BP4D train subjects, so the order-of-magnitude Table 3 reduction (TEASER 3.345 vs FIELDS 0.173) largely certifies that the encoder memorized the fitter's parameter outputs, including the elliptical-crop and regularization biases of that fitter. The paper's claim of 'calibrated expression magnitude' is therefore self-referential: the evaluation reference is the same pseudo-ground-truth that supplied the supervision, not an independent measure of facial expression fide

  2. fitted input called prediction [§3.2.2 Eq. (7); §4.1; §4.2 'Emotion Analysis'; Supplementary §6]
    "We train on six public datasets – LRW [7], CelebA [35], FFHQ [25], MEAD [55], AffectNet [40], and BP4D [63] – ... Since the official AffectNet test set is no longer public, we perform 5-fold cross-validation on the official training split ... The cross-dataset sampling ratios are LRW 20%, CelebA 26%, AffectNet 26%, FFHQ 10%, and MEAD 8%."

    Eq. (7) trains the encoder and emotion head with AffectNet discrete and valence–arousal labels, and AffectNet supplies 26% of each training batch. The AffectNet evaluation then performs 5-fold CV on the same official training split. The paper never states that the validation folds were excluded from the AffectNet training batches; as written, the images/labels used for the reported 56.0% accuracy and V-A gains in Tables 1–2 may be exactly the samples seen during training. Under that protocol, the AffectNet 'predictions' are fitted inputs: the model is trained on the evaluation split, so the gains over SMIRK/TEASER—which receive no AffectNet labels—can reflect memorization or direct supervision rather than representation quality. Even if standard CV exclusion was intended, the missing state

full rationale

The core derivation chain of FIELDS is: add L3D-GT (MSE to FLAME expression fits on BP4D) and Lemo (AffectNet emotion supervision), then show improved AffectNet emotion metrics and improved BP4D FLAME-parameter MSE. Two load-bearing links in that chain are self-referential. First, the BP4D parameter-MSE evaluation is defined by the same fitted FLAME parameters used as the L3D-GT training target; the large improvement in Table 3 is therefore partly a statement that the model learned the fitter's biases, not an independent geometric or expression-fidelity result. The paper explicitly acknowledges that explicit 3D supervision 'inherits the biases of the pseudo ground truth,' which undercuts using that same pseudo-GT as the headline fidelity measure. Second, the AffectNet evaluation is performed on the official training split while AffectNet is simultaneously a training dataset at 26% sampling; absent an explicit statement that validation folds are excluded, the central 'strongest emotion classification and V-A regression' claim is not shown to be out-of-sample. I do not find load-bearing self-citation or imported uniqueness theorems: the baselines and FLAME fitting are external prior work, and the geometric vertex-error comparisons (e.g., 0.0998 vs 0.0989 mm) give some independent evidence that geometry is not degraded. However, the two constructed links above affect the paper's two headline claims, so the overall circularity score is 6 rather than a lower score.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

FIELDS introduces no new physical or model entities; all components (FLAME, TEASER tokenizer/synthesizer, MICA, AffectNet, BP4D) are reused. The load-bearing unknowns are the hand-tuned loss weights, the expression subspace choice, and the quality of the BP4D FLAME fits that serve as ground truth. The main epistemic risk is that both the direct-supervision target (Eq. 6) and the headline evaluation metric (Table 3) live in the coordinate system of the same fitting pipeline.

free parameters (4)
  • Loss weights λemo, λ3D, λshape-reg, λexpr-reg, and w2D vector = λemo=5, λ3D=5→1 two-stage, λshape-reg=100, λexpr-reg=1e-2, w2D=(10,10,10,100,500)
    Chosen by validation/ablation (supp §6, Tabs. 5/8). The λemo sweep from 0.5 to 10 directly tunes the headline FER and VA results.
  • BP4D FLAME-fitting weights = scan 2.0, landmark 0.1, shape 1e-4, pose 1e-3, expression 1e-4
    These shape the L3D-GT targets (Eq. 6) and the evaluation reference in Table 3; different weights would produce different 'ground truth' expressions and different reported MSE.
  • Expression subspace selection = ψexp∈R50 (of 100) + ψeye∈R2 + θjaw∈R3
    The restriction to a 55-D expression subspace of FLAME's 100-D codes is a modeling choice that bounds what direct supervision can recover.
  • Pretraining weights λlmk, λmica = 100, 10
    Hand-set to anchor the base model B used for shape/expression regularization in Eq. (9).
assumptions (5)
  • domain assumption FLAME's disentanglement of identity (β), expression (Ψ), and pose (θ) is valid enough that affect signals live mainly in Ψ
    The entire method and evaluation (emotion head on (β,Ψ), parameter-MSE on {ψexp,θjaw}) presupposes this disentanglement (§3.1).
  • domain assumption BP4D FLAME fits are accurate enough to serve as expression ground truth
    Eq. (6) trains directly on these fits; the paper concedes in Related Work that explicit 3D supervision 'inherits the biases of the pseudo ground truth'.
  • domain assumption AffectNet discrete and valence-arousal labels are valid per-image affect targets
    Lemo (Eq. 7) optimizes against them; AffectNet is known to contain label noise, which would mis-calibrate the 'intensity-aware' term.
  • domain assumption 2D landmark detectors (MediaPipe, FAN, InsightFace) and MICA shape estimates are reliable
    Pretraining Eq. (12) and the 2D consistency loss Eq. (8) rely on their accuracy.
  • domain assumption Parameter-space MSE against a fitted reference is a meaningful proxy for expression fidelity
    Table 3's headline calibration evidence; the paper itself flags that geometric/parameter proxies are imperfect in §4.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision." pith.science (2026). https://pith.science/paper/ZMCUAT4U

@misc{pith2026251121245,
  author       = {Pith},
  title        = {Pith review of: FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZMCUAT4U}},
  note         = {Machine review of arXiv:2511.21245}
}
read the original abstract

Monocular 3D face reconstruction estimates a 3D morphable model (3DMM) representation from a single image, providing geometry-aware expression codes that are useful for facial expression analysis and affect understanding. Despite strong progress, most pipelines are trained with image-level self-supervision and evaluated primarily by geometric fidelity, which does not necessarily maximize the affective utility of the learned expression representation and may encourage intensity-amplifying shortcuts when affect supervision is naively coupled. We propose FIELDS (Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision), a task-driven framework that learns FLAME expression codes for facial expression recognition (FER) under a geometric plausibility constraint. Using hybrid 2D/3D supervision, FIELDS improves affect prediction in both in-domain and external evaluations while maintaining competitive geometric fidelity on held-out and out-of-domain 3D benchmarks.

Figures

Figures reproduced from arXiv: 2511.21245 by the authors.

Figure 1
Figure 1. Comparison between the baseline approach, which relies on external emotion consistency losses, and our FIELDS framework, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A circumplex model of affect [15, 48] with discrete emo￾tions [14] overlaid. Adapted from [57]. and the geometry encoder, encouraging reconstructions that are both faithful to real muscle deformations and rich in genuine affective content. This dual-supervision strategy yields 3D expression representations that improve down￾stream FER performance without sacrificing geometric ac￾curacy or naturalness. In summary, ou… view at source ↗
Figure 3
Figure 3. Illustration of the FIELDS pipeline. Given an input image [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Visual Examples of 3D Face Reconstruction. From top to bottom: Emotion class label, Valence value label, Arousal value label, [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Emotion classification, per-class accuracy (mean [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: t-SNE visualization of FIELDS expression embeddings [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Visual examples of 3D face model and reconstruction in BP4D dataset, from left to right: BP4D original image, 3D scan, fitted [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Data Pre-Processing Pipeline. Data Pre-processing Details. All datasets undergo facial landmark detection using MediaPipe [36] and FAN [5]. The Insight Face 203 landmarks [22] which is applied in TEASER [34] are detected during training. Based on the MediaPipe landmark…
Figure 9
Figure 9. Figure 9: Illustration of the pre-training pipeline. Given an input image, the reconstruction encoder predicts FLAME parameters [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: An example of the raw scan (gray), the landmarks with [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Visual examples on AffectNet. From left to right, orig [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: BP4D 3D mesh model reconstruction examples (Female). From left to right: BP4D original image, 3D scan, fitted FLAME [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: BP4D 3D mesh model reconstruction examples (Male). From left to right: BP4D original image, 3D scan, fitted FLAME [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: t-SNE Cluster Visualization of FLAME Expression Parameters. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: UMAP Cluster Visualization of FLAME Expression Parameters. [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 3 linked inside Pith

  1. [1]

    Densereg: Fully convolutional dense shape regression in-the-wild

    Riza Alp Guler, George Trigeorgis, Epameinondas Anton- akos, Patrick Snape, Stefanos Zafeiriou, and Iasonas Kokki- nos. Densereg: Fully convolutional dense shape regression in-the-wild. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017. 3

    2017

  2. [2]

    Fully automated facial expression recognition using 3D mor- phable model and mesh-local binary pattern

    Hela Bejaoui, Haythem Ghazouani, and Walid Barhoumi. Fully automated facial expression recognition using 3D mor- phable model and mesh-local binary pattern. InInternational Conference on Advanced Concepts for Intelligent Vision Sys- tems, pages 39–50, 2017. 3

    2017

  3. [3]

    A morphable model for the synthesis of 3D faces

    V olker Blanz and Thomas Vetter. A morphable model for the synthesis of 3D faces. InACM SIGGRAPH, 1999. 1, 2

    1999

  4. [4]

    Face identification across different poses and illuminations with a 3D morphable model

    V olker Blanz, Sami Romdhani, and Thomas Vetter. Face identification across different poses and illuminations with a 3D morphable model. InIEEE International Conference on Automatic Face & Gesture Recognition, 2002. 3

    2002

  5. [5]

    How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks)

    Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks). InIEEE Interna- tional Conference on Computer Vision, 2017. 1, 2

    2017

  6. [6]

    Expnet: Landmark-free, deep, 3D facial expressions

    Feng-Ju Chang, Anh Tuan Tran, Tal Hassner, Iacopo Masi, Ram Nevatia, and Gerard Medioni. Expnet: Landmark-free, deep, 3D facial expressions. InIEEE International Confer- ence on Automatic Face & Gesture Recognition, 2018. 3

    2018

  7. [7]

    Lip reading in the wild

    Joon Son Chung and Andrew Zisserman. Lip reading in the wild. InAsian Conference on Computer Vision, 2017. 5, 1

    2017

  8. [8]

    Emoca: Emotion driven monocular face capture and animation

    Radek Dan ˇeˇcek, Michael J Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 2, 3, 5, 6, 7, 8

    2022

Show all 70 references
  1. [9]

    Accurate 3D face reconstruction with weakly-supervised learning: From single image to image set

    Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3D face reconstruction with weakly-supervised learning: From single image to image set. InIEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019. 2

    2019

  2. [10]

    Accurate 3D face reconstruction with weakly-supervised learning: From single image to image set

    Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3D face reconstruction with weakly-supervised learning: From single image to image set. InIEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019. 3

    2019

  3. [11]

    3D face reconstruction based on a single image: a review.IEEE Access, 12:59450–59473, 2024

    Haojie Diao, Xingguo Jiang, Yang Fan, Ming Li, and Hongcheng Wu. 3D face reconstruction based on a single image: a review.IEEE Access, 12:59450–59473, 2024. 3

    2024

  4. [12]

    IG3D: Integrating 3D face representa- tions in facial expression inference

    Lu Dong, Xiao Wang, Srirangaraj Setlur, Venu Govindaraju, and Ifeoma Nwogu. IG3D: Integrating 3D face representa- tions in facial expression inference. InEuropean Conference on Computer Vision, 2025. 3

    2025

  5. [13]

    3D morphable face models—past, present, and future.ACM Transactions on Graphics, 39(5):1–38, 2020

    Bernhard Egger, William AP Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, et al. 3D morphable face models—past, present, and future.ACM Transactions on Graphics, 39(5):1–38, 2020. 1, 2

    2020

  6. [14]

    Universals and cultural differences in facial expressions of emotion

    Paul Ekman. Universals and cultural differences in facial expressions of emotion. InNebraska Symposium on Motiva- tion, 1971. 1, 2

    1971

  7. [15]

    Independence and bipolarity in the structure of current affect.Journal of Personality and Social Psychology, 74(4):967, 1998

    Lisa Feldman Barrett and James A Russell. Independence and bipolarity in the structure of current affect.Journal of Personality and Social Psychology, 74(4):967, 1998. 2

    1998

  8. [16]

    Learning an animatable detailed 3D face model from in-the- wild images.ACM Transactions on Graphics, 40(4):1–13,

    Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. Learning an animatable detailed 3D face model from in-the- wild images.ACM Transactions on Graphics, 40(4):1–13,

  9. [17]

    Improving human–robot interaction by en- hancing nao robot awareness of human facial expression

    Chiara Filippini, David Perpetuini, Daniela Cardone, and Ar- cangelo Merla. Improving human–robot interaction by en- hancing nao robot awareness of human facial expression. Sensors, 21(19):6438, 2021. 1

    2021

  10. [18]

    Spectre: Visual speech-informed per- ceptual 3D facial expression reconstruction from videos

    Panagiotis P Filntisis, George Retsinas, Foivos Paraperas- Papantoniou, Athanasios Katsamanis, Anastasios Roussos, and Petros Maragos. Spectre: Visual speech-informed per- ceptual 3D facial expression reconstruction from videos. InIEEE/CVF Conference on Computer Vision and Pat...

    2023

  11. [19]

    Corrective 3D reconstruction of lips from monoc- ular video.ACM Transactions on Graphics, 35(6):219–1,

    Pablo Garrido, Michael Zollh ¨ofer, Chenglei Wu, Derek Bradley, Patrick P ´erez, Thabo Beeler, and Christian Theobalt. Corrective 3D reconstruction of lips from monoc- ular video.ACM Transactions on Graphics, 35(6):219–1,

  12. [20]

    Kyle Genova, Forrester Cole, Aaron Maschinot, Aaron Sarna, Daniel Vlasic, and William T. Freeman. Unsupervised training for 3D morphable model regression. InIEEE Con- ference on Computer Vision and Pattern Recognition, 2018. 3

    2018

  13. [21]

    Morphable face models-an open framework

    Thomas Gerig, Andreas Morel-Forster, Clemens Blumer, Bernhard Egger, Marcel Luthi, Sandro Sch ¨onborn, and Thomas Vetter. Morphable face models-an open framework. InIEEE International Conference on Automatic Face & Ges- ture Recognition, 2018. 3

    2018

  14. [22]

    InsightFace: 2D and 3D face analysis project.https : / / github

    Jia Guo and Jiankang Deng. InsightFace: 2D and 3D face analysis project.https : / / github . com / deepinsight/insightface, 2022. 1

    2022

  15. [23]

    Towards fast, accurate and stable 3D dense face alignment

    Jianzhu Guo, Xiangyu Zhu, Yang Yang, Fan Yang, Zhen Lei, and Stan Z Li. Towards fast, accurate and stable 3D dense face alignment. InEuropean Conference on Computer Vi- sion, 2020. 3

    2020

  16. [24]

    Mohan Karnati, Ayan Seal, Debotosh Bhattacharjee, Anis Yazidi, and Ondrej Krejcar. Understanding deep learning techniques for recognition of human emotions using facial expressions: A comprehensive survey.IEEE Transactions on Instrumentation and Measurement, 72:1–31, 2023. 1, 2

    2023

  17. [25]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 5, 1

    2019

  18. [26]

    Drivers fatigue level prediction using facial, and 9 head behavior information.IEEE Access, 9:121686–121697,

    Haider A Kassem, Morshed Chowdhury, and Jemal H Abawajy. Drivers fatigue level prediction using facial, and 9 head behavior information.IEEE Access, 9:121686–121697,

  19. [27]

    Look Ma, no land- marks!

    Tatsuro Koizumi and William AP Smith. “Look Ma, no land- marks!” – unsupervised, model-based dense face alignment. InEuropean Conference on Computer Vision, 2020. 3

    2020

  20. [28]

    in the wild

    Mohammad Rami Koujan, Luma Alharbawee, Giorgos Gi- annakakis, Nicolas Pugeault, and Anastasios Roussos. Real- time facial expression recognition “in the wild” by disen- tangling 3D expression from identity. InIEEE Interna- tional Conference on Automatic Face & Gesture Recogni-...

    2020

  21. [29]

    The role of facial movements in emotion recogni- tion.Nature Reviews Psychology, 2(5):283–296, 2023

    Eva G Krumhuber, Lina I Skora, Harold CH Hill, and Karen Lander. The role of facial movements in emotion recogni- tion.Nature Reviews Psychology, 2(5):283–296, 2023. 1

    2023

  22. [30]

    Deep facial expression recogni- tion: A survey.IEEE Transactions on Affective Computing, 13(3):1195–1215, 2020

    Shan Li and Weihong Deng. Deep facial expression recogni- tion: A survey.IEEE Transactions on Affective Computing, 13(3):1195–1215, 2020. 1, 2

    2020

  23. [31]

    Learning a model of facial shape and expression from 4D scans.ACM Transactions on Graphics, 36(6):194– 1, 2017

    Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4D scans.ACM Transactions on Graphics, 36(6):194– 1, 2017. 1, 2, 3

    2017

  24. [32]

    Black, Hao Li, and Javier Romero

    Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. TF-FLAME.https://github.com/ TimoBolkart/TF_FLAME, 2021. 4

    2021

  25. [33]

    Dense face alignment

    Yaojie Liu, Amin Jourabloo, William Ren, and Xiaoming Liu. Dense face alignment. InIEEE International Confer- ence on Computer Vision Workshops, 2017. 3

    2017

  26. [34]

    Teaser: Token enhanced spatial modeling for expres- sions reconstruction

    Yunfei Liu, Lei Zhu, Lijian Lin, Ye Zhu, Ailing Zhang, and Yu Li. Teaser: Token enhanced spatial modeling for expres- sions reconstruction. InInternational Conference on Learn- ing Representations, 2025. 2, 3, 4, 5, 6, 7, 8, 1

    2025

  27. [35]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. InIEEE Interna- tional Conference on Computer Vision, 2015. 5, 1

    2015

  28. [36]

    Medi- apipe: A framework for building perception pipelines.arXiv preprint arXiv:1906.08172, 2019

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris Mc- Clanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo- Ling Chang, Ming Guang Yong, Juhyun Lee, et al. Medi- apipe: A framework for building perception pipelines.arXiv preprint arXiv:1906.08172, 2019. 1, 2

    1906 arXiv

  29. [37]

    3D facial expression modeling based on facial land- marks in single image.Neurocomputing, 355:155–167,

    Chenlei Lv, Zhongke Wu, Xingce Wang, and Mingquan Zhou. 3D facial expression modeling based on facial land- marks in single image.Neurocomputing, 355:155–167,

  30. [38]

    Recognition of facial expression from optical flow.IEICE Transactions on Information and Systems, 74 (10):3474–3483, 1991

    Kenji Mase. Recognition of facial expression from optical flow.IEICE Transactions on Information and Systems, 74 (10):3474–3483, 1991. 2

    1991

  31. [39]

    Wadsworth, Belmont, CA, 1st edition, 1971

    Albert Mehrabian.Silent Messages. Wadsworth, Belmont, CA, 1st edition, 1971. 1

    1971

  32. [40]

    AffectNet: A database for facial expression, valence, and arousal computing in the wild.IEEE Transactions on Affective Computing, 10(1):18–31, 2017

    Ali Mollahosseini, Behzad Hasani, and Mohammad H Ma- hoor. AffectNet: A database for facial expression, valence, and arousal computing in the wild.IEEE Transactions on Affective Computing, 10(1):18–31, 2017. 5, 1

    2017

  33. [41]

    Enhanced facial expression recognition using 3D point sets and geometric deep learning.Medical & Biologi- cal Engineering & Computing, 59(6):1235–1244, 2021

    Duc-Phong Nguyen, Marie-Christine Ho Ba Tho, and Tien- Tuan Dao. Enhanced facial expression recognition using 3D point sets and geometric deep learning.Medical & Biologi- cal Engineering & Computing, 59(6):1235–1244, 2021. 2

    2021

  34. [42]

    Shah, and Ioan- nis A

    Omar Ocegueda, Tianhong Fang, Shishir K. Shah, and Ioan- nis A. Kakadiaris. Expressive maps for 3D facial expression recognition. InIEEE International Conference on Computer Vision Workshops, 2011. 2

    2011

  35. [43]

    A 3D face model for pose and illumination invariant face recognition

    Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. A 3D face model for pose and illumination invariant face recognition. InIEEE Inter- national Conference on Advanced Video and Signal Based Surveillance, 2009. 2

    2009

  36. [44]

    Monocular 3D facial expression features for continuous affect recognition.IEEE Transac- tions on Multimedia, 23:3540–3550, 2020

    Ercheng Pei, Meshia Cedric Oveneke, Yong Zhao, Dongmei Jiang, and Hichem Sahli. Monocular 3D facial expression features for continuous affect recognition.IEEE Transac- tions on Multimedia, 23:3540–3550, 2020. 3

    2020

  37. [45]

    Pat- tichis

    Melpo Pittara, Andreas Anastasiou, Konstantinos Andreou, Andreas Panayides, Nicolai Petkov, and Constantinos S. Pat- tichis. Facial image and video pain intensity estimation. In Asilomar Conference on Signals, Systems, and Computers,

  38. [46]

    To- wards a complete 3D morphable model of the human head

    Stylianos Ploumpis, Evangelos Ververas, Eimear O’Sullivan, Stylianos Moschoglou, Haoyang Wang, Nick Pears, William AP Smith, Baris Gecer, and Stefanos Zafeiriou. To- wards a complete 3D morphable model of the human head. IEEE Transactions on Pattern Analysis and Machine Intell...

    2020

  39. [47]

    Filntisis, Radek Dan ˇeˇcek, Victoria F

    George Retsinas, Panagiotis P. Filntisis, Radek Dan ˇeˇcek, Victoria F. Abrevaya, Anastasios Roussos, Timo Bolkarr, and Petros Maragos. 3D facial expressions through analysis- by-neural-synthesis. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2, 3, 5,...

    2024

  40. [48]

    A circumplex model of affect.Journal of personality and social psychology, 39(6):1161, 1980

    James A Russell. A circumplex model of affect.Journal of personality and social psychology, 39(6):1161, 1980. 2

    1980

  41. [49]

    Learning to regress 3D face shape and expression from an image without 3D supervision

    Soubhik Sanyal, Timo Bolkart, Haiwen Feng, and Michael J Black. Learning to regress 3D face shape and expression from an image without 3D supervision. InIEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2019. 2

    2019

  42. [50]

    Self-supervised monocular 3D face reconstruction by occlusion-aware multi- view geometry consistency

    Jiaxiang Shang, Tianwei Shen, Shiwei Li, Lei Zhou, Ming- min Zhen, Tian Fang, and Long Quan. Self-supervised monocular 3D face reconstruction by occlusion-aware multi- view geometry consistency. InEuropean Conference on Computer Vision, 2020. 3

    2020

  43. [51]

    3D face reconstruction in deep learning era: A survey.Archives of Computational Methods in Engineering, 29(5):3475–3507, 2022

    Sahil Sharma and Vijay Kumar. 3D face reconstruction in deep learning era: A survey.Archives of Computational Methods in Engineering, 29(5):3475–3507, 2022. 1, 3

    2022

  44. [52]

    Are 3D face shapes expressive enough for recognising continu- ous emotions and action unit intensities?IEEE Transactions on Affective Computing, 2023

    Mani Kumar Tellamekala, ¨Omer S ¨umer, Bj ¨orn W Schuller, Elisabeth Andr´e, Timo Giesbrecht, and Michel Valstar. Are 3D face shapes expressive enough for recognising continu- ous emotions and action unit intensities?IEEE Transactions on Affective Computing, 2023. 3

    2023

  45. [53]

    MOFA: Model-based deep convolutional face au- toencoder for unsupervised monocular reconstruction

    Ayush Tewari, Michael Zollhofer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Christian Theobalt. MOFA: Model-based deep convolutional face au- toencoder for unsupervised monocular reconstruction. In IEEE International Conference on Computer Vision Work- s...

    2017

  46. [54]

    Regressing robust and discriminative 3D mor- phable models with a very deep neural network

    Anh Tuan Tran, Tal Hassner, Iacopo Masi, and G ´erard Medioni. Regressing robust and discriminative 3D mor- phable models with a very deep neural network. In 10 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017. 3

    2017

  47. [55]

    Mead: A large-scale audio-visual dataset for emotional talking-face generation

    Kaisiyuan Wang, Qianyi Wu, Linsen Song, Zhuoqian Yang, Wayne Wu, Chen Qian, Ran He, Yu Qiao, and Chen Change Loy. Mead: A large-scale audio-visual dataset for emotional talking-face generation. InEuropean Conference on Com- puter Vision, 2020. 5, 1

    2020

  48. [56]

    A survey on facial expression recog- nition of static and dynamic emotions.arXiv preprint arXiv:2408.15777, 2024

    Yan Wang, Shaoqi Yan, Yang Liu, Wei Song, Jing Liu, Yang Chang, Xinji Mai, Xiping Hu, Wenqiang Zhang, and Zhongxue Gan. A survey on facial expression recog- nition of static and dynamic emotions.arXiv preprint arXiv:2408.15777, 2024. 1, 2

    2024 arXiv

  49. [57]

    Children acquire emotion categories gradually.Cognitive development, 23(2): 291–312, 2008

    Sherri C Widen and James A Russell. Children acquire emotion categories gradually.Cognitive development, 23(2): 291–312, 2008. 2

    2008

  50. [58]

    Analysis of fa- cial expressions based on silhouettes

    JC Wojdel, A Wojdel, and LJM Rothkrantz. Analysis of fa- cial expressions based on silhouettes. InFifth Annual Con- ference of ASCI, 1999. 2

    1999

  51. [59]

    3D face reconstruction with dense landmarks.arXiv preprint arXiv:2204.02776, 2022

    E Wood, T Baltrusaitis, C Hewitt, M Johnson, J Shen, N Milosavljevic, D Wilde, S Garbin, T Sharp, I Stojiljkovic, et al. 3D face reconstruction with dense landmarks.arXiv preprint arXiv:2204.02776, 2022. 3

    2022 arXiv

  52. [60]

    MVF- Net: Multi-view 3D face morphable model regression

    Fanzi Wu, Linchao Bao, Yajing Chen, Yonggen Ling, Yib- ing Song, Songnan Li, King Ngi Ngan, and Wei Liu. MVF- Net: Multi-view 3D face morphable model regression. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 3

    2019

  53. [61]

    Facescape: a large-scale high quality 3D face dataset and detailed riggable 3D face prediction

    Haotian Yang, Hao Zhu, Yanru Wang, Mingkai Huang, Qiu Shen, Ruigang Yang, and Xun Cao. Facescape: a large-scale high quality 3D face dataset and detailed riggable 3D face prediction. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. 3

    2020

  54. [62]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. 3

    2018

  55. [63]

    BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database.Image and Vision Com- puting, 32(10):692–706, 2014

    Xing Zhang, Lijun Yin, Jeffrey F Cohn, Shaun Canavan, Michael Reale, Andy Horowitz, Peng Liu, and Jeffrey M Gi- rard. BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database.Image and Vision Com- puting, 32(10):692–706, 2014. 2, 3, 4, 5, 1

    2014

  56. [64]

    Toward label-efficient emotion and sen- timent analysis.Proceedings of the IEEE, 111(10):1159– 1197, 2023

    Sicheng Zhao, Xiaopeng Hong, Jufeng Yang, Yanyan Zhao, and Guiguang Ding. Toward label-efficient emotion and sen- timent analysis.Proceedings of the IEEE, 111(10):1159– 1197, 2023. 1

    2023

  57. [65]

    Kakadiaris, and Liming Chen

    Xi Zhao, Jianhua Zou, Huibin Li, Emmanuel Dellandr ´ea, Ioannis A. Kakadiaris, and Liming Chen. Automatic 2.5-D facial landmarking and emotion annotation for social inter- action assistance.IEEE Transactions on Cybernetics, 46(9): 2042–2055, 2016. 2

  58. [66]

    Intensity enhancement via GAN for mul- timodal facial expression recognition

    Kangkang Zhu, Yunhong Wang, Hongyu Yang, Di Huang, and Liming Chen. Intensity enhancement via GAN for mul- timodal facial expression recognition. InIEEE International Conference on Image Processing, 2020. 2

    2020

  59. [67]

    Towards metrical reconstruction of human faces

    Wojciech Zielonka, Timo Bolkart, and Justus Thies. Towards metrical reconstruction of human faces. InEuropean Con- ference on Computer Vision, 2022. 3, 5, 2 11 FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision Supplementar...

    2022

  60. [68]

    Implementation Details Datasets.The proposed method is trained and evaluated on six publicly available benchmark datasets: LRW [7], CelebA [35], FFHQ [25], MEAD [55], AffectNet [40], and BP4D [63].LRWcontains over 500,000 short video clips(each video has 29 frames, totaling 1....

  61. [69]

    Scan Pre-processing.BP4D scans contain hair regions and various noisy surface points, while the FLAME model does not include vertices corresponding to hair

    BP4D FLAME Fitting and Data Split We fit each frame of the BP4D dataset to the FLAME head model to obtain compatible pose, shape, and expression pa- rameters, as well as mesh models as ground truth that will be used for model training and evaluation. Scan Pre-processing.BP4D s...

  62. [70]

    Additional Experiments 8.1. 3D Reconstruction Performance Quantitative Evaluation.We assess reconstruction with three 3D metrics.Keypoint Error: the mean Euclidean distance (mm; lower is better) between 49 FLAME key- points (barycentric on the predicted mesh) and BP4D refer- e...

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.