Pith. sign in

REVIEW 5 major objections 4 minor 54 references

SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting

T0 review · 5 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that representing facial expressions semantically, rather than as vertex displacements, makes monocular capture more accurate across views and preserves identity during retargeting.

desk verdict Serious and mostly well-executed advance in semantic facial expression representation; the unaddressed identity-overlap risk in the MultiREX evaluation is the one thing that should decide whether the SOTA claim stands. read the letter →

arxiv 2412.14371 v3 pith:TKP5DIZD submitted 2024-12-18 cs.CV cs.GRcs.LG

Arthur Josi , Luiz Gustavo Hafemann , Abdallah Dib , Emeline Got , Rafael M. O. Cruz , Marc-Andre Carbonneau This is my paper · ORCID
classification cs.CVcs.GRcs.LG
keywords semanticexpressionrepresentationfacialperformancecaptureretargeting3DmorphablemodelsmonocularfacereconstructiondomainadaptationsyntheticdataMultiREXbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a known flaw of 3D morphable models: their expression coefficients produce identical vertex displacements on any face, so the same expression on different identities gets different codes and identity leaks into retargeted results. SEREP replaces this with a semantic expression representation, a latent code trained to mean the same expression regardless of identity, decoded into identity-specific geometry by a network conditioned on the target face. The authors learn this space from unpaired 3D scans with no FACS, emotion, or paired-expression labels, then use it to generate simple synthetic renders that train a monocular capture model together with real in-the-wild images through domain-adversarial learning. On the new MultiREX benchmark, SEREP reports a lower average per-vertex error (2.19 mm) than EMICA (2.36), DECA (2.59), and SMIRK (2.74), and higher identity-preservation during retargeting than FLAME (CSIM 0.791 vs. 0.766). The paper also releases MultiREX as a public benchmark for geometric expression evaluation.

What carries the argument

The load-bearing object is a semantic expression space learned by a mesh autoencoding triplet: an identity encoder $E_{id}$, an expression encoder $E_{exp}$, and a shared decoder $D_{mesh}$. The expression code $z_{exp}$ is aligned across identities by a cycle-consistency loss $L_{cycle} = \|z_{exp} - E_{exp}(D_{mesh}(E_{id}(N_t), z_{exp}))\|^2$, which requires retargeting the same code to another neutral mesh to yield a mesh from which the same code is recovered; a delta loss $L_{delta} = \|(\hat{E}_t - N_t) - (E_s - N_s)\|^2$ keeps the decoder from collapsing toward neutral. For image capture, a ConvNeXt-B encoder predicts the expression code from a single image, with a landmark head shared across real and synthetic images and a gradient-reversal domain classifier pulling the two feature distributions together. The same decoded mesh is used for both reconstruction and retargeting, so the representation does the work of separating what is said from who says it.

What would settle it

Gather a held-out set of identities who each perform the same instructed expression (for example, a closed-mouth smile) in front of a camera, run SEREP on those frames, and compare the distances between their expression codes with distances between codes for different expressions. If the same-expression codes across unseen identities are not substantially closer to each other than to codes of other expressions, the shared-semantic-code assumption is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that expression should be encoded as a semantic code shared across identities rather than as additive vertex displacements. Starting from unpaired 3D expressive meshes, SEREP learns encoders that map any mesh to an identity code and an expression code, and a decoder that produces identity-specific deformations from the pair. The cycle-consistency and delta losses make the same expression code recover the same semantic expression on a different neutral face. From this representation the method renders simple synthetic images with known expression codes and trains a monocular encoder with a landmark head and a gradient-reversal domain classifier, so the model can capture expressions from real in-the-wild images. On the introduced MultiREX benchmark SEREP reports average per-vertex error of 2.19 mm, lower than EMICA's 2.36, DECA's 2.59, and SMIRK's 2.74, and retargeting CSIM of 0.791 versus FLAME's 0.766.

Load-bearing premise

The claim depends on a single learned expression code genuinely meaning the same expression on every identity, with the cycle-consistency and delta losses enforcing that shared semantics without paired data or labels; if the code only aligns on training identities but drifts on unseen faces, the capture and retargeting improvements are not guaranteed.

Editorial extensions

If this is right

  • Expression capture training can rely on cheap synthetic renders plus unlabeled real images, avoiding expensive photo-real synthetic datasets and differentiable-rendering supervision.
  • Retargeting a captured performance to a new character should better preserve the target actor's identity, because the code is semantic rather than additive vertex offsets.
  • Side-view and profile capture should be more stable, since the expression is represented as a whole in 3D space rather than reconstructed from image-space cues that disappear at steep angles.
  • Larger unpaired 3D expression corpora become usable for training, since subjects no longer need to perform identical posed expressions for paired supervision.
  • MultiREX gives the field a geometric, region-based, multi-view protocol for comparing expression capture methods directly in 3D.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same semantic code should make video capture temporally smoother, because the representation is viewpoint-independent; one could test this by measuring frame-to-frame code jitter on MultiREX sequences.
  • Beyond the paper: because the code is learned without labels, its axes may not match FACS action units or emotion categories; probing which dimensions drive which visible motions would test how semantically interpretable the space is.
  • Beyond the paper: the domain-adversarial loss creates an explicit trade-off between lightstage accuracy and in-the-wild robustness; sweeping the gradient-reversal scale on MultiREX would quantify that trade-off.
  • Beyond the paper: the synthetic pipeline is deliberately minimal (random texture, environment map, no hair or clothing), so the method suggests a recipe for generating training data from any semantic face model; adding more diverse synthetic identities should improve capture further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes SEREP, a three-stage pipeline for monocular facial expression capture and retargeting. First, a mesh-based autoencoder learns a 64-dimensional expression code and an identity code from unpaired 3D expressive scans, using reconstruction, cycle-consistency, edge, eye-closure, and delta losses (Eq. 1). Second, this model is used to generate synthetic rendered training images with known expression codes (Sec. 3.2). Third, an image encoder is trained on synthetic and real images with a landmark loss and a gradient-reversal domain-adversarial loss to regress the expression code (Eq. 2). The paper also introduces MultiREX, a new 3D geometric benchmark derived from Multiface with eight identities in five viewpoints, and reports that SEREP achieves lower per-vertex error than DECA, EMICA, and SMIRK on this benchmark (Tables 2 and 3) and higher identity-preservation CSIM than FLAME in retargeting (Table 1).

Significance. If the reported results are reliable, the paper makes three useful contributions: a nonlinear, identity-conditioned expression representation that could reduce identity leakage compared to linear 3DMMs; a low-cost synthetic-data generation and domain-adaptation recipe that avoids hand-crafted assets; and a publicly released geometric benchmark for expression capture. The benchmark release and the reproducibility-oriented materials (code, assets, conversion scripts) are valuable to the community. However, the support for the central claims currently depends on two unverified assumptions: that the evaluation identities are disjoint from the identity data used to train the semantic model, and that the predicted 13k-vertex mesh can be fairly evaluated after a conversion that is not described. The paper does not ship machine-checked proofs, but it does provide code and a benchmark, which are strengths.

major comments (5)
  1. [Sec. 5.1 and 5.3] The MultiREX evaluation protocol uses meshes in the FLAME/Multiface topology, while Dmesh outputs meshes in the 13k-vertex topology used to train the semantic model (Sec. 4.1). No step is described for converting SEREP's predicted meshes to FLAME topology before applying the per-subject FLAME-to-Multiface transform. Without this conversion, the per-vertex errors in Tables 2 and 3 are not reproducible, and the comparison to FLAME-based baselines (which directly output FLAME meshes) is not well-defined. Please specify the conversion procedure (e.g., nearest-neighbor mapping, learned regression, or direct evaluation in SEREP topology after mapping ground truth) and report its effect on the metrics.
  2. [Sec. 4.1 and 5.1] The dataset used to train the semantic model is never named. The model is trained on 865 neutral scans and 10 dynamic subjects (Sec. 4.1), and MultiREX is built from 8 Multiface identities (Sec. 5.1). If the 10 dynamic subjects include any of the 8 MultiREX identities, or if any of the 865 neutral scans correspond to those identities, then the semantic encoder/decoder and the synthetic data generation (Sec. 3.2) have already seen the evaluation identity geometry. In that case, the capture model could exploit identity-specific memory rather than a generic semantic expression code. Please name the dataset, report whether the MultiREX identities are disjoint from all identities used in the semantic model and in synthetic data generation, and, if they are not disjoint, re-run the MultiREX experiment with held-out identities.
  3. [Sec. 4.2] The reconstruction experiment optimizes expression codes per test mesh to minimize Lrec. This measures the representational capacity of the decoder, not the ability of Eexp to produce codes for new identities. The sentence "our expression model is better able to capture the geometric deformations caused by expressions on unseen subjects" conflates fitting power with generalization. Please clarify the claim, and if generalization is intended, evaluate with a protocol that does not optimize over the test mesh (e.g., infer codes with Eexp, or use a per-subject calibration expression to factor out identity-specific fitting gains).
  4. [Sec. 3.1, Eq. (1)] The claim that z_exp encodes the same semantic expression across identities is an assumption. The cycle-consistency loss and delta loss are soft regularizers; they do not by themselves guarantee that the code space is identity-invariant in the intended way. Please provide a direct test: on the 56-subject/19-expression test set, check whether the optimized codes for the same expression index cluster together across identities (e.g., via nearest-neighbor retrieval or a classifier trained on z_exp to predict identity and expression), and report the degree of identity dependence in the codes. This is load-bearing for the retargeting claims.
  5. [Sec. 5.3 and 5.4] The in-the-wild robustness claim is supported only by qualitative examples (Fig. 8 and Sec. 5.4) and by the acknowledged trade-off that the full model (with domain loss Ld) underperforms the variant without Ld on MultiREX (Table 2: Ours 2.19 mm vs. Ours (no Ld) 2.10 mm). To substantiate the statement that Ld is necessary for generalization to in-the-wild conditions, please provide a quantitative in-the-wild evaluation, for example landmark alignment error against a detector on a standard test set such as WFLW's test split, or an expression classification metric. Without such a metric, the benefit of the proposed full model over its ablated variant is not measured.
minor comments (4)
  1. [Sec. 3.3] Typo: "sythetic" should be "synthetic" in the description of the domain classifier.
  2. [Sec. 4.3] The text reports 61,600 retargeted meshes, but with 56 subjects and 19 fixed expressions (Sec. 4.1) the number should be 58,520 (56 x 55 x 19). Please clarify whether the neutral expression is also included or whether another count is used.
  3. [Table 1] The caption states bold results are statistically significant via a Wilcoxon test, but no values are bolded in the table as presented in the provided text. Please ensure the final PDF marks significance consistently, and define the number of paired samples used for the test.
  4. [Sec. 5.2] The expression encoder is trained with synthetic images that include ground truth codes, but the decoder Dmesh is not fine-tuned after synthetic data generation. A short discussion of how decoder imperfections affect the synthetic captions would help readers assess the self-consistency of the pipeline.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SEREP's self-generated synthetic captions are a training device, while the headline claims are evaluated on external MultiREX geometry and held-out identity metrics.

full rationale

The claimed derivation chain is: (i) train a semantic expression model on 3D meshes with reconstruction, cycle-consistency, edge, eye-closure, and delta losses; (ii) freeze it and render synthetic images with known latent codes; (iii) train the capture model with Lcode on those synthetic codes plus landmark and domain-adversarial losses; (iv) evaluate end-to-end capture on MultiREX, a benchmark built from Multiface ground-truth meshes not used in training, and retargeting on held-out identities. The only self-referential element is that the capture model's training target (zexp) is produced by the authors' own semantic model; however, the paper's quantitative claims are not validated against those self-generated labels. MultiREX per-vertex errors (Tabs. 2-3) compare predicted meshes to external ground-truth geometry, and the CSIM retargeting metric (Tab. 1) is computed on held-out test identities against an off-the-shelf identity embedding, with FLAME as a comparable fitted baseline. The self-citation [12] is used only as an off-the-shelf neutral-mesh estimator for in-the-wild demos and does not support the MultiREX or held-out quantitative results. The absence of an explicit identity-disjoint split between the unnamed 865-neutral/10-dynamic training corpus and the 8 MultiREX identities is a generalization-validity concern, but the paper provides no quotation or equation showing that any reported number is the same quantity as a training input or a fitted parameter. No step in the derivation reduces to its own input by construction.

Assumptions & free parameters 8 free parameters · 4 assumptions · 1 invented entities

The central empirical claims depend on a large set of hand-chosen loss weights and architecture capacities. The most consequential is lambda_domain, which trades off benchmark accuracy against in-the-wild generalization. No free parameters are fit to the external MultiREX geometry; the benchmark is used only for evaluation, so the main circularity risk comes from the self-defined code space and the fitting-based reconstruction evaluation, not from parameter fitting.

free parameters (8)
  • Lambda delta (Lret) = 0.01
    Hand-picked weight in Eq. 1 (App. A); ablation (Tab. 1) shows removing delta regularization collapses the decoder to neutral meshes.
  • Lambda edge (Lret) = 10000
    Edge-preservation regularizer weight (App. A); chosen to smooth decoded meshes (Fig. 10).
  • Lambda eyes (Lret) = 0.01
    Eye-closure loss weight (App. A); fixes incomplete blinks.
  • Lambda code (Lcap) = 10
    Weight on expression-code regression for synthetic data in Eq. 2 (App. A).
  • Lambda lmks (Lcap) = 1
    Landmark loss weight in Eq. 2 (App. A).
  • Lambda domain (Lcap) = 0.005
    Gradient-reversal domain loss weight in Eq. 2 (App. A); directly controls the MultiREX/in-the-wild trade-off (Tab. 2, Fig. 8).
  • Gradient reversal scale = 1
    Scale applied to reversed gradients (App. A).
  • Expression code dimensionality = 64
    Dimension of z_exp; set by architecture (Tabs. 4-5), defines the capacity of the semantic space.
assumptions (4)
  • domain assumption A single shared mesh topology with vertex correspondence is a valid substrate for transferring expressions across identities.
    Sec. 4.1 states all meshes share the same topology with 13k vertices; Dmesh decodes into this topology for any identity. If vertex correspondence is not semantically aligned, retargeting reduces to copying displacements on misaligned vertices.
  • domain assumption Cycle consistency and delta losses, without paired data, are sufficient to make z_exp capture the same semantic expression across different identities.
    Sec. 3.1 uses Lcycle and Ldelta as the only semantic-alignment signals; no FACS, emotion labels, or paired identity-expression data are used. This is the load-bearing representational assumption.
  • domain assumption Simple synthetic renders of bare meshes (no hair, clothes, accessories; identical teeth and eye assets) plus gradient reversal close the synthetic-to-real domain gap well enough for in-the-wild capture.
    Sec. 3.2/3.3 and App. F describe deliberately minimal synthetic data; the only domain-bridging mechanism is the adversarial Ld. The ablation shows Ld hurts MultiREX accuracy, so the net benefit rests on qualitative in-the-wild examples.
  • domain assumption The manually selected neutral frame and Wrap 3D retopology provide ground-truth identity meshes accurate enough for millimeter-level evaluation.
    MultiREX ground truth depends on a manual neutral frame per subject and commercial retopology (Sec. 5.1, App. B). Any bias in this step is shared by all methods but limits the benchmark's absolute accuracy claims.
invented entities (1)
  • Semantic expression code z_exp (64-dimensional) independent evidence
    purpose: Latent representation intended to capture facial expression independently of identity, decoded by Dmesh into identity-specific vertex displacements.
    The paper provides falsifiable handles through the released MultiREX benchmark and qualitative retargeting, so others can test whether the code transfers expressions across identities. However, the code space is defined entirely by the authors' model and no external expression labels validate it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting." pith.science (2026). https://pith.science/paper/TKP5DIZD

@misc{pith2026241214371,
  author       = {Pith},
  title        = {Pith review of: SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TKP5DIZD}},
  note         = {Machine review of arXiv:2412.14371}
}
read the original abstract

Monocular facial performance capture in-the-wild is challenging due to varied capture conditions, face shapes, and expressions. Most current methods rely on linear 3D Morphable Models, which represent facial expressions independently of identity at the vertex displacement level. We propose SEREP (Semantic Expression Representation), a model that disentangles expression from identity at the semantic level. We start by learning an expression representation from high-quality 3D data of unpaired facial expressions. Then, we train a model to predict expression from monocular images relying on a novel semi-supervised scheme using low quality synthetic data. In addition, we introduce MultiREX, a benchmark addressing the lack of evaluation resources for the expression capture task. Our experiments show that SEREP outperforms state-of-the-art methods, capturing challenging expressions and transferring them to new identities.

Figures

Figures reproduced from arXiv: 2412.14371 by the authors.

Figure 1
Figure 1. SEREP extracts facial expressions from in-the-wild monocular images and applies them to a given neutral mesh, enabling [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We first train a semantic expression model to disentangle expression from identity on 3D meshes. Next, we generate a synthetic [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Expression capture model training. Given a synthetic [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Ablations on the semantic expression model. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Facial expression capture examples on MultiREX. Left: [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Impact of the domain loss Ld on in-the-wild capture. quences in our supplementary video. Ld introduces a trade￾off between performance in a controlled lightstage environ￾ment and generalization to in-the-wild data. This trade-off is difficult to quantify, given that be…
Figure 7
Figure 7. Figure 7: In-the-wild expression retargeting results. Expressions [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: The 5 camera views used in MultiREX (left-to-right), [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Ablation on Leyes and Ledge [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Visual representation of the forehead, nose, mouth, and [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 15
Figure 15. Figure 15: Additional retargeting on in-the-wild images [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 14
Figure 14. Figure 14: Additional captures on the proposed MultiREX bench [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 17
Figure 17. Figure 17: Random synthetic data samples. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 47 canonical work pages

  1. [1]

    SPARK: Self-supervised Personalized Real-time Monocular Face Capture

    Kelian Baert, Shrisha Bharadwaj, Fabien Castan, Benoit Maujean, Marc Christie, Victoria Abrevaya, and Adnane Boukhayma. Spark: Self-supervised personalized real-time monocular face capture. arXiv preprint arXiv:2409.07984 ,

    work page Pith review arXiv

  2. [2]

    Timo Bolkart, Tianye Li, and Michael J. Black. Instant multi-view head capture through learnable registration. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 768–779, 2023. 4, 12

    work page 2023

  3. [3]

    Neural 3d mor- phable models: Spiral convolutional networks for 3d shape representation learning and generation

    Giorgos Bouritsas, Sergiy Bokhnyak, Stylianos Ploumpis, Michael Bronstein, and Stefanos Zafeiriou. Neural 3d mor- phable models: Spiral convolutional networks for 3d shape representation learning and generation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7213–7222, 2019. 4

    work page 2019

  4. [4]

    Facewarehouse: A 3d facial expression database for visual computing

    Chen Cao, Yanlin Weng, Shun Zhou, Yiying Tong, and Kun Zhou. Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics, 20(3):413–425, 2013. 3

    work page 2013

  5. [5]

    Realy: Rethinking the evaluation of 3d face reconstruction

    Zenghao Chai, Haoxian Zhang, Jing Ren, Di Kang, Zhengzhuo Xu, Xuefei Zhe, Chun Yuan, and Linchao Bao. Realy: Rethinking the evaluation of 3d face reconstruction. In European conference on computer vision , pages 74–92. Springer, 2022. 3, 6, 12

    work page 2022

  6. [6]

    Hiface: High-fidelity 3d face recon- struction by learning static and dynamic details

    Zenghao Chai, Tianke Zhang, Tianyu He, Xu Tan, Tadas Baltrusaitis, HsiangTao Wu, Runnan Li, Sheng Zhao, Chun Yuan, and Jiang Bian. Hiface: High-fidelity 3d face recon- struction by learning static and dynamic details. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 9087–9098, 2023. 2, 8

    work page 2023

  7. [7]

    Local anatomically-constrained facial perfor- mance retargeting

    Prashanth Chandran, Lo ¨ıc Ciccone, Markus Gross, and Derek Bradley. Local anatomically-constrained facial perfor- mance retargeting. ACM Transactions on Graphics (TOG), 41(4):1–14, 2022. 2

    work page 2022

  8. [8]

    Facial animation with disen- tangled identity and motion using transformers

    Prashanth Chandran, Gaspard Zoss, Markus Gross, Paulo Gotardo, and Derek Bradley. Facial animation with disen- tangled identity and motion using transformers. InComputer Graphics Forum, volume 41, pages 267–277. Wiley Online Library, 2022. 3

    work page 2022

Show all 54 references
  1. [9]

    Emoca: Emotion driven monocular face capture and animation

    Radek Dan ˇeˇcek, Michael J Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20311–20322, 2022. 1, 2, 3, 7

    2022

  2. [10]

    Emotional speech- driven animation with content-emotion disentanglement

    Radek Dan ˇeˇcek, Kiran Chhatre, Shashank Tripathi, Yandong Wen, Michael Black, and Timo Bolkart. Emotional speech- driven animation with content-emotion disentanglement. In SIGGRAPH Asia 2023 Conference Papers , SA ’23, New York, NY , USA, 2023. Association for Computing Mach...

    2023

  3. [11]

    Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set

    Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition workshops, pages 0–0,...

    2019

  4. [12]

    Mosar: Monocular semi-supervised model for avatar reconstruction using differentiable shading

    Abdallah Dib, Luiz Gustavo Hafemann, Emeline Got, Trevor Anderson, Amin Fadaeinejad, Rafael MO Cruz, and Marc- Andr´e Carbonneau. Mosar: Monocular semi-supervised model for avatar reconstruction using differentiable shading. In Proceedings of the IEEE/CVF Conference on Compute...

    2024

  5. [13]

    Megaportraits: One-shot megapixel neural head avatars

    Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Alek- sei Ivakhnenko, Victor Lempitsky, and Egor Zakharov. Megaportraits: One-shot megapixel neural head avatars. In Proceedings of the 30th ACM International Conference on Multimedia, pages 2663–2671, 2022. 3, 5

    2022

  6. [14]

    3d morphable face models-past, present, and future

    Bernhard Egger, William AP Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, et al. 3d morphable face models-past, present, and future. ACM Transactions on Graphics (TOG), 39(5):1–38, 2020. 2

    2020

  7. [15]

    Learning an animatable detailed 3d face model from in- the-wild images

    Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in- the-wild images. ACM Transactions on Graphics (ToG) , 40(4):1–13, 2021. 1, 2, 3, 7

    2021

  8. [16]

    Black, and Timo Bolkart

    Yao Feng, Haiwen Feng, Michael J. Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in-the-wild images. ACM Trans. Graph., 40(4), July 2021. 7, 9

    2021

  9. [17]

    Facelift: Semi-supervised 3d facial landmark localization

    David Ferman, Pablo Garrido, and Gaurav Bharaj. Facelift: Semi-supervised 3d facial landmark localization. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1781–1791, June

  10. [18]

    Spectre: Visual speech-informed per- ceptual 3d facial expression reconstruction from videos

    Panagiotis P Filntisis, George Retsinas, Foivos Paraperas- Papantoniou, Athanasios Katsamanis, Anastasios Roussos, and Petros Maragos. Spectre: Visual speech-informed per- ceptual 3d facial expression reconstruction from videos. In Proceedings of the IEEE/CVF Conference on Com...

    2023

  11. [19]

    Cycles renderer

    Blender Foundation. Cycles renderer. https://www. cycles- renderer.org/, 2021. Accessed: 2024-11-

    2021

  12. [20]

    Unsupervised domain adaptation by backpropagation

    Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180–1189. PMLR, 2015. 5

    2015

  13. [21]

    Learning neural parametric head models

    Simon Giebenhain, Tobias Kirschstein, Markos Georgopou- los, Martin R ¨unz, Lourdes Agapito, and Matthias Nießner. Learning neural parametric head models. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR),

  14. [22]

    Mononphm: Dynamic head reconstruction from monocu- lar videos

    Simon Giebenhain, Tobias Kirschstein, Markos Georgopou- los, Martin R ¨unz, Lourdes Agapito, and Matthias Nießner. Mononphm: Dynamic head reconstruction from monocu- lar videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10747– ...

    2024

  15. [23]

    Identity mappings in deep residual networks

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Computer Vision–ECCV 2016: 14th European Conference, Amster- dam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 630–645. Springer, 2016. 4

    2016

  16. [24]

    Look ma, no markers: holistic performance capture without the hassle

    Charlie Hewitt, Fatemeh Saleh, Sadegh Aliakbarian, Lohit Petikam, Shideh Rezaeifar, Louis Florentin, Zafiirah Hose- nie, Thomas J Cashman, Julien Valentin, Darren Cosker, et al. Look ma, no markers: holistic performance capture without the hassle. arXiv preprint arXiv:2410.115...

    arXiv

  17. [25]

    Dual- generator face reenactment

    Gee-Sern Hsu, Chun-Hung Tsai, and Hung-Yi Wu. Dual- generator face reenactment. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 642–650, 2022. 5

    2022

  18. [26]

    A 3D Face Model for Pose and Illumination Invariant Face Recognition, Genova, Italy, 2009

    IEEE. A 3D Face Model for Pose and Illumination Invariant Face Recognition, Genova, Italy, 2009. 1, 2, 8

    2009

  19. [27]

    Differentiable rendering: A survey

    Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. Differentiable rendering: A survey. arXiv preprint arXiv:2006.12057, 2020. 3

    2006 arXiv

  20. [28]

    Adam: A method for stochastic optimization

    Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR) , San Diega, CA, USA,

  21. [29]

    A hierarchical representation network for accurate and detailed face reconstruction from in-the-wild images

    Biwen Lei, Jianqiang Ren, Mengyang Feng, Miaomiao Cui, and Xuansong Xie. A hierarchical representation network for accurate and detailed face reconstruction from in-the-wild images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 394–403,

  22. [30]

    Learning forma- tion of physically-based face attributes

    Ruilong Li, Karl Bladin, Yajie Zhao, Chinmay Chinara, Owen Ingraham, Pengda Xiang, Xinglei Ren, Pratusha Prasad, Bipin Kishore, Jun Xing, et al. Learning forma- tion of physically-based face attributes. In Proceedings of the IEEE/CVF conference on computer vision and pattern r...

    2020

  23. [31]

    Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and ex- pression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):194:1–194:17, 2017. 1, 2, 5

    2017

  24. [32]

    Teaser: Token enhanced spatial modeling for ex- pressions reconstruction

    Yunfei Liu, Lei Zhu, Lijian Lin, Ye Zhu, Ailing Zhang, and Yu Li. Teaser: Token enhanced spatial modeling for ex- pressions reconstruction. arXiv preprint arXiv:2502.10982,

    arXiv

  25. [33]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 11976–11986,

  26. [34]

    Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J. Black. Generating 3D faces using convolutional mesh autoencoders. In European Conference on Computer Vision (ECCV), pages 725–741, 2018. 4

    2018

  27. [35]

    Hr-net: a landmark based high realistic face reenactment network

    Qiuyu Ren, Zhiying Lu, Haopeng Wu, Jianfeng Zhang, and Zijian Dong. Hr-net: a landmark based high realistic face reenactment network. IEEE Transactions on Circuits and Systems for Video Technology, 33(11):6347–6359, 2023. 5

    2023

  28. [36]

    3d facial expressions through analysis-by- neural-synthesis

    George Retsinas, Panagiotis P Filntisis, Radek Danecek, Victoria F Abrevaya, Anastasios Roussos, Timo Bolkart, and Petros Maragos. 3d facial expressions through analysis-by- neural-synthesis. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognitio...

    2024

  29. [37]

    Unsupervised performance analysis of 3d face align- ment with a statistically robust confidence test

    Mostafa Sadeghi, Xavier Alameda-Pineda, and Radu Ho- raud. Unsupervised performance analysis of 3d face align- ment with a statistically robust confidence test. Neurocom- puting, 564:126941, 2024. 2

    2024

  30. [38]

    Learning to regress 3D face shape and expression from an image without 3D supervision

    Soubhik Sanyal, Timo Bolkart, Haiwen Feng, and Michael Black. Learning to regress 3D face shape and expression from an image without 3D supervision. InProceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 7763–7772, June 2019. 3

    2019

  31. [39]

    State of the art on neural rendering

    Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin- Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, et al. State of the art on neural rendering. In Computer Graphics Forum, volume 39, pages 701–727. Wiley Online...

    2020

  32. [40]

    Face transfer with multilinear models

    Daniel Vlasic, Matthew Brand, Hanspeter Pfister, and Jovan Popovic. Face transfer with multilinear models. In ACM SIGGRAPH 2006 Courses, pages 24–es. 2006. 2

    2006

  33. [41]

    3d face reconstruction with the geometric guidance of facial part segmentation

    Zidu Wang, Xiangyu Zhu, Tianshuo Zhang, Baiqin Wang, and Zhen Lei. 3d face reconstruction with the geometric guidance of facial part segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1672–1682, 2024. 1, 2, 3, 7, 9

    2024

  34. [42]

    Fake it till you make it: face analysis in the wild using synthetic data alone

    Erroll Wood, Tadas Baltru ˇsaitis, Charlie Hewitt, Sebastian Dziadzio, Thomas J Cashman, and Jamie Shotton. Fake it till you make it: face analysis in the wild using synthetic data alone. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3681–36...

    2021

  35. [43]

    3d face reconstruction with dense landmarks

    Erroll Wood, Tadas Baltru ˇsaitis, Charlie Hewitt, Matthew Johnson, Jingjing Shen, Nikola Milosavljevi´c, Daniel Wilde, Stephan Garbin, Toby Sharp, Ivan Stojiljkovi´c, et al. 3d face reconstruction with dense landmarks. In European Confer- ence on Computer Vision , pages 160–1...

    2022

  36. [44]

    3d face reconstruction with dense landmarks

    Erroll Wood, Tadas Baltru ˇsaitis, Charlie Hewitt, Matthew Johnson, Jingjing Shen, Nikola Milosavljevi´c, Daniel Wilde, 10 Stephan Garbin, Toby Sharp, Ivan Stojiljkovi´c, et al. 3d face reconstruction with dense landmarks. In European Confer- ence on Computer Vision , pages 16...

    2022

  37. [45]

    Look at boundary: A boundary-aware face alignment algorithm

    Wayne Wu, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, and Qiang Zhou. Look at boundary: A boundary-aware face alignment algorithm. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2129– 2138, 2018. 6

    2018

  38. [46]

    Group normalization

    Yuxin Wu and Kaiming He. Group normalization. In Pro- ceedings of the European conference on computer vision (ECCV), pages 3–19, 2018. 4

    2018

  39. [47]

    Multiface: A dataset for neural face rendering

    Cheng-hsin Wuu, Ningyuan Zheng, Scott Ardisson, Rohan Bali, Danielle Belko, Eric Brockmeyer, Lucas Evans, Tim- othy Godisart, Hyowon Ha, Xuhua Huang, Alexander Hy- pes, Taylor Koska, Steven Krenn, Stephen Lombardi, Xi- aomin Luo, Kevyn McPhail, Laura Millerschoen, Michal Perdo...

    2022

  40. [48]

    Vasa-1: Lifelike audio-driven talking faces generated in real time

    Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, and Baining Guo. Vasa-1: Lifelike audio-driven talking faces generated in real time. arXiv preprint arXiv:2404.10667 ,

    arXiv

  41. [49]

    Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face pre- diction

    Haotian Yang, Hao Zhu, Yanru Wang, Mingkai Huang, Qiu Shen, Ruigang Yang, and Xun Cao. Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face pre- diction. In Proceedings of the ieee/cvf conference on com- puter vision and pattern recognition , pag...

    2020

  42. [50]

    Media2face: Co-speech facial animation gen- eration with multi-modality guidance

    Qingcheng Zhao, Pengyu Long, Qixuan Zhang, Dafei Qin, Han Liang, Longwen Zhang, Yingliang Zhang, Jingyi Yu, and Lan Xu. Media2face: Co-speech facial animation gen- eration with multi-modality guidance. In ACM SIGGRAPH 2024 Conference Papers, pages 1–13, 2024. 3

    2024

  43. [51]

    Tinghui Zhou, Philipp Kr ¨ahenb¨uhl, Mathieu Aubry, Qixing Huang, and Alexei A. Efros. Learning dense correspondence via 3d-guided cycle consistency. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 117–126, 2016. 4

    2016

  44. [52]

    Star loss: Reducing semantic am- biguity in facial landmark detection

    Zhenglin Zhou, Huaxia Li, Hong Liu, Nanyang Wang, Gang Yu, and Rongrong Ji. Star loss: Reducing semantic am- biguity in facial landmark detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15475–15484, 2023. 6

    2023

  45. [53]

    Celebv- hq: A large-scale video facial attributes dataset

    Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. Celebv- hq: A large-scale video facial attributes dataset. InEuropean conference on computer vision , pages 650–667. Springer,

  46. [54]

    Towards metrical reconstruction of human faces

    Wojciech Zielonka, Timo Bolkart, and Justus Thies. Towards metrical reconstruction of human faces. In European confer- ence on computer vision, pages 250–269. Springer, 2022. 2, 3 11 Appendices A. Implementation details For training the semantic expression model, we use the fo...

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.