Pith. sign in

REVIEW 4 major objections 5 minor 4 references

EVA: Expressive Virtual Avatars from Multi-view Videos

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-layer avatar model separates facial expressions from body and hand control, and renders the result in real time.

desk verdict Solid real-time full-body avatar system with a genuinely disentangled face/body Gaussian model, but the expressive-control claim hinges on an unvalidated closed-loop expression tracker. read the letter →

arxiv 2505.15385 v1 pith:5NTMPFXF submitted 2025-05-21 cs.CV cs.GR

Hendrik Junkawitsch , Guoxing Sun , Heming Zhu , Christian Theobalt , Marc Habermann This is my paper · ORCID
classification cs.CVcs.GR
keywords expressivevirtualavatars3DGaussiansplattingmulti-viewreconstructionhumanavataranimationfacialexpressiontrackingdisentangledbody-facecontrolreal-timerenderingdeformabletemplate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EVA is an actor-specific avatar system that takes multi-view video of a person and produces a drivable full-body model in which body motion, hand gestures, and facial expressions are controlled independently. The paper argues that this independence is what existing full-body avatar methods lack: they either tie facial appearance to body pose or require image-based facial drivers, so they cannot faithfully re-enact new expressions on a moving body. EVA's answer is to split the avatar into an expressive deformable template mesh and a decoupled 3D Gaussian appearance layer, with separate networks predicting body and face appearance. The authors report that this design beats the DDC and ASH baselines on perceptual quality (lower LPIPS) in novel-view, novel-pose, and novel-expression settings, while rendering at over 30 frames per second.

What carries the argument

The load-bearing object is the expressive template mesh $\boldsymbol{\Phi}_{\mathrm{mesh}}(\boldsymbol{\theta}^{\triangleright}, \boldsymbol{\psi}) = C(\boldsymbol{\theta}^{\triangleright}) \bowtie H(\boldsymbol{\psi})$, a watertight union of a learned deformable body character $C$ and a personalized FLAME head $H$, stitched at the neck. This mesh is the shared geometric proxy for everything downstream: it is tracked to multi-view video via 2D facial landmarks and Chamfer losses, and it is rendered into UV-space normal and position maps that feed the two disentangled Gaussian predictors. The second central object is the appearance function $\Phi_{\mathrm{app}} = \Phi^{\mathrm{body}}_{\mathrm{app}}(C^{\triangleright}) \cup \Phi^{\mathrm{head}}_{\mathrm{app}}(H^{\triangleright}(\boldsymbol{\psi}), \hat{\boldsymbol{\vartheta}}^{\triangleright})$, which emits body Gaussians and face Gaussians separately so that pose and expression can be edited without mutual interference.

What would settle it

Render the same trained avatar from the same multi-view test frames while using FLAME parameters obtained from an independent, high-accuracy face measurement (for example, a calibrated facial capture rig or manually annotated landmarks) and compare the rendered face region to the recorded images. If the appearance network's outputs degrade when the independently measured parameters replace the paper's tracked ones, that indicates the learned face mapping has absorbed errors from the closed tracking loop rather than truly grounding expression control.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that photorealistic, real-time avatars with full expression control are achievable by disentangling geometry and appearance along body/face lines rather than by improving a single unified model. Concretely, a FLAME-based personalized head is stitched onto a learned deformable character model, so one template mesh responds to both skeletal motion windows and facial expression parameters. The same split is repeated in appearance: two independent U-Nets predict 3D Gaussian parameters in UV space, one conditioned on body motion and one on facial expression, and their outputs are combined at splatting time. The empirical claim is that this explicit separation yields better rendering quality than DDC and ASH, and the ablations show that removing the separation or feeding body features into the head network hurts quality, indicating the model is learning the intended independent control.

Load-bearing premise

The method assumes that the FLAME expression parameters recovered from 2D landmarks and Chamfer fitting are accurate enough to serve as the true control signal, with no independent face-rig or manual validation to catch systematic tracking errors.

Editorial extensions

If this is right

  • An actor's expression can be changed without retouching the body, because the head and body Gaussian parameters are produced by separate networks.
  • The same avatar can be driven by any source of FLAME expression parameters, including audio-based expression synthesis, rather than requiring front-facing images at inference.
  • Loose clothing remains dynamic: the body appearance network consumes motion windows of the deformed character mesh, so garment motion is not baked into a static template.
  • The reported runtimes put the full pipeline at roughly 35 fps for the appearance layer and 41 fps for the geometry layer, i.e., real-time 2K rendering on the tested hardware.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test the paper does not run: re-train on a subject whose expressions are measured by an independent rig, then compare the recovered FLAME parameters; a mismatch would reveal how much of the apparent expressiveness comes from the appearance network absorbing tracking error.
  • The same disentanglement could be turned into a quantitative controllability metric, measuring how much changing expression parameters changes body-region pixels and vice versa, which the paper reports only qualitatively.
  • Because the dataset combines full-body motion, hand gestures, detailed expressions, and close-up face views, it could serve as a shared benchmark for future expressive-avatar work, though the paper does not frame it that way.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. EVA proposes an actor-specific, real-time animatable avatar system that separates the human representation into an expressive template geometry layer (a DDC-style deformable body model stitched to a personalized FLAME-based head avatar) and a disentangled 3D Gaussian appearance layer in which separate U-Nets predict body and face Gaussian parameters. The method takes skeletal motion and FLAME expression parameters as inputs, aiming for independent control of body, hands, and face. The authors introduce a new multi-view dataset of four subjects with strong facial expressions, evaluate against DDC, ASH, AnimatableGaussians, and a reimplemented GaussianAvatars, and report quantitative improvements in LPIPS/PSNR/SSIM plus qualitative demonstrations of novel-view and novel-expression synthesis.

Significance. If the expressive-control claim is validated, EVA would be a practically significant contribution: it combines a deformable, expressive template with Gaussian splatting in a two-module design, supports direct FLAME-expression driving (enabling audio-driven avatars), and reaches real-time 2K rendering. The paper is unusually thorough in documenting implementation details, including the head-stitching procedure, the optimization losses, and several ablations. The main load-bearing weakness is that the facial expression parameters used both for training targets and for test-time control signals come from the same unvalidated tracking pipeline, so the reported gains do not yet demonstrate faithful, generalizable expression control. The quantitative comparison also lacks variance analysis and is confined to the authors' dataset, which tempers the strength of the 'surpasses state-of-the-art' claim.

major comments (4)
  1. [Sec. 3.2, Sec. D.2, Eq. (3)] The expression parameters psi are recovered from 2D landmarks (MediaPipe/FAN) and Chamfer fitting to per-frame 3D reconstructions, with no independent validation. The same tracking pipeline is used to create the training targets and the test-time control signals, so the 'Novel Pose/Expr.' numbers in Tab. 1 may reflect the model learning the specific noisy distribution of this tracker rather than a generalizable map from psi to appearance. Please add an independent check of psi, for example manual landmark annotation on held-out frames, a high-quality multi-view FLAME fit using the 20 close-up cameras, or a cross-driving experiment where psi comes from an external source (audio model, retargeted motion, or user sliders) and is evaluated for expression fidelity.
  2. [Tab. 1 and Sec. 4.2] The quantitative comparison aggregates metrics over four subjects without error bars, per-subject breakdowns, or significance tests. Several reported differences are small relative to the likely subject-to-subject variation (for instance, DDC actually leads PSNR in the Novel View row, 46.87 vs. 46.39), yet the text concludes that EVA 'surpasses' the baselines. Please report per-subject values and variance; if the advantage is not consistent, soften the claim to match the evidence.
  3. [Sec. 4, Dataset and Evaluation Protocol] All quantitative evaluations are on the authors' new dataset, and the public DynaCap benchmark is excluded because its actors show limited facial expressions. Since the body-pose rendering quality is a central part of the claim, the absence of any public-benchmark body/pose comparison makes it hard to assess generalization. Please either report body-only metrics on DynaCap or a similar public sequence, or explicitly scope the claims to the new dataset.
  4. [Sec. K.3, Tab. 4] The comparison against GaussianAvatars is based on the authors' own reimplementation (the text states the original method cannot faithfully track the full-body motions, so they implemented a version with constant head Gaussians). Because a reimplementation may not reflect the original method's performance, this comparison is not a verifiable substitute for running the published code. Please label this row as an ablation of constant head Gaussians rather than as a direct comparison, or provide a more faithful baseline.
minor comments (5)
  1. [Sec. 4, Implementation and Runtime] The real-time claim is '35 fps on two A40 GPUs with one frame delay'; the abstract says 'over 30 fps' without this qualification. Please state the GPU count and latency explicitly in the abstract and in the runtime paragraph.
  2. [Sec. 3.2 and Sec. F] The eyelid postprocessing relies on empirically chosen hyperparameters zeta = 0.75 and omega (the downscale factor). Please add a sensitivity analysis or a short ablation to show the results are not sensitive to these choices.
  3. [Sec. 4 vs. Sec. H] The main text says the system comprises '100 full-frame cameras' while the supplementary (Sec. H) says 'approximately 120 synced 25 fps cameras'; these numbers should be reconciled.
  4. [Sec. 4.2] The explanation that DDC's higher novel-view PSNR is 'likely due to training on all frames rather than sampled ones' is speculative; if this is the cause, it can be tested by training EVA with a comparable frame schedule, and the result should be reported.
  5. [Throughout] The title and headers repeatedly display 'EV A' with a space; this should be 'EVA' consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EVA's rendering claims are evaluated against held-out views and external baselines; the unvalidated expression tracker is a verification gap, not a self-referential reduction.

full rationale

I traced the claimed derivation chain: expression tracking (Sec. 3.2, Eq. 2, and Sec. D.2) produces psi; Eq. (3) feeds psi into the head Gaussian predictor; Sec. 3.3 trains the predictors with L1/SSIM/IDMRF losses against multi-view images; Sec. 4.2 evaluates on four held-out camera views against ASH, DDC, AG, and GA. No step defines a predicted quantity as a function of the fitted psi. The psi parameters are conditioning inputs, not outputs of the appearance model, and the reported LPIPS/PSNR/SSIM numbers compare rendered images to ground-truth pixels on excluded views. Using the same refined skeletal poses for EVA and the baselines is a controlled comparison, not a circular one. The citations to DDC and ASH are to published, implementable baselines; no load-bearing uniqueness claim is imported from the authors' prior work. The legitimate concern, that tracked psi is never validated against independent ground truth (face mocap, manual annotation, or a close-up multi-view FLAME fit), is a correctness and robustness gap in the 'faithful expression control' claim, not a circular derivation. The paper's own Limitations section identifies different weaknesses (topology, neck seam, cross-region lighting), and none of these reduces the method to its inputs. I therefore find no specific circular step that can be quoted and exhibited.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claims rest on standard parametric models (FLAME, DDC), 2D landmark detectors, 3D Gaussian splatting, and the multi-view capture setup. The only explicit empirical knobs are the eyelid postprocessing parameters and the camera-selection threshold. The paper does not introduce new physical entities; the global position latent is a learned feature, not an entity.

free parameters (3)
  • Eyelid closure threshold zeta = 0.75
    Empirically chosen in Sec. F of the supplement to binarize normalized eye-opening differences; directly affects eyelid tracking and hence expression control.
  • Eyelid opening downscale factor omega = not reported
    Empirically chosen in Sec. F to downweight opening adjustments; affects eyelid parameter updates.
  • Camera selection angle threshold = 70 degrees
    Heuristic in Sec. 3.2 to select views where the face is visible for landmark tracking; affects tracking quality.
assumptions (6)
  • domain assumption FLAME provides a valid parametric model of head shape and expression, and its expression space (gamma, jaw, eyelids) is sufficient for the facial expressions in the dataset.
    Invoked in Sec. 3.1 and used throughout for head avatar fitting and expression tracking.
  • domain assumption The DDC deformation model (Habermann et al. 2021), trained with NeuS2 geometric supervision, accurately captures pose-dependent and clothing deformations including loose garments.
    Invoked in Sec. 3.1 and 3.2; the template layer for body and clothing relies on this.
  • domain assumption 2D landmarks from MediaPipe and FAN are sufficiently accurate, and optimizing against them yields correct FLAME expression parameters.
    Sec. 3.2, Motion Optimization & Expression Tracking; no independent ground-truth validation is provided.
  • domain assumption 3D Gaussian splatting with UV-space parameter prediction and dual quaternion skinning can represent dynamic clothed-human appearance.
    Sec. 3.1 and 3.3; borrowed from ASH/3DGS and assumed to hold.
  • domain assumption The stitching operator produces a watertight, artifact-free connection between the DDC body and FLAME head meshes.
    Sec. E of the supplement; no quantitative evaluation of stitching artifacts is given.
  • domain assumption The newly captured multi-view dataset is representative and the actors' instructed expressions are genuine and varied.
    Sec. 4 and Dataset section; used for training and evaluation; no comparison on existing public benchmarks like DynaCap (excluded by the authors).

how reviews work

0 comments
Cite this review

Pith. "Pith review of EVA: Expressive Virtual Avatars from Multi-view Videos." pith.science (2026). https://pith.science/paper/5NTMPFXF

@misc{pith2026250515385,
  author       = {Pith},
  title        = {Pith review of: EVA: Expressive Virtual Avatars from Multi-view Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5NTMPFXF}},
  note         = {Machine review of arXiv:2505.15385}
}
read the original abstract

With recent advancements in neural rendering and motion capture algorithms, remarkable progress has been made in photorealistic human avatar modeling, unlocking immense potential for applications in virtual reality, augmented reality, remote communication, and industries such as gaming, film, and medicine. However, existing methods fail to provide complete, faithful, and expressive control over human avatars due to their entangled representation of facial expressions and body movements. In this work, we introduce Expressive Virtual Avatars (EVA), an actor-specific, fully controllable, and expressive human avatar framework that achieves high-fidelity, lifelike renderings in real time while enabling independent control of facial expressions, body movements, and hand gestures. Specifically, our approach designs the human avatar as a two-layer model: an expressive template geometry layer and a 3D Gaussian appearance layer. First, we present an expressive template tracking algorithm that leverages coarse-to-fine optimization to accurately recover body motions, facial expressions, and non-rigid deformation parameters from multi-view videos. Next, we propose a novel decoupled 3D Gaussian appearance model designed to effectively disentangle body and facial appearance. Unlike unified Gaussian estimation approaches, our method employs two specialized and independent modules to model the body and face separately. Experimental results demonstrate that EVA surpasses state-of-the-art methods in terms of rendering quality and expressiveness, validating its effectiveness in creating full-body avatars. This work represents a significant advancement towards fully drivable digital human models, enabling the creation of lifelike digital avatars that faithfully replicate human geometry and appearance.

Figures

Figures reproduced from arXiv: 2505.15385 by the authors.

Figure 1
Figure 1. EVA generates high-fidelity, real-time human renderings from arbitrary camera views, skeletal motion, and expression parameters. It leverages an expressive template geometry as a geometric proxy for its 3D Gaussian appearance model, enabling disentangled control over body, hands, and face. With recent advancements in neural rendering and motion capture algo￾rithms, remarkable progress has been made in photorealistic… view at source ↗
Figure 2
Figure 2. Method Overview. EVA generates high-fidelity renderings from a virtual viewpoint, skeletal motion, and expression parameters. Using a personalized head avatar and a deformable character model, we control body movements and facial expressions to drive an actor-specific mesh. This mesh generates motion-aware textures, and separate modules independently predict the Gaussian parameters for the face and body. The 3D Gaus… view at source ↗
Figure 3
Figure 3. Illustration of the head fitting: (b-d) results after each iteration of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Qualitative result and comparison of our expressive mesh template and appearance (green box) with the ground truth image (GT), alongside results [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results of the complete EVA model. The first set of results demonstrates EVA’s ability to render images of a character with previously [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of EVA with two real-time human rendering approaches: ASH [Pang et al [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Qualitative ablation results highlighting the impact of our design and training strategies. The fully disentangled modeling of EVA’s appearance delivers [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Comparison between the high-resolution scan, the uniformly down [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: The top row visualizes the one-sided Chamfer distance to the tar [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Visualization of 2D Face Landmark Predictions: 2D face landmarks [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Visualization of the normal and positional motion-aware textures [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Textured high-resolution 3D scans of all subjects in our dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Cropped frames showcasing different actions performed by actors: [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Visualization of our personalized head avatar [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Personalized head avatars 𝐻 for all subjects, showcasing four ran￾domly sampled facial expressions [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: illustrates the capability of the learned deformable character model to predict physically plausible clothing deformations. This (a) (b) (c) (d) [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: EVA uses skeletal motions and facial expressions generated from [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 19
Figure 19. Figure 19: Comparison of the appearance model trained with random back [PITH_FULL_IMAGE:figures/full_fig_p019_19.png]
Figure 18
Figure 18. Figure 18: Qualitative comparison of EVA with AnimatableGaussians (AG) [Li [PITH_FULL_IMAGE:figures/full_fig_p019_18.png]
Figure 21
Figure 21. Figure 21: Comparison between (b) our method, EVA, and (c) a reimplemen [PITH_FULL_IMAGE:figures/full_fig_p020_21.png]
Figure 20
Figure 20. Figure 20: Comparison of the rendered avatar under varying lighting condi [PITH_FULL_IMAGE:figures/full_fig_p020_20.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

4 extracted references · 3 canonical work pages

  1. [2]

    Based on these inputs, the 3D face landmarks on the FLAME mesh are computed, projected to 2D (𝜋V), and com- pared with the corresponding 2D landmarks l𝑖

    (12) Here, V represents the vertices of the expressive deformable tem- plate mesh, and LV denotes the predicted 2D landmarks for a given camera viewpointV. Based on these inputs, the 3D face landmarks on the FLAME mesh are computed, projected to 2D (𝜋V), and com- pared with the corresponding 2D landmarks l𝑖. To ensure temporal smoothness, a total variatio...

    work page 2019

  2. [2019]

    Computer Vision and Pattern Recognition (CVPR) (2019), 10101–10111

    Capture, Learning, and Synthesis of 3D Speaking Styles. Computer Vision and Pattern Recognition (CVPR) (2019), 10101–10111. http://voca.is.tue.mpg.de/ M. Garland and P.S. Heckbert. 1998. Simplifying surfaces with color and texture using quadric error metrics. In Proceedings Visualization ’98 (Cat. No.98CB36276) . 263–269. https://doi.org/10.1109/VISUAL.19...

    work page arXiv 2019

  3. [2023]

    Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesis

    Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4015–4026. Youngjoong Kwon, Dahun Kim, Duygu Ceylan, and Henry Fuchs. 2021. Neural human performer: Learning generalizable radiance fields for human performance rendering. Advances in Neural Information Processing Systems 34 (2021). Jiefeng Li, Siyuan Bian, Chao ...

    work page Pith review arXiv 2021

  4. [2024]

    MetaCap: Meta-learning Priors from Multi-View Imagery for Sparse-view Human Performance Capture and Rendering. In ECCV. Guoxing Sun, Rishabh Dabral, Heming Zhu, Pascal Fua, Christian Theobalt, and Marc Habermann. 2025. Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Textures. InProceedings of the IEEE/CVF Conferenc...

    arXiv 2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.