{"id":"d47bedb5-1a50-4b15-bd92-71808bed320d","arxiv_id":"2509.02141","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GRMM combines a classic 3D face template with learned fine detail residuals to render controllable full-head avatars in real time.","lead":"A new head model, GRMM, renders full-head avatars in real time by fusing a standard 3D face template with learned fine detail layers for wrinkles, hair, and the mouth interior. The authors also collected a 50-person, 60-expression dataset to keep identity and expression controls separate.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appearance decoder is not conditioned on expression code, so expression-specific appearance residuals are absent from the 3D model; only geometry and screen-space refinement can carry expression appearance.","rationale":"The reader identified expression alignment noise as the weakest assumption. That is a valid data-quality concern, but the appearance-decoder issue is more load-bearing because it is an internal architectural property that directly contradicts the claimed decomposition of residuals into geometry and appearance. If the appearance decoder ignores zexp, then the model cannot represent expression-specific appearance in a 3D-consistent manner, which limits fine-grained expression control and transfer—central contributions. The concern is testable via a simple ablation. The verdict should remain CONDITIONAL because the issue may be fixable by adding expression conditioning to Φapp and retraining, or by clarifying that expression appearance is intentionally left to the screen-space network; either way, the current paper overstates its residual appearance capabilities. I do not advocate rejection because the geometric residuals and overall pipeline may still deliver plausible results, and the user study shows good performance against Morphable Diffusion, albeit without this ablation.","tokens_in":15569,"tokens_out":7597,"duration_ms":95204,"concrete_test":"Train a variant where Φapp takes zexp (and optionally αexp) as additional conditioning (e.g., concatenated to the decoder input). On the held-out expression-transfer set (EXPRESS-50 / NeRSemble), compare LPIPS/PSNR against the published model. If the expression-conditioned variant improves LPIPS by more than ~0.01 or is preferred in the user study, the omission is material and the paper's architecture needs revision. As a simpler diagnostic, render the same fitted identity with two different zexp codes while keeping zid, pose, view, and Ψref fixed, and compare the pre-refinement Irgb: if the per-Gaussian RGB maps are strictly identical, the appearance decoder indeed ignores expression.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of GRMM is disentangled residual geometry and appearance that capture subject- and expression-specific detail. However, in Sec. 3.3 and Fig. 2, the appearance decoder Φapp is explicitly conditioned only on zid and viewing direction d, and the opacity decoder Φα only on zid. Neither takes the expression code zexp nor the FLAME expression coefficients αexp. Consequently, the per-Gaussian RGB and 32-D feature vectors are identical for all expressions of a given identity (up to view direction). Expression-dependent appearance—wrinkles, creases, teeth/tongue visibility, skin-stretch shading—must therefore be produced either by geometric warping of a fixed identity texture or by the screen-space CNN Ψref from Inorm_depth. The latter is a 2D post-process that can hallucinate expression cues from depth, but it is not a learned 3D appearance residual and may not be multiview consistent. This weakens the paper's claim of 'residual geometry and appearance components' that 'capture subject- and expression-specific detail.' The overview sentence in Sec. 3.3 even states that decoders 'map identity and expression codes to geometric and appearance attributes,' which is inconsistent with the actual Φapp input. This is an internal architectural gap, independent of the expression-alignment quality highlighted by the reader.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"GRMM proposes a full-head Gaussian morphable head model that combines a FLAME-based mesh decoder with learned residual identity and expression latent codes, UV-anchored 3D Gaussian primitives, CNN decoders, and a screen-space refinement network. The method is trained on a newly introduced EXPRESS-50 dataset (50 identities, 60 aligned expressions) together with RenderMe-360, and is evaluated on monocular reconstruction, novel-view synthesis, and expression transfer against HeadNeRF, MoFaNeRF, and Morphable Diffusion. The paper claims state-of-the-art fidelity, disentangled control, and real-time 1K rendering at 75 FPS.","tokens_in":15903,"tokens_out":4604,"duration_ms":52918,"significance":"The work has several strengths: it is explicitly positioned as open-source, introduces a novel expression-aligned dataset (EXPRESS-50) that addresses a real gap in the literature, and provides ablations supporting the value of the residual parameterization, the mesh decoder, the refinement network, and joint dataset training. If the claims are substantiated, GRMM would be a practically useful controllable head model with photorealistic real-time rendering. However, the quantitative evidence is incomplete: the closest Gaussian-based head models (GPHM, HeadGAP) are not compared, and the reported numbers lack error bars and are based on small test sets. In addition, there is an internal architectural mismatch between the stated inputs to the decoders and their actual conditioning, which affects the central claim of expression-dependent appearance residuals.","major_comments":[{"comment":"Equation (4) states that all decoders take zid, zexp, αexp, θneck, θjaw, and d. However, the detailed specification shows that the appearance decoder Φapp is conditioned only on zid and view direction d, and the opacity decoder Φα is conditioned only on zid. Consequently, the per-Gaussian RGB and 32-D feature vectors are identical across expressions for a fixed identity, up to view direction. Expression-dependent appearance—wrinkles, creases, teeth/tongue visibility, skin-stretch shading—must therefore be carried by geometric warping of a fixed identity texture or by the screen-space CNN Ψref, not by a 3D appearance residual. This contradicts the abstract's claim of 'residual geometry and appearance components' that capture 'subject- and expression-specific detail', and the overview sentence in Sec. 3.3 stating that the decoders 'map identity and expression codes to geometric and appeara","section":"Sec. 3.3, Eqs. (4), (13)-(14)"},{"comment":"The quantitative comparisons are against HeadNeRF and MoFaNeRF, both NeRF-based methods from 2022, plus a qualitative comparison to Morphable Diffusion. The closest Gaussian head models—GPHM and HeadGAP, which are discussed in Related Work—are not included in any quantitative comparison. Since the paper's central claim is to outperform prior morphable face models, including Gaussian-based ones, the evidence is incomplete. In addition, Table 3 reports RMSE and FID on only 10 randomly selected RAVDESS identities without variance; Table 2 does not state test-set size or error bars. Please add the missing Gaussian baselines and report mean and standard deviation over multiple seeds or splits.","section":"Sec. 4.2, Tables 2 and 3"},{"comment":"The disentanglement of identity and expression is partly built into the architecture: z_exp is shared per expression across identities, z_id is per identity, and L_z regularizes both. The semantic validity of this separation depends entirely on the quality of the manual expression alignment. For EXPRESS-50, peak frames are selected by subjective visual matching against a reference identity; for RenderMe-360, the last frame of each video is used, with the assertion that it 'typically captures the peak'. No quantitative alignment accuracy or inter-annotator agreement is reported. The supplementary material itself shows the risk: Sec. 7 states that before adding mouth-interior geometry, zeroing z_exp severely distorts the mouth, indicating that intra-oral appearance was entangled with the expression channel. This does not invalidate the method, but it means the disentanglement and transfer","section":"Sec. 3.1; Supp. Sec. 7"},{"comment":"Depth supervision uses ProbeSDF, a learned surface reconstruction method, as 'ground-truth' depth. No validation of ProbeSDF's accuracy on the specific datasets, nor an analysis of potential systematic bias, is provided. Since the depth loss directly supervises the mesh and Gaussian geometry, geometric errors in ProbeSDF could propagate into the learned residuals and affect the claimed fidelity. A quantitative comparison of ProbeSDF against known geometry (for example, on a dataset with 3D scans) or a sensitivity analysis would strengthen the geometric claims.","section":"Sec. 3.1 and 3.4; Supp. Sec. 6"}],"minor_comments":[{"comment":"The 75 FPS figure is reported without hardware specification. Please state the GPU and rendering resolution used.","section":"Abstract; Sec. 1"},{"comment":"Typo: 'Distangled' should be 'Disentangled'.","section":"Fig. 17 caption"},{"comment":"References [28] and [37] appear to have malformed page markers ('4, 1' and '4, 1'); please clean up the bibliography.","section":"References"},{"comment":"The user study pooling is reported as percentages, but it is unclear how ties are handled across the 'Expression' and 'Identity' rows. Please clarify the denominator and whether the 600 judgments are 30 per participant or another allocation.","section":"Supp. Sec. 8, Table 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses an important problem and the EXPRESS-50 dataset is a valuable contribution. The main concerns are the missing comparisons to GPHM and HeadGAP, the lack of statistical robustness in the quantitative results, and the architectural mismatch in expression-conditioned appearance. If the authors can add the missing baselines and clarify or rectify the appearance-conditioning issue, the paper could become acceptable. I recommend major revision rather than rejection because the core ideas are defensible and the issues appear addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a well-engineered system and EXPRESS-50 is a real contribution, but the paper oversells its disentangled appearance residuals and the evaluation doesn't yet support the SOTA claims. Worth engaging, but it needs revision plus artifact release.\n\nWhat's new: combining a base 3DMM with learned per-vertex and per-Gaussian residuals, the full-head mouth-interior UV topology, and the manually expression-aligned multi-view dataset. The ablations are useful and honestly reported: residuals, the mesh decoder, refinement, and joint dataset training each earn their keep. The integration of convolutional decoders for per-Gaussian attributes instead of heavy MLPs is sensible and plausibly helps the real-time claim.\n\nThe main soft spot is an internal architectural gap that the paper itself papers over. The overview says all decoders take both identity and expression codes, but the appearance decoder Φapp is conditioned only on zid and view direction, and the opacity decoder only on zid. So expression-specific appearance—wrinkles, teeth/tongue visibility, skin-stretch shading—cannot be produced as 3D appearance residuals. It has to come from geometric warping of a fixed identity texture or from the screen-space CNN Ψref, which is a 2D post-process and not multiview consistent. That undercuts the claim that residual appearance components capture subject- and expression-specific detail. It doesn't kill the method, but the paper needs to be honest about where expression appearance actually lives.\n\nAlso real: the comparisons omit the closest Gaussian head models, GPHM and HeadGAP. Comparing against HeadNeRF and MoFaNeRF is not enough for a claim of state of the art. Test sets are small (10 identities for monocular), there are no error bars, and the 75 FPS figure ships without hardware specs. The open-source claim is currently just a promise—no code or dataset link in the paper. The manual expression alignment, especially the RenderMe-360 last-frame shortcut, is a legitimate concern for the disentanglement claim, though not a fatal one.\n\nOn balance, the central method is credible and the dataset is worthwhile. The problems are mostly in the framing and the evaluation, not in the core idea. I'd send this to peer review, but I'd expect a major revision: add the missing baselines, report statistics, release the artifacts, and fix the description of the appearance decoder. This is a paper for people working on parametric head models and 3D Gaussian avatars; they'll want to see it, but the claims need tightening before it becomes citable as-is.","headline":"A solid engineering contribution with a valuable new dataset, but the paper oversells its disentangled appearance residuals and the SOTA claims outrun the evidence: missing closest baselines, no error bars, and an architecture text that contradicts the actual decoder inputs.","tokens_in":16406,"tokens_out":2124,"would_cite":true,"duration_ms":24449,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GRMM augments a classical 3DMM with learned residual geometry and appearance to render photorealistic full-head avatars at 75 fps.","keywords":["gaussian splatting","3D morphable model","head avatar","expression disentanglement","residual learning","novel-view synthesis","expression transfer","multi-view head dataset"],"falsifier":"Take two subjects whose mouth shapes differ strongly, fit GRMM to the same labeled expression from each, then render each subject with the other subject's expression code. If the resulting mouth reveals the source identity's tooth or tongue geometry rather than the intended expression, identity has leaked into the shared expression residual. A quantitative check is to train a simple identity classifier on mouth-crop renderings generated with z_exp held constant; above-chance classification would indicate identity leakage.","tokens_in":15501,"feed_emoji":"🎭","tokens_out":5813,"duration_ms":63377,"temperature":0.7,"pith_summary":"The paper sets out to establish GRMM, an open-source full-head morphable model that layers learned residual geometry and appearance on top of a classic mesh-based 3DMM. If correct, it gives one parametric model fine-grained control over identity, expression, and pose while rendering detailed full-head portraits at 1K resolution in real time at 75 FPS. The central design claim is that separating interpretable coarse 3DMM parameters from low-dimensional residual codes keeps the model editable and expressive. To make those residuals transferable, the authors introduce EXPRESS-50, a multiview dataset with 50 identities performing 60 manually aligned expressions, and argue that this cross-identity alignment is what allows identity and expression to be disentangled.","feed_headline":"Residual Gaussians put photorealistic head avatars at 75 fps","feed_subtitle":"A base 3DMM plus learned residual detail gives disentangled, real-time control over identity and expression.","key_machinery":"UV-anchored Gaussian primitives on an enhanced FLAME mesh. Gaussians are initialized by uniform sampling in the mesh's UV space, so they deform coherently with predicted mesh displacements; a transformation decoder refines their position, rotation, and scale, an opacity decoder conditions only on identity, and an appearance decoder conditions on identity and view direction. The residual codes z_id and z_exp are the disentanglement mechanism: expression-specific detail is carried by a shared global code, while identity-specific detail stays in the per-identity code.","core_discovery":"GRMM's central claim is that a full-head Gaussian morphable model can be built by additive residual refinement of a base FLAME 3DMM: a shared identity code z_id and a shared expression code z_exp drive lightweight decoders that predict vertex-level displacements and per-Gaussian transforms, while a screen-space CNN cleans up the rasterized image. The deformed mesh v_d = v_rec + v_delta,id + M_face v_delta,exp and the final image I = Psi_ref([I_rgb, I_feature, I_norm_depth]) capture the two stages. Against state-of-the-art morphable head models, the paper reports higher reconstruction fidelity and novel-view quality, and its user study shows expression transfer that preserves identity across","pith_inferences":["A natural extension is to automate peak-expression selection: if the manual alignment in EXPRESS-50 is replaced by a learned expression embedding, the same residual decomposition could scale to thousands of identities.","The architecture implies a direct control knob for hairstyle and face shape through z_id while z_exp controls expression; one could test this by interpolating two identities' z_id codes and checking whether rendered hair and face shape interpolate independently of expression.","Because opacity is conditioned only on identity, occluding features like hair are decoupled from expression; a testable corollary is that editing z_id alone should change occlusion boundaries without disturbing the expression readout.","The reported 75 FPS for 1K rendering suggests the same coarse-to-fine residual design could be carried into relighting or gaze editing with additional latent codes."],"forward_implications":["A single fitted portrait can be re-rendered from arbitrary viewpoints with consistent identity and expression, because the same 3D Gaussian representation is shared across views.","Expression transfer reduces to copying the shared z_exp code: the paper shows that this preserves the target expression and the source identity, including subtle mouth articulation.","Fine details like wrinkles, hairline variation, and inner-mouth geometry are recovered by the learned residuals rather than by raising the 3DMM's PCA dimensionality, so they stay locally editable.","Because the full head, including teeth and inner mouth, is modeled, GRMM can be used for tasks face-only models cannot handle, such as avatars with talking or expressive mouths."],"supporting_citations":[{"why":"Supplies the base FLAME 3DMM mesh, identity/expression parameter space, and tracked mesh v_rec to which residuals are added.","marker":"[20]"},{"why":"Provides differentiable 3D Gaussian splatting rasterization used for real-time rendering of RGB, depth, and feature images.","marker":"[15]"},{"why":"GaussianHeads architecture is adapted for UV-anchored Gaussian primitive prediction, providing the initial formulation of primitive attributes conditioned on expression.","marker":"[34]"},{"why":"RenderMe-360 dataset supplies 280 identities with 12 expressions, forming the identity-diversity half of joint training.","marker":"[25]"},{"why":"VHAP tracking estimates FLAME parameters and extends the mesh with teeth vertices, supporting expression alignment and mouth-interior modeling.","marker":"[28]"},{"why":"ProbeSDF generates ground-truth depth maps used to supervise the model's geometry.","marker":"[37]"},{"why":"GPHM is the closest Gaussian parametric head prior whose limitations the paper targets with its coarse/fine residual separation.","marker":"[45]"},{"why":"HeadGAP is the FLAME-based Gaussian head baseline that GRMM contrasts against in expressivity and generative flexibility.","marker":"[50]"}],"fun_headline_variants":["Full-head Gaussian 3DMM with residual detail hits 75 FPS","Residual Gaussians give real-time photorealistic head avatars","GRMM: Gaussian morphable head model with learned residuals at 75 FPS","Gaussian head avatars with disentangled real-time control","Real-time full-head Gaussian model with residual refinements"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The model's claim that expressions transfer cleanly across people depends on its hand-picked 'peak expression' frames actually matching semantically for every subject; if those alignments are off, the expression code silently carries identity information and the edit fails.","fun_headline_variants_meta":{"raw":{"variants":["Full-head Gaussian 3DMM with residual detail hits 75 FPS","Residual Gaussians give real-time photorealistic head avatars","GRMM: Gaussian morphable head model with learned residuals at 75 FPS","Gaussian head avatars with disentangled real-time control","Real-time full-head Gaussian model with residual refinements"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000285,"raw_usage":{"total_tokens":1568,"prompt_tokens":852,"completion_tokens":716,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":625}},"tokens_in":596,"tokens_out":716,"duration_ms":7314,"temperature":1.0,"reasoning_tokens":625,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:49:57.904679+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two subjects whose mouth shapes differ strongly, fit GRMM to the same labeled expression from each, then render each subject with the other subject's expression code. If the resulting mouth reveals the source identity's tooth or tongue geometry rather than the intended expression, identity has leaked into the shared expression residual. A quantitative check is to train a simple identity classifier on mouth-crop renderings generated with z_exp held constant; above-chance classification would indicate identity leakage.","supporting_citations":[{"cited_title":"Learning a model of facial shape and expression from 4d scans","cited_arxiv_id":null,"evidence_quote":"Supplies the base FLAME 3DMM mesh, identity/expression parameter space, and tracked mesh v_rec to which residuals are added."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Provides differentiable 3D Gaussian splatting rasterization used for real-time rendering of RGB, depth, and feature images."},{"cited_title":"Gaus- sianheads: End-to-end learning of drivable gaussian head avatars from coarse-to-fine representations","cited_arxiv_id":null,"evidence_quote":"GaussianHeads architecture is adapted for UV-anchored Gaussian primitive prediction, providing the initial formulation of primitive attributes conditioned on expression."},{"cited_title":"Renderme-360: A large dig- ital asset library and benchmarks towards high-fidelity head avatars","cited_arxiv_id":null,"evidence_quote":"RenderMe-360 dataset supplies 280 identities with 12 expressions, forming the identity-diversity half of joint training."},{"cited_title":"Vhap: Versatile head alignment with adaptive appearance priors, 2024","cited_arxiv_id":null,"evidence_quote":"VHAP tracking estimates FLAME parameters and extends the mesh with teeth vertices, supporting expression alignment and mouth-interior modeling."},{"cited_title":"Probesdf: Light field probes for neural surface reconstruc- tion","cited_arxiv_id":null,"evidence_quote":"ProbeSDF generates ground-truth depth maps used to supervise the model's geometry."},{"cited_title":"3d gaussian parametric head model","cited_arxiv_id":null,"evidence_quote":"GPHM is the closest Gaussian parametric head prior whose limitations the paper targets with its coarse/fine residual separation."}],"review_version":1}