REVIEW 4 major objections 6 minor 3 references
Relightable Full-Body Gaussian Codec Avatars
T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper introduces the first relightable full-body avatar model that jointly handles the body, face, and hands, using 3D Gaussian splatting with light transport decomposed into local and non-local effects.
desk verdict Solid full-body relightable avatar system whose novel-illumination claim outruns the quantitative evidence; still worth a serious referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is the three-way decomposition of light transport. Zonal harmonics (spherical harmonics with one axis of symmetry) can be rotated to world space far more cheaply than full SH coefficients, so the model learns diffuse transfer per Gaussian in its local tangent frame and rotates it under articulation. The shadow network is a UV-space convolutional network, adapted from earlier full-body avatar work, that takes a normalized irradiance map (visibility-weighted irradiance divided by unshadowed irradiance, Eq. 15) on a coarse tracked mesh and predicts a single shadow factor per texel, rendering non-local occlusion. Deferred shading rasterizes per-Gaussian specular normals, roughness, and visibility to screen space, then integrates the environment against a spherical Gaussian lobe centered at the reflected view direction, so specular detail is not limited by local Gaussian density.
What would settle it
Train the model on light-stage point lights whose directions cover only a limited angular range, then relight under an environment map containing a bright source from a direction outside that range and compare the rendered shadows to a path-traced reference. If shadow boundaries or darkness move to the wrong location, the irradiance normalization has not made the network illumination-invariant and the novel-illumination generalization claim fails.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that full-body relighting can be factored into three learned components that each handle what they are good at: zonal-harmonic diffuse radiance transfer in local coordinates for orientation-dependent local shading, an irradiance-conditioned shadow network for non-local self-occlusion between body parts, and deferred-shading-based spherical-Gaussian specular transfer for high-frequency highlights. The paper demonstrates on captured light-stage sequences that this combination outperforms a physically based rendering appearance model and its own ablations (SH instead of ZH, no shadow network, no deferred shading) in perceptual quality, and that it generalizes to unseen poses and to held-out point-light configurations.
Load-bearing premise
The shadow network, trained on normalized irradiance computed from a coarse tracked mesh, must generalize to lighting conditions it never saw during training; the quantitative evidence covers held-out point-light configurations from the same light stage, while environment-map relighting is shown only qualitatively.
Editorial extensions
If this is right
- A full-body avatar can be relit and animated in near-real time without multi-bounce path tracing, since global illumination effects are captured by learned light transport rather than simulation.
- The diffuse light transport parameter count drops from 113 to 51 per texel compared to the SH variant, while perceptual quality improves, making the representation more memory-efficient.
- The shadow network generalizes to held-out point-light configurations and the method shows environment-map relighting qualitatively, which the authors take as evidence that the normalization scheme makes the network illumination-invariant.
- Deferred shading recovers high-frequency specular effects such as eye glints that per-Gaussian shading misses when the Gaussian count is limited, closing a fidelity gap for full-body models.
- The avatar can be animated to unseen poses and still render correctly under novel lighting, a combination that prior full-body methods did not jointly achieve.
Reading between the lines
- A stress test the paper leaves implicit: feed the shadow network an environment map with a sharp directional source from an elevation not present in the light-stage training distribution; the normalization in Eq. (15) is only as illumination-invariant as the coverage of training light directions, so failure would show up as a misplaced shadow boundary rather than a brightness error.
- The ZH local-coordinate design suggests a path toward avatars that are relightable from monocular or sparse-view video: if rotation is the only pose-dependent operation, the same transfer coefficients could be trained with far fewer lighting conditions.
- The deferred-shading specular branch could be extended to fit a fuller BRDF (normal, roughness, plus a second specular lobe) without changing the splatting, which might push eye and jewelry highlights closer to path-traced quality.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Relightable Full-Body Gaussian Codec Avatars, a relightable and animatable full-body avatar model built on 3D Gaussian splatting. The appearance pipeline combines three components: zonal-harmonic (ZH) diffuse radiance transfer learned in each Gaussian's local frame and evaluated in the world frame via the Gaussian rotation (Sec. 3.2.1); a UV-space shadow network that predicts a scalar shadow map from physically normalized incoming irradiance on a coarse tracked mesh, targeting non-local shadows from body-part occlusion (Sec. 3.3); and deferred shading for spherical-Gaussian specular radiance transfer, argued to preserve high-frequency reflections such as eye glints under a limited Gaussian budget (Sec. 3.2.2). The model is trained on multi-view light-stage data (512 cameras, 1024 controllable lights, roughly 5000-6000 frames per subject) with known illumination, using L1 plus LPIPS reconstruction and several regularizers. The authors claim this is the first relightable full-body avatar jointly modeling body, face, and hands, and they report quantitative results on held-out point-light frames for three subjects, ablating against PBR, unrotated SH, no shadow network, no deferred shading, and mesh normals, together with qualitative environment-map relighting.
Significance. If correct, this would be a meaningful advance over head-only relightable Gaussian avatars: the ZH formulation gives an efficient way to obtain pose-dependent diffuse transfer for articulated bodies without per-joint SH rotation, the irradiance-normalized shadow network is a principled alternative to per-iteration ray tracing for non-local shadows, and deferred shading decouples reflection frequency from Gaussian density for specular radiance transfer. The experimental methodology is generally solid: the ablations isolate each component, the held-out camera and pose splits constitute genuine generalization tests, and the Appendix A demonstration that a single Monte Carlo sample suffices for the irradiance input is a useful robustness check. The limitations section is candid, conceding that cloth dynamics may be physically implausible, that eye/face/hand fidelity is below specialized methods, and that the capture setup has limited scalability, which appropriately tempers the 'high-fidelity' and 'first' claims.
major comments (4)
- [Sec. 3.3, Eq. (15), Table 1, Fig. 9] The abstract's claim of 'superior generalization ability under novel illumination conditions' is supported quantitatively only for held-out point-light configurations (Table 1, 'Unseen Motion' column); environment-map relighting appears only qualitatively in Fig. 9. This distinction is load-bearing because the shadow network of Sec. 3.3 is trained on normalized irradiance maps computed from 10-20 sparse point lights, which are near-binary, whereas environment maps produce smooth, dense normalized-irradiance inputs; the physical normalization of Eq. (15) plausibly reduces but does not eliminate this distribution shift, and the network's invariance to it is asserted rather than demonstrated. The 1-sample robustness result of Appendix A (Table A.1) addresses noise tolerance, not illumination-class transfer. In addition, the shadow map is scalar and view-independent (Eq. 16) yet multiplies the specular term, so specular occlusion is modeled only approximately. I request either a quantitative environment-map evaluation (e.g., light-stage frames lit by representative environment maps, or comparisons against path-traced references) or a restriction of the generalization claim to the point-light setting.
- [Sec. 4.2, Table 1] No direct comparison is made to any published full-body relighting method, although Sec. 2.2 cites several as state of the art (Relighting4D [Chen and Liu 2022], IntrinsicAvatar [Wang et al. 2024], Animatable and Relightable Gaussians [Li et al. 2024b], MeshAvatar [Chen et al. 2024c]). The only external baseline is a PBR appearance model built on the authors' own learned geometry, so the 'first relightable full-body avatar' and 'superior quality' claims are not directly supported by comparison with prior systems. The dataset-incompatibility justification is understandable, but the paper would be materially stronger if one or two such methods were evaluated on a shared public benchmark (for example, ZJU-MoCap or monocular video data) with the proposed method adapted to that setting; otherwise the comparative claims should be explicitly scoped to the PBR baseline.
- [Sec. 4.3, Table 1 (SH row)] The ZH-versus-SH ablation conflates the representation choice with the rotation strategy, because the SH baseline is explicitly unrotated ('SH is not rotated as discussed in Sec. 3.2.1'); its degraded appearance on the hands and arms in Fig. 4 could therefore be due to the absence of pose-dependent transfer rather than to the SH basis itself. Since efficient rotation is precisely the motivation for ZH, the comparison supports the full design but does not isolate the basis choice. A rotated SH baseline at reduced order (e.g., third or fourth order, using Wigner-D matrices per Gaussian or per body part) would separate the two effects and substantiate the paper's second contribution.
- [Table 1, Sec. 4.3] The quantitative differences between ablations are small: Ours versus w.o. deferred differ by -0.07 dB PSNR and -0.0015 LPIPS on training motion, and Ours versus Mesh normal by 0.05 dB PSNR, while no error bars, per-subject breakdown, or repeated runs are reported. With only three quantitative subjects, the individual component claims (deferred shading, Gaussian normals) are supported at a level that might lie within run-to-run variation. The paper's explanation of the PSNR/SSIM reversal for deferred shading is plausible and the qualitative evidence in Fig. 7 is helpful, but per-subject metrics or multiple seeds would materially solidify the ablation conclusions.
minor comments (6)
- [Appendix B] The sentence 'Lalbedo and Lneg_color are the squared losses on negative diffuse color values and albedo values, respectively' contradicts the preceding sentence defining Lalbedo as an L1 loss on albedo values; please clarify which loss applies to which quantity.
- [Sec. 3.2.1] The notation z_k in R^{3x3l} for the learned ZH coefficient set is ambiguous; please state the dimensions explicitly (three colored lobes through third order plus three monochromatic lobes through eighth order).
- [Sec. 4.2 and Sec. 1] The typos 'PNSR/SSMI' and 'religthtable' should read 'PSNR/SSIM' and 'relightable'.
- [Sec. 2.2] The sentence 'Neural relighting with shadow conditioning has also been explored for relightable hands [Chen et al. 2024b; Iwase et al. 2023] exhibit more articulation compared to the human head' is grammatically incomplete; consider rewriting it.
- [Eq. (13)] The normalization constant 1/sqrt(2pi) * 2/(3 sigma) of the spherical Gaussian is unusual for a lobe parameterized by arccos(p . q)/sigma; please cite the standard spherical-Gaussian formulation or justify the constant.
- [Sec. 4.1] The paper does not state how the quantitative metrics are aggregated across the three subjects or how those subjects were selected; a brief statement would aid reproducibility.
Circularity Check
No circular derivation: the learned components are fitted to light-stage data and evaluated on held-out conditions; self-citations are building blocks, not load-bearing evidence.
full rationale
This paper is a learning-based reconstruction system rather than a derivation, so the central equations do not reduce to their inputs by construction. The diffuse radiance transfer (Eqs. 7-9) learns zonal-harmonic coefficients from captured images; the specular model (Eqs. 12-14) is a spherical-Gaussian deferred-shading formulation; and the shadow network (Sec. 3.3) learns a mapping from normalized irradiance (Eq. 15) to a shadow multiplier (Eq. 16), supervised by captured images and tested on held-out poses and point-light configurations. Although normalized irradiance already contains the visibility function, the network output is not algebraically equal to the input, so this is a learned approximation rather than a self-fulfilling prediction. The paper relies on prior work by overlapping authors, especially Saito et al. 2024 and Bagautdinov et al. 2021, but only as architectural building blocks and baseline components; no load-bearing conclusion is justified solely by a self-citation. The abstract's claim of 'superior generalization ability under novel illumination conditions' is partly an evaluation-scope concern, since environment-map relighting is shown qualitatively only and the quantitative numbers come from held-out point lights; that is a correctness and generalization risk, not circularity. Overall, no step was found in which a prediction is identical by construction to a fitted input or to a self-cited uniqueness claim.
Assumptions & free parameters
free parameters (4)
- Zonal harmonics configuration =
n=8, colored to order 3, monochromatic orders 4-8 (51 coefficients per texel)
- Regularization loss weights =
LPIPS 0.1, offset 0.05, mask 0.1, normal_orient 0.1, alpha_sparsity 0.1, bound 0.01, albedo 0.01, neg_color 0.01…
- Roughness and scale bounds =
scale lb=0.0001, ub=0.01; roughness lb=0.01, ub=0.25
- Shadow network input samples =
N=1 sample per pixel in the reported variant
assumptions (5)
- ad hoc to paper Zonal harmonics can express the per-point diffuse radiance transfer as a sum of three ZH lobes aligned with tangent, bitangent, and normal directions (Eq. 9).
- domain assumption The normalized irradiance computed on a coarse tracked mesh (Eq. 15) contains enough information for a network to predict non-local shadows that generalize to new illuminations.
- domain assumption 3D Gaussians associated with texels of a UV map on a tracked template mesh can represent the geometry and appearance of body, face, and hands with the allowed budget.
- standard math Standard SH/ZH orthogonality and rotation properties (used in Sec 3.2.1) are correct.
- domain assumption Light-stage captures provide calibrated known light sources and ground-truth illumination during training.
Cite this review
Pith. "Pith review of Relightable Full-Body Gaussian Codec Avatars." pith.science (2026). https://pith.science/paper/N4T2KROS
@misc{pith2026250114726,
author = {Pith},
title = {Pith review of: Relightable Full-Body Gaussian Codec Avatars},
year = {2026},
howpublished = {\url{https://pith.science/paper/N4T2KROS}},
note = {Machine review of arXiv:2501.14726}
}
read the original abstract
We propose Relightable Full-Body Gaussian Codec Avatars, a new approach for modeling relightable full-body avatars with fine-grained details including face and hands. The unique challenge for relighting full-body avatars lies in the large deformations caused by body articulation and the resulting impact on appearance caused by light transport. Changes in body pose can dramatically change the orientation of body surfaces with respect to lights, resulting in both local appearance changes due to changes in local light transport functions, as well as non-local changes due to occlusion between body parts. To address this, we decompose the light transport into local and non-local effects. Local appearance changes are modeled using learnable zonal harmonics for diffuse radiance transfer. Unlike spherical harmonics, zonal harmonics are highly efficient to rotate under articulation. This allows us to learn diffuse radiance transfer in a local coordinate frame, which disentangles the local radiance transfer from the articulation of the body. To account for non-local appearance changes, we introduce a shadow network that predicts shadows given precomputed incoming irradiance on a base mesh. This facilitates the learning of non-local shadowing between the body parts. Finally, we use a deferred shading approach to model specular radiance transfer and better capture reflections and highlights such as eye glints. We demonstrate that our approach successfully models both the local and non-local light transport required for relightable full-body avatars, with a superior generalization ability under novel illumination conditions and unseen poses.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[2022]
InConference on Computer Vision and Pattern Recognition (CVPR)
High-Resolution Image Synthesis With Latent Diffusion Models. InConference on Computer Vision and Pattern Recognition (CVPR) . Boxiang Rong, Artur Grigorev, Wenbo Wang, Michael J. Black, Bernhard Thomaszewski, Christina Tsalicoglou, and Otmar Hilliges. 2024. Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Mu...
arXiv 2024
-
[2023]
In Advances in Neural Information Processing Systems (NeurIPS)
PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation. In Advances in Neural Information Processing Systems (NeurIPS) . Zhaoxi Chen and Ziwei Liu. 2022. Relighting4D: Neural Relightable Human from Videos. In European Conference on Computer Vision (ECCV) . Zhaoxi Chen, Gyeongsik Moon, Kaiwen Guo, Chen Cao, Stanislav Pidhorskyi, Tomas Simon...
work page 2015
-
[2024]
SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting. arXiv preprint arXiv:2402.18848 (2024). Gengyan Li, Abhimitra Meka, Franziska Mueller, Marcel C Buehler, Otmar Hilliges, and Thabo Beeler. 2022a. EyeNeRF: a hybrid representation for photorealistic synthesis, animation and relighting of human ey...
work page Pith review arXiv 2024
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.