{"id":"3bc04832-973b-4cff-a8f4-491e1f8789d8","arxiv_id":"2501.14726","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A full-body 3D avatar model that combines learned zonal-harmonic light transport, a shadow network, and deferred shading for relighting and animation of body, face, and hands.","lead":"Researchers built a system for making realistic, relightable 3D avatars of an entire person, including face and hands, which can then be animated. It learns how light bounces off skin, cloth, and eyes from a massive multi-camera light-stage capture, allowing the avatar to be placed under new lighting conditions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Novel-illumination generalization is asserted but only point-light relighting is measured; the shadow network's Eq. 15 normalization may not transfer to environment maps.","rationale":"Good-faith reading: the paper's actual contribution is a full-body relightable Gaussian codec avatar with a novel zonal-harmonic local transport plus a dedicated shadow network; that contribution is technically plausible and internally consistent. The strongest evidence (Table 1) shows consistent, though small, gains over PBR and ablation variants on held-out point-light relighting, and several ablations support the design choices (ZH vs SH, shadow network, deferred shading, specular normals). The central risk is not internal inconsistency but external validity: the claimed generalization to novel environment-map illumination is exactly the step where Eq. 15's normalization must make the input lighting-invariant, and the paper gives only a qualitative figure. This is the same weakest assumption the reader identified, so our pass agrees with the CONDITIONAL verdict rather than moving it. We do not object to the first-claim or to the absence of full-body SOTA comparisons as a correctness issue; those are scope and positioning matters. The one concrete check that would settle the concern is a quantitative environment-map test, preferably using the light stage's LEDs to emulate held-out maps.","tokens_in":19837,"tokens_out":6549,"duration_ms":93452,"concrete_test":"Use the light stage's 1024 controllable LEDs to emulate 3-5 held-out environment maps by setting LED intensities to environment-map radiance, and capture novel poses and views under those maps. Evaluate foreground-cropped PSNR/SSIM/LPIPS for Ours vs PBR and vs w.o. shadow, reporting per-subject standard deviations. If the Ours-vs-baseline gaps on environment maps are not comparable to the Table 1 point-light gaps, the claimed novel-illumination generalization is not supported and the conclusion should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim of 'superior generalization ability under novel illumination conditions' rests on the shadow network of Sec. 3.3 being illumination-invariant. Equation 15 feeds the network the normalized irradiance, i.e. the integral of L times visibility divided by the integral of L, and the training data are point-light configurations from the light stage. Table 1 quantifies relighting only for held-out point-light frames; environment-map relighting appears only qualitatively in Fig. 9. This matters because normalized irradiance is not the visibility function: it is a lighting-weighted projection of visibility. A sparse point-light mixture produces near-binary normalized-irradiance maps, while a smooth environment map produces a different input distribution, so a CNN trained on the former has no demonstrated extrapolation to the latter. The scalar shadow map is also view-independent and multiplies specular color, although specular occlusion is view-dependent; the network is never given the viewing direction. Thus the quantitative results support relighting under held-out point lights, not the abstract's stronger novel-illumination claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Relightable Full-Body Gaussian Codec Avatars, a relightable and animatable full-body avatar model built on 3D Gaussian splatting. The appearance pipeline combines three components: zonal-harmonic (ZH) diffuse radiance transfer learned in each Gaussian's local frame and evaluated in the world frame via the Gaussian rotation (Sec. 3.2.1); a UV-space shadow network that predicts a scalar shadow map from physically normalized incoming irradiance on a coarse tracked mesh, targeting non-local shadows from body-part occlusion (Sec. 3.3); and deferred shading for spherical-Gaussian specular radiance transfer, argued to preserve high-frequency reflections such as eye glints under a limited Gaussian budget (Sec. 3.2.2). The model is trained on multi-view light-stage data (512 cameras, 1024 controllable lights, roughly 5000-6000 frames per subject) with known illumination, using L1 plus LPIPS reconstruction and several regularizers. The authors claim this is the first relightable full-body avatar jointly modeling body, face, and hands, and they report quantitative results on held-out point-light frames for three subjects, ablating against PBR, unrotated SH, no shadow network, no deferred shading, and mesh normals, together with qualitative environment-map relighting.","tokens_in":20084,"tokens_out":17767,"duration_ms":135722,"significance":"If correct, this would be a meaningful advance over head-only relightable Gaussian avatars: the ZH formulation gives an efficient way to obtain pose-dependent diffuse transfer for articulated bodies without per-joint SH rotation, the irradiance-normalized shadow network is a principled alternative to per-iteration ray tracing for non-local shadows, and deferred shading decouples reflection frequency from Gaussian density for specular radiance transfer. The experimental methodology is generally solid: the ablations isolate each component, the held-out camera and pose splits constitute genuine generalization tests, and the Appendix A demonstration that a single Monte Carlo sample suffices for the irradiance input is a useful robustness check. The limitations section is candid, conceding that cloth dynamics may be physically implausible, that eye/face/hand fidelity is below specialized methods, and that the capture setup has limited scalability, which appropriately tempers the 'high-fidelity' and 'first' claims.","major_comments":[{"comment":"The abstract's claim of 'superior generalization ability under novel illumination conditions' is supported quantitatively only for held-out point-light configurations (Table 1, 'Unseen Motion' column); environment-map relighting appears only qualitatively in Fig. 9. This distinction is load-bearing because the shadow network of Sec. 3.3 is trained on normalized irradiance maps computed from 10-20 sparse point lights, which are near-binary, whereas environment maps produce smooth, dense normalized-irradiance inputs; the physical normalization of Eq. (15) plausibly reduces but does not eliminate this distribution shift, and the network's invariance to it is asserted rather than demonstrated. The 1-sample robustness result of Appendix A (Table A.1) addresses noise tolerance, not illumination-class transfer. In addition, the shadow map is scalar and view-independent (Eq. 16) yet multiplies the specular term, so specular occlusion is modeled only approximately. I request either a quantitative environment-map evaluation (e.g., light-stage frames lit by representative environment maps, or comparisons against path-traced references) or a restriction of the generalization claim to the point-light setting.","section":"Sec. 3.3, Eq. (15), Table 1, Fig. 9"},{"comment":"No direct comparison is made to any published full-body relighting method, although Sec. 2.2 cites several as state of the art (Relighting4D [Chen and Liu 2022], IntrinsicAvatar [Wang et al. 2024], Animatable and Relightable Gaussians [Li et al. 2024b], MeshAvatar [Chen et al. 2024c]). The only external baseline is a PBR appearance model built on the authors' own learned geometry, so the 'first relightable full-body avatar' and 'superior quality' claims are not directly supported by comparison with prior systems. The dataset-incompatibility justification is understandable, but the paper would be materially stronger if one or two such methods were evaluated on a shared public benchmark (for example, ZJU-MoCap or monocular video data) with the proposed method adapted to that setting; otherwise the comparative claims should be explicitly scoped to the PBR baseline.","section":"Sec. 4.2, Table 1"},{"comment":"The ZH-versus-SH ablation conflates the representation choice with the rotation strategy, because the SH baseline is explicitly unrotated ('SH is not rotated as discussed in Sec. 3.2.1'); its degraded appearance on the hands and arms in Fig. 4 could therefore be due to the absence of pose-dependent transfer rather than to the SH basis itself. Since efficient rotation is precisely the motivation for ZH, the comparison supports the full design but does not isolate the basis choice. A rotated SH baseline at reduced order (e.g., third or fourth order, using Wigner-D matrices per Gaussian or per body part) would separate the two effects and substantiate the paper's second contribution.","section":"Sec. 4.3, Table 1 (SH row)"},{"comment":"The quantitative differences between ablations are small: Ours versus w.o. deferred differ by -0.07 dB PSNR and -0.0015 LPIPS on training motion, and Ours versus Mesh normal by 0.05 dB PSNR, while no error bars, per-subject breakdown, or repeated runs are reported. With only three quantitative subjects, the individual component claims (deferred shading, Gaussian normals) are supported at a level that might lie within run-to-run variation. The paper's explanation of the PSNR/SSIM reversal for deferred shading is plausible and the qualitative evidence in Fig. 7 is helpful, but per-subject metrics or multiple seeds would materially solidify the ablation conclusions.","section":"Table 1, Sec. 4.3"}],"minor_comments":[{"comment":"The sentence 'Lalbedo and Lneg_color are the squared losses on negative diffuse color values and albedo values, respectively' contradicts the preceding sentence defining Lalbedo as an L1 loss on albedo values; please clarify which loss applies to which quantity.","section":"Appendix B"},{"comment":"The notation z_k in R^{3x3l} for the learned ZH coefficient set is ambiguous; please state the dimensions explicitly (three colored lobes through third order plus three monochromatic lobes through eighth order).","section":"Sec. 3.2.1"},{"comment":"The typos 'PNSR/SSMI' and 'religthtable' should read 'PSNR/SSIM' and 'relightable'.","section":"Sec. 4.2 and Sec. 1"},{"comment":"The sentence 'Neural relighting with shadow conditioning has also been explored for relightable hands [Chen et al. 2024b; Iwase et al. 2023] exhibit more articulation compared to the human head' is grammatically incomplete; consider rewriting it.","section":"Sec. 2.2"},{"comment":"The normalization constant 1/sqrt(2pi) * 2/(3 sigma) of the spherical Gaussian is unusual for a lobe parameterized by arccos(p . q)/sigma; please cite the standard spherical-Gaussian formulation or justify the constant.","section":"Eq. (13)"},{"comment":"The paper does not state how the quantitative metrics are aggregated across the three subjects or how those subjects were selected; a brief statement would aid reproducibility.","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"This paper is a strong entry in the Codec Avatars lineage and reports honestly: the ablations are informative, the PSNR/SSIM reversal for deferred shading is disclosed and explained, and the limitations section is unusually candid. The primary correctness risk is the novel-illumination generalization claim, which currently rests on point-light experiments plus qualitative environment-map results; I have asked for quantitative evidence or a scoped claim. An editorial concern is reproducibility: the light-stage dataset is proprietary and no code release is mentioned, so outside groups cannot reproduce the results from the paper alone; a detailed configuration appendix would partially mitigate this. The reliance on Saito et al. 2024 is appropriate given the direct technical lineage, and the novelty over that work, namely ZH-based diffuse transfer, the shadow network, and deferred shading for specular transfer, is clearly delineated. The paper fits the scope of a graphics journal such as ACM TOG."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid, well-engineered extension of Saito et al. 2024 from head avatars to full bodies, and the first system I know of that does relightable body+face+hands in one 3DGS pipeline. The three new pieces—ZH-based diffuse transfer in local frames, a shadow network for non-local occlusion, and deferred shading for specular—are each motivated by a real problem, and each ablation supports its role. The ZH rotation argument is correct and the parameter savings are real (51 vs 113 per texel). The 1spp robustness check in the appendix is a nice touch.\n\nThe main soft spot is the novel-illumination claim. The abstract promises \"superior generalization ability under novel illumination conditions,\" but the quantitative evaluation is held-out point-light configurations from the same light stage. Environment-map relighting appears only qualitatively in Fig. 9. The stress-test concern is legitimate: Eq. 15 feeds the shadow network normalized irradiance (integral of L times visibility divided by integral of L), which is not the visibility function. Under a sparse set of point lights this is near-binary; under a smooth environment map it is a different input distribution. Nothing in the training protocol demonstrates that the CNN extrapolates across that gap, and the view-independent shadow map multiplying specular color is also a simplification. So the evidence supports relighting under held-out point lights, not the stronger claim.\n\nSecond soft spot: the baselines. They compare only to a PBR model they build on their own geometry, not to existing full-body neural relighting methods (Relighting4D, Animatable Gaussians relighting, IntrinsicAvatar). The PBR gap is expected. A comparison to a learned relighting baseline would calibrate how much of the gain is representation versus method. Also no error bars, and some metric gaps are small—Ours vs mesh-normal is 29.48 vs 29.43 PSNR. Minor, but worth noting.\n\nNone of this is fatal. The core method is sound, the ablations are honest, and the limitations section is candid. The paper deserves a serious referee. I would accept it conditionally: either add quantitative environment-map relighting results (or a held-out lighting condition not from the light-stage point-light family) and revise the generalization claim, or soften the claim. Ideally also add one learned full-body relighting baseline. I would bring it to a reading group; there is good discussion material here about what counts as illumination generalization.","headline":"Solid full-body relightable avatar system whose novel-illumination claim outruns the quantitative evidence; still worth a serious referee.","tokens_in":20659,"tokens_out":3232,"would_cite":true,"duration_ms":29889,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces the first relightable full-body avatar model that jointly handles the body, face, and hands, using 3D Gaussian splatting with light transport decomposed into local and non-local effects.","keywords":["relightable avatars","full-body avatars","3D Gaussian splatting","zonal harmonics","light transport","shadow network","deferred shading","neural rendering"],"falsifier":"Train the model on light-stage point lights whose directions cover only a limited angular range, then relight under an environment map containing a bright source from a direction outside that range and compare the rendered shadows to a path-traced reference. If shadow boundaries or darkness move to the wrong location, the irradiance normalization has not made the network illumination-invariant and the novel-illumination generalization claim fails.","tokens_in":19662,"feed_emoji":"💡","tokens_out":5451,"duration_ms":43015,"temperature":0.7,"pith_summary":"This paper claims that a full-body avatar—body, face, and hands together—can be made relightable and drivable by decomposing light transport into a local part and a non-local part. The local part is a diffuse radiance transfer learned in each Gaussian's own coordinate frame using zonal harmonics, which rotate cheaply when the body articulates. The non-local part is a learned shadow network that predicts occlusion shadows from a normalized irradiance map on a coarse tracked mesh. The specular part is rendered in screen space via deferred shading, capturing high-frequency reflections like eye glints without inflating the Gaussian count. If correct, this is the first method to jointly relight and animate the whole body from light-stage data at higher quality than a standard physically based rendering baseline.","feed_headline":"Full-body avatars relight under unseen poses and lights","feed_subtitle":"Zonal harmonics and a shadow network handle global illumination without ray tracing, beating a PBR baseline.","key_machinery":"The central machinery is the three-way decomposition of light transport. Zonal harmonics (spherical harmonics with one axis of symmetry) can be rotated to world space far more cheaply than full SH coefficients, so the model learns diffuse transfer per Gaussian in its local tangent frame and rotates it under articulation. The shadow network is a UV-space convolutional network, adapted from earlier full-body avatar work, that takes a normalized irradiance map (visibility-weighted irradiance divided by unshadowed irradiance, Eq. 15) on a coarse tracked mesh and predicts a single shadow factor per texel, rendering non-local occlusion. Deferred shading rasterizes per-Gaussian specular normals, roughness, and visibility to screen space, then integrates the environment against a spherical Gaussian lobe centered at the reflected view direction, so specular detail is not limited by local Gaussian density.","core_discovery":"On the paper's own terms, the central discovery is that full-body relighting can be factored into three learned components that each handle what they are good at: zonal-harmonic diffuse radiance transfer in local coordinates for orientation-dependent local shading, an irradiance-conditioned shadow network for non-local self-occlusion between body parts, and deferred-shading-based spherical-Gaussian specular transfer for high-frequency highlights. The paper demonstrates on captured light-stage sequences that this combination outperforms a physically based rendering appearance model and its own ablations (SH instead of ZH, no shadow network, no deferred shading) in perceptual quality, and that it generalizes to unseen poses and to held-out point-light configurations.","pith_inferences":["A stress test the paper leaves implicit: feed the shadow network an environment map with a sharp directional source from an elevation not present in the light-stage training distribution; the normalization in Eq. (15) is only as illumination-invariant as the coverage of training light directions, so failure would show up as a misplaced shadow boundary rather than a brightness error.","The ZH local-coordinate design suggests a path toward avatars that are relightable from monocular or sparse-view video: if rotation is the only pose-dependent operation, the same transfer coefficients could be trained with far fewer lighting conditions.","The deferred-shading specular branch could be extended to fit a fuller BRDF (normal, roughness, plus a second specular lobe) without changing the splatting, which might push eye and jewelry highlights closer to path-traced quality."],"forward_implications":["A full-body avatar can be relit and animated in near-real time without multi-bounce path tracing, since global illumination effects are captured by learned light transport rather than simulation.","The diffuse light transport parameter count drops from 113 to 51 per texel compared to the SH variant, while perceptual quality improves, making the representation more memory-efficient.","The shadow network generalizes to held-out point-light configurations and the method shows environment-map relighting qualitatively, which the authors take as evidence that the normalization scheme makes the network illumination-invariant.","Deferred shading recovers high-frequency specular effects such as eye glints that per-Gaussian shading misses when the Gaussian count is limited, closing a fidelity gap for full-body models.","The avatar can be animated to unseen poses and still render correctly under novel lighting, a combination that prior full-body methods did not jointly achieve."],"supporting_citations":[{"why":"Supplies the base relightable Gaussian codec-avatar framework and learned radiance transfer that this work extends to full-body articulation.","marker":"[Saito et al. 2024]"},{"why":"Provides the 3D Gaussian splatting representation used for geometry and alpha blending.","marker":"[Kerbl et al. 2023]"},{"why":"Introduces zonal harmonics for efficient rotation of precomputed radiance transfer, the basis of the diffuse transport model.","marker":"[Sloan et al. 2005]"},{"why":"Supplies the UV-space convolutional network architecture that the shadow network adapts for predicting non-local shadows.","marker":"[Bagautdinov et al. 2021]"},{"why":"Gives the spherical Gaussian specular light transport model used for deferred specular shading.","marker":"[Wang et al. 2009]"},{"why":"Demonstrates deferred shading with 3D Gaussian splatting, the technique adopted for screen-space specular rendering.","marker":"[Ye et al. 2024]"}],"fun_headline_variants":["Full-body avatars relight with decomposed light transport","Zonal harmonics plus shadow network for avatar relighting","Relightable avatars handle new poses and lighting","Deferred shading and ZH for full-body relighting","Avatar relighting that generalizes beyond training poses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The shadow network, trained on normalized irradiance computed from a coarse tracked mesh, must generalize to lighting conditions it never saw during training; the quantitative evidence covers held-out point-light configurations from the same light stage, while environment-map relighting is shown only qualitatively.","fun_headline_variants_meta":{"raw":{"variants":["Full-body avatars relight with decomposed light transport","Zonal harmonics plus shadow network for avatar relighting","Relightable avatars handle new poses and lighting","Deferred shading and ZH for full-body relighting","Avatar relighting that generalizes beyond training poses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000434,"raw_usage":{"total_tokens":2211,"prompt_tokens":943,"completion_tokens":1268,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":1193}},"tokens_in":559,"tokens_out":1268,"duration_ms":11203,"temperature":1.0,"reasoning_tokens":1193,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:51:05.236632+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the model on light-stage point lights whose directions cover only a limited angular range, then relight under an environment map containing a bright source from a direction outside that range and compare the rendered shadows to a path-traced reference. If shadow boundaries or darkness move to the wrong location, the irradiance normalization has not made the network illumination-invariant and the novel-illumination generalization claim fails.","supporting_citations":[],"review_version":1}