{"id":"67c1ef31-5a78-4119-a817-a51fb2c866a1","arxiv_id":"2507.18231","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PS-GS jointly estimates geometry, materials, and lighting from multi-view multi-light images by regularizing 2D Gaussian splatting with uncalibrated photometric stereo normals and directional-light ray-traced visibility.","lead":"PS-GS is a method that combines 2D Gaussian splatting with multi-view photometric stereo to reconstruct an object's shape, materials, and lighting from images taken under many different light directions. It promises much faster and more memory-efficient inverse rendering than prior NeRF-based approaches, with direct applications in relighting, novel-view synthesis, and material editing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The regularizer in Eq. (10) assumes UPS normals are metric camera-coordinate normals; uncalibrated PS has a GBR ambiguity the paper neither resolves nor tests, so the headline normal accuracy is conditional on that assumption.","rationale":"The reader's weakest_assumption matches the load-bearing point: Eq. (10) silently promotes UPS normals to metric truth. I agree that this is the central risk, so no new objection is needed. The concern is not that UPS is generally wrong; it is that uncalibrated PS is theoretically ambiguous up to the GBR family, and any data-driven resolution is a prior rather than a calibration. The paper's own ablation in Table 3 shows how much of the reported MAE gain comes from Ln,r, so a systematic bias in the guidance normals would propagate directly into geometry and undermine the main quantitative claim. This does not make the engineering contribution worthless; it makes the headline conditional on a verification the paper does not provide. The proposed concrete test uses the synthetic split, where ground truth is available, and quantifies both the GBR residual and its downstream effect. I therefore recommend keeping the CONDITIONAL verdict rather than accepting the headline claim as presently supported.","tokens_in":12111,"tokens_out":7179,"duration_ms":88051,"concrete_test":"On the synthetic PS-NeRF dataset, where ground-truth world normals are available, compute the UPS guidance normals G_v = Tc2w(Ne_v) for each training view. Fit the best per-view invertible 3x3 linear map from the GBR family that maps G_v to the ground-truth world normal field on masked pixels, and record the residual. Then rerun the full pipeline with G_v replaced by ground-truth world normals. If the fitted map is near identity and the residual is comparable to the reported normal MAE, the concern is refuted. If the GBR-corrected variant improves MAE materially (e.g. by more than 10% relative) or the fitted map deviates by more than a few degrees, the UPS bias is injected through Eq. (10) and the headline normal-accuracy claim is not independently supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation (10) applies the camera-to-world transform Tc2w to the UPS normals Ne and penalizes rendered normals against that target. This is only valid if Ne is expressed in the same metric camera coordinates as the posed images. Uncalibrated photometric stereo recovers normals only up to an unknown generalized-bas-relief (GBR) transform. Chen et al. 2019 is a deep uncalibrated PS method that may select one member of the GBR family, but the paper neither shows that this member is the metric one nor cites a GBR-resolution step. If the per-view transform G_v is not identity, the Ln,r term in both Stage I (Eqs. 9-10) and Stage II (Eq. 15) pushes the Gaussian tangent frames and positions toward GBR-warped normals. The reported normal MAE then measures how close UPS's GBR selection is to ground truth rather than the quality of PS-GS itself. Table 3 shows Ln,r is responsible for a large MAE improvement (7.02 -> 5.57), so this assumption is load-bearing: if it fails, the central claim of state-of-the-art normal accuracy is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents PS-GS, a two-stage inverse rendering method based on 2D Gaussian Splatting for multi-view photometric stereo. Stage I pretrains a standard 2DGS model on light-averaged images; Stage II jointly optimizes geometry, BRDF materials, and incident lighting using the rendering equation, an MLP-based lighting model, a normal regularization from uncalibrated photometric stereo (UPS) estimates, and a visibility-based lighting regularizer from 2D Gaussian ray-tracing adapted to single directional lights. The authors report improved normal accuracy, NVS/relighting quality, and lower training time and GPU memory relative to PS-NeRF, DPIR, R3DG, and IRGS on DiLiGenT-MV and PS-NeRF synthesis datasets.","tokens_in":12405,"tokens_out":5268,"duration_ms":60505,"significance":"If the results are reproducible, this is a useful and timely contribution: it appears to be the first Gaussian-splatting-based MVPS inverse rendering method, and the efficiency gains (0.7 hours, 6.5 GB GPU memory) over PS-NeRF are substantial. The explicit 2DGS representation also enables material and geometry editing. However, the headline claims are conditional on the UPS-estimated normals being metrically correct in camera coordinates, an assumption the paper neither justifies nor tests; the main quantitative claims therefore need additional evidence before they can be considered established.","major_comments":[{"comment":"The normal regularization Ln,r transforms the UPS normals Ne only by the rigid camera-to-world transform Tc2w and then penalizes the rendered normals against that target. This is valid only if Ne is expressed in the same metric camera coordinates as the posed images. Uncalibrated photometric stereo recovers normals only up to a generalized bas-relief (GBR) ambiguity, i.e., an unknown 3x3 linear transform per view; the cited Chen et al. 2019 deep UPS method may select one member of the GBR family, but the paper neither shows that this member is the metric one nor cites a GBR-resolution step. This assumption is load-bearing: Table 3 attributes a large part of the final normal accuracy to Ln,r (MAE 7.02 without it versus 5.57 with it), and Table 2 reports state-of-the-art MAE values against ground-truth normals. If the UPS normals are GBR-warped, the reported MAE partly measures the quality of the UPS prior rather than the contribution of PS-GS. The authors should either add a GBR-resolution/canonicalization step, or provide a concrete test on the actual benchmark objects (e.g., report the MAE of the raw UPS normals against ground truth after only a rigid alignment on the DiLiGenT-MV test views) to demonstrate that the assumption holds.","section":"Stage I: 2D Gaussian Pretraining, Eq. (10); Stage II loss, Eq. (15)"},{"comment":"The paper repeatedly defers implementation details to a supplementary document and ships no code, but the empirical claims depend on several unspecified choices: the loss weights lambda_n,c, lambda_n,r, lambda_o, lambda_inc, the MLP architecture and Fourier encoding for the incident lighting network, how the light direction omega_in is obtained for each multi-light image, the optimizer and learning-rate schedule, and the number of training iterations. Without these details, the quantitative results in Tables 1-3 cannot be reproduced or independently checked, and the sensitivity of the central regularizers to their weights cannot be assessed. The authors should include these details in the main paper or make the supplementary material and code available.","section":"Experimental"},{"comment":"All quantitative results are reported from single runs with no error bars or variance across seeds. Some comparisons are close (e.g., Bear PSNR 40.22 for DPIR versus 39.95 for PS-GS in Table 1, and Cow LPIPS 0.642 for DPIR versus 0.931 for PS-GS), so the state-of-the-art claims are not robustly supported without some measure of variability. The authors should report at least mean and standard deviation over multiple runs, or state if only one run was performed and temper the superiority claims accordingly.","section":"Tables 1 and 2"}],"minor_comments":[{"comment":"The loss term Ln,r is written as a sum of vectors without an explicit norm; it should specify L1 or L2 loss, otherwise the loss value is not well defined.","section":"Eq. (10)"},{"comment":"There are several typos and caption errors: 'devided' and 'rednering' in the Preliminary section, 'uesd' in the rendering equation text, and Table 2 is captioned 'Qualitative comparison' even though it reports quantitative MAE values. These should be corrected.","section":"Captions and text"},{"comment":"The rightmost columns 'GPU memory' and 'Time' are not normal-accuracy metrics and should be moved to a separate efficiency table or clearly labeled in the caption to avoid mixing metric types.","section":"Table 2"},{"comment":"The paper describes the rendering model as the 'full rendering equation without simplification,' but Eq. (14) accounts only for direct illumination from a single directional light; there is no interreflection or global illumination term. The authors should soften this claim or clarify that 'full' refers only to the local BRDF evaluation.","section":"Eq. (14) and contribution claim"}],"recommendation":"major_revision","confidential_remarks":"The primary technical risk is the GBR ambiguity in the UPS normals used by Eq. (10). If the authors can show on the DiLiGenT-MV benchmark that the raw UPS normals are metrically correct up to the rigid transform, or add an explicit GBR resolution step, I would be willing to accept a revised version. I also recommend that the editor verify that the referenced supplementary material actually contains the implementation details, since the current manuscript text relies on it for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a legitimate first integration of 2D Gaussian splatting with multi-view photometric stereo, and it delivers real efficiency gains plus good normal accuracy on public benchmarks. The soft spot is the UPS-normal regularizer: it assumes the estimated normals live in metric camera coordinates, and the paper neither resolves nor even cites the generalized bas-relief ambiguity that uncalibrated PS inherits. That makes the headline MAE numbers partly a statement about UPS's gauge choice, not purely about PS-GS.\n\nWhat's actually new: the 2DGS+MVPS combination itself, the SDL-specific 2D Gaussian ray-tracing that regularizes an incident-light MLP, and the two-stage pretrain-then-deferred-PBR pipeline. The experiments cover DiLiGenT-MV and the PS-NeRF synthesis dataset, with ablations showing each loss term earns its place. Training in 0.7h with 6.5GB GPU memory versus PS-NeRF's 22h/9.6GB is a meaningful practical win.\n\nThe stress-test concern lands. Equation (10) applies the camera-to-world transform to UPS normals, which is only valid if those normals are in true metric camera coordinates. Uncalibrated PS recovers normals up to an unknown 3x3 linear transform; Chen et al. 2019 self-calibrates lighting but still leaves that ambiguity. The paper doesn't discuss it. Table 3 shows removing the normal regularizer worsens MAE from 5.57 to 7.02, so the regularizer is doing heavy lifting. A proper revision should either cite a GBR-resolution step or empirically show invariance to the gauge. That said, the comparison to PS-NeRF is fair because PS-NeRF uses the same UPS normals, so the relative gains are not an artifact.\n\nOther soft spots are more minor: no code, no error bars, loss weights and MLP details deferred to an unavailable supplementary, and the phrase \"full rendering equation\" overstates what is actually a simplified Disney BRDF. None of these sink the paper; they are standard revision requests.\n\nWho is this for? Inverse rendering and 3D reconstruction researchers, especially anyone working on MVPS or Gaussian-based relighting. It deserves a serious referee: the integration is novel, the evaluations are on the right benchmarks, and the open problems are addressable. Send it to peer review with a request for major revision.\n\nRecommendation: engage with it, but expect the GBR issue to be the crux of any discussion.","headline":"First 2DGS+MVPS integration with strong benchmark numbers; the UPS-normal regularizer is load-bearing and its GBR ambiguity is unaddressed, so the accuracy claims are conditional.","tokens_in":12947,"tokens_out":1612,"would_cite":true,"duration_ms":19566,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PS-GS is the first Gaussian-splatting method for multi-view photometric stereo, jointly estimating geometry, materials, and lighting from multi-light images in about 0.7 hours while achieving state-of-the-art normal accuracy and enabling…","keywords":["multi-view photometric stereo","Gaussian splatting","inverse rendering","surface normal estimation","relighting","material editing","2D Gaussian ray-tracing","uncalibrated photometric stereo"],"falsifier":"A reader could run PS-GS on a synthetic object with ground-truth normals and deliberately feed the UPS guidance normal maps through a fixed, non-identity 3x3 linear transform (the generalized bas-relief ambiguity) before the alignment step. If the final reconstructed normals and mesh shift predictably with that transform, the assumption is load-bearing; if the reconstruction remains accurate, the optimization is absorbing the ambiguity elsewhere.","tokens_in":11921,"feed_emoji":"📐","tokens_out":7618,"duration_ms":68618,"temperature":0.7,"pith_summary":"The paper introduces PS-GS, an inverse-rendering method that combines 2D Gaussian splatting with multi-view photometric stereo to reconstruct an object's geometry, surface materials, and incident lighting from multi-view images captured under multiple directional lights. It claims to be the first Gaussian-splatting-based MVPS inverse-rendering approach, and reports state-of-the-art normal accuracy (for example, MAE 3.68 on the Bear object versus 4.86 for the DPIR baseline), top or near-top novel-view synthesis and relighting quality, and far lower cost than prior MVPS renderers: about 0.7 hours and 6.5 GB of GPU memory, versus over 22 hours and 9.6 GB for PS-NeRF. The practical payoff is that accurate, editable 3D models with fine surface detail can be produced quickly on modest hardware, and the method degrades gracefully with as few as five training views. The paper's route to that result is to regularize splat-rendered normals with uncalibrated photometric stereo normals and to regularize a neural lighting field with visibility computed by ray-tracing the Gaussian model under a single directional light.","feed_headline":"One pipeline recovers 3D shape, materials, and light in 42 minutes","feed_subtitle":"PS-GS fuses 2D Gaussian splatting and multi-light photometric stereo, beating prior normal accuracy at 6.5 GB memory.","key_machinery":"The load-bearing machinery is the 2D Gaussian primitive, whose surface normal is defined directly as the cross product of its two tangent vectors, giving per-primitive geometry that participates in alpha-blended feature maps; the full rendering equation with a simplified Disney BRDF (diffuse albedo, metallic, roughness) evaluated via deferred shading; an MLP that predicts incident radiance as a function of position, tangents, normal, and in/out directions with Fourier-encoded position; and two regularizers. The first regularizer is the UPS-estimated normal map, aligned to world coordinates by a fixed transform and compared with the rendered normal map at the pixel level. The second is visibility computed by a 2D Gaussian ray-tracing technique modified to sample a single directional light, used to constrain the MLP's predicted lighting. Together these components let multi-view and multi-light observations constrain material, shape, and lighting simultaneously.","core_discovery":"PS-GS's central claim is that 2D Gaussian splatting can carry the full inverse-rendering workload for multi-view photometric stereo without sacrificing the efficiency of splatting or the physical fidelity of a full rendering equation. After a standard 2DGS pretraining stage that fixes a reliable initial geometry, the method jointly optimizes per-Gaussian BRDF parameters (albedo, roughness, metallic), the Gaussian normals, and an MLP that predicts incident lighting, using deferred shading in image space for speed. Two regularizers anchor the otherwise ill-posed optimization: the rendered normal maps are pushed toward normals estimated by an uncalibrated photometric stereo method, and the MLP-predicted lighting is pushed toward visibility computed by a modified 2D Gaussian ray-tracer adapted to a single directional light. On the DiLiGenT-MV and PS-NeRF synthetic datasets, the method reports the best normal MAE on five of seven objects, the best PSNR on four, and the best SSIM on all seven, while completing training in 0.7 hours and 6.5 GB of GPU memory. The result, if accepted, means a single optimized model supports novel-view synthesis, relighting, and direct material and shape editing.","pith_inferences":["If the UPS normals are affected by the generalized bas-relief ambiguity, the fixed alignment transform could systematically skew the recovered geometry; a testable extension is to estimate that linear transform jointly instead of hard-coding it.","The two-stage design suggests a general recipe: pretrain a geometric splat model, then run deferred inverse rendering with external normal and visibility regularizers, which could transfer to other light configurations beyond single directional lights.","Because ray-tracing is performed only against the Stage-I geometry, large geometry changes in Stage II could make the visibility regularizer stale; re-running ray-tracing periodically would test whether that matters.","Scene-level effects such as inter-reflections between separate objects are out of scope; extending the method to multi-object scenes would require coupling the lighting MLP with light transport beyond the current single-directional-light visibility model."],"forward_implications":["Training time of about 0.7 hours and GPU memory of 6.5 GB make MVPS-based inverse rendering practical on a single modest GPU, a sharp drop from PS-NeRF's more than 22 hours.","The same optimized model supports novel-view synthesis, relighting, and material and shape editing by directly editing Gaussian primitive attributes.","Quality degrades gracefully with sparse views: the method still produces usable reconstructions with five training views.","Ablations show both regularizers matter: dropping the UPS normal regularization smooths away detail, and dropping the lighting regularization bakes lighting into albedo.","Relying on multi-light inputs rather than fixed environment illumination yields sharper shadows and more detailed normals than prior splatting-based inverse renderers."],"supporting_citations":[{"why":"Supplies the 2D Gaussian Splatting representation whose per-Gaussian normals and alpha-blending pipeline PS-GS builds on.","marker":"(Huang et al. 2024)"},{"why":"PS-NeRF, the MVPS inverse-rendering baseline that provides the UPS-normal regularization strategy and the synthetic dataset used for evaluation.","marker":"(Yang et al. 2022)"},{"why":"DPIR, the point-based MVPS baseline that PS-GS compares against and outperforms on normal accuracy and efficiency.","marker":"(Chung, Choi, and Baek 2024)"},{"why":"IRGS, the source of the 2D Gaussian ray-tracing technique that PS-GS modifies for single-directional-light visibility regularization.","marker":"(Gu et al. 2025)"},{"why":"Self-calibrating deep photometric stereo network used to estimate the guidance normals in the UPS step.","marker":"(Chen et al. 2019)"},{"why":"3D Gaussian Splatting, the base splatting framework that motivates the explicit Gaussian representation and real-time rendering.","marker":"(Kerbl et al. 2023)"},{"why":"DiLiGenT-MV benchmark dataset used for the real-world evaluation of normal accuracy, relighting, and novel-view synthesis.","marker":"(Li et al. 2020)"}],"fun_headline_variants":["PS-GS: Full inverse rendering with splatting in 0.7 hours","Gaussian splatting for multi-view photometric stereo done fast","Joint geometry, materials, and lighting via splatting","Multi-view photometric stereo with 2D Gaussian splatting","PS-GS: Efficient 3D reconstruction with physical lighting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The strongest load-bearing assumption is that the uncalibrated photometric stereo normals are recovered in true metric camera coordinates, so that the fixed camera-to-world transform in the normal regularization aligns them with the Gaussian model's world-space normals; if those normals carry an unknown linear ambiguity, that bias is injected directly into the optimized geometry.","fun_headline_variants_meta":{"raw":{"variants":["PS-GS: Full inverse rendering with splatting in 0.7 hours","Gaussian splatting for multi-view photometric stereo done fast","Joint geometry, materials, and lighting via splatting","Multi-view photometric stereo with 2D Gaussian splatting","PS-GS: Efficient 3D reconstruction with physical lighting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"total_tokens":1478,"prompt_tokens":1051,"completion_tokens":427,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":667,"completion_tokens_details":{"reasoning_tokens":335}},"tokens_in":667,"tokens_out":427,"duration_ms":4596,"temperature":1.0,"reasoning_tokens":335,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:39:13.823078+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could run PS-GS on a synthetic object with ground-truth normals and deliberately feed the UPS guidance normal maps through a fixed, non-identity 3x3 linear transform (the generalized bas-relief ambiguity) before the alignment step. If the final reconstructed normals and mesh shift predictably with that transform, the assumption is load-bearing; if the reconstruction remains accurate, the optimization is absorbing the ambiguity elsewhere.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 2D Gaussian Splatting representation whose per-Gaussian normals and alpha-blending pipeline PS-GS builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PS-NeRF, the MVPS inverse-rendering baseline that provides the UPS-normal regularization strategy and the synthetic dataset used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DPIR, the point-based MVPS baseline that PS-GS compares against and outperforms on normal accuracy and efficiency."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Self-calibrating deep photometric stereo network used to estimate the guidance normals in the UPS step."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"3D Gaussian Splatting, the base splatting framework that motivates the explicit Gaussian representation and real-time rendering."}],"review_version":1}