{"id":"78c10f56-c74a-4183-8c59-e59ee485d45c","arxiv_id":"2507.23162","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A single-stage neural inverse rendering pipeline jointly recovers geometry, spatially varying reflectance, and per-light direction and intensity from multi-view one-light-at-a-time images.","lead":"Researchers built a method that reconstructs a 3D object's shape, surface material, and lighting directions from many photos taken with one light on at a time, without requiring a calibration sphere or normal maps. It could make photometric 3D scanning practical with a handheld camera and flash.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Inter-reflection/directional-light assumption in Eq. (1) is the load-bearing risk: under model mismatch, shape, BRDF, and light estimates can co-degrade, as Sec. 4.2 admits; a synthetic global-illumination test would settle it.","rationale":"The reader's weakest_assumption matches the main correctness risk I see. The paper is honest about the limitation: Sec. 4.2 explicitly reports breakdown in concave reflective regions, and Eq. (3) already makes a simplified shadow approximation. The concern is therefore not an internal inconsistency but a boundary on the central claim. Since the abstract and Sec. 1 promise handling of 'challenging geometry and reflectance' and real view-unaligned capture, the absence of any quantitative test with indirect illumination means the advertised scope is not yet evidenced. The synthetic global-illumination-versus-direct test would either confirm that boundary or show the method tolerates modest inter-reflections. I do not see a reason to move the verdict: the benchmark results, ablations, and released code support the core single-stage joint optimization claim under the stated direct-illumination assumption. The reader's CONDITIONAL verdict is appropriate; my stress test would keep it unchanged.","tokens_in":17697,"tokens_out":9326,"duration_ms":118785,"concrete_test":"Render synthetic concave objects with known GT geometry, material, and light, using path tracing with full global illumination (including inter-reflections and color bleeding) and also with direct-only illumination of the same scenes. Run the released code on both renderings, using identical masks and camera poses, and compare estimated mesh Chamfer distance, normal MAE, light-direction MAE, and intensity SI-MSE to GT. If errors are substantially larger on the global-illumination renderings and grow with concavity or reflectance, the inter-reflection limitation is load-bearing; if the method remains within benchmark-level error despite indirect light, the assumption is not the critical risk.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that geometry, SVBRDF, and per-light directions/intensities can be recovered from raw OLAT pixels by single-stage inverse rendering—rests on Eq. (1), which models each view as direct illumination from one directional light. Three model choices inherit this assumption: Eq. (4) multiplies the whole volume-rendered radiance by a single surface shadow factor; Eq. (13) lets the shadow MLP refine that factor into s' in [0,1]; and no term accounts for inter-reflections. When the assumption fails, as in concave reflective regions (the paper's own Sec. 4.2 and Fig. 12), indirect light is not an additive field that the model can separate. It is channeled through s' and through errors in the BRDF, latent, geometry, and light estimates. There is no mechanism in L_color to distinguish 'the surface is brighter because of indirect light' from 'the light is stronger, the BRDF is more specular, or the normal is tilted.' Consequently, geometry, reflectance, and lighting can co-degrade while still reproducing the input images. The concern is not that DiLiGenT-MV violates the assumption badly—directional lighting is a reasonable approximation there—but that the paper's broader claims about 'challenging geometry and reflectance' and real-world view-unaligned capture inherit this boundary without any quantitative characterization of how much model mismatch the optimization tolerates.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a neural inverse rendering method for multi-view photometric stereo (MVPS) that jointly reconstructs geometry, spatially varying reflectance, and per-light direction and intensity from raw one-light-at-a-time (OLAT) images in a single-stage optimization. Geometry is represented as a neural SDF with a per-point BRDF latent code, reflectance is predicted by a latent-driven BRDF MLP, and lighting is parameterized as per-light directions and RGB intensities; a shadow MLP refines volume-rendered shadow factors. The method is evaluated on DiLiGenT-MV for shape, normal, lighting, and test-view rendering quality, and on a new self-captured view-unaligned OLAT dataset. The central claim is that discarding intermediate photometric-stereo cues and light calibration, and optimizing all scene parameters directly against pixels, outperforms normal-guided MVPS baselines in shape and lighting accuracy while also supporting sparse lighting and view-unaligned capture.","tokens_in":18009,"tokens_out":6956,"duration_ms":80885,"significance":"If the result holds, the paper is a useful step for MVPS: it removes the need for light calibration and per-view normal maps, demonstrates that a neural latent-driven BRDF can be trained from scratch from OLAT images, and provides a practical view-unaligned capture protocol. The authors are commendably explicit about the main limitation (inter-reflections, Sec. 4.2) and release code. The evaluation uses a public benchmark with external ground-truth meshes and calibrated lights, and the derivation is not circular: all scene parameters are optimized against raw pixels and evaluated against external references. However, several load-bearing empirical claims lack support: no error bars or multiple seeds, no quantitative evaluation of light intensities, no quantitative reflectance error, and no controlled test of the directional-light / no-interreflection assumption that underlies the forward model.","major_comments":[{"comment":"The forward model assumes a single directional light and neglects inter-reflections, and the paper's own Sec. 4.2 reports that the method breaks down in concave regions of reflective surfaces. Because Eq. (14) multiplies the entire volume-rendered color by a single MLP-refined shadow factor s' from Eq. (13), indirect light has no dedicated channel; it can be absorbed by s', by the BRDF latent code, or by the estimated light intensities. Under model mismatch, geometry, reflectance, and lighting can co-degrade while still fitting the input images. The paper should add a synthetic experiment with known ground truth and path-traced global illumination to quantify how much model mismatch the optimization tolerates, or at least provide a quantitative analysis of the failure mode in Fig. 12. This is load-bearing for the abstract's claims about 'challenging geometry and reflectance' and real-world applicability.","section":"Sec. 3.1, Eq. (1); Sec. 4.2, Fig. 12"},{"comment":"The lighting evaluation reports only direction MAE, yet the abstract and conclusion claim recovery of per-light intensities (relative intensities). The supplementary defines scale-invariant MSE (SI-MSE) in Eq. (22), but no SI-MSE results are reported anywhere in the paper. Since DiLiGenT-MV provides calibrated light intensities via a white board, the authors should report SI-MSE for the configurations in Table 1 and for the ablation in Table 2; without this, the intensity component of the lighting-recovery claim is unsupported.","section":"Sec. 4.1.3, Table 1; Supplementary Eq. (22)"},{"comment":"The central empirical claim is that the method 'outperforms' baselines across lighting counts, but no error bars or multiple seeds are reported. Some comparisons are close (e.g., Fig. 4 CD curves for bear and pot2 at dense lighting, and several light-direction MAE differences in Table 1 below 0.3 degrees). A single run cannot support fine-grained comparative claims. At minimum, three seeds with mean and standard deviation for the headline metrics (CD, normal MAE, light MAE) should be reported for the main configurations, or the claims should be scoped to qualitative trends.","section":"Sec. 4.1.1, Fig. 4; Sec. 4.1.3, Table 1"},{"comment":"Reflectance recovery is evaluated only through qualitative BRDF latent maps and BRDF spheres, plus test-view PSNR, where DPIR actually achieves higher PSNR than the proposed method. The explanation that weighted-L1 loss is less correlated with PSNR is plausible, but no quantitative reflectance error (e.g., albedo or BRDF error on a synthetic object with known material, or a perceptual metric such as LPIPS on held-out renderings) is provided. Since the abstract explicitly claims recovery of spatially varying reflectance, the paper should either add a quantitative reflectance metric or clearly scope the claim to 'plausible relighting' rather than 'accurate SVBRDF'.","section":"Sec. 4.1.2, Fig. 8"}],"minor_comments":[{"comment":"The y-axis labels in Fig. 4 are difficult to read or missing; the top row should clearly say 'Chamfer distance (mm)' and the bottom row 'Normal MAE (deg)', and the caption should state the units for each row.","section":"Fig. 4"},{"comment":"The column header 'R EADING' contains an unintended space and should be 'READING'.","section":"Table 1"},{"comment":"The text contains typos: 'stobes' should be 'strobes', and '1 maway' should be '1 m away'.","section":"Supplementary Sec. 7.1"},{"comment":"The sentence 'the bar over a vector indicates it is normalized' is confusing because the normalized gradient is written with the same symbol as the unnormalized one; please define explicitly, e.g., n = ∇g / ||∇g||.","section":"Sec. 3.2, Eq. (7)"},{"comment":"The halfway vector is defined as h = ℓ + v, but it is not normalized; since the angular encoding uses n^T h, the unnormalized magnitude affects the feature value. Please state whether h is meant to be normalized, and if not, justify why the unnormalized sum is used.","section":"Sec. 3.2, Eq. (10)"},{"comment":"The ablation study is performed on a single scene (POT2 with 32L18V); the main text should state this limitation, or add a second scene, before drawing general conclusions about the loss and angular encoding.","section":"Sec. 4.1.4, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is sound and the benchmark results are promising, but the empirical support needs strengthening in several places that are central to the claims: variance reporting, intensity-error quantification, reflectance-error quantification, and a controlled test of the interreflection/directional-light assumption. These are all fixable within the manuscript's scope. The novelty relative to DPIR is moderate but genuine: the method removes light calibration and uses an SDF-based volume renderer with a learned shadow factor. The authors are transparent about the main failure mode, which is a positive sign. Fit with a CV/geometry venue is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Paper worth a serious look. The genuinely new thing is that it runs MVPS as one joint optimization over raw OLAT pixels: SDF geometry, a latent-driven BRDF trained from scratch, per-light directions and intensities, and a shadow refinement MLP, with no per-view normal maps and no light calibration. That is a real step beyond DPIR, which is end-to-end but needs known light directions, and beyond PS-NeRF and SVNL, which lean on normal or shadow cues. On DiLiGenT-MV the method beats normal-guided SuperNormal on three of five objects, holds up as lighting drops to one or two sources, and estimates light directions more accurately than PS-NeRF across the board. Code on a public benchmark makes the main experiments re-runnable. The paper is also honest: it shows a failure case in concave reflective regions and attributes it to unmodeled inter-reflections.\n\nThe soft spots are real but not fatal. The abstract says \"outperforms\" state-of-the-art normal-guided approaches; Fig. 4 shows SuperNormal wins on Bear and Pot2 under dense lighting, so the claim should be qualified. There are no error bars or multiple seeds, and the ablation is a single scene. The reflectance evaluation is the weakest part: DPIR gets higher PSNR on held-out views, and the \"perceptually more accurate\" claim is not backed by any user study. The real-world view-unaligned results are qualitative only, with no baseline and no released data, so the capture-protocol benefit is demonstrated but not measured. The stress-test concern about Eq. (1) is accurate: with only one directional light and no inter-reflections, model mismatch can co-degrade geometry, BRDF, and lighting. The paper's own limitation section confirms this. It does not sink the benchmark results, because DiLiGenT-MV is close to directional with weak inter-reflections, but it should bound the claims about \"challenging geometry and reflectance.\"\n\nI do not see circularity. Evaluation uses external scanned ground truth and held-out views and lights, so predictions are not fitted values masquerading as tests. The math is straightforward and clearly presented.\n\nThis is a paper for the MVPS and inverse-rendering crowd. It deserves a serious referee, and I would accept it for review with a request for seeds and error bars, a synthetic global-illumination stress test, and quantitative real-world evaluation. I would cite it.","headline":"A genuinely end-to-end MVPS method that mostly delivers on its central claim; referee it, but ask for seeds, error bars, and a global-illumination stress test.","tokens_in":18591,"tokens_out":3134,"would_cite":true,"duration_ms":34813,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Multi-view photometric stereo can be solved as a single joint optimization, recovering shape, material, and uncalibrated per-light directions from raw images alone.","keywords":["multi-view photometric stereo","neural inverse rendering","signed distance field","neural BRDF","self-calibrated lighting","view-unaligned capture","shadow-aware volume rendering","raw OLAT images"],"falsifier":"Render a synthetic concave reflective object with a renderer that includes inter-reflections, using known geometry, material, and lighting, then run the method on those images: if reconstruction errors concentrate in the concavity and grow with the ratio of indirect to direct radiance, the single-reflection model of Eq. (1) is the failing component.","tokens_in":17425,"feed_emoji":"💡","tokens_out":8464,"duration_ms":87745,"temperature":0.7,"pith_summary":"Multi-view photometric stereo is usually solved in stages: calibrate the lights with special objects, estimate per-view normal maps, then fuse them into a shape. This paper claims all of that can be skipped. It presents a neural inverse rendering method that takes raw one-light-at-a-time (OLAT) images from multiple views and jointly optimizes a signed distance field, a spatially varying neural reflectance model, and per-light directions and relative intensities in one stage. If the claim holds, 3D scanning under varying illumination no longer needs light calibration or intermediate normal estimation, and capture becomes flexible enough for camera-light rigs whose viewpoints are not aligned across lights.","feed_headline":"One stage recovers shape, material, and lights from raw photos","feed_subtitle":"No normal maps, no light calibration: raw multi-light images alone drive the reconstruction.","key_machinery":"The load-bearing object is the single-reflection image formation model of Eq. (1), $r = e f(x, n, v, \\ell)(n^\\top \\ell)_+$, rendered through shadow-aware volume rendering. A spatial multilayer perceptron with multi-resolution hash encoding outputs the signed distance and a reflectance latent code per point; a reflectance (bidirectional reflectance distribution function, BRDF) multilayer perceptron predicts RGB BRDF values from that latent code and the angular encoding $A(n, v, \\ell) = [n^\\top h, \\ell^\\top h, n^\\top \\ell, n^\\top v, (n^\\top h)^{10}]$ with $h = \\ell + v$; and a shadow multilayer perceptron refines SDF-transmittance shadow factors into smooth values that tolerate inter-reflection fill. The joint optimization has one channel through which photometric information flows into geometry: the spatial MLP's analytic SDF gradient acts as the surface normal in the BRDF evaluation, so raw pixel loss supervises shape directly.","core_discovery":"The central claim is that multi-view photometric stereo can be solved as one inverse rendering problem, without photometric stereo cues. Given posed multi-view OLAT images and foreground masks, the method optimizes a neural signed distance field for geometry, a latent-driven reflectance network for spatially varying material, per-light camera-space directions and RGB intensities, and a shadow-refining network, all from raw pixel values under a weighted L1 loss. On a standard multi-view photometric stereo benchmark, it reports lower surface-distance error and normal-angle error than stage-by-stage normal-guided baselines and point-based inverse rendering, and lower light-direction error than a baseline that initializes lighting from a pretrained network, while remaining accurate with as few as one light per view. The same pipeline is demonstrated on self-captured view-unaligned OLAT images of reflective objects, where viewpoints differ from light to light.","pith_inferences":["Editorial inference: recovered camera-space light directions could serve as a calibration signal for structure-from-motion or hand-eye calibration, since a light rigidly fixed to the camera gives each frame a known directional reference.","Editorial inference: the tolerance for view-unaligned capture suggests a handheld camera-and-strobe rig could replace turntable capture, because object rotation no longer has to be synchronized with light switching.","Editorial inference: a direct stress test is to replace the directional-light assumption with an inverse-square point light model; if the joint optimization fails to separate distance falloff from BRDF, that would identify the lighting model as the next bottleneck.","Editorial inference: because the angular encoding is rotation-invariant, the reflectance network is effectively learning a local-coordinate material model, which raises a testable question of whether latent codes transfer across scenes that share a material."],"forward_implications":["Light calibration objects such as chrome spheres and white boards can be dropped; each light's direction and relative intensity is an optimizable parameter recovered from pixels.","The capture rig no longer needs aligned viewpoints across lights: a light can stay fixed to the camera while the object rotates, and the next light starts a fresh multi-view pass.","The method remains accurate when lighting variation is sparse, down to a single light per view, a regime where per-view normal-map estimation degrades.","Because material is represented by a per-point latent code instead of a fixed analytic model, the same pipeline handles diffuse, ceramic, metallic, and hybrid surfaces, and supports unshadowed rendering and relighting."],"supporting_citations":[{"why":"Supplies the benchmark dataset with 20 viewpoints, 96 lights, calibrated ground-truth lighting, and scanned meshes used for all quantitative geometry and lighting comparisons.","marker":"[27]"},{"why":"The stage-by-stage normal-guided MVPS baseline the method is compared against and reported to outperform in shape and normal accuracy.","marker":"[8]"},{"why":"The end-to-end point-based inverse rendering baseline that avoids intermediate PS cues but needs known light directions and intensities.","marker":"[13]"},{"why":"Neural inverse rendering baseline for light-direction estimation, which uses pretrained initialization for lighting.","marker":"[49]"},{"why":"Per-view photometric stereo estimator whose normal maps feed the normal-guided baseline, anchoring the established PS-cue pipeline.","marker":"[20]"},{"why":"Supplies the SDF-to-opacity conversion used for shadow-aware volume rendering.","marker":"[46]"},{"why":"Multi-resolution hash encoding provides the spatial MLP's input features for high-frequency detail and training efficiency.","marker":"[37]"},{"why":"Motivates the weighted intensity loss, adapted here to weighted L1 for OLAT inputs.","marker":"[35]"}],"fun_headline_variants":["Single-stage neural inverse rendering for photometric stereo without cues","No normal maps or calibration: one-stage photometric stereo","End-to-end multi-view photometric stereo without photometric cues","Unified neural inverse rendering for multi-view photometric stereo","Joint shape, material, and light from raw multi-view images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every pixel records one surface point lit by one directional source, with inter-reflections and ambient light negligible; the paper itself reports that concave regions of reflective surfaces break under this assumption.","fun_headline_variants_meta":{"raw":{"variants":["Single-stage neural inverse rendering for photometric stereo without cues","No normal maps or calibration: one-stage photometric stereo","End-to-end multi-view photometric stereo without photometric cues","Unified neural inverse rendering for multi-view photometric stereo","Joint shape, material, and light from raw multi-view images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000181,"raw_usage":{"total_tokens":1266,"prompt_tokens":860,"completion_tokens":406,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":323}},"tokens_in":476,"tokens_out":406,"duration_ms":4627,"temperature":1.0,"reasoning_tokens":323,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T11:00:31.709585+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a synthetic concave reflective object with a renderer that includes inter-reflections, using known geometry, material, and lighting, then run the method on those images: if reconstruction errors concentrate in the concavity and grow with the ratio of indirect to direct radiance, the single-reflection model of Eq. (1) is the failing component.","supporting_citations":[{"cited_title":"Multi-view photometric stereo: A ro- bust solution and benchmark dataset for spatially varying isotropic materials","cited_arxiv_id":null,"evidence_quote":"Supplies the benchmark dataset with 20 viewpoints, 96 lights, calibrated ground-truth lighting, and scanned meshes used for all quantitative geometry and lighting comparisons."},{"cited_title":"Supernormal: Neural sur- face reconstruction via multi-view normal integration","cited_arxiv_id":null,"evidence_quote":"The stage-by-stage normal-guided MVPS baseline the method is compared against and reported to outperform in shape and normal accuracy."},{"cited_title":"Differentiable point-based inverse rendering","cited_arxiv_id":null,"evidence_quote":"The end-to-end point-based inverse rendering baseline that avoids intermediate PS cues but needs known light directions and intensities."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Neural inverse rendering baseline for light-direction estimation, which uses pretrained initialization for lighting."},{"cited_title":"Scalable, detailed and mask-free universal photometric stereo","cited_arxiv_id":null,"evidence_quote":"Per-view photometric stereo estimator whose normal maps feed the normal-guided baseline, anchoring the established PS-cue pipeline."},{"cited_title":"NeuS: Learning neural im- plicit surfaces by volume rendering for multi-view recon- struction","cited_arxiv_id":null,"evidence_quote":"Supplies the SDF-to-opacity conversion used for shadow-aware volume rendering."},{"cited_title":"Instant neural graphics primitives with a multires- olution hash encoding","cited_arxiv_id":null,"evidence_quote":"Multi-resolution hash encoding provides the spatial MLP's input features for high-frequency detail and training efficiency."},{"cited_title":"Srinivasan, and Jonathan T","cited_arxiv_id":null,"evidence_quote":"Motivates the weighted intensity loss, adapted here to weighted L1 for OLAT inputs."}],"review_version":1}