{"id":"3cf02e9b-c05d-489c-b4a2-54ea242ba2e8","arxiv_id":"2411.17235","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MLI-NeRF generates physics-based pseudo reflectance and shading labels from multi-light images to train an intrinsic-aware neural radiance field without ground truth intrinsic data.","lead":"This paper trains a neural radiance field to decompose images into reflectance and shading by using photos taken under multiple light positions as free supervision. The result is an editable 3D scene model that can be relit or recolored without hand-labeled ground truth.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is load-bearing on Eq. 5's direct-Lambertian shading model: pseudo-labels ignore indirect/specular light at Step C (R = I/S), yet real-scene results are only qualitative, so the synthetic win may not transfer.","rationale":"The reader's verdict is CONDITIONAL, and I agree with its weakest_assumption. The strongest claim is that MLI-NeRF generates physics-based pseudo-labels that train a disentangled decomposition without GT and outperforms SOTA on synthetic and real datasets. This is a pipeline claim: pseudo-labels are the only supervision for intrinsic separation in Stage 2. If the pseudo-shading formula (Eq. 5) is wrong for the capture conditions, the pseudo-reflectance labels are corrupted by the ignored residual, and Stage 2 cannot correct the error because its residual term is defined as the leftover of I−R⊗S and is only L1-regularized, not physically modeled. I considered other potential issues—the under-specified K-means merging, the unverified code link, and the unequal baseline settings—but they are less central: the K-means details are implementation choices, the code is promised, and the baseline inequality undermines only the magnitude, not the direction, of the improvement. The direct-Lambertian concern is more structural because it attacks the mechanism by which the method claims to generalize to real scenes. The concrete test I propose isolates this mechanism by adding global illumination to the synthetic benchmark; if the method's advantage survives, the concern is resolved, and if not, the paper should be revised to state the assumption and either add indirect-light awareness or restrict claims to direct-light scenes. This is an UNCHANGED situation: the reader's conditional verdict already captures the needed revisions, and my stress-test does not move it to accept or reject.","tokens_in":6,"tokens_out":6616,"duration_ms":128241,"concrete_test":"Render the four synthetic scenes twice in Blender with identical geometry, materials, cameras, and light positions: one pass with direct illumination only and one pass with full global illumination (interreflections and, optionally, a small specular lobe). Run the complete MLI-NeRF pipeline on both passes and compare predicted reflectance/shading against the shared ground truth. If the full-GI pass degrades reflectance or shading PSNR by more than ~2 dB relative to the direct-only pass, or if the L1 norm of the recovered residual grows by a comparable margin, then the Eq. 5 + Step C pseudo-label assumption is the active bottleneck and the real-scene claim is not yet supported. If the two passes match, the direct-Lambertian concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that physics-based pseudo-labels from multiple light positions replace statistical priors and yield accurate intrinsic decomposition on real scenes. That claim depends on the pseudo-shading model in Eq. 5, S* = (max(N·L,0)⊗V)^γ, being an adequate description of the captured shading. Eq. 1 allows a residual Re for specular/indirect light, but Step C explicitly computes R* = I ⊘ S* 'as a simplified version of Eq. (1) ignoring at this point the residual component.' Thus when Re ≠ 0, the pseudo-reflectance is R* = R + Re/S*, so any interreflection, specular highlight, or soft-shadow/penumbra effect is baked into the label. The K-means merge over multiple lights only protects pixels where most lights have small Re; with broad-area lights or strong indirect bounce, contamination can be present under every light. Stage 2 is then trained to match these corrupted labels, with the residual term defined as I − R⊗S and regularized by L1, so the optimization can push modeling error into R and S rather than expose it. The quantitative synthetic evaluation (Tab. 1) uses scenes generated under the same direct-light protocol, so it does not falsify this concern; the ReNe real-scene evaluation (Sec. 4.6, Fig. 6) is qualitative only, with no quantitative intrinsic metrics. The paper's limitation section lists light-position and training-cost limitations but not this validity boundary. Consequently, the claimed robustness to 'challenging real-world data' is currently unverified at exactly the point where the method's physics assumption is weakest.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MLI-NeRF, a two-stage neural radiance field method for intrinsic image decomposition (reflectance, shading, residual) that uses images captured under multiple known light positions. Stage 1 trains a relightable Neuralangelo-style SDF network conditioned on light position. Post-processing derives pseudo-shading from normals, light visibility (via sphere tracing), and a gamma-corrected Lambertian term, and pseudo-reflectance by dividing the input image by that pseudo-shading and merging across lights with K-means and hole-filling. Stage 2 freezes the geometry and adds reflectance and shading MLPs, supervised by the pseudo-labels with weight maps and a residual regularization term. The authors evaluate on a synthetic Blender dataset, a real object relighting dataset, and the real ReNe dataset, reporting strong quantitative results on synthetic intrinsic decomposition and NVS/relighting, and qualitative results on real scenes, plus editing applications.","tokens_in":15471,"tokens_out":2817,"duration_ms":28742,"significance":"If the pseudo-label strategy is robust, MLI-NeRF would be a meaningful advance: it replaces statistical priors for intrinsic decomposition with multi-light physics-based supervision inside a NeRF, and it targets full scenes rather than isolated objects. The paper includes a public code release, re-rendered synthetic data with ground truth intrinsic images, and comparisons against several baselines across multiple light settings. The main significance is therefore conditional on whether the direct-Lambertian pseudo-shading model is accurate enough for real scenes and whether the reported synthetic gains transfer beyond the same rendering protocol.","major_comments":[{"comment":"The pseudo-reflectance label is computed as R* = I ⊘ S* while explicitly ignoring the residual term in Eq. (1). Under the paper's own model I = R⊗S + Re, this yields R* = R + Re/S*, so any specular highlight, interreflection, or penumbra that the direct-Lambertian shading model (Eq. 5) does not capture is baked into the pseudo-reflectance label. The synthetic evaluation in Table 1 uses scenes rendered under the same direct-light, gamma-corrected protocol, so it does not test this contamination, and the ReNe experiments in Sec. 4.6 report only qualitative intrinsic results. To support the claimed robustness on real-world data, the authors should provide either quantitative intrinsic evaluation on real scenes with measured or known albedo (e.g., OpenIllumination-style objects) or a controlled synthetic experiment with indirect illumination and specular materials, showing that the pseudo-labels and final decomposition remain accurate when Re is nonzero.","section":"§3.3, Eq. (5), Step C"},{"comment":"The pseudo-label generation pipeline relies on several under-specified heuristics: the gamma exponent γ in Eq. (5), the number of K-means clusters and initialization used for the pixel-level merge, the distance weights in the hole-filling strategy, and the construction of the weight maps W_R and W_S. These are load-bearing because they directly determine the supervision targets in Eq. (7), yet no values, sensitivity analysis, or ablation are reported. The authors should state all hyperparameters and provide an ablation showing how pseudo-label quality and final decomposition metrics vary with these choices, at least for the number of lights used in the merge and the gamma value.","section":"§3.3, Step C and §3.4, Eq. (7)"},{"comment":"There is a self-distillation aspect to the training: the pseudo-shading S* is computed from the Stage-1 SDF normals and visibility, and Stage 2 trains a shading MLP on the same frozen geometry to reproduce S*. The pseudo-reflectance is then I ⊘ S*, so Stage 2 largely learns a function of Stage 1's geometry rather than being independently supervised. The paper does not report how accurate the pseudo-labels themselves are on the synthetic dataset (e.g., PSNR/LPIPS of S* and R* against the re-rendered ground truth), nor whether Stage 2 actually improves over the pseudo-labels. Adding these numbers would clarify how much of the final performance comes from the pseudo-label generation versus the Stage-2 network and would strengthen the claim that the decomposition is genuinely learned rather than inherited from the geometry.","section":"§3.4, Stage 2"},{"comment":"The ReNe experiments are the primary evidence for real-world full-scene performance, but the intrinsic decomposition results there are qualitative only; Table 3 reports NVS metrics and does not include reflectance or shading error. Since the test set is not public and the validation set is used for inference, the authors should at least provide a quantitative intrinsic evaluation on a publicly available real multi-light dataset or on a held-out portion of ReNe with manual annotation, if available. As written, the claim of 'superior performance in intrinsic decomposition on real scenes' is not backed by numbers.","section":"§4.6, Fig. 6 and Table 3"}],"minor_comments":[{"comment":"In the multiple-light track, TensoIR is run with three lights while MLI-NeRF uses four, because TensoIR 'occasionally failed' with four; this should be stated as a limitation in the comparison fairness rather than only in the text.","section":"§4.2, 'Additional Lighting Setups'"},{"comment":"The paper calls γ 'gamma correction' but does not specify whether it is a fixed constant (e.g., 2.2) or a learned/estimated value; please clarify and justify the choice.","section":"§3.3, Eq. (5)"},{"comment":"The loss weights weik, wcurv, wintrinsic, wreg are listed with values, but there is no discussion of how they were chosen or whether results are sensitive to them; a brief note or supplementary ablation would help.","section":"§4.3 and §3.2"},{"comment":"Some labels are inconsistent: 'NRHInts' appears in Fig. 5 and the supplementary, and 'MLPref lectance' appears in Sec. 3.4; please unify the notation.","section":"Fig. 2 and Fig. 6 captions"},{"comment":"The NVS/relighting comparison in the Random setting reports slightly lower PSNR/SSIM for MLI-NeRF than NRHints but better LPIPS; the text says 'comparable performance,' which is fair, but the shading and reflectance errors are not reported there, so the intrinsic decomposition contribution to relighting quality is not visible.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible and well-scoped contribution, but the central claim of robust real-world intrinsic decomposition is currently supported mainly by synthetic experiments and qualitative real results. The load-bearing assumptions in Eq. (5) and Step C need either a quantitative real-data validation or a controlled synthetic stress test. I also note that the comparison with TensoIR in the multiple-light setting uses different numbers of lights, and the pseudo-label hyperparameters are not fully documented; these are fixable within a revision, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read MLI-NeRF. The core idea is genuinely new: instead of statistical priors, multiple light positions generate physics-based pseudo-labels for reflectance and shading, then train an intrinsic-aware NeRF. The synthetic results in Table 1 are strong across single, multiple, and random light settings, and the code/data are public. That's a practical, well-motivated contribution.\n\nThe two-stage pipeline is clearly described. Stage 1 extends Neuralangelo with light input; post-processing derives normals, visibility, and gamma-corrected pseudo-shading, then merges pseudo-reflectance via K-means across lights. Stage 2 correctly restricts light input to the shading MLP, so reflectance is illumination-invariant. The editing applications are a nice plus.\n\nThe main soft spot is exactly what the stress-test note flags: the pseudo-label generation assumes direct Lambertian shading. Equation 5 uses S* = (max(N·L,0)⊗V)^γ, and Step C computes R* = I ⊘ S*, explicitly ignoring the residual term. Any indirect light, specularity, or penumbra gets baked into R*. In real scenes with interreflections that is a real contamination risk, and the ReNe evaluation is qualitative only. The synthetic scenes are rendered under the same direct-light model, so they do not test this boundary. The limitation section mentions light positions and training cost but not this validity assumption. That is the biggest gap.\n\nSecond, the TensoIR comparison is slightly unequal: TensoIR used three lights when four were intended because it 'occasionally failed.' Minor, but disclosed.\n\nThird, several heuristics are under-specified: the gamma value, K-means merge details, hole-filling weights, and the weight maps WR/WS. These parameters could affect results substantially, and they are not reported. A reader cannot reproduce the pseudo-labels from the description alone.\n\nFourth, there is a self-distillation aspect: Stage 2 uses the same frozen geometry from Stage 1 for both pseudo-labels and predictions, so geometry errors can propagate. The residual regularization helps, but this is worth stating clearly.\n\nOverall, this is a solid, useful paper for anyone working on intrinsic decomposition or neural relighting. It deserves peer review. I would recommend accept with major revisions: add ablations on the pseudo-label heuristics, report quantitative intrinsic metrics on real scenes where possible (or at least a more systematic qualitative evaluation), and address the indirect-light assumption directly.","headline":"Genuinely new multi-light pseudo-label approach with impressive synthetic results; the real-world claim rests on a direct-Lambertian assumption that is not yet tested quantitatively.","tokens_in":16018,"tokens_out":2722,"would_cite":true,"duration_ms":26155,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Multiple light positions teach a neural radiance field to separate reflectance from shading without ground-truth intrinsic images, using physics-based pseudo labels.","keywords":["neural radiance fields","intrinsic image decomposition","reflectance","shading","multi-light relighting","pseudo-label generation","neural rendering","scene editing"],"falsifier":"A concrete test that would falsify the central claim: run MLI-NeRF on a real scene containing a visible glossy or mirror-like surface under the multi-light capture setup, and check whether the predicted reflectance contains specular highlights or color from the environment, and whether the residual term remains large. If the residual absorbs a large fraction of the image energy or the reflectance inherits lighting-dependent colors, the direct-Lambertian pseudo-shading assumption fails.","tokens_in":14919,"feed_emoji":"🔦","tokens_out":2859,"duration_ms":20948,"temperature":0.7,"pith_summary":"The paper tries to establish that the ill-posed problem of intrinsic image decomposition can be solved by using a neural radiance field that sees the scene under several known light positions. The core claim is that the extra lighting information substitutes for ground-truth intrinsic images: the method generates physics-based pseudo reflectance and shading labels from a relightable NeRF, then retrains the network with dedicated reflectance and shading modules supervised by those labels. If this works, intrinsic decomposition becomes feasible on real full scenes with backgrounds and cast shadows, not just isolated synthetic objects. The paper reports state-of-the-art results across synthetic, real-object, and real-scene datasets, and demonstrates editing applications such as relighting, reflectance editing, and shading editing.","feed_headline":"Multiple lights split a scene into reflectance and shading","feed_subtitle":"A NeRF trained under several light positions generates its own training labels, removing the need for ground-truth intrinsic images.","key_machinery":"The central machinery is physics-based pseudo label generation: use the SDF geometry to compute normals and sphere-traced visibility, convert them into a Lambertian pseudo shading $S^* = (\\max(\\vec{N}\\cdot\\vec{L},0) \\otimes V)^\\gamma$, and then derive pseudo reflectance by dividing each observed image by its pseudo shading and merging multiple lighting observations with K-means. This replaces statistical priors with a direct geometric-optical constraint; the labels then supervise dedicated reflectance and shading MLPs in Stage 2.","core_discovery":"MLI-NeRF claims that multiple light positions, combined with multi-view geometry, provide enough information to disentangle reflectance and shading without ground-truth intrinsic data. In Stage 1, a Neuralangelo-style SDF network takes light position as an extra input, producing a relightable scene representation. Post-processing then derives pseudo shading from the SDF normal and sphere-traced light visibility as $S^* = (\\max(\\vec{N}\\cdot\\vec{L},0)\\otimes V)^\\gamma$, and pseudo reflectance by per-light division $R = I \\oslash S$ followed by a K-means merge across lights with hole filling. In Stage 2, separate MLPs predict reflectance (independent of light and view) and shading (conditioned on light position), supervised by these labels with reliability weight maps, while the residual term $Re$ is regularized toward zero so the image is explained by reflectance and shading. The paper's experiments on synthetic, real-object, and ReNe full-scene datasets report that this approach outperforms prior intrinsic decomposition and inverse rendering methods in reflectance and shading metrics, and performs competitively in relighting.","pith_inferences":["The pseudo-shading formula assumes direct Lambertian illumination only; real interreflections and specular effects are pushed into the residual term, which the Stage 2 regularization tries to minimize, potentially corrupting estimated reflectance in scenes with significant indirect light. This is an inference about the method's behavior beyond the paper's own claims.","Because Stage 1 is trained on known light positions, the same pseudo-label generation could in principle be applied to any relightable NeRF backbone, not just the Neuralangelo base; the paper notes the method is applicable to 'different similar baselines'.","A testable extension would be to replace the K-means merge of pseudo reflectances with a learned fusion or a robust median that accounts for specular highlights, which might improve performance on scenes with glossy materials."],"forward_implications":["Intrinsic decomposition can be performed on real full scenes with backgrounds and cast shadows, where statistical-prior methods and inverse rendering methods struggle.","Relighting and intrinsic decomposition can be learned in a single NeRF pipeline from multi-light captures without requiring ground-truth reflectance maps.","A small number of light positions, around four, is sufficient to approach the quality achieved with many random light positions in the tested scenes.","The learned decomposition enables editing applications such as reflectance editing, relighting, and shading editing on scenes from diverse datasets."],"supporting_citations":[{"why":"IntrinsicNeRF is the key baseline and prior work integrating intrinsic decomposition into NeRF; MLI-NeRF compares against it and extends its framework.","marker":"[44]"},{"why":"Neuralangelo provides the SDF-based geometry backbone used in Stage 1 and Stage 2.","marker":"[24]"},{"why":"L-Tracing supplies the sphere tracing algorithm used to estimate light visibility for pseudo shading generation.","marker":"[8]"},{"why":"NRHints provides the relighting baseline, the synthetic dataset setup for light and camera poses, and the real-object relighting dataset.","marker":"[45]"},{"why":"TensoIR is a primary inverse rendering baseline under multiple light settings and the source of the observation that performance improves with more light positions.","marker":"[18]"},{"why":"The ReNe dataset provides the real full-scene multi-light images used for evaluation.","marker":"[39]"},{"why":"InvRender is a baseline inverse rendering method for comparison and motivates the handling of indirect illumination.","marker":"[49]"},{"why":"PIE-Net is a baseline data-driven intrinsic decomposition method using photometric invariant edge guidance.","marker":"[9]"}],"fun_headline_variants":["Multi-light NeRF splits reflectance from shading","NeRF with multiple lights auto-labels intrinsic images","Light positions teach NeRF to separate shading and reflectance","No ground truth: MLI-NeRF disentangles scene components","MLI-NeRF: multi-light supervision for intrinsic decomposition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that real scene shading can be modeled as direct Lambertian illumination, with all indirect light and specular effects relegated to a residual term that regularization can suppress.","fun_headline_variants_meta":{"raw":{"variants":["Multi-light NeRF splits reflectance from shading","NeRF with multiple lights auto-labels intrinsic images","Light positions teach NeRF to separate shading and reflectance","No ground truth: MLI-NeRF disentangles scene components","MLI-NeRF: multi-light supervision for intrinsic decomposition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000212,"raw_usage":{"total_tokens":1416,"prompt_tokens":938,"completion_tokens":478,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":399}},"tokens_in":554,"tokens_out":478,"duration_ms":4673,"temperature":1.0,"reasoning_tokens":399,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:20:56.641555+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test that would falsify the central claim: run MLI-NeRF on a real scene containing a visible glossy or mirror-like surface under the multi-light capture setup, and check whether the predicted reflectance contains specular highlights or color from the environment, and whether the residual term remains large. If the residual absorbs a large fraction of the image energy or the reflectance inherits lighting-dependent colors, the direct-Lambertian pseudo-shading assumption fails.","supporting_citations":[{"cited_title":"IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis","cited_arxiv_id":null,"evidence_quote":"IntrinsicNeRF is the key baseline and prior work integrating intrinsic decomposition into NeRF; MLI-NeRF compares against it and extends its framework."},{"cited_title":"Neuralangelo: High-fidelity neural surface reconstruction","cited_arxiv_id":null,"evidence_quote":"Neuralangelo provides the SDF-based geometry backbone used in Stage 1 and Stage 2."},{"cited_title":"L-tracing: Fast light visibility estimation on neural surfaces by sphere tracing","cited_arxiv_id":null,"evidence_quote":"L-Tracing supplies the sphere tracing algorithm used to estimate light visibility for pseudo shading generation."},{"cited_title":"Relighting neural radiance fields with shadow and highlight hints","cited_arxiv_id":null,"evidence_quote":"NRHints provides the relighting baseline, the synthetic dataset setup for light and camera poses, and the real-object relighting dataset."},{"cited_title":"Tensoir: Tensorial inverse rendering","cited_arxiv_id":null,"evidence_quote":"TensoIR is a primary inverse rendering baseline under multiple light settings and the source of the observation that performance improves with more light positions."},{"cited_title":"Relight my nerf: A dataset for novel view synthesis and relighting of real world objects","cited_arxiv_id":null,"evidence_quote":"The ReNe dataset provides the real full-scene multi-light images used for evaluation."},{"cited_title":"Modeling indirect illumination for inverse rendering","cited_arxiv_id":null,"evidence_quote":"InvRender is a baseline inverse rendering method for comparison and motivates the handling of indirect illumination."},{"cited_title":"Pie-net: Pho- tometric invariant edge guided network for intrinsic image decomposition","cited_arxiv_id":null,"evidence_quote":"PIE-Net is a baseline data-driven intrinsic decomposition method using photometric invariant edge guidance."}],"review_version":1}