{"id":"997da7d8-006c-41c8-a5cd-05eb4176baec","arxiv_id":"2411.17994","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A camera-flash inverse rendering system uses 2D Gaussians plus sparse basis materials, automatically adjusting the material count to give interpretable, separable materials.","lead":"An inverse rendering method reconstructs 3D shape and material from multi-view flash photos while separating materials into a small set of editable basis materials. The approach aims to make reconstructed scenes easier to relight and edit than previous inverse rendering systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed automatic, scene-adaptive basis BRDF control is not yet established: the merge/removal thresholds are manually tuned per scene (Supplemental Sections 3.6 and 3.7), so fixed-parameter evaluation is needed before the scalability claim holds.","rationale":"The reader's verdict is CONDITIONAL with moderate confidence, and the weakest_assumption focuses on the expressiveness of the simplified Disney basis BRDFs and the fidelity cost of the sparsity regularizer. This stress-test identifies a different but closely related load-bearing concern: the adaptive basis-count mechanism is not shown to work without per-scene manual threshold selection. The supplemental document explicitly states that the merge threshold is chosen per scene (Section 3.6) and that removal thresholds are reported 'for this scene' (Section 3.7). Since the paper's second contribution is precisely the dynamic adjustment of the number of basis BRDFs, this dependence on hand-tuned thresholds weakens the generality claim. The concern is not an internal contradiction but an evidence gap: the method may still work well with a fixed threshold on a wide range of scenes, but that has not been demonstrated. The proposed concrete test would settle it by running a fixed-threshold configuration across all datasets and checking whether the basis count adapts and quality holds. This concern reinforces the reader's CONDITIONAL verdict rather than overturning it, so the verdict is left unchanged. Credit is due where the paper is careful: the ablation table reports the PSNR drop from the sparsity loss, and the supplemental includes detailed discussions of shadow and global-illumination limitations, which are honest about the model's scope. The central unresolved issue is that the headline contribution of automatic scalability lacks a fixed-parameter evaluation, and the interpretability claim lacks a quantitative metric; both gaps should be addressed before the results are relied upon.","tokens_in":961,"tokens_out":1536,"duration_ms":41144,"concrete_test":"Run the full pipeline on all four synthetic scenes and the real-world scene with a single fixed threshold set (e.g., tau_merge=0.4, tau_removal-weight=0.1, tau_removal-number=0.005) and no per-scene manual tuning. Report the final number of basis BRDFs per scene, PSNR, normal MAE, and a quantitative spatial-separation metric such as the mean entropy of per-pixel weight vectors. If final counts do not track scene complexity or reconstruction quality drops substantially relative to the per-scene-tuned results, the automatic-scaling claim is falsified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the method 'dynamically adjusts the number of basis BRDFs to fit the target scene' rests on the merge and removal thresholds defined in Section 3.2. The supplemental ablation (Sections 3.6 and 3.7) states that 'we find appropriate merge threshold tau_merge for each complex scene' and reports removal thresholds 'for this scene.' This indicates that the final basis count is not a purely automatic emergent property but a function of per-scene hyperparameters chosen with knowledge of the results. No experiment reports the sensitivity of the final number of basis BRDFs to these thresholds, nor does the paper show that a single fixed threshold works across scenes of varying complexity. If the adaptive-count behavior disappears or degrades under fixed thresholds, the headline contribution of scalability is overstated. This is load-bearing because the paper's second contribution is specifically the 'basis BRDF control method' that 'dynamically adjusts the number of basis BRDFs.' The unquantified interpretability claim is a separate concern, but the threshold dependence more directly undermines the method's claimed generality.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a differentiable inverse rendering approach that represents a scene with 2D Gaussians whose reflectance is a sparse blend of basis BRDFs modeled by a simplified Disney BRDF. During optimization, the method applies a low-temperature softmax, an entropy sparsity regularizer, and heuristic merge/removal rules to reduce the number of basis BRDFs and encourage spatially separated weights. The authors evaluate on four synthetic multi-view flash scenes and one real-world scene, reporting improved normal reconstruction (MAE 9.81 vs. 15.28 for GS3), faster training (0.5h vs. 2h for GS3), and novel-view relighting results (31.78 PSNR vs. 38.93 for GS3, which is lower than GS3). They also demonstrate reflectance editing and mesh extraction.","tokens_in":17286,"tokens_out":4811,"duration_ms":42179,"significance":"If substantiated, the paper's contributions would provide an efficient inverse rendering pipeline with editable and relightable scene representations, and a mechanism that adapts the number of material bases. Strengths include the clear formulation of an interpretability objective via sparsity, the merge/removal machinery, an evaluation showing substantial geometry improvements over strong baselines, and a very competitive training time. The main weaknesses are that the adaptive basis-count behavior relies on per-scene tuning of thresholds, the interpretability is not quantified, and the fidelity cost of sparsity is not isolated in the relighting comparison. These gaps currently prevent the paper from fully supporting its headline claims.","major_comments":[{"comment":"The central claim that the method 'dynamically adjusts the number of basis BRDFs to fit the target scene' is not yet supported by a fixed-parameter evaluation. The supplemental states that 'we find appropriate merge threshold τ_merge for each complex scene' (Supp. 3.6) and reports that removal thresholds are set 'for this scene' (Supp. 3.7), indicating that the final basis count depends on manual, per-scene hyperparameters. As a result, the adaptive behavior could be an artifact of threshold tuning rather than an emergent property of the optimization. I request a sensitivity study that sweeps τ_merge and the removal thresholds while keeping them fixed across scenes of varying complexity, and a report of the resulting basis counts and reconstruction quality.","section":"Section 3.2 / Supplemental 3.6–3.7"},{"comment":"The interpretability claim is supported only by qualitative visualizations. Figure 6 and the supplemental figures show spatially separated weight maps, but there is no quantitative metric, no user study, and no comparison with ground-truth material or object segmentations. This matters because the sparsity loss and low-temperature softmax (Eqs. 9–10) directly enforce sparse weights, so observing sparse weight maps could simply reflect the optimization objective. I recommend adding a quantitative evaluation, e.g., measuring agreement of weight-map clusters with object masks, computing a spatial-separation index, or running a perceptual user study.","section":"Section 4 / Figure 6"},{"comment":"The paper does not isolate the fidelity cost of interpretability in relighting. Supplemental Table 1 shows that adding L_sparse lowers PSNR from 34.13 to 31.78, but the main-text relighting comparison (Table 2) only reports the full method, and the paper does not report relighting PSNR for the variant without L_sparse. Since the interpretation is that sparse representation sacrifices fidelity, the reader cannot tell whether the 31.78 PSNR is acceptable or whether the sparse prior is the cause. Additionally, no error bars or multiple-seed runs are provided for Table 1, so the significance of the MAE improvement cannot be assessed. Please report results with confidence intervals and a no-sparsity relighting baseline.","section":"Supplemental Table 1 / Table 2"}],"minor_comments":[{"comment":"The row labelled 'Train' is ambiguous; it should be labelled 'Training time' and the units (hours) should be stated explicitly.","section":"Section 4.1, Table 1"},{"comment":"The notation 'GS3' and 'GS 3' is used inconsistently in the caption and text; please unify the notation.","section":"Figure 6 caption"},{"comment":"The phrase 'especially for thin and convex objects' is vague; please specify how thin and convex objects are identified or give per-object results to substantiate the claim.","section":"Section 4.1"},{"comment":"The description of Zhou et al. [49] would benefit from a clear statement of what is inherited and what is novel beyond adding a differentiable pipeline; as written, the distinction is implicit.","section":"Section 2"},{"comment":"The supplementary correctly lists global illumination as a limitation; please also mention this limitation in the main text's conclusion or discussion so that readers of the main paper are aware of it.","section":"Supplemental 4.2"},{"comment":"The number of initialized basis BRDFs N is a free parameter, but the ablation only tests N=9 and N=15; please report the effect of N on the final number of basis BRDFs across different scene complexities.","section":"Section 3.1 / Supplemental 3.5"}],"recommendation":"major_revision","confidential_remarks":"The per-scene threshold tuning issue is the most serious concern: the paper's second contribution, the adaptive number of basis BRDFs, needs a fixed-threshold evaluation to be credible. The interpretability evaluation also needs quantitative grounding. The lack of error bars is worrying for the quantitative claims, and the lower relighting PSNR relative to GS3 should be discussed more candidly. These are fixable with additional experiments and a revised narrative, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a credible incremental contribution to basis-BRDF inverse rendering. The useful novelty is the control loop—dynamic merge and removal of basis BRDFs plus a low-temperature softmax and entropy sparsity regularizer inside a 2D Gaussian splatting pipeline. That combination is new relative to DPIR and GS3, which fix the basis count, and it delivers the best normal reconstruction on the authors' synthetic benchmark (MAE 9.81 vs 15.28 for GS3). The paper is clearly written, the method is described in enough detail to reimplement, and the editing demos give some evidence the representation is actually usable.\n\nThe soft spots are real but not fatal. First, the 'dynamically adjusts the number of basis BRDFs' claim overstates what is demonstrated. The supplemental says the merge threshold is chosen per scene (\"we find appropriate merge threshold for each complex scene\") and the removal thresholds are also scene-specific. So the final basis count is partly a function of tuned hyperparameters, not purely an emergent property. A fixed-threshold sensitivity study across scenes of varying complexity is needed to support the scalability claim. Second, interpretability is only shown visually; there is no quantitative measure of spatial separation or semantic alignment, so the main qualitative claim is hard to verify independently. Third, the sparsity regularizer measurably lowers reconstruction fidelity (their own ablation: PSNR 34.13 without it, 31.78 with it) and relighting PSNR is below GS3. That is an acceptable trade-off, but it should be framed as one. Minor: only four synthetic scenes and one real scene, no error bars, no code release.\n\nThe math and the citation pattern look fine. The Disney BRDF basis is standard, the merge/removal criteria are precisely defined, and the comparison to Zhou et al. 2016 as prior art is honest. The stated limitations around global illumination and cast shadows are acknowledged in the supplemental.\n\nWho this is for: researchers in differentiable inverse rendering, material editing, and Gaussian-splatting relighting. It should go to peer review. My recommendation: send it out, and expect the authors to tighten the adaptive-count claim, add a fixed-threshold experiment, and provide at least one quantitative interpretability metric or a code release.","headline":"Credible incremental inverse-rendering paper with a real geometry win; the 'automatic' basis-count adaptation is undercut by per-scene threshold tuning, but the core method deserves a serious referee.","tokens_in":17812,"tokens_out":2144,"would_cite":true,"duration_ms":18871,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A differentiable inverse-rendering method yields spatially separated, interpretable basis BRDFs while reconstructing accurate geometry.","keywords":["inverse rendering","basis BRDF","2D Gaussian splatting","sparsity regularization","SVBRDF","relighting","scene editing","differentiable rendering"],"falsifier":"Render or capture a scene with strong interreflections or a material whose BRDF is far from the Disney family, run the method, and check whether the recovered basis BRDFs stay spatially separated and whether novel-view relighting under different lights matches ground truth; a failure would show up either as entangled weight maps or as a large relighting error that the sparsity loss alone cannot explain.","tokens_in":16806,"feed_emoji":"🎨","tokens_out":5243,"duration_ms":44143,"temperature":0.7,"pith_summary":"From multi-view flash photographs, the authors reconstruct both 3D geometry and surface reflectance by representing the scene as a collection of 2D Gaussian splats, each with a reflectance given by a weighted blend of a small set of basis BRDFs. The core claim is that adding a sparsity prior on the per-splat weights and letting the number of basis BRDFs grow or shrink during optimization makes those basis BRDFs align with actual materials in the scene, so one basis BRDF tends to describe one object or surface region. If correct, this turns an inverse-rendering pipeline into a scene editor: materials can be selected, recolored, or replaced by editing a few parameters, and the reconstructed geometry supports physically based relighting. The authors report the most accurate surface normals among the compared methods and demonstrate editing on synthetic and real scenes.","feed_headline":"Materials separate themselves during inverse rendering","feed_subtitle":"A sparsity penalty and adaptive basis count turn multi-view flash photos into relightable, editable 3D scenes.","key_machinery":"The load-bearing machinery is the pair of a low-temperature softmax on basis weights and an entropy sparsity regularizer, combined with merge and removal rules keyed to radiometric and geometric similarity. The low-temperature softmax pushes each Gaussian's weight vector toward one-hot; the sparsity regularizer penalizes diffuse weight assignments both per Gaussian and in the rendered weight images; merge deletes a basis when its sampled BRDF values are close to another basis's and its associated Gaussians occupy overlapping space; removal deletes a basis whose rendered weight map exceeds a coverage threshold on too few pixels. These mechanisms convert a continuous optimization into a discrete, adaptive selection of materials.","core_discovery":"On the paper's own terms, the discovery is that interpretable material decomposition is not a separate post-processing step but an outcome of the optimization itself. The method keeps a fixed number of basis BRDFs during a warm start, then repeatedly merges pairs that are both radiometrically similar and geometrically similar, removes bases that cover almost no pixels, and applies an entropy-based sparsity loss to per-Gaussian weights and to rendered weight maps. The result is a representation in which each spatial point is described by one or a few basis BRDFs, the bases correspond to visually distinct materials, and the number of bases adapts to scene complexity. This is what enables editing operations such as changing a glass material to silver and extracting a single object's mesh for environment relighting.","pith_inferences":["The merge and removal thresholds are hand-set per scene in the paper, so a natural extension is to choose them automatically via a held-out validation loss, which could remove the per-scene tuning described in the supplement.","The same sparse-basis formulation might transfer to other primitive types beyond 2D Gaussians, or to appearance properties beyond BRDFs, wherever spatial coherence can be exploited.","Because sparsity directly trades reconstruction fidelity, an interesting testable extension is an adaptive sparsity weight that tightens only after material separation stabilizes, potentially recovering some of the lost PSNR without losing interpretability.","If the method is right, it suggests that interpretability in inverse rendering is largely a sparsity property of the representation rather than a property of the network or basis choice."],"forward_implications":["Novel-view relighting becomes physically based, using reconstructed normals and basis BRDFs, without a residual network to hide geometry errors.","Scene editing reduces to editing the parameters of a few basis BRDFs, such as base color, roughness, and metallic, and selecting Gaussians by their dominant basis weight.","Object extraction is possible by pruning Gaussians whose highest-weight basis BRDF is not the selected one, enabling mesh extraction and environment relighting.","The number of basis BRDFs scales with scene complexity: simple scenes end with fewer bases, while complex multi-object scenes keep more.","Training time is much shorter than neural-SDF baselines because the representation is rasterization-based and does not require a large implicit network."],"supporting_citations":[{"why":"Provides the 2D Gaussian splatting primitive, differentiable rasterization, and geometric regularization losses that the scene representation and rendering rest on.","marker":"[13]"},{"why":"Defines the simplified Disney BRDF model whose parameters (base color, roughness, metallic) parameterize each basis BRDF.","marker":"[41]"},{"why":"Introduced the idea of adjusting the number of basis BRDFs during optimization, which the paper adapts to a differentiable inverse-rendering pipeline.","marker":"[49]"},{"why":"Supplies a Gaussian-splatting baseline with fixed, non-interpretable basis BRDFs that motivates the proposed sparsity and basis-control scheme.","marker":"[2]"},{"why":"Provides a differentiable point-based inverse-rendering baseline with fixed basis BRDFs whose spatially entangled materials the paper compares against.","marker":"[7]"},{"why":"Supplies a neural-SDF inverse-rendering baseline and the synthetic dataset configuration used for geometry and relighting evaluation.","marker":"[44]"}],"fun_headline_variants":["Materials self-separate during inverse rendering","Sparsity-driven decomposition yields editable materials","Basis BRDFs adapt automatically to scene complexity","Interpretable material maps from differentiable rendering","Editable relighting via adaptive basis BRDFs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that real surface reflectance can be approximated well by a small number of simplified Disney BRDFs blended sparsely per point, under direct point-light illumination only, without giving up reconstruction fidelity.","fun_headline_variants_meta":{"raw":{"variants":["Materials self-separate during inverse rendering","Sparsity-driven decomposition yields editable materials","Basis BRDFs adapt automatically to scene complexity","Interpretable material maps from differentiable rendering","Editable relighting via adaptive basis BRDFs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1433,"prompt_tokens":937,"completion_tokens":496,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":426}},"tokens_in":553,"tokens_out":496,"duration_ms":4861,"temperature":1.0,"reasoning_tokens":426,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:37:08.370440+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render or capture a scene with strong interreflections or a material whose BRDF is far from the Disney family, run the method, and check whether the recovered basis BRDFs stay spatially separated and whether novel-view relighting under different lights matches ground truth; a failure would show up either as entangled weight maps or as a large relighting error that the sparsity loss alone cannot explain.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the simplified Disney BRDF model whose parameters (base color, roughness, metallic) parameterize each basis BRDF."},{"cited_title":"Modeling indirect illumination for inverse rendering","cited_arxiv_id":null,"evidence_quote":"Introduced the idea of adjusting the number of basis BRDFs during optimization, which the paper adapts to a differentiable inverse-rendering pipeline."},{"cited_title":"Neural-pil: Neural pre-integrated lighting for reflectance decomposition","cited_arxiv_id":null,"evidence_quote":"Provides a differentiable point-based inverse-rendering baseline with fixed basis BRDFs whose spatially entangled materials the paper compares against."},{"cited_title":"Inverserendernet: Learning single image inverse rendering","cited_arxiv_id":null,"evidence_quote":"Supplies a neural-SDF inverse-rendering baseline and the synthetic dataset configuration used for geometry and relighting evaluation."}],"review_version":1}