Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Three learned priors stabilize Gaussian-splatting inverse rendering from just four camera views.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 17:18 UTC pith:BNPVC6TT

load-bearing objection A well-engineered, genuinely new prior combination for sparse-view Gaussian inverse rendering, but the paper's own Table 2 contradicts its albedo claim and the ablation is too thinly supported to justify the headline — still worth a serious referee. the 3 major comments →

arxiv 2512.09925 v2 pith:BNPVC6TT submitted 2025-12-10 cs.CV

GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures

Patrick Noras , Jun Myeong Choi , Didier Stricker , Pieter Peers , Roni Sengupta This is my paper
classification cs.CV
keywords inverse renderingGaussian splattingsparse viewsmaterial estimationrelightingdiffusion priorsintrinsic image decompositionnovel-view synthesis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Inverse rendering—recovering a scene's geometry, materials, and lighting from photographs—turns ill-posed when only a handful of camera views are available, because many different combinations of shape, reflectance, and light reproduce the same images. GAINS claims this ambiguity can be substantially removed by injecting three complementary learning-based priors into a Gaussian-splatting optimizer: segmentation that enforces within-region consistency of specular parameters, a monocular intrinsic-image decomposition that anchors diffuse albedo, and a diffusion-based score-distillation loss that penalizes unrealistic appearances at novel views and novel lights. On synthetic and real datasets, with 4 to 32 input views, the method reports consistent gains over existing Gaussian-based inverse rendering methods for albedo, roughness, relighting, and novel-view synthesis, with the largest advantages in the sparse regime of 4–8 views. A sympathetic reader would care because the recipe could turn ordinary sparse captures into relightable 3D assets that a phone or a robot can produce.

Core claim

The central claim is that sparse-view inverse rendering is stabilized when Gaussian-splatting-based physically based rendering is coupled with three complementary learned priors, each correcting one failure mode of the others. Segmentation guidance, lifted from per-view masks and self-supervised image features, reduces noise and enforces multi-view consistency in specular roughness and metallicity; an intrinsic-image-decomposition prior regularizes diffuse albedo, with its weight linearly annealed over iterations to tolerate view inconsistencies; and a multi-illumination score-distillation loss re-renders the material estimate under shuffled environment maps at novel viewpoints, penalizing u

What carries the argument

The load-bearing mechanism is the Stage II joint optimization of per-point surface reflectance parameters (albedo, roughness, metallicity) and a small environment map, regularized by three complementary losses. An intra-class consistency loss, computed over groups of Gaussians sharing a lifted segmentation label, reduces variance in roughness and metallicity within semantically similar regions and, for mirror-like materials, biases textural detail toward specular reflection. An intrinsic-image-decomposition loss pulls rendered diffuse albedo toward a monocular per-image albedo estimate, with weight linearly annealed to tolerate cross-view inconsistency. A multi-illumination score-distillatio

Load-bearing premise

The method's gains depend on the assumption that the pretrained monocular depth/normal, segmentation, intrinsic-image-decomposition, and diffusion priors are reliable for the scene being reconstructed—if any prior systematically misfires, the optimization can be pulled toward visually plausible but physically wrong materials.

What would settle it

Run GAINS with only 4–8 views on a scene where the monocular depth prior is deliberately misled (e.g., an object whose depth the estimator systematically mispredicts) and compare relighting error against the same pipeline with priors disabled; the claim that priors stabilize sparse-view recovery would be falsified if the prior-laden version is worse. A cleaner test: a 4-view benchmark on object categories absent from the priors' training distribution, measuring whether the gains over prior-free baselines persist.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Sparse multi-view capture (as few as 4–8 images) becomes a practical input for relightable 3D reconstruction, opening the door to phone- or robot-made assets.
  • Improved material–lighting separation removes baked-in reflections: relit images show plausible diffuse-versus-specular movement under new environment maps rather than frozen highlights.
  • The segmentation prior keeps roughness and metallicity stable across view counts, at the documented cost of losing some high-frequency roughness detail when 16–32 views are available.
  • Ablations show the three priors are genuinely complementary: dropping any one degrades at least one task, and only the full combination is best across novel-view synthesis, albedo, and relighting.
  • The two-stage geometry-then-material ordering indicates that strong geometry priors should come first; future inverse-rendering pipelines can adopt staged prior injection.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The complementary-prior recipe likely transfers to other scene representations (neural implicit fields or large reconstruction models), where the albedo-lighting ambiguity is equally severe and learned priors are less explored.
  • A testable extension would replace fixed prior weights with an uncertainty-aware or learned schedule, potentially smoothing the documented trade-off between segmentation-driven robustness and dense-view high-frequency detail.
  • Because the IID prior is annealed, the method leans on it only early; a multi-view-consistent intrinsic decomposition, estimated jointly across views, could serve as a stronger anchor and remove the annealing crutch.
  • The reliance on monocular depth/normal priors suggests GAINS's gains may be largest for categories on which those priors were trained; evaluating on out-of-distribution objects or unusual (translucent, anisotropic) materials would delineate true generality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes GAINS, a two-stage inverse rendering framework built on 2D Gaussian Splatting that targets sparse multi-view inputs. Stage I refines geometry with monocular depth/normal losses, a depth ranking loss, normal smoothness, and score distillation sampling (SDS). Stage II jointly optimizes material parameters and environment lighting with three complementary priors: a segmentation-based intra-class consistency loss, an intrinsic image decomposition (IID) albedo loss, and a multi-illumination SDS loss. The method is evaluated against Ref-GS and GI-GS on TensorIR, Synthetic4Relight, and Ref-Real, with view-count sweeps and ablations. The central claim is that combining these learning-based priors significantly improves material accuracy, relighting, and novel-view synthesis under sparse captures.

Significance. If the reported results are accurate, the paper would make a useful contribution by showing that foundation-model priors can stabilize a long-standing ambiguity in inverse rendering—albedo/lighting disambiguation under sparse views. The paper contains a broad set of experiments: two synthetic benchmarks with ground-truth albedo/roughness/relighting, a real-world dataset, quantitative view-count sweeps, and an ablation of each proposed prior. The qualitative comparisons, especially the relighting examples in Figs. 1 and 4, are illustrative and suggest the method has practical value. However, the quantitative support is uneven: the abstract's headline claim of 'significantly improves material parameter accuracy' is not consistently supported by the tables, and the ablation table does not support the claim that every component causes 'clear and measurable degradation.' These issues are fixable but require either corrected claims or additional experimental evidence.

major comments (3)
  1. [Section 6, Table 2] The text states 'On TensorIR dataset, our method estimates better albedo than existing approaches,' but Table 2 reports albedo PSNR for GAINS as 27.913 versus 28.969 for GI-GS—a 1.06 dB deficit on the primary fidelity metric. GAINS is slightly better on albedo SSIM (0.912 vs 0.901) and LPIPS (0.118 vs 0.121), but the claim as written is not supported by the PSNR column. Because the abstract's 'significantly improves material parameter accuracy' rests on albedo/roughness numbers, this overstatement must be corrected, or the metric choice justified, before the headline claim is credible.
  2. [Section 6, Table 3 and ablation discussion] The ablation narrative says 'the full model consistently achieves the strongest results across NVS, albedo, and relighting, and ablating any component leads to clear and measurable performance degradation.' Table 3 does not show this. Removing IID gives NVS PSNR 25.148 vs 25.106 for the full model; removing segmentation gives albedo PSNR 24.214 vs 23.991; removing MI-SDS gives albedo PSNR 24.022 vs 23.991. All differences are below 0.3 dB, and no variance, repeated runs, or significance tests are reported. The data therefore do not support 'clear and measurable' degradation for every component. The authors should either add repeated runs with error bars/statistical testing, or substantially soften the ablation claims and rely on qualitative evidence.
  3. [Section 7, Conclusion vs Abstract] The conclusion says GAINS achieves 'state-of-the-art relighting accuracy and competitive novel-view synthesis,' while the abstract claims 'significantly improves ... novel-view synthesis.' These statements are in tension. Given Table 2 shows a larger NVS PSNR gain on TensorIR but the Supplementary Fig. 8-9 show mixed per-scene relighting numbers, the manuscript should align the abstract and conclusion with the actual pattern of results, especially after the Table 2 albedo issue is resolved.
minor comments (4)
  1. [Throughout] There are several typos and grammatical issues: 'enviorment cubemap' (Sec. 3), 'lindear decrease' (Sec. 5.2), 'metalicity' (multiple), 'GI-GS almost slightly competes in NVS' (Sec. 6). These should be corrected.
  2. [Figures 6 and 18] The view-count bar charts report single runs without error bars. Adding variance across scenes or seeds would make the sparse-versus-dense trends more convincing.
  3. [Section 6, Evaluation Framework] The paper mentions that albedo evaluation uses scale-invariant losses but does not state whether the same scaling is applied to all methods consistently and how the scale is computed for the reported PSNR/SSIM/LPIPS values. A short clarification would improve reproducibility.
  4. [Section 5.1, Eq. (5)] The notation S_i is used both for a set of masks and for the cardinality term in γ(|s_i|). This is confusing; a distinct symbol for the mask region and the set of masks would help.

Circularity Check

0 steps flagged

No significant circularity: GAINS is an empirical optimization framework whose priors are external and whose evaluation targets are held-out ground truth.

full rationale

GAINS does not derive any prediction from fitted constants; it is an optimization framework that combines external pretrained priors (monocular depth/normal [12], RGB-X IID [40], SAM/CLIP/DINOv2 segmentation, Stable Diffusion SDS) with a 2DGS-based shape and PBR material optimization. The losses L_DC, L_DR, L_NS, L_SDS, L_ICC, L_IID, and L_MI-SDS regularize towards external priors or internal consistency; none encodes the evaluation ground truth (TensorIR albedo/relight, Synthetic4Relight roughness, Ref-Real NVS). Albedo evaluation uses scale-invariant scaling, but this is a standard metric normalization, not a fitted parameter renamed as a prediction. The central claim is validated by comparing the optimized outputs against held-out views and ground-truth intrinsics, which are not inputs to the optimization. The only self-citations (e.g., GaNI [32]) appear in related work and are not load-bearing. The noted Table 2 albedo PSNR discrepancy is a reporting/consistency issue, not a circularity. Hand-tuned loss weights are calibration choices, not a derivation that assumes its own conclusion. No equation reduces to a fitted value, and no load-bearing argument rests on a self-citation chain.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central claim rests on many hand-chosen hyperparameters and on the trustworthiness of pretrained foundation models. No new physical entities are introduced. The method is an empirical optimization system, not a derivation from first principles.

free parameters (4)
  • Stage I loss weights (λ_DC, λ_DR, λ_NC, λ_NS, λ_SDS, λ_BCE) = 0.005, 10, 1, 0.005, 0.0001, 0.1
    Chosen by hand in Section 4 with no sensitivity analysis. These weights balance re-rendering against monocular depth/normal and SDS priors, directly affecting geometry quality.
  • Stage II loss weights (λ_ICC, λ_IID, λ_MI-SDS, λ_TV) = 0.1, 1, 0.0001, 1
    Chosen by hand in Section 5.4. The material recovery quality depends on these balances; no grid search or variance analysis is reported.
  • Activation schedules for SDS/MI-SDS and IID decay beta(τ) = SDS after 10000 of 16000 iterations; MI-SDS after 3000 of 7000; beta decreases linearly with τ
    Empirically chosen; the paper states SDS is not effective in early optimization and starts it later. The schedule strongly influences results but is not ablated.
  • Diffusion and segmentation sampling settings = 100 novel viewpoints; timestep t ~ U(0.02,0.98); gamma exponent 25
    Introduced ad hoc in Sections 4 and 5.1. The supplementary shows segmentation quality depends on the number of views used for lifting (25/50/100), yet 100 is used as a fixed default.
axioms (5)
  • domain assumption A simplified Disney BRDF with split-sum approximation and a distant 128x128 environment map sufficiently represents scene reflectance and lighting
    Used in Section 3 to define L_diffuse and L_specular. This ignores global illumination, interreflections, near-field lighting, and complex shadowing, all of which occur in real scenes.
  • domain assumption Foundation-model priors (monocular depth/normal, RGB-X IID, SAM/CLIP/DINOv2 segmentation, Stable Diffusion) provide reliable and unbiased guidance
    The method's core contribution is to trust these priors (Sections 4, 5.1-5.3). If a prior systematically misfires on an object/material class, optimization will be pulled away from the true solution.
  • domain assumption Camera intrinsics and extrinsics are known and masks are available or can be assumed
    Stated in Section 3 as input. Real captures often have pose error; the method does not address pose uncertainty.
  • domain assumption Two-stage optimization (geometry first, then materials and lighting) is a valid decomposition
    The pipeline in Sections 3-4 assumes Stage I geometry is accurate enough for Stage II material recovery. The supplementary itself notes segmentation consistency relies on the reconstructed geometry.
  • domain assumption Specular roughness and metallicity are approximately constant within semantically similar subparts
    The ICC loss in Section 5.1 enforces this. The paper concedes this limits high-frequency roughness recovery at dense views.

pith-pipeline@v1.3.0-alltime-deepseek · 18151 in / 11778 out tokens · 117683 ms · 2026-08-03T17:18:09.408663+00:00 · methodology

0 comments
read the original abstract

Recent advances in Gaussian Splatting-based inverse rendering extend Gaussian primitives with shading parameters and physically grounded light transport, enabling high-quality material recovery from dense multi-view captures. However, the accuracy of these methods degrades under sparse-view settings, where limited observations lead to severe ambiguity between geometry, reflectance, and lighting. We introduce GAINS (Gaussian-based Inverse rendering from Sparse multi-view captures), a two-stage inverse rendering framework that leverages foundation models as priors to stabilize geometry and material estimation. The core technical contribution of this paper is an inverse rendering framework that unifies foundation model priors with physically-based representations in an optimization scheme. GAINS first refines geometry using monocular depth, normal, and diffusion priors, and then employs segmentation, intrinsic image decomposition (IID), and diffusion priors to regularize material recovery. Extensive experiments on synthetic and real-world datasets show that GAINS significantly improves material parameter accuracy, relighting quality, and novel-view synthesis compared to state-of-the-art Gaussian-based inverse rendering methods. While GAINS outperforms and remains competitive across a wide range of objects captured with 4 to 32 cameras, the improvement is particularly pronounced under sparse-view settings, where ambiguity is high and learning-based priors become especially beneficial. Project page: https://patrickbail.github.io/gains/

Figures

Figures reproduced from arXiv: 2512.09925 by Didier Stricker, Jun Myeong Choi, Patrick Noras, Pieter Peers, Roni Sengupta.

Figure 1
Figure 1. Figure 1: We introduce GAINS, GAussian-based INverse rendering from Sparse multi-view captures, which synergizes learning-based priors related to monocular depth/normal, segmentation, intrinsic image decomposition (IID), and diffusion, to better disambiguate re￾flectance from lighting, leading to better intrinsics, novel view synthesis and relighting compared to existing state-of-the-art approaches such as Ref-GS [3… view at source ↗
Figure 2
Figure 2. Figure 2: GAINS follows a two-stage inverse rendering pipeline: Stage I reconstructs geometry, and Stage II estimates material parameters and lighting. In Stage I, we enhance geometry using learning-based priors from monocular depth, normal, and diffusion predictors. In Stage II, we introduce three complementary priors: segmentation, intrinsic image decomposition (IID), and diffusion, to improve material estimation,… view at source ↗
Figure 3
Figure 3. Figure 3: Ablation studies on the gardensphere scene from the Ref-Real [29] dataset. In absence of learning-based priors (Ours w/o IID, Seg, SDS in 1st col) reflectance maps are poorly reconstructed, especially metallicity and roughness. Without the IID prior (3rd col) results in weaker specular effects (compared to the 2nd col). Without segmentation guidance (4th col) results in noise material maps across objects. … view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison of intrinsic estimation and relighting on the sedan scene from the Ref-Real dataset [29] recon￾structed from 8 views. Column 1 shows novel-view intrinsic renderings in a 2×2 layout: (top) rendered albedo and surface normals, (bottom) specular roughness and metallicity. Columns 2–4 show relighting results under three different environment maps from novel viewpoints. GAINS recovers sig… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison of albedo and roughness estimation and novel-view synthesis (NVS) on Synthetic4Relight [44] dataset trained with 8 views. While all methods produce reasonable NVS, our method’s estimates significantly better albedo and rough￾ness than GI-GS and Ref-GS that overfit to limited training views and fail to disentangle reflectance from lighting. 5.4. Final Loss Each loss contributes to a s… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of our method with Ref-GS [36] and GI-GS [3] on the Synthetic4Relight [44] dataset for an increasing numbers of input views. The figure is organized into: PSNR for novel-view synthesis (top left), albedo (top right), and relighting (bottom left), and MSE for roughness (bottom right). Each bar chart shows results for 4, 8, 16, and 32 input views, with blue indicating Ref-GS, orange indicating GI-… view at source ↗
Figure 7
Figure 7. Figure 7: Relighting and material estimation comparison for varying number of views on the gardenspheres scene from the Ref-Real [29] dataset. We visualize for each method (rows: Ref-GS [36], GI-GS [3], and GAINS (ours)) relit results for 4, 8, 16, and 32 input views. For each result, we show the relit result under the bridge environment map, as well as (from left to right) the estimated albedo, metallic, and roughn… view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of our method with Ref-GS [36] and GI-GS [3] on the hotdog scene from the TensoIR [10] dataset, recon￾structed from 8 input views. From top to bottom, the rows show the ground-truth reference, Ref-GS, GI-GS and GAINS (our method). The columns present, from left to right: NVS renderings, predicted surface normals, albedo reconstruction, and relighting results. To com￾pensate for the albedo-lighti… view at source ↗
Figure 9
Figure 9. Figure 9: Comparison of our method with Ref-GS [36] and GI-GS [3] on the lego scene from the TensoIR [10] dataset, reconstructed from 8 input views. From top to bottom, the rows show the ground-truth reference, Ref-GS, GI-GS and GAINS (our method). The columns present, from left to right: NVS renderings, predicted surface normals, albedo reconstruction, and relighting results. To compensate for the albedo-lighting i… view at source ↗
Figure 10
Figure 10. Figure 10: Reconstructed segmentation maps generated by our iterative lifting procedure. The first row shows the reference ground truth images, while the second row displays our rendered segmentation maps. For each scene, we render 100 novel viewpoints in an orbital trajectory around the object to perform iterative segmentation lifting and Gaussian-object merging. Despite the challenging setting of using only 8 inpu… view at source ↗
Figure 11
Figure 11. Figure 11: Impact of the number of novel views used for the segmentation lifting and merging mechanism. Each row presents the resulting segmentation render in column 1, the metallic map in column 2, and the roughness map in column 3. From top to bottom, we show results using 100, 50, and 25 novel views utilized for lifting and merging of gaussian objects. As the number of views decreases, some regions become noticea… view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative comparison of intrinsic estimation and relighting on the sedan scene from the Ref-Real dataset [29], recon￾structed from 8 views. The rows show results from top to bottom for: GIR [26], Ref-GS [36], GI-GS [3], and GAINS (Ours). Column 1 presents novel-view intrinsic renderings in a 2 × 2 layout: (top) albedo and surface normals, (bottom) specular roughness and metallic￾ity. Columns 2–4 show re… view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative comparison on the toycar scene trained with different numbers of input views (top: 4 views, bottom: 16 views). Results for the 8-view setting are provided in [PITH_FULL_IMAGE:figures/full_fig_p018_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Relighting comparison across different lighting conditions on the hotdog scene from the TensoIR [10] dataset. Each column shows different relighting results. Rows correspond to Ref-GS [36], GI-GS [3], and our method. 7 [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Relighting comparison across different lighting conditions on the lego scene from the TensoIR [10] dataset. Each column shows different relighting results. Rows correspond to Ref-GS [36], GI-GS [3], and our method. 8 [PITH_FULL_IMAGE:figures/full_fig_p020_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Relighting comparison across different lighting conditions on the ficus scene from the TensoIR [10] dataset. Each column shows different relighting results. Rows correspond to Ref-GS [36], GI-GS [3], and our method. 9 [PITH_FULL_IMAGE:figures/full_fig_p021_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Relighting comparison across different lighting conditions on the armadillo scene from the TensoIR [10] dataset. Each column shows different relighting results. Rows correspond to Ref-GS [36], GI-GS [3], and our method. 10 [PITH_FULL_IMAGE:figures/full_fig_p022_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Comparison of our method with Ref-GS [36] and GI-GS [3] on the TensoIR [10] dataset across increasing numbers of input views. Columns (left to right) report PSNR, SSIM, and LPIPS. Rows correspond to novel-view synthesis (NVS), albedo estimation, and relighting. Each bar chart shows results for 4, 8, 16, and 32 input views, with blue indicating Ref-GS, orange indicating GI-GS, and green indicating GAINS (o… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TRON: Tracing Rays to Orchestrate a Neural Renderer for 3D Gaussian Reconstructions

    cs.CV 2026-06 unverdicted novelty 5.0

    Hybrid system that uses ray-traced 3D Gaussians to supply radiometric guidance and material regularization to a neural renderer for editable, realistic output from captured scenes.

Reference graph

Works this paper leans on

45 extracted references · 4 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Physically-based shading at disney

    Brent Burley and Walt Disney Animation Studios. Physically-based shading at disney. InAcm siggraph, pages 1–7. vol. 2012, 2012. 3

    2012

  2. [2]

    Lifting by gaussians: A simple, fast and flexible method for 3d instance segmentation

    Rohan Chacko, Nicolai H ¨ani, Eldar Khaliullin, Lin Sun, and Douglas Lee. Lifting by gaussians: A simple, fast and flexible method for 3d instance segmentation. In2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3497–3507. IEEE, 2025. 5

    2025

  3. [3]

    Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering

    Hongze Chen, Zehong Lin, and Jun Zhang. Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering. InICLR, 2025. 1, 2, 3, 5, 8, 9, 7, 10, 11

    2025

  4. [4]

    Estimating surface reflectance properties of a complex scene under cap- tured natural illumination

    Paul Debevec, Chris Tchou, Andrew Gardner, Tim Hawkins, Charis Poullis, Jessi Stumpfel, Andrew Jones, Nathaniel Yun, Per Einarsson, Therese Lundgren, et al. Estimating surface reflectance properties of a complex scene under cap- tured natural illumination. 2004. 2

    2004

  5. [5]

    IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing, 2025

    Chun Gu, Xiaofei Wei, Zixuan Zeng, Yuxuan Yao, and Li Zhang. IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing, 2025. 2

    2025

  6. [6]

    Shape, Light, and Material Decomposition from Images us- ing Monte Carlo Rendering and Denoising

    Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. Shape, Light, and Material Decomposition from Images us- ing Monte Carlo Rendering and Denoising. 2

  7. [7]

    2d gaussian splatting for geometrically accu- rate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accu- rate radiance fields. InSIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024. 3

    2024

  8. [8]

    Fatesgs: Fast and accurate sparse-view surface reconstruction using gaussian splatting with depth- feature consistency

    Han Huang, Yulun Wu, Chao Deng, Ge Gao, Ming Gu, and Yu-Shen Liu. Fatesgs: Fast and accurate sparse-view surface reconstruction using gaussian splatting with depth- feature consistency. InProceedings of the AAAI Conference on Artificial Intelligence, 2025. 2, 3, 4

    2025

  9. [9]

    Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces.arXiv preprint arXiv:2311.17977, 2023

    Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaox- iao Long, Wenping Wang, and Yuexin Ma. Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces.arXiv preprint arXiv:2311.17977, 2023. 2

    Pith/arXiv arXiv 2023

  10. [10]

    Tensoir: Tensorial inverse rendering

    Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Song- fang Han, Sai Bi, Xiaowei Zhou, Zexiang Xu, and Hao Su. Tensoir: Tensorial inverse rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2, 5, 8, 10, 1, 3, 7, 9, 11

    2023

  11. [11]

    Real shading in unreal engine 4.Proc

    Brian Karis and Epic Games. Real shading in unreal engine 4.Proc. Physically Based Shading Theory Practice, 4(3):1,

  12. [12]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 4, 6

    2024

  13. [13]

    3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42 (4), 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42 (4), 2023. 2, 3

    2023

  14. [14]

    Inverse Rendering for Complex Indoor Scenes: Shape, Spatially- Varying Lighting and SVBRDF From a Single Image

    Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Inverse Rendering for Complex Indoor Scenes: Shape, Spatially- Varying Lighting and SVBRDF From a Single Image. In 2020 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 2472–2481, Seattle, W A, USA, 2020. IEEE. 2, 3

    2020

  15. [15]

    Gs-ir: 3d gaussian splatting for inverse rendering

    Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. Gs-ir: 3d gaussian splatting for inverse rendering. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21644–21653, 2024. 2

    2024

  16. [16]

    Shape and material capture at home

    Daniel Lichy, Jiaye Wu, Soumyadip Sengupta, and David W Jacobs. Shape and material capture at home. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6123–6133, 2021. 2

    2021

  17. [17]

    Zero-1-to-3: Zero-shot one image to 3d object, 2023

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object, 2023. 4

    2023

  18. [18]

    NeRO: Neural Geometry and BRDF Reconstruction of Re- flective Objects from Multiview Images.ACM Trans

    Yuan Liu, Peng Wang, Cheng Lin, Xiaoxiao Long, Jiepeng Wang, Lingjie Liu, Taku Komura, and Wenping Wang. NeRO: Neural Geometry and BRDF Reconstruction of Re- flective Objects from Multiview Images.ACM Trans. Graph., 42(4):114:1–114:22, 2023. 2

    2023

  19. [19]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis, 2020. 2

    2020

  20. [20]

    Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Rus- sell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nico- las Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patri...

    2023

  21. [21]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 5

    2021

  22. [22]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junt- ing Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Doll´ar, and Christoph Feicht- enhofer. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:...

    Pith/arXiv arXiv 2024

  23. [23]

    Object shape and reflectance modeling from observation

    Yoichi Sato, Mark D Wheeler, and Katsushi Ikeuchi. Object shape and reflectance modeling from observation. InPro- ceedings of the 24th annual conference on Computer graph- ics and interactive techniques, pages 379–387, 1997. 2

    1997

  24. [24]

    Castillo, and David W

    Soumyadip Sengupta, Angjoo Kanazawa, Carlos D. Castillo, and David W. Jacobs. SfSNet: Learning Shape, Reflectance and Illuminance of Faces ‘in the Wild’. InProceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 6296–6305, 2018. 2, 3

    2018

  25. [25]

    Neural inverse rendering 11 of an indoor scene from a single image

    Soumyadip Sengupta, Jinwei Gu, Kihwan Kim, Guilin Liu, David W Jacobs, and Jan Kautz. Neural inverse rendering 11 of an indoor scene from a single image. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 8598–8607, 2019. 2, 3

    2019

  26. [26]

    Gir: 3d gaussian inverse rendering for relightable scene factorization.IEEE Transactions on Trans- actions on Pattern Analysis and Machine Intelligence, 2025

    Yahao Shi, Yanmin Wu, Chenming Wu, Xing Liu, Chen Zhao, Haocheng Feng, Jian Zhang, Bin Zhou, Errui Ding, and Jingdong Wang. Gir: 3d gaussian inverse rendering for relightable scene factorization.IEEE Transactions on Trans- actions on Pattern Analysis and Machine Intelligence, 2025. 2, 3, 8, 1, 5

    2025

  27. [27]

    Nerv: Neural reflectance and visibility fields for relighting and view synthesis

    Pratul P Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan T Barron. Nerv: Neural reflectance and visibility fields for relighting and view synthesis. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 7495–7504, 2021. 2

    2021

  28. [28]

    Neural-PBIR Reconstruction of Shape, Mate- rial, and Illumination

    Cheng Sun, Guangyan Cai, Zhengqin Li, Kai Yan, Cheng Zhang, Carl Marshall, Jia-Bin Huang, Shuang Zhao, and Zhao Dong. Neural-PBIR Reconstruction of Shape, Mate- rial, and Illumination. In2023 IEEE/CVF International Con- ference on Computer Vision (ICCV), pages 18000–18010, Paris, France, 2023. IEEE. 2

    2023

  29. [29]

    Barron, and Pratul P

    Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T. Barron, and Pratul P. Srinivasan. Ref-NeRF: Structured view-dependent appearance for neural radiance fields.CVPR, 2022. 2, 5, 6, 7, 8, 9, 10, 1

    2022

  30. [30]

    Sparsenerf: Distilling depth ranking for few-shot novel view synthesis

    Guangcong Wang, Zhaoxi Chen, Chen Change Loy, and Zi- wei Liu. Sparsenerf: Distilling depth ranking for few-shot novel view synthesis. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 9065–9076,

  31. [31]

    Haoyuan Wang, Wenbo Hu, Lei Zhu, and Rynson W.H. Lau. Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19999–20008, Seattle, W A, USA, 2024. IEEE. 2

    2024

  32. [32]

    Gani: Global and near field illumination aware neural inverse rendering.arXiv preprint arXiv:2403.15651, 2024

    Jiaye Wu, Saeed Hadadan, Geng Lin, Matthias Zwicker, David Jacobs, and Roni Sengupta. Gani: Global and near field illumination aware neural inverse rendering.arXiv preprint arXiv:2403.15651, 2024. 2

    Pith/arXiv arXiv 2024

  33. [33]

    Recover- ing shape and spatially-varying surface reflectance under un- known illumination.ACM Transactions on Graphics (ToG), 35(6):1–12, 2016

    Rui Xia, Yue Dong, Pieter Peers, and Xin Tong. Recover- ing shape and spatially-varying surface reflectance under un- known illumination.ACM Transactions on Graphics (ToG), 35(6):1–12, 2016. 2

    2016

  34. [34]

    Sparsegs: Real- time 360° sparse view synthesis using gaussian splatting

    Haolin Xiong, Sairisheek Muttukuru, Rishi Upadhyay, Pradyumna Chari, and Achuta Kadambi. Sparsegs: Real- time 360° sparse view synthesis using gaussian splatting. Arxiv, 2023. 2, 3, 4

    2023

  35. [35]

    Gaussianob- ject: High-quality 3d object reconstruction from four views with gaussian splatting.ACM Transactions on Graphics, 43 (6), 2024

    Chen Yang, Sikuang Li, Jiemin Fang, Ruofan Liang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, and Qi Tian. Gaussianob- ject: High-quality 3d object reconstruction from four views with gaussian splatting.ACM Transactions on Graphics, 43 (6), 2024. 2, 3, 4

    2024

  36. [36]

    Reflective gaussian splatting.arXiv preprint, 2024

    Yuxuan Yao, Zixuan Zeng, Chun Gu, Xiatian Zhu, and Li Zhang. Reflective gaussian splatting.arXiv preprint, 2024. 1, 2, 3, 5, 8, 9, 7, 10, 11

    2024

  37. [37]

    Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models

    Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models. InCVPR,

  38. [38]

    Ye Yu and William A. P. Smith. Inverserendernet: Learn- ing single image inverse rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2

    2019

  39. [39]

    Relighting neural radiance fields with shadow and highlight hints

    Chong Zeng, Guojun Chen, Yue Dong, Pieter Peers, Hongzhi Wu, and Xin Tong. Relighting neural radiance fields with shadow and highlight hints. InACM SIGGRAPH 2023 Con- ference Proceedings, pages 1–11, 2023. 2

    2023

  40. [40]

    Rgb↔x: Image decomposition and synthesis using material- and lighting-aware diffusion models, 2024

    Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, and Miloˇs Haˇsan. Rgb↔x: Image decomposition and synthesis using material- and lighting-aware diffusion models, 2024. 6

    2024

  41. [41]

    NeILF++: Inter-Reflectable Light Fields for Geometry and Material Es- timation

    Jingyang Zhang, Yao Yao, Shiwei Li, Jingbo Liu, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. NeILF++: Inter-Reflectable Light Fields for Geometry and Material Es- timation. In2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 3578–3587, Paris, France,

  42. [42]

    Relitlrm: Generative relightable radiance for large reconstruction models.arXiv preprint arXiv:2410.06231, 2024

    Tianyuan Zhang, Zhengfei Kuang, Haian Jin, Zexiang Xu, Sai Bi, Hao Tan, He Zhang, Yiwei Hu, Milos Hasan, William T Freeman, et al. Relitlrm: Generative relightable radiance for large reconstruction models.arXiv preprint arXiv:2410.06231, 2024. 3

    Pith/arXiv arXiv 2024

  43. [43]

    Ner- factor: Neural factorization of shape and reflectance under an unknown illumination.ACM Transactions on Graphics (ToG), 40(6):1–18, 2021

    Xiuming Zhang, Pratul P Srinivasan, Boyang Deng, Paul De- bevec, William T Freeman, and Jonathan T Barron. Ner- factor: Neural factorization of shape and reflectance under an unknown illumination.ACM Transactions on Graphics (ToG), 40(6):1–18, 2021. 2

    2021

  44. [44]

    Modeling indirect illumination for inverse rendering

    Yuanqing Zhang, Jiaming Sun, Xingyi He, Huan Fu, Rongfei Jia, and Xiaowei Zhou. Modeling indirect illumination for inverse rendering. InCVPR, 2022. 2, 5, 8, 9, 10, 1

    2022

  45. [45]

    Fsgs: Real-time few-shot view synthesis using gaussian splatting, 2023

    Zehao Zhu, Zhiwen Fan, Yifan Jiang, and Zhangyang Wang. Fsgs: Real-time few-shot view synthesis using gaussian splatting, 2023. 3, 4 12 GAINS: Gaussian-based Inverse Rendering from Sparse Multi-View Captures Supplementary Material A. Overview of Appendices We categorize our appendices in the following way: • Sec. B provides additional details of our exper...

    2023