pith. sign in

arxiv: 2606.11314 · v1 · pith:BMTROBQ6new · submitted 2026-06-09 · 💻 cs.CV · cs.GR

TRON: Tracing Rays to Orchestrate a Neural Renderer for 3D Gaussian Reconstructions

Or Perel , Hassan Abu Alhaija , Zian Wang , Jacob Munkberg , Matan Atzmon , Sanja Fidler , Masha Shugrina This is my paper

Pith reviewed 2026-06-27 13:24 UTC · model grok-4.3

classification 💻 cs.CV cs.GR
keywords neural rendering3D Gaussian splattingray tracingrelightingscene editinginverse renderingphotorealistic rendering
0
0 comments X

The pith

TRON combines 3D Gaussian ray tracing with neural rendering to enable controllable photorealistic output for real scenes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that a hybrid system can deliver both the realism of neural synthesis and the editability of explicit 3D representations when rendering captured environments. It does so by feeding a ray tracer's radiometric output as structured guidance into a lightweight neural renderer whose material estimates are first regularized by learned intrinsic decomposition priors. A reader would care because this setup supports interactive changes such as novel lighting, object insertion, motion, and material edits without the domain-gap artifacts typical of pure physically based or pure neural approaches. The method is trained in stages on a dataset of 2.1 million frames that mixes synthetic and real reconstructions to close remaining gaps.

Core claim

TRON shows that treating ray-tracer output as an explicit 3D scaffold, together with intrinsic material priors, allows a neural renderer to synthesize photorealistic images from imperfect Gaussian reconstructions while retaining fine-grained control over geometry, lighting, and materials.

What carries the argument

The ray-traced radiometric scaffold that structures input to the neural renderer, regularized by intrinsic decomposition priors on Gaussian material properties.

If this is right

  • Outperforms pure Gaussian PBR methods in realism under novel lighting and materials.
  • Outperforms prior neural renderers in editability and inference speed.
  • Supports interactive object insertion, dynamic motion, and material changes in captured scenes.
  • Enables the first practical interactive applications in real-world 3D reconstructions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same scaffold-plus-neural pattern may transfer to other explicit scene representations such as meshes or point clouds.
  • The multi-stage pretrain-and-fine-tune schedule could be reused for domain adaptation in other rendering tasks.
  • Success would indicate that explicit geometric guidance can anchor neural methods for more reliable editing across reconstruction techniques.

Load-bearing premise

Intrinsic decomposition priors and ray-tracer guidance are sufficient for the neural renderer to close the domain gap without new artifacts or loss of editability.

What would settle it

A test scene where material or lighting edits produce visible inconsistencies or artifacts absent from both pure PBR Gaussian renderings and pure neural baselines.

Figures

Figures reproduced from arXiv: 2606.11314 by Hassan Abu Alhaija, Jacob Munkberg, Masha Shugrina, Matan Atzmon, Or Perel, Sanja Fidler, Zian Wang.

Figure 1
Figure 1. Figure 1: TRON Architecture During training, we (1) apply an intrinsic decomposition model on all input view to get G-buffers and then (2) reconstruct a multiview scene as a set of 2D Gaussians, and (3) extract G-buffers diffusion priors per view, which we lift and bake in 3D. At inference time, given an envmap and camera view we (1) render a pair of buffers: PBR shaded and and irradiance. (2) Then the Neural Render… view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative Material Decompositions Comparison. As inverse-rendering is an ill-posed problem, previous Gaussian based methods [11, 51] that rely on direct optimization typically suffer from shadows baked into the albedo layer. Neural rendering methods [50, 32] either sacrifice spatial resolution to maintain a temporal context window (in the case of video-based models), or suffer from multi-view inconsisten… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative Relighting Comparisons. Top: Gaussian Relighting baselines. To achieve interactive￾speed shading, previous Gaussian-based relighting methods apply split-sum [42], which uses convolved lighting and is incompatible for modeling occlusions. TRON separates pbr and irradiance channels to avoid these pitfalls. Bottom: Neural Rendering baselines. Contemporary neural methods rely on priors embedded in … view at source ↗
Figure 4
Figure 4. Figure 4: Results Gallery. Top & middle: show TRON results for Novel Relighting on real-world captures. Bottom: editing global light intensity (left) and rendering shadows of a dynamic harmonized object (right). Diffusion Renderer GS-ID Ours Ours [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Harmonization Comparisons: Our method dynamically computes the highlight and shadow condi￾tions of the scene for a harmonized object. Furthermore, we validate our method on the Video Generative Models benchmark of VBench [98], with 5 criteria that allow evaluation directly on rendered videos without an accompanying prompt. Summarized in Tb.3, we compare smooth trajectories, using the first 100 frames per r… view at source ↗
Figure 6
Figure 6. Figure 6: Random seed sensitivity comparison: We evaluate the sensitivity of each method to random initialization by running each model four times on the same input image with different random seeds. This test measures whether a model consistently follows the input conditioning or instead hallucinates lighting and shadow details. UniRelight [32] and Diffusion Renderer [50] are both multi-step diffusion models, makin… view at source ↗
Figure 7
Figure 7. Figure 7: Additional material decomposition results. Showing gbuffer representation baked into the Gaussian field, each channel rendered individually. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison to Gaussian based methods on the Relighting Synthetic Benchmark. All methods were presented different illumination conditions during reconstruction. Top example: baselines exhibit baked shadows, in incorrect location and direction. Bottom row: long shadows due to novel relighting are not modeled by baselines, compared to TRON. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Comparison to Neural based methods on the Relighting Synthetic Benchmark. By adhering to irradiance cues rendered according to the geometry of the scene, TRON models the correct silhouttes, faithful to the scene geometry. In comparison, other neural baselines condition on priors embedded in their weights, resulting in shadows that do not adhere to the intricate geometries of the scene. 21 [PITH_FULL_IMAGE… view at source ↗
Figure 10
Figure 10. Figure 10: Comparisons and Ablation of Tracer v.s. Full Pipeline on mip-360 [4]. Left to right, columns 1-3: With shadows and highlights baked into the albedo, artifacts emerge during novel relighting. Column 4: Even with a clean albedo map, the shading model is limited in expressivity. Right column: the full pipeline benefits from the realism the neural renderer introduces [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Latent-fusion weight sweep. The Neural Renderer fuses the two input modalities by Ifused = wpbr Ipbr +wirr Iimg, where Ipbr and Iimg are the standardized VAE latents of the rendered PBR and irradiance images shown at the top. The centre cell (outlined in magenta) is the default fusion settings (wpbr, wirr) = (0.5, 0.5). The two axes carry distinct shading roles: increasing wirr (left→right) progressively … view at source ↗
Figure 12
Figure 12. Figure 12: Additional latent-fusion weight sweep. Same setup as [PITH_FULL_IMAGE:figures/full_fig_p026_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Overall Pipeline: (a) We extract per-frame material priors (b), and bake them into a 2D Gaussian representation as additional channels. (c) Given any novel lighting (envmap insets, rightmost), we can compute shading passes, PBR and irradiance (d), and use them to render a high-quality relit image with a Neural Renderer. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Agent Test Explainability. Showing rationale for final verdict of agent score, as presented in A/B photorealism benchmark of [PITH_FULL_IMAGE:figures/full_fig_p033_14.png] view at source ↗
read the original abstract

We introduce TRON, a rendering framework that combines 3D Gaussian ray tracing with neural rendering to enable realistic and controllable rendering of real-world 3D scenes under novel lighting, dynamic object motion, object insertion, and material editing. Prior approaches that rely solely on physically based rendering (PBR) of Gaussian representations struggle to achieve realistic relighting due to imperfections in reconstructed geometry, material estimates, and light transport estimation. At the same time, neural rendering methods often lack an explicit scene representation, limiting their ability to support interactive editing with fine-grained manipulation. TRON bridges these two paradigms. We use intrinsic decomposition priors from a learned inverse rendering model to regularize the material properties of a Gaussian field, and repurpose a ray tracer to provide radiometric guidance rather than final pixels. By treating this output as a structured 3D scaffold, we empower a lightweight neural renderer to bridge the domain gap between shading-model constrained estimates and photorealistic output. Our key insight is that the combination of explicit 3D knowledge with robust material priors provides speed and controllability, while neural rendering enables the synthesis of photorealistic images. To support real-world scenarios, we train our neural renderer with a multi-stage strategy consisting of large-scale pretraining and targeted fine-tuning on a newly constructed dataset of 2.1M rendered synthetic and real-world frames from 3D reconstructions. TRON outperforms Gaussian-based relighting methods in realism, and prior neural renderers in editability and speed. To the best of our knowledge, TRON is the first method to enable practical interactive applications in captured 3D environments, offering realistic appearance under dynamic geometric, lighting and material conditions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The paper introduces TRON, a hybrid rendering framework that combines 3D Gaussian ray tracing with neural rendering. It regularizes Gaussian material properties using intrinsic decomposition priors from a learned inverse rendering model, repurposes a ray tracer to provide radiometric guidance as a 3D scaffold for a lightweight neural renderer, and trains via multi-stage pretraining and fine-tuning on a new dataset of 2.1M rendered synthetic and real-world frames. The method claims to bridge PBR limitations (imperfect geometry/materials/light transport) and neural rendering limitations (lack of explicit editability), outperforming Gaussian-based relighting methods in realism and prior neural renderers in editability and speed, while enabling the first practical interactive applications in captured 3D environments under dynamic geometric, lighting, and material conditions.

Significance. If the empirical claims hold with proper validation, the work would be significant for enabling controllable photorealistic rendering of real-world captured scenes, combining explicit 3D structure and material priors with neural synthesis to support editing operations like relighting, object insertion, and material changes at interactive speeds.

major comments (3)
  1. [Abstract] Abstract: the central claims of outperformance in realism, editability and speed, plus being the first method for practical interactive applications, are asserted without any quantitative results, baselines, ablation studies, or dataset statistics. This makes the soundness of the contribution impossible to assess from the provided text.
  2. [Abstract] Abstract (and implied results section): the multi-stage training strategy on the 2.1M frame dataset is presented as the mechanism that closes the domain gap while retaining editability, yet no details on dataset construction, train/test splits, loss formulations, or comparisons to pure PBR or pure neural baselines are supplied to support this.
  3. [Abstract] Abstract: the key insight that 'the combination of explicit 3D knowledge with robust material priors provides speed and controllability, while neural rendering enables the synthesis of photorealistic images' is stated without derivation, equations, or empirical tests showing that the ray-tracer scaffold plus priors actually avoids artifacts that pure PBR or pure neural methods suffer from.
minor comments (1)
  1. [Abstract] The abstract is clearly written but would benefit from a brief mention of the specific neural renderer architecture or loss terms used in the scaffold-guided stage.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for highlighting issues with the abstract's presentation of claims. We agree that the abstract should better indicate the empirical grounding for our assertions and will revise it to incorporate key quantitative results, dataset scale, and a more precise statement of the core insight, drawing from the detailed experiments and method sections in the full manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claims of outperformance in realism, editability and speed, plus being the first method for practical interactive applications, are asserted without any quantitative results, baselines, ablation studies, or dataset statistics. This makes the soundness of the contribution impossible to assess from the provided text.

    Authors: We agree the abstract would be strengthened by concrete support. In revision we will add brief quantitative highlights (e.g., PSNR/SSIM gains and runtime figures from Section 5) and a mention of the 2.1M-frame dataset scale while preserving the high-level summary. Full baselines, ablations, and statistics appear in Sections 5 and 6; the 'first practical interactive applications' claim rests on the explicit editability and speed comparisons shown there. revision: yes

  2. Referee: [Abstract] Abstract (and implied results section): the multi-stage training strategy on the 2.1M frame dataset is presented as the mechanism that closes the domain gap while retaining editability, yet no details on dataset construction, train/test splits, loss formulations, or comparisons to pure PBR or pure neural baselines are supplied to support this.

    Authors: Dataset construction (synthetic + real captures from 3D Gaussian reconstructions), 80/20 train/test splits, loss terms (radiometric guidance + perceptual + material regularization), and direct comparisons to pure PBR and pure neural baselines are fully specified in Section 4.2 and the experiments of Section 5. Because abstracts have strict length limits we will add a concise clause on the multi-stage pretraining/fine-tuning strategy and dataset size; the complete supporting details remain in the body. revision: partial

  3. Referee: [Abstract] Abstract: the key insight that 'the combination of explicit 3D knowledge with robust material priors provides speed and controllability, while neural rendering enables the synthesis of photorealistic images' is stated without derivation, equations, or empirical tests showing that the ray-tracer scaffold plus priors actually avoids artifacts that pure PBR or pure neural methods suffer from.

    Authors: The derivation, including the ray-tracing scaffold formulation and material-prior regularization equations, is given in Section 3; artifact reduction (shading errors, editability failures) is quantified and visualized against pure PBR and pure neural baselines in Section 5 ablations. We will revise the abstract wording to state the insight more precisely and reference the observed artifact mitigation without adding equations, which are infeasible in the abstract format. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The abstract and available description outline a hybrid pipeline using Gaussian ray tracing, intrinsic priors, ray-tracer guidance as scaffold, and multi-stage training on a 2.1M frame dataset. No equations, derivations, fitted parameters renamed as predictions, or self-citation chains are present that reduce any claimed result to its inputs by construction. The central claim of combining explicit 3D knowledge with neural rendering for controllability and photorealism is presented as an empirical engineering combination without load-bearing self-referential steps. This matches the expectation that most papers are non-circular when no such reduction is exhibited.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are stated.

pith-pipeline@v0.9.1-grok · 5867 in / 1087 out tokens · 18461 ms · 2026-06-27T13:24:40.427605+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

129 extracted references · 15 canonical work pages

  1. [1]

    Barron, Jia-Bin Huang, Pratul P

    Hadi Alzayer, Philipp Henzler, Jonathan T. Barron, Jia-Bin Huang, Pratul P. Srinivasan, and Dor Verbin. Generative multiview relighting for 3d reconstruction under extreme illumination variation. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 10933–10942, June 2025

    2025

  2. [2]

    Gare: Relightable 3d gaussian splatting for outdoor scenes from unconstrained photo collections.ICCV, 2025

    Haiyang Bai, Jiaqi Zhu, Songru Jiang, Wei Huang, Tao Lu, Yuanqi Li, Jie Guo, Runze Fu, Yanwen Guo, and Lijun Chen. Gare: Relightable 3d gaussian splatting for outdoor scenes from unconstrained photo collections.ICCV, 2025

    2025

  3. [3]

    SLANG.D: Fast, Modular and Differentiable Shader Programming.ACM Trans

    Sai Praveen Bangaru, Lifan Wu, Tzu-Mao Li, Jacob Munkberg, Gilbert Bernstein, Jonathan Ragan-Kelley, Frédo Durand, Aaron Lefohn, and Yong He. SLANG.D: Fast, Modular and Differentiable Shader Programming.ACM Trans. Graph., 42(6), 2023

    2023

  4. [4]

    Barron, Ben Mildenhall, Dor Verbin, Pratul P

    Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields.CVPR, 2022

    2022

  5. [5]

    2024 , isbn =

    Zoubin Bi, Yixin Zeng, Chong Zeng, Fan Pei, Xiang Feng, Kun Zhou, and Hongzhi Wu. Gs3: Efficient relighting with triple gaussian splatting. InSIGGRAPH Asia 2024 Conference Papers, SA ’24, New York, NY , USA, 2024. Association for Computing Machinery. ISBN 9798400711312. doi: 10.1145/3680528.3687576. URL https://doi.org/10.1145/3680528. 3687576

    work page doi:10.1145/3680528.3687576 2024

  6. [6]

    Blender Foundation, Stadhouderskade 592, Amsterdam, 2024

    Blender Online Community.Blender - a 3D modelling and rendering package. Blender Foundation, Stadhouderskade 592, Amsterdam, 2024. URL http://www.blender.org

    2024

  7. [7]

    Gaslight: Gaussian splats for spatially-varying lighting in hdr.ArXiv, 2025

    Christophe Bolduc, Yannick Hold-Geoffroy, Zhixin Shu, and Jean-François Lalonde. Gaslight: Gaussian splats for spatially-varying lighting in hdr.ArXiv, 2025

    2025

  8. [8]

    Barron, Ce Liu, and Hendrik P.A

    Mark Boss, Raphael Braun, Varun Jampani, Jonathan T. Barron, Ce Liu, and Hendrik P.A. Lensch. Nerd: Neural reflectance decomposition from image collections. InIEEE International Conference on Computer Vision (ICCV), 2021

    2021

  9. [9]

    Physically controllable relighting of photographs

    Chris Careaga and Ya˘gız Aksoy. Physically controllable relighting of photographs. InProc. SIGGRAPH, 2025

    2025

  10. [10]

    Sam 3: Segment anything with concepts,

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman Rädle, Triantafyllos Afouras, Effrosyni Mavroudi, Katherine Xu, Tsung-Han Wu, Yu Zhou, Liliane ...

  11. [11]

    URL https://arxiv.org/abs/2511.16719

    Pith/arXiv arXiv

  12. [12]

    Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering

    Hongze Chen, Zehong Lin, and Jun Zhang. Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering. InICLR, 2025

    2025

  13. [13]

    Fast feedforward 3d gaussian splatting compression

    Yihang Chen, Qianyi Wu, Mengyao Li, Weiyao Lin, Mehrtash Harandi, and Jianfei Cai. Fast feedforward 3d gaussian splatting compression. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  14. [14]

    Differentiable inverse rendering with interpretable basis brdfs

    Hoon-Gyu Chung, Seokjun Choi, and Seung-Hwan Baek. Differentiable inverse rendering with interpretable basis brdfs. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 10

    2025

  15. [15]

    Cook and Kenneth E

    Robert L. Cook and Kenneth E. Torrance. A reflectance model for computer graphics.ACM Transactions on Graphics (ToG), 1(1):7–24, 1982

    1982

  16. [16]

    V omp: Predicting volumetric mechanical property fields.arXiv preprint arXiv:2510.22975, 2025

    Rishit Dagli, Donglai Xiang, Vismay Modi, Charles Loop, Clement Fuji Tsang, Anka He Chen, Anita Hu, Gavriel State, David IW Levin, and Maria Shugrina. V omp: Predicting volumetric mechanical property fields.arXiv preprint arXiv:2510.22975, 2025

    arXiv 2025

  17. [17]

    The triangle processor and normal vector shader: a vlsi system for high performance graphics

    Michael Deering, Stephanie Winner, Bic Schediwy, Chris Duffy, and Neil Hunt. The triangle processor and normal vector shader: a vlsi system for high performance graphics. InProceed- ings of the 15th Annual Conference on Computer Graphics and Interactive Techniques, SIG- GRAPH ’88, pages 21–30, New York, NY , USA, 1988. Association for Computing Machinery....

    work page doi:10.1145/54852.378468 1988

  18. [18]

    Ppisp: Physically- plausible compensation and control of photometric variations in radiance field reconstruction,

    Isaac Deutsch, Nicolas Moënne-Loccoz, Gavriel State, and Zan Gojcic. Ppisp: Physically- plausible compensation and control of photometric variations in radiance field reconstruction,

  19. [19]

    URL https://arxiv.org/abs/2601.18336

    Pith/arXiv arXiv

  20. [20]

    Jan-Niklas Dihlmann, Arjun Majumdar, Andreas Engelhardt, Raphael Braun, and Hen- drik P.A. Lensch. Subsurface scattering for gaussian splatting. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors,Ad- vances in Neural Information Processing Systems, volume 37, pages 121765–121789. Cur- ran Associates, Inc., 2024. ...

    2024

  21. [21]

    Gs-id: Illumination decomposition on gaussian splatting via diffusion prior and parametric light source optimization, 2024

    Kang Du, Zhihao Liang, and Zeyu Wang. Gs-id: Illumination decomposition on gaussian splatting via diffusion prior and parametric light source optimization, 2024. URL https: //arxiv.org/abs/2408.08524

    arXiv 2024

  22. [22]

    Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans

    Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10786–10796, 2021

    2021

  23. [23]

    Bermano, and Christian Theobalt

    Yotam Erel, Rishabh Dabral, Vladislav Golyanik, Amit H. Bermano, and Christian Theobalt. Practilight: Practical light control using foundational diffusion models.ACM Transactions on Graphics (TOG), 44(6):1–11, 2025

    2025

  24. [24]

    Rng: Relightable neural gaussians.Proceedings of CVPR 2025, 2025

    Jiahui Fan, Fujun Luan, Jian Yang, Milos Hasan, and Beibei Wang. Rng: Relightable neural gaussians.Proceedings of CVPR 2025, 2025

    2025

  25. [25]

    Spotlight: Shadow-guided object relighting via diffusion

    Frédéric Fortier-Chouinard, Zitian Zhang, Louis-Etienne Messier, Mathieu Garon, Anand Bhattad, and Jean-François Lalonde. Spotlight: Shadow-guided object relighting via diffusion. InInternational Conference on 3D Vision (3DV). IEEE, March 2026

    2026

  26. [26]

    Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray tracing

    Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray tracing. arXiv:2311.16043, 2023

    arXiv 2023

  27. [27]

    Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing

    Jian Gao, Chun Gu, Youtian Lin, Zhihao Li, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing. InEuropean Conference on Computer Vision, pages 73–89. Springer, 2024

    2024

  28. [28]

    Radiant foam: Real-time differentiable ray tracing.arXiv:2502.01157, 2025

    Shrisudhan Govindarajan, Daniel Rebain, Kwang Moo Yi, and Andrea Tagliasacchi. Radiant foam: Real-time differentiable ray tracing.arXiv:2502.01157, 2025

    arXiv 2025

  29. [29]

    Irgs: Inter-reflective gaussian splatting with 2d gaussian ray tracing

    Chun Gu, Xiaofei Wei, Zixuan Zeng, Yuxuan Yao, and Li Zhang. Irgs: Inter-reflective gaussian splatting with 2d gaussian ray tracing. InCVPR, 2025

    2025

  30. [30]

    Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering.CVPR, 2024

    Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering.CVPR, 2024

    2024

  31. [31]

    Milo: Mesh-in-the-loop gaussian splatting for detailed and efficient surface reconstruction.ACM Transactions on Graphics, 2025

    Antoine Guédon, Diego Gomez, Nissim Maruani, Bingchen Gong, George Drettakis, and Maks Ovsjanikov. Milo: Mesh-in-the-loop gaussian splatting for detailed and efficient surface reconstruction.ACM Transactions on Graphics, 2025. URL https://anttwo.github.io/milo/. 11

    2025

  32. [32]

    Radiomet- rically consistent gaussian surfels for inverse rendering.arXiv preprint arXiv:2603.01491, 2026

    Kyu Beom Han, Jaeyoon Kim, Woo Jae Kim, Jinhwan Seo, and Sung-eui Yoon. Radiomet- rically consistent gaussian surfels for inverse rendering.arXiv preprint arXiv:2603.01491, 2026

    arXiv 2026

  33. [33]

    Shape, Light, and Material De- composition from Images using Monte Carlo Rendering and Denoising.arXiv:2206.03380, 2022

    Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. Shape, Light, and Material De- composition from Images using Monte Carlo Rendering and Denoising.arXiv:2206.03380, 2022

    arXiv 2022

  34. [34]

    Unirelight: Learning joint decomposition and synthesis for video relighting.arXiv preprint arXiv:2506.15673, 2025

    Kai He, Ruofan Liang, Jacob Munkberg, Jon Hasselgren, Nandita Vijaykumar, Alexander Keller, Sanja Fidler, Igor Gilitschenski, Zan Gojcic, and Zian Wang. Unirelight: Learning joint decomposition and synthesis for video relighting.arXiv preprint arXiv:2506.15673, 2025

    arXiv 2025

  35. [35]

    Metags: A meta-learned gaussian-phong model for out-of-distribution 3d scene relighting.arXiv preprint arXiv:2405.20791, 2024

    Yumeng He, Yunbo Wang, and Xiaokang Yang. Metags: A meta-learned gaussian-phong model for out-of-distribution 3d scene relighting.arXiv preprint arXiv:2405.20791, 2024

    arXiv 2024

  36. [36]

    Lin, Marc Van Droogenbroeck, and Andrea Tagliasacchi

    Jan Held, Renaud Vandeghen, Sanghyun Son, Daniel Rebain, Matheus Gadelha, Yi Zhou, Ming C. Lin, Marc Van Droogenbroeck, and Andrea Tagliasacchi. Triangle splatting+: Differentiable rendering with opaque triangles.arXiv, 2025

    2025

  37. [37]

    2d gaussian splatting for geometrically accurate radiance fields,

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. InSIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024. doi: 10.1145/3641519.3657428

    work page doi:10.1145/3641519.3657428 2024

  38. [38]

    Kaiwen Jiang, Jia-Mu Sun, Zilu Li, Dan Wang, Tzu-Mao Li, and Ravi Ramamoorthi. Differ- entiable light transport with gaussian surfels via adapted radiosity for efficient relighting and geometry reconstruction.ACM Transactions on Graphics (TOG), 44(6), December 2025

    2025

  39. [39]

    Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces.arXiv preprint arXiv:2311.17977, 2023

    Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, and Yuexin Ma. Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces.arXiv preprint arXiv:2311.17977, 2023

    arXiv 2023

  40. [40]

    Tensoir: Tensorial inverse rendering

    Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei Zhou, Zexiang Xu, and Hao Su. Tensoir: Tensorial inverse rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  41. [41]

    Neural gaffer: Relighting any object via diffusion

    Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neural gaffer: Relighting any object via diffusion. InAdvances in Neural Information Processing Systems, 2024

    2024

  42. [42]

    James T. Kajiya. The rendering equation.SIGGRAPH Comput. Graph., 20(4):143–150, August

  43. [43]

    , title =

    ISSN 0097-8930. doi: 10.1145/15886.15902. URL https://doi.org/10.1145/15886.15902

    work page doi:10.1145/15886.15902

  44. [44]

    LumiGauss: Re- lightable Gaussian Splatting in the Wild

    Joanna Kaleta, Kacper Kania, Tomasz Trzci ´nski, and Marek Kowalski. LumiGauss: Re- lightable Gaussian Splatting in the Wild. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2025

    2025

  45. [45]

    Real shading in unreal engine 4.Proc

    Brian Karis and Epic Games. Real shading in unreal engine 4.Proc. Physically Based Shading Theory Practice, 4(3):1, 2013

    2013

  46. [46]

    Repurposing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Kon- rad Schindler. Repurposing diffusion-based image generators for monocular depth estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  47. [47]

    3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), July

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), July

  48. [48]

    URL https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/

  49. [49]

    3d gaussian splatting with deferred reflection

    Ye Keyang, Hou Qiming, and Zhou Kun. 3d gaussian splatting with deferred reflection. In ACM SIGGRAPH Conference Proceedings, Denver , CO, United States, July 28 - August 1, 2024, 2024. 12

    2024

  50. [50]

    3d gaussian splatting as markov chain monte carlo

    Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Yang-Che Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3d gaussian splatting as markov chain monte carlo. InAdvances in Neural Information Processing Systems (NeurIPS),

  51. [51]

    Spotlight Presentation

  52. [52]

    Intrinsic image diffusion for indoor single-view material estimation.Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    Peter Kocsis, Vincent Sitzmann, and Matthias Niessner. Intrinsic image diffusion for indoor single-view material estimation.Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  53. [53]

    Glossygs: Inverse rendering of glossy objects with 3d gaussian splatting.IEEE Transactions on Visualization and Computer Graphics, pages 1–14,

    Shuichang Lai, Letian Huang, Jie Guo, Kai Cheng, Bowen Pan, Xiaoxiao Long, Jiangjing Lyu, Chengfei Lv, and Yanwen Guo. Glossygs: Inverse rendering of glossy objects with 3d gaussian splatting.IEEE Transactions on Visualization and Computer Graphics, pages 1–14,

  54. [54]

    doi: 10.1109/TVCG.2025.3547063

    work page doi:10.1109/tvcg.2025.3547063 2025

  55. [55]

    Envidr: Implicit differentiable renderer with neural environment lighting.arXiv preprint arXiv:2303.13022, 2023

    Ruofan Liang, Huiting Chen, Chunlin Li, Fan Chen, Selvakumar Panneer, and Nandita Vijaykumar. Envidr: Implicit differentiable renderer with neural environment lighting.arXiv preprint arXiv:2303.13022, 2023

    arXiv 2023

  56. [56]

    Diffusionrenderer: Neural inverse and forward rendering with video diffusion models

    Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Zhi-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Diffusionrenderer: Neural inverse and forward rendering with video diffusion models. InThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2025

    2025

  57. [57]

    Gs-ir: 3d gaussian splatting for inverse rendering.arXiv preprint arXiv:2311.16473, 2023

    Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. Gs-ir: 3d gaussian splatting for inverse rendering.arXiv preprint arXiv:2311.16473, 2023

    arXiv 2023

  58. [58]

    GUS-IR: Gaussian Splatting With Unified Shading for Inverse Rendering .IEEE Transactions on Pattern Analysis & Machine Intelligence, 47(10):8364–8378, October 2025

    Zhihao Liang, Hongdong Li, Kui Jia, Kailing Guo, and Qi Zhang. GUS-IR: Gaussian Splatting With Unified Shading for Inverse Rendering .IEEE Transactions on Pattern Analysis & Machine Intelligence, 47(10):8364–8378, October 2025. ISSN 1939-3539. doi: 10. 1109/TPAMI.2025.3578416. URL https://doi.ieeecomputersociety.org/10.1109/TPAMI.2025. 3578416

    work page doi:10.1109/tpami.2025 2025

  59. [59]

    IRIS: Inverse rendering of indoor scenes from low dynamic range images

    Chih-Hao Lin, Jia-Bin Huang, Zhengqin Li, Zhao Dong, Christian Richardt, Tuotuo Li, Michael Zollhöfer, Johannes Kopf, Shenlong Wang, and Changil Kim. IRIS: Inverse rendering of indoor scenes from low dynamic range images. InConference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  60. [60]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22160–22169, 2024

    2024

  61. [61]

    Lightswitch: Multi-view relighting with material-guided diffusion

    Yehonathan Litman, Fernando De la Torre, and Shubham Tulsiani. Lightswitch: Multi-view relighting with material-guided diffusion. InICCV, 2025

    2025

  62. [62]

    Materialfusion: Enhancing inverse rendering with material diffusion priors

    Yehonathan Litman, Or Patashnik, Kangle Deng, Aviral Agrawal, Rushikesh Zawar, Fer- nando De la Torre, and Shubham Tulsiani. Materialfusion: Enhancing inverse rendering with material diffusion priors. In3DV, 2025

    2025

  63. [63]

    Intrinsicedit: Precise generative image manipulation in intrinsic space.ACM Transactions on Graphics, 44(4), 2025

    Linjie Lyu, Valentin Deschaintre, Yannick Hold-Geoffroy, Miloš Hašan, Jae Shin Yoon, Thomas Leimküehler, Christian Theobalt, and Iliyan Georgiev. Intrinsicedit: Precise generative image manipulation in intrinsic space.ACM Transactions on Graphics, 44(4), 2025

    2025

  64. [64]

    Lightlab: Controlling light sources in images with diffusion models

    Nadav Magar, Amir Hertz, Eric Tabellion, Yael Pritch, Alex Rav-Acha, Ariel Shamir, and Yedid Hoshen. Lightlab: Controlling light sources in images with diffusion models. 2025. doi: 10.1145/3721238.3730696

    work page doi:10.1145/3721238.3730696 2025

  65. [65]

    Neural microfacet fields for inverse rendering.2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 408–418, 2023

    Alexander Mai, Dor Verbin, Falko Kuester, and Sara Fridovich-Keil. Neural microfacet fields for inverse rendering.2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 408–418, 2023. URL https://api.semanticscholar.org/CorpusID:257900923

    2023

  66. [66]

    Ever: Exact volumetric ellipsoid rendering for real-time view synthesis, 2024

    Alexander Mai, Peter Hedman, George Kopanas, Dor Verbin, David Futschik, Qiangeng Xu, Falko Kuester, Jon Barron, and Yinda Zhang. Ever: Exact volumetric ellipsoid rendering for real-time view synthesis, 2024. URL https://arxiv.org/abs/2410.01804. 13

    arXiv 2024

  67. [67]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. InECCV, 2020

    2020

  68. [68]

    Simplicits: Mesh-free, geometry-agnostic elastic simulation.ACM Transactions on Graphics (TOG), 43 (4):1–11, 2024

    Vismay Modi, Nicholas Sharp, Or Perel, Shinjiro Sueda, and David IW Levin. Simplicits: Mesh-free, geometry-agnostic elastic simulation.ACM Transactions on Graphics (TOG), 43 (4):1–11, 2024

    2024

  69. [69]

    Tackling gaus- sian splats, physics simulation and visualization with nvidia kaolin and warp libraries

    Vismay Modi, Clement Fuji Tsang, Masha Shugrina, and Gilles Daviet. Tackling gaus- sian splats, physics simulation and visualization with nvidia kaolin and warp libraries. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Tech- niques Conference Labs, SIGGRAPH Labs ’25, New York, NY , USA, 2025. Association for Computing Ma...

    work page doi:10.1145/3721251.3736529 2025

  70. [70]

    3d gaussian ray tracing: Fast tracing of particle scenes.ACM Transactions on Graphics and SIGGRAPH Asia, 2024

    Nicolas Moenne-Loccoz, Ashkan Mirzaei, Or Perel, Riccardo de Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp, and Zan Gojcic. 3d gaussian ray tracing: Fast tracing of particle scenes.ACM Transactions on Graphics and SIGGRAPH Asia, 2024

    2024

  71. [71]

    Instant neural graphics primitives with a multiresolution hash encoding , volume=

    Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding.ACM Trans. Graph., 41(4):102:1–102:15, July 2022. doi: 10.1145/3528223.3530127. URL https://doi.org/10.1145/3528223.3530127

    work page doi:10.1145/3528223.3530127 2022

  72. [72]

    Extracting Triangular 3D Models, Materials, and Lighting From Images

    Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, and Sanja Fidler. Extracting Triangular 3D Models, Materials, and Lighting From Images. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8280–8290, June 2022

    2022

  73. [73]

    Gains: Gaussian-based inverse rendering from sparse multi-view captures, 2025

    Patrick Noras, Jun Myeong Choi, Didier Stricker, Pieter Peers, and Roni Sengupta. Gains: Gaussian-based inverse rendering from sparse multi-view captures, 2025. URL https://arxiv. org/abs/2512.09925

    arXiv 2025

  74. [74]

    Cosmos world foundation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    NVIDIA, Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, Daniel Dworakowski, Jiaojiao Fan, Michele Fenzi, Francesco Ferroni, Sanja Fidler, Dieter Fox, Songwei Ge, Yunhao Ge, Jinwei Gu, Siddharth Gururani, Ethan He, Jiahui Huang, Jacob Huffman, Pooya Jannaty, Jingy...

    Pith/arXiv arXiv 2025

  75. [75]

    Painting with 3d gaussian splat brushes

    Karran Pandey, Anita Hu, Humbert Hardy, Frank Nadeau, Eloi Champagne, Karan Singh, and Maria Shugrina. Painting with 3d gaussian splat brushes. InProceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Real-Time Live!, SIGGRAPH Real-Time Live! ’25, New York, NY , USA, 2025. Association for Computing Machinery. ...

    work page doi:10.1145/3721243.3735989 2025

  76. [76]

    Painting with 3d gaussian splat brushes

    Karran Pandey, Anita Hu, Clement Fuji Tsang, Or Perel, Karan Singh, and Maria Shugrina. Painting with 3d gaussian splat brushes. InProceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pages 1–10, 2025

    2025

  77. [77]

    Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 3rd edition,

    Matt Pharr, Wenzel Jakob, and Greg Humphreys.Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 3rd edition,

  78. [78]

    Multi-view relighting using a geometry-aware network.ACM Transactions on Graphics (SIGGRAPH Conference Proceedings), 38(4), July 2019

    Julien Philip, Michaël Gharbi, Tinghui Zhou, Alexei Efros, and George Drettakis. Multi-view relighting using a geometry-aware network.ACM Transactions on Graphics (SIGGRAPH Conference Proceedings), 38(4), July 2019. URL http://www-sop.inria.fr/reves/Basilic/2019/ PGZED19

    2019

  79. [79]

    A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis.Computer Graphics F orum, 2024

    Yohan Poirier-Ginter, Alban Gauthier, Julien Philip, Jean-François Lalonde, and George Drettakis. A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis.Computer Graphics F orum, 2024. ISSN 1467-8659. doi: 10.1111/cgf.15147

    work page doi:10.1111/cgf.15147 2024

  80. [80]

    Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields

    Yohan Poirier-Ginter, Jeffrey Hu, Jean-François Lalonde, and George Drettakis. Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields. InSIGGRAPH Asia 2025 - 18th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, Hong Kong, Hong Kong SAR China, December 2025. doi: 10.1145/ 3757377.3763971....

    arXiv 2025

Showing first 80 references.