Pith. sign in

REVIEW 3 major objections 5 minor 40 references

LightCrafter relights video by translating a PBR re-render of the input, not the raw footage, and beats prior state-of-the-art on real benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 07:56 UTC pith:L5TKEEUN

load-bearing objection A solid hybrid idea for video relighting — PBR proxy translation plus a diffusion refiner — with good synthetic evidence, but the real-world evaluation is the weak link: the DL3DV round-trip can be satisfied by pipeline-specific correction rather than true relighting. the 3 major comments →

arxiv 2607.08016 v2 pith:L5TKEEUN submitted 2026-07-09 cs.CV cs.GR

LightCrafter: PBR-Conditioned Video Diffusion Refinement for Controllable and Consistent Relighting

Zixin Guo , Yehonathan Litman , Yifeng He , John Miller , Chuhan Chen , Deva Ramanan This is my paper
classification cs.CV cs.GR
keywords video relightingphysically based renderingdiffusion modelsinverse renderingtemporal consistencyartifact-matched traininggenerative video translationlong-form video
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

LightCrafter tries to establish that video relighting is best treated as refinement of a physically based proxy: first recover a scene's intrinsics, re-render it under the target light with a forward renderer, and let a video diffusion model clean up that noisy render into a photorealistic output. The point is that the PBR proxy already carries the structured lighting change — shadows, highlights, shading — and is temporally consistent by construction, so the diffusion model only needs to correct inverse-rendering artifacts instead of learning what an environment map is. The authors argue this hybrid outperforms both inverse-rendering-only and direct generative translation on real-world relighting benchmarks, and provides long-form consistency without chunk-boundary drift. If right, it moves the hard part of relighting from the generative model back to the renderer, making relighting control a matter of rendering rather than prompting.

Core claim

LightCrafter's central claim is that a noisy physically based rendering of a video under target illumination is a better conditioning signal for generative relighting than the input video or intrinsics alone. The pipeline first recovers scene intrinsics — depth, camera pose, albedo, normals, roughness/metalness — via off-the-shelf inverse renderers, optimizes a source environment map so a PBR render matches the input, then forward-renders the recovered scene under the target illumination using a physically based renderer with a Cook-Torrance BRDF and explicit visibility. It then fine-tunes a pretrained video diffusion transformer to translate that PBR proxy into a photorealistic video, train

What carries the argument

The central object is the PBR proxy video: a physically based rendering of the recovered scene state under the target illumination, computed with the rendering equation using a Lambertian diffuse lobe and a Cook-Torrance microfacet specular term, with visibility from meshified depth. Because incident illumination is additive, the target can encode environment maps, point lights, area lights, or combinations, and the rendering is frame-aligned and temporally consistent by construction. The diffusion refiner takes the latent of the proxy concatenated with the noisy target latent and predicts noise; it learns artifact correction, not relighting. The artifact-matched data curation — synthetic pa

Load-bearing premise

The load-bearing premise is the real-world pseudo-pair assumption: after optimizing the source environment map so the PBR render matches the input, any remaining discrepancy is treated as inverse-rendering error rather than lighting mismatch, so pairing the render with the input teaches artifact correction without ground-truth relit videos; if the optimized lighting absorbs scene-dependent appearance instead, the refiner may learn a pipeline-specific correction rather than tr

What would settle it

Take a real multi-illumination video with captured ground-truth light B and input light A, run the full pipeline from A to B, and compare the refined output to the captured B frames and to the PBR proxy alone. If the refinement does not clearly beat the proxy, or if a round-trip (A→B→A) succeeds mainly by restoring a learned pipeline-specific correction, the pseudo-pair assumption and the central hybrid claim are undermined.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Relighting control becomes a rendering problem: any target light that a renderer can express — environment maps, point lights, area lights, combinations — is baked into the proxy, so the diffusion model never has to interpret illumination concepts.
  • Long-form relighting is stabilized: because the proxy is consistent across all frames and the refiner uses overlap-fused temporal tiling, sequences far longer than the training window can be relit without chunk-boundary drift or independent interpretation of the lighting.
  • Scene editing reduces to re-rendering: material edits, light insertion, and object compositing are applied to the recovered intrinsics or lighting, re-rendered, and refined by the same trained refiner with no retraining.
  • Training-data composition matters: the ablations show that removing synthetic pairs hurts relighting control while removing real-world pseudo-pairs hurts photorealism, implying the artifact-matched pairing is what makes the refiner generalize.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the pseudo-pair assumption doesn't hold — i.e., the optimized source light absorbs scene-dependent appearance rather than leaving only inverse-rendering error — the real-world training could teach the refiner a pipeline-specific correction, making the round-trip evaluation partially a test of that learned mapping rather than of true relighting.
  • The proxy-translation recipe likely transfers to other video edits where a cheap forward model exists (material swapping, weather simulation, object compositing), since the diffusion model only needs residual artifact correction.
  • The overlap-fused temporal tiling is a general drop-in for extending any windowed video diffusion model to arbitrary length when a consistent conditioning signal is available; it could be tested independently of relighting.
  • A cleaner evaluation would use captured real multi-illumination video pairs; if the refiner's gain over the PBR proxy shrinks on such pairs, the contribution of generative refinement would need to be rebalanced.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. LightCrafter proposes a hybrid video relighting pipeline that first recovers scene intrinsics via inverse rendering, renders a physically-based rendering (PBR) proxy under the target illumination, and then fine-tunes a video diffusion model (CogVideoX-5B) to refine the proxy into a photorealistic relit video. The key idea is to bake the illumination target into the PBR render, so the diffusion model only needs to correct rendering artifacts rather than learn lighting concepts. Training uses synthetic paired videos (with ground-truth relit targets) and real-world pseudo-pairs, where an environment map is optimized so the PBR render matches the input video. The paper also contributes a synthetic benchmark, a long-video overlap-fusion inference scheme, and reports improvements over existing baselines on synthetic and real-world benchmarks.

Significance. If the claims are validated, the hybrid reformulation—translation of a PBR proxy rather than direct video-to-video translation—is a conceptually appealing way to combine explicit light control with generative realism. The artifact-matched data curation idea is useful, and the release of dataset, code, and benchmark promises reproducibility. The synthetic paired evaluation is a genuine contribution with ground-truth relit targets, and the full method does improve on it. However, the real-world video evaluation rests on a round-trip reconstruction proxy rather than captured relit videos, and the pseudo-pair training relies on an unproven assumption that optimized lighting does not absorb inverse-rendering errors. These issues affect the strength of the central 'real-world SOTA' claim.

major comments (3)
  1. [§4.1, Table 1] The 'Real-World' results average MIT Multi-Illumination (captured ground truth, but static images) with DL3DV (a round-trip reconstruction proxy). The DL3DV protocol relights via Light-A-Video and then asks the method to map back to the original input; metrics are computed against the original frames. A model trained on pseudo-pairs (Sec. 3.4) that has learned its own pipeline's render-to-input mapping can score highly on this task without performing general relighting. Since the abstract claims to 'outperform prior SOTA on existing real-world relighting benchmarks,' the video-relighting claim is not supported by the DL3DV numbers. Report MIT and DL3DV separately and either add a captured multi-illumination video benchmark or explicitly scope the claim to image relighting plus a reconstruction proxy.
  2. [§3.4, Eq. (1)] The pseudo-pair construction optimizes a 64×128 HDR environment map to minimize ||R(S,L_source)−I|| with S held fixed. With imperfect normals, albedo, or geometry, this underconstrained optimization can absorb scene-dependent appearance errors into the lighting (e.g., shifting light direction to compensate for wrong normals, boosting intensity for dark albedo). The three scalar alignment parameters only correct global yaw/pitch/exposure. Thus the assertion that 'any discrepancy between Î and I comes from inverse rendering errors rather than lighting change' is an unsupported axiom. The pair (Î,I) may instead teach a pipeline-specific identity/correction mapping. Validate by comparing optimized envmaps to ground-truth lighting in synthetic scenes, or by testing on held-out scenes with known lighting.
  3. [Tables 1 and 2] All quantitative claims are point estimates without error bars, number of test videos, or significance tests. The real-world differences (e.g., LightCrafter 19.91 vs. PCRP-video 17.97 PSNR) could be within noise given the small evaluation set (10 MIT scenes, unstated DL3DV clip count). Report standard errors or confidence intervals across scenes and illuminations, and state the number of DL3DV test clips used.
minor comments (5)
  1. [Abstract / §5] The claim of outperforming prior SOTA on real-world benchmarks should be qualified given the proxy nature of the DL3DV evaluation. Section 5 also contains a typo: 'This division of yields higher fidelity' appears to be missing a word (likely 'labor').
  2. [Figures 9–13] The captions of Figures 9–13 contain garbled strings such as '/uni0000002c/uni00000051/...' that appear to be accidental text; please clean these up.
  3. [§4.1] Please specify the number of DL3DV test clips and report MIT and DL3DV metrics separately. Also define T-CLIP, which is used but not described.
  4. [§A.1, Eq. (9)] The conditioning noise augmentation is described as applied in pixel space before encoding; clarify whether the noise is added before or after tonemapping, and whether the same noise schedule is used at inference.
  5. [§A.3] PCRP-video is an author re-implementation since the original PCRP is image-only and has no public code. State clearly that this baseline is a video extension that may not reflect the original method's full performance, and consider labeling it 'PCRP-video (our impl.)' in tables.

Circularity Check

2 steps flagged

Real-world pseudo-pair training and the DL3DV evaluation both reduce, by construction, to fitting the PBR render to the input and then refining it back to that same input; the claimed real-world SOTA rests partly on this round-trip rather than on captured relit video ground truth.

specific steps
  1. fitted input called prediction [Sec. 3.4 (Real-World Pseudo-Pairs), Eq. (1)]
    "For real videos, we run inverse rendering to recover intrinsics, then optimize an environment map ˆLsource such that the PBR render video matches the input frames. This yields the rendering ˆI=R(S, ˆLsource) depicting the same illumination as the input. Any discrepancy between ˆI and I comes from inverse rendering errors rather than lighting change, so pairing (ˆI,I) teaches the model to correct artifacts on real-world appearance without ground-truth relit videos."

    Eq. (1) defines Lsource as the minimizer of ||R(S,Lsource)-I||, so the render Î is matched to the input I by optimization. The pair (Î,I) then uses I as both the quantity that determined the fitted envmap and the training target, so the refiner learns a pipeline-specific inverse/correction map Î→I rather than a general relighting from a novel illumination. The added per-scene yaw/pitch/exposure parameters in Sec. A.1 make the fit even more able to absorb scene-dependent appearance, undermining the claim that all discrepancy is inverse-rendering error.

  2. fitted input called prediction [Sec. 4.1 (Metrics); Table 1 Real-World [22,16]]
    "For real videos, we take unseen clips from DL3DV, 1) inverse render, forward render, and refine to match the input video, and 2), following [20] first relight a video based on a text prompt using [39] and use the relit video as input and map it back to the original after repeating inverse and forward rendering. ... On DL3DV, metrics are computed against original videos as proxy measures of preservation and temporal stability."

    The DL3DV evaluation is a round-trip: the input is inverse-rendered, re-rendered, and refined back to the original video, and the metrics are computed against that original. This is the same operation as the real-world pseudo-pair training (fit envmap to input, render, refine back to input). A model that has learned to correct its own renderer's artifacts can therefore score highly on this task without demonstrating general relighting under novel illumination. Since Table 1 reports DL3DV and MIT in the same 'Real-World' column, part of the claimed real-world SOTA is attributable to this reconstruction-style proxy rather than captured relit video ground truth.

full rationale

The synthetic training branch (Sec. 3.4, Sec. A.1) provides genuine ground-truth relit supervision: target videos are rendered in Blender and the PBR proxy is produced by running the same inverse-rendering stack on the source video, so the refiner learns to correct realistic artifacts while the target is a true relit video. The MIT Multi-Illumination evaluation is also real captured ground truth, and the PBR-proxy formulation itself is a legitimate design choice. However, the real-world pseudo-pair branch and the DL3DV evaluation are self-referential: the source envmap is optimized so the PBR render matches the input, and the model is then trained and evaluated on mapping that render back to the same input. This makes the real-world component of the headline claim partially forced by construction, though the method retains independent content from the synthetic and MIT results. Accordingly, the overall circularity is partial, not total: score 6.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 0 invented entities

The central claim rests on a chain of off-the-shelf inverse-rendering estimates, an assumed artifact distribution match, and several hand-chosen hyperparameters. No genuinely new physical entity is introduced. The most fragile entries are the per-scene fitted environment maps and the pseudo-pair identity assumption, because they couple training and evaluation to the paper's own pipeline.

free parameters (7)
  • Per-scene source environment map Lsource (64x128 HDR in log space) = Optimized per real clip with Adam lr 2e-3 for 2,000 iterations
    Used to construct real-world pseudo-pairs and to define the source illumination; can absorb scene-dependent appearance and gauge ambiguities, which is the main circularity risk.
  • Three scalar alignment parameters (global yaw, pitch, exposure) per real clip = Optimized jointly with envmap
    Absorb gauge ambiguities in recovered geometry and tonemapping; directly affect the pseudo-pair render and therefore what the refiner learns.
  • Per-synthetic-scene global similarity transform and depth-scale alignment (Eq. 7) = Estimated per synthetic scene from ground-truth coordinate system
    Changes shadow direction and cast-shadow length in the synthetic proxy; the paper explicitly does not replace recovered buffers, but the fitted alignment partly determines proxy quality.
  • Conditioning noise sigma for PBR proxy = log sigma ~ N(-3, 0.5), bulk near 0.05
    Regularizes the refiner against overfitting to specific inverse-rendering artifacts; the noise level is chosen by hand.
  • Temporal overlap kappa for long-video tiling = 12 pixel frames
    Controls fusion strength and compute; part of the long-form consistency recipe.
  • Guidance scale and DDIM steps = omega=6, 50 steps
    Sampling hyperparameters that trade fidelity against mode collapse; reported without sensitivity analysis.
  • TV regularization weight for envmap = lambda_TV=1e-3
    Smooths the estimated environment map during pseudo-pair creation; affects lighting structure and therefore the training signal.
axioms (6)
  • standard math The rendering equation (Eq. 2) and Cook-Torrance BRDF (Eq. 3) are an adequate physical model for the relighting task.
    Used for all PBR proxies; global illumination, interreflections, and transparency are explicitly named as effects the PBR render does not capture, so this axiom is only partially satisfied.
  • domain assumption Off-the-shelf inverse renderers (DiffusionRenderer, DiffusionLight, MegaSAM) yield sufficiently accurate intrinsics for the proxy to be useful.
    Invoked in Sec. 3.1 and in the finding that PBR-alone already outperforms some prior art. If intrinsics are too noisy, the proxy misleads the refiner.
  • domain assumption Artifact-matched PBR renders produced by running the same inverse-rendering stack at training time are representative of inference-time artifacts.
    Central claim of Sec. 3.4; the entire data-curation design depends on this distribution match.
  • ad hoc to paper Any discrepancy between the optimized PBR pseudo-render and the original real video comes from inverse rendering errors, not lighting estimation error.
    Explicitly stated in Sec. 3.4. This is the load-bearing assumption behind real-world pseudo-pairs and is not independently verified.
  • domain assumption Synthetic Blender scenes built from Objaverse assets and procedural materials transfer to real-world relighting.
    The synthetic branch of training relies on this; the paper's own ablation (No Real vs Full) shows real data is needed, partially validating the assumption but also showing synthetic-only is insufficient.
  • standard math The latent video diffusion objective (Eq. 4) and the frozen VAE faithfully encode and decode the necessary appearance information.
    Standard latent-diffusion assumption; the paper follows CogVideoX practice and does not re-derive it.

pith-pipeline@v1.3.0-alltime-deepseek · 14780 in / 10750 out tokens · 99064 ms · 2026-08-02T07:56:17.985988+00:00 · methodology

0 comments
read the original abstract

Video relighting requires balancing long-form temporal consistency with a physically grounded understanding of light transport, which depends on accurate estimation of intrinsic scene properties such as materials, geometry, and illumination. Existing methods follow two paradigms: (1) reconstruct a video's photometric properties via inverse rendering and relight them to a target illumination via forward rendering, using physically-based rendering (PBR) or a neural renderer; these suffer from noisy reconstructions and struggle with hard-to-model effects such as global illumination. (2) Frame the task as generative video-to-video translation conditioned on relighting targets (a target environment map or text); this limits relighting control and temporal stability, since diffusion models struggle to translate long-form videos, and is constrained by the availability of input/relit training pairs. We propose LightCrafter, a hybrid pipeline that reformulates video relighting as video translation of a proxy video: rather than translating the input video directly to the target, we translate a PBR rendering of the input under the target illumination to the final target. This bakes illumination targets into the PBR proxy, removing the need to teach the diffusion model illumination concepts like environment maps, and enables more intricate lighting control while naturally providing long-form temporal consistency. We show PBR renders alone already outperform some prior art but struggle with effects like global illumination; to capture these, we leverage photometric priors in video generation models by post-training CogVideoX on synthetic video pairs and real-world unpaired videos. We outperform prior state-of-the-art on existing real-world relighting benchmarks and contribute a synthetic benchmark for further analysis. We will release our dataset, benchmark, metrics, and code.

Figures

Figures reproduced from arXiv: 2607.08016 by Chuhan Chen, Deva Ramanan, John Miller, Yehonathan Litman, Yifeng He, Zixin Guo.

Figure 1
Figure 1. Figure 1: Controllable Video Relighting as a Rendering Refinement Task. Given an input video, LightCrafter utilizes inverse-rendered photometric and geometric scene properties and refines a physically-based rendering (PBR) proxy for video relighting control. The PBR rendering captures most scene-light interaction, while a video diffusion model translates the rendering to a photorealistic relit video with coherent sh… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed method. Our method recovers a relightable scene state from an input video, including photometric properties, geometry, camera motion, and illumination, and uses it to render a frame-aligned PBR render. The rendering provides an explicit interface for environment relighting, indoor light insertion, and G-buffer or scene-state editing, while a video diffusion refiner removes artifact… view at source ↗
Figure 3
Figure 3. Figure 3: Relighting Comparison on Synthetic & Real-World Data. Our method successfully relights synthetic (Rows 1,3) and real-world (Row 2: MIT Illumination [23], Row 4: DL3DV [18]) scenes across diverse scenarios while baselines exhibit relighting errors in structure and baked-in lighting from the source. 4.1 Experimental Setup. Datasets. We curate and train on a dataset of paired videos and PBR renderings. It con… view at source ↗
Figure 4
Figure 4. Figure 4: Long-Form Consistent Generation. Our method produces temporally consistent relighting across long sequences as scene relighting is explicitly controlled and grounded through the PBR renderer. In contrast, the baselines suffer from inconsistencies across time, as shown by two relit frames produced when processing the video in independent batches. Despite only a small camera rotation between frames, the base… view at source ↗
Figure 5
Figure 5. Figure 5: Scene-Editing Applications. LightCrafter’s underlying explicit controllability allows for downstream scene editing applications, such as modifying lighting, materials, and inserting objects. Input PBR Ours (No Synthetic) Ours (No Real) Ours (Full) [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Training Data Ablation Studies. Removing real-world data from training leads to unrealistic structures and hallucinations in the relighting, while removing synthetic data leads to poor relighting accuracy. Training with both data types leads to accurate and faithful relighting. illumination conditioning, while inverse rendering methods accumulate geometry and material es￾timation errors that produce flicke… view at source ↗
Figure 7
Figure 7. Figure 7: Synthetic and Real-World Training Data Curation. A Supplementary We show additional qualitative comparisons on synthetic and real-world relighting data in Figs. 9-13. We also provide a supplementary anonymized webpage containing video relighting comparisons at https://lightcrafterr.github.io/lightcrafter. A.1 Artifact-Matched Training Data Our training data has two complementary sources. The synthetic bran… view at source ↗
Figure 7
Figure 7. Figure 7: Synthetic and Real-World Training Data Curation. A Supplementary We show additional qualitative comparisons on synthetic and real-world relighting data in Figs. 9-13. A.1 Artifact-Matched Training Data Our training data has two complementary sources. The synthetic branch provides paired ground-truth relighting supervision, while the real branch exposes the model to in-the-wild reconstruction and rendering … view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative comparison with the PCRP-video extension baseline on static-scene orbiting [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative comparison with the PCRP-video extension baseline on static-scene orbiting [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative comparison on synthetic relighting videos with a camera orbiting a static scene. [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative comparison on synthetic relighting videos with a camera orbiting a static scene. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative comparison on synthetic videos with moving cameras and dynamic scenes. [PITH_FULL_IMAGE:figures/full_fig_p018_10.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative comparison on synthetic videos with moving cameras and dynamic scenes. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative comparison on synthetic videos captured by a static camera under rotating [PITH_FULL_IMAGE:figures/full_fig_p019_11.png] view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative comparison on synthetic videos captured by a static camera under rotating [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative comparison on real-world video pseudo-pairs from DL3DV. We first relight [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative comparison on real-world video pseudo-pairs from DL3DV. We first relight [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative comparison on paired real-image relighting examples from the MIT Multi [PITH_FULL_IMAGE:figures/full_fig_p020_13.png] view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative comparison on paired real-image relighting examples from the MIT Multi [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 7 linked inside Pith

  1. [1]

    Gs3: Efficient relighting with triple gaussian splatting

    Zoubin Bi, Yixin Zeng, Chong Zeng, Fan Pei, Xiang Feng, Kun Zhou, and Hongzhi Wu. Gs3: Efficient relighting with triple gaussian splatting. InSIGGRAPH Asia, 2024

    2024

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Physically-based shading at disney

    Brent Burley. Physically-based shading at disney. InSIGGRAPH Courses, 2012

    2012

  4. [4]

    Physically controllable relighting of photographs

    Chris Careaga and Ya˘gız Aksoy. Physically controllable relighting of photographs. InSIG- GRAPH, 2025

    2025

  5. [5]

    Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering

    Hongze Chen, Zehong Lin, and Jun Zhang. Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering. InICLR, 2025

    2025

  6. [6]

    Cook and Kenneth E

    Robert L. Cook and Kenneth E. Torrance. A reflectance model for computer graphics.ACM TOG, 1982. 10

    1982

  7. [7]

    Objaverse: A universe of annotated 3d objects.arXiv preprint arXiv:2212.08051, 2022

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects.arXiv preprint arXiv:2212.08051, 2022

    Pith/arXiv arXiv 2022

  8. [8]

    FlowR: Flowing from sparse to dense 3d reconstructions

    Tobias Fischer, Samuel Rota Bulò, Yung-Hsu Yang, Nikhil Keetha, Lorenzo Porzi, Norman Müller, Katja Schwarz, Jonathon Luiten, Marc Pollefeys, and Peter Kontschieder. FlowR: Flowing from sparse to dense 3d reconstructions. InICCV, 2025

    2025

  9. [9]

    Unirelight: Learning joint decomposition and synthesis for video relighting

    Kai He, Ruofan Liang, Jacob Munkberg, Jon Hasselgren, Nandita Vijaykumar, Alexander Keller, Sanja Fidler, Igor Gilitschenski, Zan Gojcic, and Zian Wang. Unirelight: Learning joint decomposition and synthesis for video relighting. InNeurIPS, 2025

    2025

  10. [10]

    Mitsuba 3 renderer,

    Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. Mitsuba 3 renderer,

  11. [11]

    Tensoir: Tensorial inverse rendering

    Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei Zhou, Zexiang Xu, and Hao Su. Tensoir: Tensorial inverse rendering. InCVPR, 2023

    2023

  12. [12]

    Real shading in unreal engine 4

    Brian Karis. Real shading in unreal engine 4. InSIGGRAPH Courses, 2013

    2013

  13. [13]

    Megasam: Accurate, fast, and robust structure and motion from casual dynamic videos.arXiv preprint arXiv:2412.04463, 2024

    Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast, and robust structure and motion from casual dynamic videos.arXiv preprint arXiv:2412.04463, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Diffusion renderer: Neural inverse and forward rendering with video diffusion models

    Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Chih-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Diffusion renderer: Neural inverse and forward rendering with video diffusion models. InCVPR, 2025

    2025

  15. [15]

    LuxRemix: Lighting decomposition and remixing for indoor scenes.arXiv preprint arXiv:2601.15283, 2026

    Ruofan Liang, Normal Müller, Ethan Weber, Duncan Zauss, Nandita Vijaykumar, Peter Kontschieder, and Christian Richardt. LuxRemix: Lighting decomposition and remixing for indoor scenes.arXiv preprint arXiv:2601.15283, 2026

    Pith/arXiv arXiv 2026

  16. [16]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InCVPR, 2024

    2024

  17. [17]

    Lightswitch: Multi-view relighting with material-guided diffusion

    Yehonathan Litman, Fernando De la Torre, and Shubham Tulsiani. Lightswitch: Multi-view relighting with material-guided diffusion. InICCV, 2025

    2025

  18. [18]

    Materialfusion: Enhancing inverse rendering with material diffusion priors

    Yehonathan Litman, Or Patashnik, Kangle Deng, Aviral Agrawal, Rushikesh Zawar, Fer- nando De la Torre, and Shubham Tulsiani. Materialfusion: Enhancing inverse rendering with material diffusion priors. In3DV, 2025

    2025

  19. [19]

    Unilumos: Fast and unified image and video relighting with physics-plausible feedback

    Pengwei Liu, Hangjie Yuan, Bo Dong, Jiazheng Xing, Jinwang Wang, Rui Zhao, Weihua Chen, and Fan Wang. Unilumos: Fast and unified image and video relighting with physics-plausible feedback. InNeurIPS, 2025

    2025

  20. [20]

    Light-x: Generative 4d video rendering with camera and illumination control

    Tianqi Liu, Zhaoxi Chen, Zihao Huang, Shaocong Xu, Saining Zhang, Chongjie Ye, Bohan Li, Zhiguo Cao, Wei Li, Hao Zhao, and Ziwei Liu. Light-x: Generative 4d video rendering with camera and illumination control. InICLR, 2026

    2026

  21. [21]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Pith/arXiv arXiv 2017

  22. [22]

    A multi-illumination dataset of indoor object appearance

    Lukas Murmann, Michael Gharbi, Miika Aittala, and Fredo Durand. A multi-illumination dataset of indoor object appearance. InICCV, 2019

    2019

  23. [23]

    Diffusionlight: Light probes for free by painting a chrome ball

    Pakkapon Phongthawee, Worameth Chinchuthakun, Nontaphat Sinsunthithet, Varun Jampani, Amit Raj, Pramook Khungurn, and Supasorn Suwajanakorn. Diffusionlight: Light probes for free by painting a chrome ball. InCVPR, 2024

    2024

  24. [24]

    Cc0 textures, 2026

    Public Domain. Cc0 textures, 2026. https://url-to-texture.com. 11

    2026

  25. [25]

    Marschner, Hongsong Li, and Kenneth E

    Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. Microfacet models for refraction through rough surfaces. InEGSR, 2007

    2007

  26. [26]

    Image quality assessment: From error visibility to structural similarity.IEEE TIP, 2004

    Zhou Wang, Alan Bovik, Hamid Sheikh, and Eero Simoncelli. Image quality assessment: From error visibility to structural similarity.IEEE TIP, 2004

    2004

  27. [27]

    Difix3d+: Improving 3d reconstructions with single-step diffusion models

    Jay Zhangjie Wu, Yuxuan Zhang, Haithem Turki, Xuanchi Ren, Jun Gao, Mike Zheng Shou, Sanja Fidler, Zan Gojcic, and Huan Ling. Difix3d+: Improving 3d reconstructions with single-step diffusion models. InCVPR, 2025

    2025

  28. [28]

    Lumosx: Relate any identities with their attributes for personalized video generation

    Jiazheng Xing, Fei Du, Hangjie Yuan, Pengwei Liu, Hongbin Xu, Hai Ci, Ruigang Niu, Weihua Chen, Fan Wang, and Yong Liu. Lumosx: Relate any identities with their attributes for personalized video generation. InICLR, 2026

    2026

  29. [29]

    Barron, Pratul P

    Xiaoyan Xing, Philipp Henzler, Junhwa Hur, Runze Li, Jonathan T. Barron, Pratul P. Srinivasan, and Dor Verbin. Gr3en: Generative relighting for 3d environments. InSIGGRAPH, 2026

    2026

  30. [30]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2025

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Yuxuan Zhang, Weihan Wang, Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Neilf: Neural incident light field for physically-based material estimation

    Yao Yao, Jingyang Zhang, Jingbo Liu, Yihang Qu, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. Neilf: Neural incident light field for physically-based material estimation. In ECCV, 2022

    2022

  32. [32]

    Poly haven - the public 3d asset library, 2024

    Greg Zaal and et al. Poly haven - the public 3d asset library, 2024. https://polyhaven.com/

    2024

  33. [33]

    Dilightnet: Fine-grained lighting control for diffusion-based image generation

    Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained lighting control for diffusion-based image generation. InSIGGRAPH, 2024

    2024

  34. [34]

    Lumen: Consistent video relighting and harmonious background replacement with video generative models.arXiv preprint arXiv:2508.12945, 2025

    Jianshu Zeng, Yuxuan Liu, Yutong Feng, Chenxuan Miao, Zixiang Gao, Jiwang Qu, Jianzhang Zhang, Bin Wang, and Kun Yuan. Lumen: Consistent video relighting and harmonious background replacement with video generative models.arXiv preprint arXiv:2508.12945, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Neilf++: Inter-reflectable light fields for geometry and material estimation

    Jingyang Zhang, Yao Yao, Shiwei Li, Jingbo Liu, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. Neilf++: Inter-reflectable light fields for geometry and material estimation. In ICCV, 2023

    2023

  36. [36]

    The unreason- able effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as a perceptual metric. InCVPR, 2018

    2018

  37. [37]

    Srinivasan, Boyang Deng, Paul Debevec, William T

    Xiuming Zhang, Pratul P. Srinivasan, Boyang Deng, Paul Debevec, William T. Freeman, and Jonathan T. Barron. Nerfactor: Neural factorization of shape and reflectance under an unknown illumination.ACM TOG, 2021

    2021

  38. [38]

    Diffusionharmonizer: Bridging neural reconstruction and photorealistic simulation with online diffusion enhancer.arXiv preprint arXiv:2602.24096, 2026

    Yuxuan Zhang, Katarína Tóthová, Zian Wang, Kangxue Yin, Haithem Turki, Riccardo de Lutio, Yen-Yu Chang, Or Litany, Sanja Fidler, and Zan Gojcic. Diffusionharmonizer: Bridging neural reconstruction and photorealistic simulation with online diffusion enhancer.arXiv preprint arXiv:2602.24096, 2026

    arXiv 2026

  39. [39]

    Light-a-video: Training-free video relighting via progressive light fusion

    Yujie Zhou, Jiazi Bu, Pengyang Ling, Pan Zhang, Tong Wu, Qidong Huang, Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Anyi Rao, Jiaqi Wang, and Li Niu. Light-a-video: Training-free video relighting via progressive light fusion. InICCV, 2025. 12 Target GT PBR Input Materials Envmaps DL3DV Dataset Synthetic Dataset PBR Renderer Envmap Estimator Optimized...

    2025

  40. [2022]

    https://mitsuba-renderer.org