Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LumiNet claims that a diffusion model conditioned on latent intrinsic and extrinsic codes can relight an indoor scene from a target image alone, preserving geometry and albedo while transferring shadows, specular highlights, and indirect…

desk verdict LumiNet is a promising engineering contribution whose qualitative results are compelling, but the abstract overstates what the MIIW table shows and the cross-scene guarantee is an untested assumption; referee it, but expect revisions. read the letter →

arxiv 2412.00177 v3 pith:SQRY5AQW submitted 2024-11-29 cs.CV cs.GRcs.LG

Xiaoyan Xing , Konrad Groh , Sezer Karaoglu , Theo Gevers , Anand Bhattad This is my paper · ORCID
classification cs.CVcs.GRcs.LG
keywords indoorscenerelightinglatentintrinsicsdiffusionmodelsControlNetlightingtransfercross-attentionStyleGANimage-based
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LumiNet sets out to prove that indoor scene relighting can be done from two ordinary images alone: a source image whose geometry and materials are kept, and a target image whose lighting is imported. The paper argues that by conditioning a diffusion model on latent intrinsic features—an illumination-invariant scene map plus a 16-dimensional lighting code extracted from the target—the model can reproduce complex light behavior such as cast shadows, specular highlights, and inter-reflections, even across scenes with different layouts. It also claims that a variational StyleGAN-based data-generation pipeline supplies enough paired relighting examples to train this cross-scene transfer, and that the resulting method beats existing image-only relighters on indoor benchmarks and in-the-wild images. If true, LumiNet would make relighting a practical image-editing operation rather than a 3D-reconstruction problem.

What carries the argument

The load-bearing mechanism is the latent intrinsic decomposition: a pretrained encoder $f_\lambda$ that turns an image into a lighting-invariant feature map $A_o$ of shape $H \times W \times 128$ and a 16-dimensional lighting code $I_L$. This split is what lets LumiNet hold the source scene fixed while importing the target's lighting. On top of it, the Latent Intrinsic ControlNet concatenates $\{A_o, I_{L_t}'\}$ and passes the result through convolutions into the denoiser, and a learned MLP adaptor ($3072 \to 4096 \to 4096 \to 4096 \to 3072$) injects the lighting code into the cross-attention layers of a frozen latent diffusion model, so the lighting condition acts both as a spatial map and as a global embedding.

What would settle it

Render a single synthetic indoor scene twice under two known lighting conditions with a physically based renderer, extract the latent intrinsic features from both renders, and run LumiNet to relight the first render with the second render's lighting. If the extracted feature maps differ noticeably between the two lighting conditions, or if the relit output deviates substantially from the ground-truth second render, then the illumination-invariance premise that carries the method is false.

Watch

Extended reading notes

Core claim

The central discovery is that lighting transfer can be decomposed in latent space and re-injected through a diffusion model without explicit geometry, materials, or 3D reconstruction. Given a source image and a target image, a pretrained latent-intrinsic encoder maps each into a lighting-invariant feature map (geometry and albedo) and a low-dimensional extrinsic lighting code. LumiNet's Latent Intrinsic ControlNet concatenates the source's invariant map with the target's lighting code as the condition for a latent diffusion denoiser, while a learned MLP adaptor projects the lighting code into the cross-attention dimension of the pretrained text-to-image backbone. Training pairs are generated by a variational StyleGAN relighter that maps real bedroom images into a relightable latent space, together with multi-illumination real captures, and the model is trained only on same-scene pairs yet transfers lighting across different scenes at inference. The paper's claim is that this preserves geometry and albedo while synthesizing direct and indirect lighting effects, with a bypass decoder and post-processing steps (nearest-neighbor seed selection and flow-based inversion) added to clean up artifacts.

Load-bearing premise

The load-bearing premise is that the pretrained latent-intrinsic encoder returns a genuinely illumination-invariant, scene-agnostic split—the feature map holds only geometry and albedo while the 16-dimensional lighting code transfers across scenes—so if residual lighting leaks into the map or scene-specific content leaks into the code, the cross-scene conditioning fails.

Editorial extensions

If this is right

  • Indoor relighting becomes a two-image editing operation: no 3D reconstruction, multi-view capture, or per-scene optimization is needed.
  • Because training uses same-scene pairs but inference works cross-scene, the latent intrinsic split appears to capture lighting as a separable, transferable quantity rather than a scene-specific texture.
  • The method can handle light sources that are not visible in the source image, inferring where lamps should turn on and casting their light plausibly.
  • Second-order effects such as specular highlights, reflections, and inter-reflections can be produced by a latent-conditioned diffusion model, not only by explicit light transport simulation.
  • Pixel-wise metrics understate the gain: a one-pixel shift in a highlight can dominate RMSE and SSIM, so perceptual and user-study evaluation is the more meaningful comparison.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: the same latent intrinsic split could drive other lighting-conditioned edits, such as harmonizing a composited object into a scene or transferring daylight direction outdoors, since the conditioning signal is already separated from geometry.
  • Extension: interpolating single dimensions of the 16-dimensional lighting code and inspecting the relit images would test whether the code dimensions are semantically interpretable as light direction, intensity, or color.
  • Extension: because the method's ceiling is set by the pretrained encoder's invariance, an obvious stress test is to feed the same scene under extreme lighting (near-dark and overexposed) and measure whether the extracted feature map stays stable; the paper's failure cases with small or occluded lamps and chromatic shifts suggest that invariance is only partial.
  • Extension: the training data comes largely from StyleGAN-generated bedroom scenes and multi-illumination captures, so relighting kitchens or offices with unusual luminaires would test whether lighting transfer generalizes beyond the furniture and light-fixture distribution seen in training.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. LumiNet addresses single-image indoor relighting by combining a pretrained latent-intrinsic encoder with a ControlNet-style latent diffusion model. Given a source image and a target lighting image, it extracts a latent intrinsic map A_o and a 16-dimensional lighting code I_L, concatenates the spatially expanded lighting code with A_o as ControlNet conditioning, and additionally injects an MLP-transformed lighting embedding into cross-attention. Training uses same-scene relit pairs from MIIW and BigTime plus synthetic pairs generated by a variational StyleGAN-based pipeline; a bypass decoder and seed-selection/rectified-flow post-processing are used at inference. The paper reports MIIW RMSE/SSIM, an in-the-wild surface-normal proxy and a user study, and ablations. The central claim is that, despite same-scene-only training, LumiNet generalizes to cross-scene lighting transfer in the wild, preserving geometry and albedo while synthesizing shadows, specularities, and interreflections.

Significance. Relighting indoor scenes from a single image with no 3D or multi-view input is an important problem, and LumiNet's architecture is a sensible combination of existing components: latent intrinsics provide a compact conditioning signal, and the diffusion prior supplies plausible light transport. The paper is transparent in places: it evaluates on an external MIIW test split, uses the official evaluation code, acknowledges the sensitivity of pixel metrics, and explicitly states in §6.1 that it does not surpass the MIIW-specialist methods Latent-Intrinsic and SA-AE on the benchmark. The qualitative examples, especially lamp switching and second-order effects such as reflections, are striking. However, the evidence for the headline claims is currently incomplete: the abstract overstates Table 1, the cross-scene generalization rests on an unvalidated invariance/transferability assumption about the frozen encoder, and the in-the-wild quantitative metrics rely on a proxy normal estimator and an unreported small user study. These are fixable with additional experiments and more careful claims.

major comments (5)
  1. [Abstract, §6.1, Table 1] The abstract and introduction claim LumiNet 'outperforms existing approaches,' but §6.1 and Table 1 show the opposite on the main benchmark: on the MIIW test set, Ours has RMSE 0.310 and SSIM 0.440 (color-corrected 0.240 and 0.527), while Latent-Intrinsic [75] achieves RMSE 0.297 and SSIM 0.473 (color-corrected 0.222 and 0.571), and SA-AE [25] is also better in most rows. The only MIIW methods LumiNet clearly beats are RGB↔X variants. The central quantitative claim should be revised to 'competitive with specialist methods and better than generalist diffusion baselines,' or supported by new cross-scene quantitative evidence.
  2. [§5.1, §5.3, Eq. (3)] The method's cross-scene generalization rests on two properties of the frozen latent-intrinsic encoder f_λ that are never measured: (i) the source intrinsic map A_o is illumination-invariant, and (ii) the 16-dimensional lighting code I_L^t is transferable across scenes with different layouts and materials. Because training uses same-scene pairs only (Eq. (3) conditions on A_o and I_L^t extracted from the same scene), the model could learn a same-scene shortcut in which A_o retains residual lighting or I_L^t encodes scene-specific information such as luminaire positions or viewpoint. The paper should validate these properties directly, for example by measuring A_o stability across the 25 illuminations of the same MIIW scene, by testing whether I_L^t codes from one scene produce consistent relighting when applied to another scene's A_o, and by training or evaluating on explicitly cross-scene held-out pairs.
  3. [§6.2, Table 2] The surface-normal consistency evaluation uses RGB↔X to estimate normals for both the original and relit images and treats the original-image normals as ground truth. This is not ground-truth geometry, and since RGB↔X is itself one of the baselines in the table, the metric conflates relighting quality with one estimator's behavior on the relit images; it could favor outputs whose appearance leads that estimator to return normals similar to the original. At minimum, use an independent normal or depth estimator or multi-view geometric data if available, and report the full error distribution, not just the median, with confidence intervals.
  4. [§6.2, §6.3, Table 2] The user study (31 participants) is the main evidence for open-world superiority, but the paper reports only mean rank scores with no number of test scenes, no variance, and no significance tests. Since I-PQ, L-PQ, and P-PQ are ordinal rankings, the authors should report per-scene distributions, inter-rater agreement, and pairwise Wilcoxon signed-rank tests (or equivalent) to show that the reported advantages, such as L-PQ 1.30 versus 2.52 for the nearest baseline, are not noise.
  5. [§5.2, §5.3, §6.2] There is no quantitative cross-scene evaluation anywhere in the paper: Table 1 is same-scene MIIW relighting with ground truth, and Table 2's in-the-wild metrics have no lighting ground truth. The paper should add a controlled cross-scene experiment, for example by taking a source image from one MIIW scene and a target lighting condition from another MIIW scene and reporting perceptual metrics or a forced-choice user study against same-scene-trained baselines. Without this, the generalizable-relighting claim is supported only by qualitative examples.
minor comments (5)
  1. [§4.1] Report the number of lighting directions per scene, the CLIP similarity threshold, and the number of scenes kept after filtering; currently 'seven lighting variations' and '≈1K' are the only details, and no sensitivity analysis is given for these choices.
  2. [§5.3, Eq. (3)] The symbol ε is used both for the target noise and for the encoder in expressions such as ε(S_{L_t}); rename the encoder, for example E, and define all conditioning tensors explicitly.
  3. [Table 1] The row 'Ours (previous)' is not defined in the text; clarify what differs from 'Ours,' or remove the row if it refers to an earlier version of the method.
  4. [§6.2, §6.3] The user study is described in both §6.2 and §6.3 with overlapping content; consolidate the descriptions and specify whether the 31-participant study is the same one reported in Table 2.
  5. [Figure 2] The notation S_o^{L_o}, S_t^{L_t}, A_o, and I_L should be defined in the caption, and the 'Loss' arrow and the VAE-omission note are difficult to read at the current resolution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claimed cross-scene relighting is trained on same-scene pairs and evaluated on external MIIW ground truth and in-the-wild images; self-cited components are used as frozen tools rather than as conclusions that force the result.

full rationale

The derivation chain is not circular. LumiNet's training objective (Eq. 3) is a standard conditional diffusion loss on same-scene relight pairs; the cross-scene generalization claim is an inductive empirical assertion, not an analytical consequence of the objective. The latent-intrinsic encoder f_lambda and the StyLitGAN generator are cited from prior work, including papers with overlapping authorship, but they function as fixed feature extractors and data generators, and the method's central claim is tested against the external MIIW benchmark (held-out test set), in-the-wild qualitative comparisons, and user studies. No equation reduces to its own inputs by construction: the lighting code IL_t is an input condition, not the output objective; the pixel-level RMSE/SSIM and perceptual rankings are not defined in terms of IL_t. The geometry-consistency evaluation uses RGB↔X's normal predictions as a proxy ground truth, which is a methodological weakness (a competitor's model as evaluator) but not circularity, because those normals are not used to train or fit LumiNet. The acknowledged limitations (small/occluded lamps, chromatic failure, no intensity control) are empirical failure modes, not evidence of definitional circularity. Self-citation is present but the load-bearing components are independently published, code-available models whose outputs are checked against real captured data; therefore the paper does not reduce to its citations.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two upstream artifacts the paper does not train or validate independently: the pretrained latent-intrinsic encoder from [75] and the StyLitGAN generator used to synthesize relit training pairs. In addition, several inference-time constants such as eta, seed count, and CLIP threshold are chosen by hand. No new physical entities are introduced.

free parameters (5)
  • Number of StyLitGAN lighting directions per scene = 7
    Section 4.1: seven lighting variations per scene are generated for training; chosen by hand with no ablation on the number of directions.
  • Rectified-flow inversion noise level eta = 0.99
    Section 5.5: set by hand for artifact removal; no sensitivity analysis is reported.
  • Number of seed candidates in nearest-neighbor selection = 30
    Supplementary Section B: 30 random seeds are sampled and ranked; no study of how the number of seeds affects quality or cost.
  • CLIP similarity filtering threshold = not reported
    Section 4.1: images are curated by CLIP similarity to keywords but the threshold or selection rule is not specified.
  • Loss weights for variational StyLitGAN = not reported
    Eq. (2) combines MSE, LPIPS, and KL losses without stated weights; default equal weighting is an unstated hand choice.
assumptions (4)
  • domain assumption The latent-intrinsic encoder f_lambda from [75] produces a lighting-invariant intrinsic map and a transferable 16-dimensional lighting code.
    Section 5.1; if the decomposition is imperfect or scene-specific, cross-scene conditioning fails.
  • domain assumption StyLitGAN's latent lighting manipulation produces plausible relit images for real LSUN bedroom images mapped by the variational encoder.
    Section 4.1; synthetic pairs are the primary source of diverse relighting pairs and the paper notes the mapping is approximate.
  • domain assumption Stable Diffusion 2.1 can accept a non-text lighting embedding in its cross-attention space when text prompts are empty.
    Section 5.2; the MLP adaptor is designed to match text-embedding dimensions, but no evidence is given that this space is appropriate for lighting codes.
  • domain assumption Rectified-flow inversion with eta=0.99 removes artifacts without substantially altering the lighting that LumiNet produced.
    Section 5.5; used at inference on real-world images, with visual evidence only.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting." pith.science (2026). https://pith.science/paper/SQRY5AQW

@misc{pith2026241200177,
  author       = {Pith},
  title        = {Pith review of: LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SQRY5AQW}},
  note         = {Machine review of arXiv:2412.00177}
}
read the original abstract

We introduce LumiNet, a novel architecture that leverages generative models and latent intrinsic representations for effective lighting transfer. Given a source image and a target lighting image, LumiNet synthesizes a relit version of the source scene that captures the target's lighting. Our approach makes two key contributions: a data curation strategy from the StyleGAN-based relighting model for our training, and a modified diffusion-based ControlNet that processes both latent intrinsic properties from the source image and latent extrinsic properties from the target image. We further improve lighting transfer through a learned adaptor (MLP) that injects the target's latent extrinsic properties via cross-attention and fine-tuning. Unlike traditional ControlNet, which generates images with conditional maps from a single scene, LumiNet processes latent representations from two different images - preserving geometry and albedo from the source while transferring lighting characteristics from the target. Experiments demonstrate that our method successfully transfers complex lighting phenomena including specular highlights and indirect illumination across scenes with varying spatial layouts and materials, outperforming existing approaches on challenging indoor scenes using only images as input.

Figures

Figures reproduced from arXiv: 2412.00177 by the authors.

Figure 1
Figure 1. LumiNet transfers complex lighting conditions from a target image (a) to a source image (b), synthesizing a relit version of the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. LumiNet’s Architecture and Training Pipeline. Left: LumiNet’s inference pipeline transfers lighting from a target image to a source image while preserving structure and materials (a). Right: Training utilizes latent intrinsic representations from a pretrained model [75], which decomposes images into lighting-invariant features and low-dimensional extrinsic lighting vectors. We train a condi￾tional latent diffusion m… view at source ↗
Figure 3
Figure 3. Training Framework of Variational StyLitGAN. (a) Traditional StyleGAN suffers from mode collapse, generating similar outputs every 10–20 iterations despite different latent codes. (b) Our variational approach maps real images to StyleGAN’s latent space via an encoder (qe), using a frozen pretrained generator (pg) from StyLitGAN [7]. The colored bars represent StyLitGAN’s disentangled lighting codes, enabling diverse… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison on the MIIW dataset [45]. To facilitate a direct comparison with prior work, we evaluate on the same images from the MIIW benchmark as those reported in Latent-Intrinsics [75]. Our method produces more realistic and consistent lighting effects. I…
Figure 5
Figure 5. Figure 5: In-the-wild image relighting visual comparison. We evaluate LumiNet on diverse indoor scenes under various target lighting conditions, more in the supplemental. Both RGB↔X [68] and IC-Light-v2 [71] require text prompts to achieve relighting, where we use descriptions d…
Figure 6
Figure 6. Figure 6: Same Scene Under Various Lighting. LumiNet can relight the same scene under different lighting conditions while preserving the overall layout, demonstrating effective disentanglement of intrinsic properties and lighting. ControlNet (w/o latent intrinsic) w/o. Flow inve…
Figure 7
Figure 7. Figure 7: Ablation Study. Left: Target light. Second: Source image. Third: Vanilla ControlNet (without latent intrinsic) fails at relighting, altering average color while losing details. Fourth: Without variational StyleGAN data, LumiNet fails to recognize light sources (e.g., s…
Figure 8
Figure 8. Figure 8: Failure case. Our method fails to recognize the lamp when it is either too small or positioned with its back to the camera. It also fails to transfer dramatic lighting color (chromaticity), such as the lighting of a Karaoke room. ows, specular highlights, and indirect …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A feed-forward two-stage compositing framework that harmonizes inserted objects across views using a Hilbert-ordered Gaussian color mapping, trained and evaluated on a new 480k-scene synthetic dataset.

Reference graph

Works this paper leans on

79 extracted references · 67 canonical work pages · cited by 1 Pith paper

  1. [75]

    Latent intrinsics emerge from training to relight

    Xiao Zhang, William Gao, Seemandhar Jain, Michael Maire, David Forsyth, and Anand Bhattad. Latent intrinsics emerge from training to relight. In NeurIPS, 2024. 2, 3, 4, 5, 6, 7

    work page 2024

  2. [25]

    Sa- ae for any-to-any relighting

    Zhongyun Hu, Xin Huang, Yaning Li, and Qing Wang. Sa- ae for any-to-any relighting. In ECCV. Springer, 2020. 3, 6, 7

    work page 2020

  3. [1]

    Shape, illumination, and reflectance from shading

    Jonathan T Barron and Jitendra Malik. Shape, illumination, and reflectance from shading. IEEE TPAMI, 37(8):1670– 1687, 2015. 3

    work page 2015

  4. [2]

    Barrow and J.M

    H.G. Barrow and J.M. Tenenbaum. Recovering intrinsic scene characteristics from images. In Computer Vision Sys- tems, 1978. 3

    work page 1978

  5. [3]

    Shadingnet: Image intrinsics by fine-grained shading decomposition

    Anil S Baslamisli, Partha Das, Hoang-An Le, Sezer Karaoglu, and Theo Gevers. Shadingnet: Image intrinsics by fine-grained shading decomposition. IJCV, 2021. 3

    work page 2021

  6. [4]

    Cut-and-paste object insertion by enabling deep image prior for reshading

    Anand Bhattad and David A Forsyth. Cut-and-paste object insertion by enabling deep image prior for reshading. In 3DV, 2022. 2

    work page 2022

  7. [5]

    Stylegan knows normal, depth, albedo, and more

    Anand Bhattad, Daniel McKee, Derek Hoiem, and David Forsyth. Stylegan knows normal, depth, albedo, and more. In NeurIPS, 2023. 3

    work page 2023

  8. [6]

    Make It So: Steering StyleGAN for Any Image Inversion and Editing

    Anand Bhattad, Viraj Shah, Derek Hoiem, and DA Forsyth. Make it so: Steering stylegan for any image inversion and editing. arXiv preprint arXiv:2304.14403, 2023. 2, 3, 4

    work page Pith review arXiv 2023

Show all 79 references
  1. [7]

    Anand Bhattad, James Soole, and David A. Forsyth. Styl- itgan: Image-based relighting via latent control. In CVPR,

  2. [8]

    Real-time 3d-aware portrait video relighting

    Ziqi Cai, Kaiwen Jiang, Shu-Yu Chen, Yu-Kun Lai, Hongbo Fu, Boxin Shi, and Lin Gao. Real-time 3d-aware portrait video relighting. In CVPR, 2024. 3

    2024

  3. [9]

    Intrinsic image decomposi- tion via ordinal shading

    Chris Careaga and Ya ˘gız Aksoy. Intrinsic image decomposi- tion via ordinal shading. ACM ToG, 2023. 3

    2023

  4. [10]

    Colorful diffuse intrinsic image decomposition in the wild

    Chris Careaga and Ya ˘gız Aksoy. Colorful diffuse intrinsic image decomposition in the wild. ACM ToG, 2024. 3

    2024

  5. [11]

    Intrinsic harmonization for illumination-aware image com- positing

    Chris Careaga, S Mahdi H Miangoleh, and Ya ˘gız Aksoy. Intrinsic harmonization for illumination-aware image com- positing. In SIGGRAPH Asia, 2023. 2

    2023

  6. [12]

    Scribblelight: Single image indoor relighting with scribbles

    Jun Myeong Choi, Annie Wang, Pieter Peers, Anand Bhat- tad, and Roni Sengupta. Scribblelight: Single image indoor relighting with scribbles. In CVPR, 2025. 3

    2025

  7. [13]

    Pie-net: Pho- tometric invariant edge guided network for intrinsic image decomposition

    Partha Das, Sezer Karaoglu, and Theo Gevers. Pie-net: Pho- tometric invariant edge guided network for intrinsic image decomposition. In CVPR, 2022. 3

    2022

  8. [14]

    Flashtex: Fast relightable mesh texturing with lightcontrolnet

    Kangle Deng, Timothy Omernick, Alexander Weiss, Deva Ramanan, Jun-Yan Zhu, Tinghui Zhou, and Maneesh Agrawala. Flashtex: Fast relightable mesh texturing with lightcontrolnet. In ECCV, 2024. 3

    2024

  9. [15]

    Intrinsic single-image hdr reconstruction

    Sebastian Dille, Chris Careaga, and Ya ˘gız Aksoy. Intrinsic single-image hdr reconstruction. In ECCV, 2024. 3

    2024

  10. [16]

    Generative models: What do they know? do they know things? let’s find out! arXiv preprint arXiv:2311.17137, 2023

    Xiaodan Du, Nicholas Kolkin, Greg Shakhnarovich, and Anand Bhattad. Generative models: What do they know? do they know things? let’s find out! arXiv preprint arXiv:2311.17137, 2023. 3

    2023 arXiv

  11. [17]

    Multi-view intrinsic im- ages of outdoors scenes with an application to relighting

    Sylvain Duch ˆene, Clement Riant, Gaurav Chaurasia, Jorge Lopez-Moreno, Pierre-Yves Laffont, Stefan Popov, Adrien Bousseau, and George Drettakis. Multi-view intrinsic im- ages of outdoors scenes with an application to relighting. ACM ToG, 2015. 3

    2015

  12. [18]

    Intrinsic image decom- position using paradigms

    David Forsyth and Jason J Rock. Intrinsic image decom- position using paradigms. IEEE TPAMI, 44(11):7624–7637,

  13. [19]

    Spotlight: Shadow-guided object relighting via dif- fusion

    Fr ´ed´eric Fortier-Chouinard, Zitian Zhang, Louis-Etienne Messier, Mathieu Garon, Anand Bhattad, and Jean-Franc ¸ois Lalonde. Spotlight: Shadow-guided object relighting via dif- fusion. arXiv preprint arXiv:2411.18665, 2024. 2, 3

    2024

  14. [20]

    Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray trac- ing

    Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray trac- ing. arXiv preprint arXiv:2311.16043, 2023. 3

    2023 arXiv

  15. [21]

    A survey on intrinsic images: Delv- ing deep into lambert and beyond

    Elena Garces, Carlos Rodriguez-Pardo, Dan Casas, and Jorge Lopez-Moreno. A survey on intrinsic images: Delv- ing deep into lambert and beyond. IJCV, 130(3), 2022. 3

    2022

  16. [22]

    The sky’s the limit: Relightable outdoor scenes via a sky-pixel constrained illumination prior and outside-in visibility

    James Gardner, Evgenii Kashin, Bernhard Egger, and William Alfred Peter Smith. The sky’s the limit: Relightable outdoor scenes via a sky-pixel constrained illumination prior and outside-in visibility. In ECCV. Springer, 2024. 3

    2024

  17. [23]

    Towards a perceptual evaluation framework for lighting estimation

    Justine Giroux, Mohammad Reza Karimi Dastjerdi, Yannick Hold-Geoffroy, Javier Vazquez-Corral, and Jean-Franc ¸ois Lalonde. Towards a perceptual evaluation framework for lighting estimation. In CVPR, 2024. 6, 7

    2024

  18. [24]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. NeurIPS, 2020. 2

    2020

  19. [26]

    Nerf- facelighting: Implicit and disentangled face lighting rep- resentation leveraging generative prior in neural radiance fields

    Kaiwen Jiang, Shu-Yu Chen, Hongbo Fu, and Lin Gao. Nerf- facelighting: Implicit and disentangled face lighting rep- resentation leveraging generative prior in neural radiance fields. ACM ToG, 42, 2023. 3

    2023

  20. [27]

    Neu- ral gaffer: Relighting any object via diffusion

    Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neu- ral gaffer: Relighting any object via diffusion. In NeurIPS,

  21. [28]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR, 2019. 2, 4

    2019

  22. [29]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM ToG, 42(4), 2023. 3

    2023

  23. [30]

    Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting

    Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting. In CVPR, 2024. 2, 3

    2024

  24. [31]

    Lightit: Illumination modeling and control for diffusion models

    Peter Kocsis, Julien Philip, Kalyan Sunkavalli, Matthias Nießner, and Yannick Hold-Geoffroy. Lightit: Illumination modeling and control for diffusion models. arXiv preprint arXiv:2403.10615, 2024. 3, 7

    2024 arXiv

  25. [32]

    In- trinsic image diffusion for single-view material estimation

    Peter Kocsis, Vincent Sitzmann, and Matthias Nießner. In- trinsic image diffusion for single-view material estimation. In CVPR, 2024. 3

    2024

  26. [33]

    The retinex theory of color vision

    Edwin H Land. The retinex theory of color vision. Scientific american, 1977. 3

    1977

  27. [34]

    Lighting, reflectance and geometry estimation from 360 ◦ panoramic stereo, 2021

    Junxuan Li, Hongdong Li, and Yasuyuki Matsushita. Lighting, reflectance and geometry estimation from 360 ◦ panoramic stereo, 2021. 2

    2021

  28. [35]

    Learning intrinsic image de- composition from watching the world

    Zhengqi Li and Noah Snavely. Learning intrinsic image de- composition from watching the world. In CVPR, 2018. 4

    2018

  29. [36]

    Physically-based editing of indoor scene lighting from a single image

    Zhengqin Li, Jia Shi, Sai Bi, Rui Zhu, Kalyan Sunkavalli, Miloˇs Haˇsan, Zexiang Xu, Ravi Ramamoorthi, and Manmo- han Chandraker. Physically-based editing of indoor scene lighting from a single image. In ECCV, pages 555–572. Springer, 2022. 2

    2022

  30. [37]

    Multi-view inverse rendering for large-scale real- world indoor scenes, 2023

    Zhen Li, Lingli Wang, Mofang Cheng, Cihui Pan, and Ji- aqi Yang. Multi-view inverse rendering for large-scale real- world indoor scenes, 2023. 2

    2023

  31. [38]

    Photorealistic object insertion with diffusion-guided inverse rendering

    Ruofan Liang, Zan Gojcic, Merlin Nimier-David, David Acuna, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Photorealistic object insertion with diffusion-guided inverse rendering. In ECCV. Springer, 2024. 2

    2024

  32. [39]

    Urbanir: Large-scale urban scene inverse ren- dering from a single video

    Zhi-Hao Lin, Bohan Liu, Yi-Ting Chen, Kuan-Sheng Chen, David Forsyth, Jia-Bin Huang, Anand Bhattad, and Shen- long Wang. Urbanir: Large-scale urban scene inverse ren- dering from a single video. In 3DV, 2025. 3

    2025

  33. [40]

    Learning to factorize and relight a city

    Andrew Liu, Shiry Ginosar, Tinghui Zhou, Alexei A Efros, and Noah Snavely. Learning to factorize and relight a city. In ECCV. Springer, 2020. 3

    2020

  34. [41]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, 2022. 4

    2022

  35. [42]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. In ICLR, 2017. 1

    2017

  36. [43]

    Intrinsicdiffusion: joint in- trinsic layers from latent diffusion models

    Jundan Luo, Duygu Ceylan, Jae Shin Yoon, Nanxuan Zhao, Julien Philip, Anna Fr ¨uhst¨uck, Wenbin Li, Christian Richardt, and Tuanfeng Wang. Intrinsicdiffusion: joint in- trinsic layers from latent diffusion models. In SIGGRAPH,

  37. [44]

    Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang, Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, and Vishal M. Patel. Lightpainter: Interactive portrait relighting with freehand scribble. In CVPR, 2023. 3

    2023

  38. [45]

    A multi-illumination dataset of indoor object ap- pearance

    Lukas Murmann, Michael Gharbi, Miika Aittala, and Fredo Durand. A multi-illumination dataset of indoor object ap- pearance. In ICCV, 2019. 3, 4, 6, 7

    2019

  39. [46]

    Learning physics-guided face relighting under directional light

    Thomas Nestmeyer, Jean-Franc ¸ois Lalonde, Iain Matthews, Epic Games, Andreas Lehrmann, and AI Borealis. Learning physics-guided face relighting under directional light. 2020. 3

    2020

  40. [47]

    Multi-view relighting using a geometry-aware network

    Julien Philip, Micha ¨el Gharbi, Tinghui Zhou, Alexei A Efros, and George Drettakis. Multi-view relighting using a geometry-aware network. ACM ToG, 2019. 3

    2019

  41. [48]

    Diffusionlight: Light probes for free by painting a chrome ball

    Pakkapon Phongthawee, Worameth Chinchuthakun, Non- taphat Sinsunthithet, Amit Raj, Varun Jampani, Pramook Khungurn, and Supasorn Suwajanakorn. Diffusionlight: Light probes for free by painting a chrome ball. In CVPR,

  42. [49]

    A Diffu- sion Approach to Radiance Field Relighting using Multi- Illumination Synthesis

    Yohan Poirier-Ginter, Alban Gauthier, Julien Philip, Jean- Franc ¸ois Lalonde, and George Drettakis. A Diffu- sion Approach to Radiance Field Relighting using Multi- Illumination Synthesis. Computer Graphics Forum , 2024. 3

    2024

  43. [50]

    Difareli: Diffusion face relighting

    Puntawat Ponglertnapakorn, Nontawat Tritrong, and Supa- sorn Suwajanakorn. Difareli: Diffusion face relighting. In ICCV, 2023. 2, 3, 5

    2023

  44. [51]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML. PMLR, 2021. 4

    2021

  45. [52]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 2, 3, 1

    2022

  46. [53]

    Semantic image inversion and editing using rectified stochastic differential equations

    L Rout, Y Chen, N Ruiz, C Caramanis, S Shakkottai, and W Chu. Semantic image inversion and editing using rectified stochastic differential equations. 2024. 5

    2024

  47. [54]

    Nerf for outdoor scene relighting

    Viktor Rudnev, Mohamed Elgharib, William Smith, Lingjie Liu, Vladislav Golyanik, and Christian Theobalt. Nerf for outdoor scene relighting. In ECCV, 2022. 3

    2022

  48. [55]

    Shadows don’t lie and lines can’t bend! generative models don’t know projective geometry

    Ayush Sarkar, Hanlin Mai, Amitabh Mahapatra, Svetlana Lazebnik, David A Forsyth, and Anand Bhattad. Shadows don’t lie and lines can’t bend! generative models don’t know projective geometry... for now. In CVPR, 2024. 9

    2024

  49. [56]

    A light stage on every desk

    Soumyadip Sengupta, Brian Curless, Ira Kemelmacher- Shlizerman, and Steven M Seitz. A light stage on every desk. In ICCV, 2021. 3

    2021

  50. [57]

    Single image portrait relighting

    Tiancheng Sun, Jonathan T Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul Debevec, and Ravi Ramamoorthi. Single image portrait relighting. ACM ToG, 2019. 3

    2019

  51. [58]

    Relight my nerf: A dataset for novel view synthesis and re- lighting of real world objects

    Marco Toschi, Riccardo De Matteo, Riccardo Spezialetti, Daniele De Gregorio, Luigi Di Stefano, and Samuele Salti. Relight my nerf: A dataset for novel view synthesis and re- lighting of real world objects. In CVPR, 2023. 3

    2023

  52. [59]

    Zero-reference low-light enhancement via physical quadru- ple priors

    Wenjing Wang, Huan Yang, Jianlong Fu, and Jiaying Liu. Zero-reference low-light enhancement via physical quadru- ple priors. In CVPR, 2024. 5, 6, 7

    2024

  53. [60]

    Chen Xi, Peng Sida, Yang Dongchen, Liu Yuan, Pan Bowen, Lv Chengfei, and Zhou. Xiaowei. Intrinsicanything: Learn- ing diffusion priors for inverse rendering under unknown il- lumination. In ECCV, 2024. 3

    2024

  54. [61]

    Intrinsic appearance decomposition using point cloud representation

    Xiaoyan Xing, Konrad Groh, Sezer Karaoglu, and Theo Gev- ers. Intrinsic appearance decomposition using point cloud representation. In ICCVW, 2023. 3

    2023

  55. [62]

    Retinex-diffusion: On controlling illumination conditions in diffusion models via retinex theory

    Xiaoyan Xing, Vincent Tao Hu, Jan Hendrik Metzen, Konrad Groh, Sezer Karaoglu, and Theo Gevers. Retinex-diffusion: On controlling illumination conditions in diffusion models via retinex theory. arXiv preprint arXiv:2407.20785, 2024. 3

    2024 arXiv

  56. [63]

    Good seed makes a good crop: Discovering secret seeds in text-to- image diffusion models

    Katherine Xu, Lingzhi Zhang, and Jianbo Shi. Good seed makes a good crop: Discovering secret seeds in text-to- image diffusion models. arXiv preprint arXiv:2405.14828,

    arXiv

  57. [64]

    S3net: A single stream structure for depth guided image relighting

    Hao-Hsiang Yang, Wei-Ting Chen, and Sy-Yen Kuo. S3net: A single stream structure for depth guided image relighting. In CVPR, 2021. 3, 7

    2021

  58. [65]

    Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop

    Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianx- iong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. 4

    2015 arXiv

  59. [66]

    Self- supervised outdoor scene relighting

    Ye Yu, Abhimitra Meka, Mohamed Elgharib, Hans-Peter Seidel, Christian Theobalt, and William AP Smith. Self- supervised outdoor scene relighting. In ECCV. Springer,

  60. [67]

    Dilightnet: Fine-grained light- ing control for diffusion-based image generation

    Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained light- ing control for diffusion-based image generation. In SIG- GRAPH, 2024. 2, 3

    2024

  61. [68]

    Rgb-x: Image decomposition and synthesis using material-and lighting-aware diffusion models

    Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, and Miloˇs Ha ˇsan. Rgb-x: Image decomposition and synthesis using material-and lighting-aware diffusion models. In SIG- GRAPH, 2024. 2, 3, 6, 7

    2024

  62. [69]

    Cohen, and Brian Curless

    Edward Zhang, Michael F. Cohen, and Brian Curless. Emp- tying, refurnishing, and relighting indoor spaces. ACM ToG, 35(6), 2016. 2

    2016

  63. [70]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 2, 5, 8

    2023

  64. [71]

    Scaling in-the-wild training for diffusion-based illumination harmo- nization and editing by imposing consistent light transport

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion-based illumination harmo- nization and editing by imposing consistent light transport. In ICLR, 2025. 2, 3, 5, 6, 7

    2025

  65. [72]

    Efros, Eli Shecht- man, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 4

    2018

  66. [73]

    Neu- ral light transport for relighting and view synthesis

    Xiuming Zhang, Sean Fanello, Yun-Ta Tsai, Tiancheng Sun, Tianfan Xue, Rohit Pandey, Sergio Orts-Escolano, Philip Davidson, Christoph Rhemann, Paul Debevec, et al. Neu- ral light transport for relighting and view synthesis. ACM ToG, 40(1), 2021. 2

    2021

  67. [74]

    Nerfac- tor: Neural factorization of shape and reflectance under an unknown illumination

    Xiuming Zhang, Pratul P Srinivasan, Boyang Deng, Paul De- bevec, William T Freeman, and Jonathan T Barron. Nerfac- tor: Neural factorization of shape and reflectance under an unknown illumination. ACM ToG, 40(6), 2021. 3

    2021

  68. [76]

    Zerocomp: Zero-shot object compositing from image intrinsics via dif- fusion

    Zitian Zhang, Fr ´ed´eric Fortier-Chouinard, Mathieu Garon, Anand Bhattad, and Jean-Franc ¸ois Lalonde. Zerocomp: Zero-shot object compositing from image intrinsics via dif- fusion. arXiv preprint arXiv:2410.08168, 2024. 2, 3

    2024 arXiv

  69. [77]

    Srinivasan, Dor Verbin, Keunhong Park, Ricardo Martin Brualla, and Philipp Henzler

    Xiaoming Zhao, Pratul P. Srinivasan, Dor Verbin, Keunhong Park, Ricardo Martin Brualla, and Philipp Henzler. IllumiN- eRF: 3D Relighting Without Inverse Rendering. InNeruIPS,

  70. [78]

    Deep single-image portrait relighting

    Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, and David W Ja- cobs. Deep single-image portrait relighting. In ICCV, 2019. 3

    2019

  71. [79]

    I 2-sdf: Intrinsic indoor scene reconstruction and editing via raytracing in neural sdfs, 2023

    Jingsen Zhu, Yuchi Huo, Qi Ye, Fujun Luan, Jifan Li, Dian- bing Xi, Lisha Wang, Rui Tang, Wei Hua, Hujun Bao, and Rui Wang. I 2-sdf: Intrinsic indoor scene reconstruction and editing via raytracing in neural sdfs, 2023. 2 LumiNet: Latent Intrinsics Meets Diffusion Models for I...

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.