Pith. sign in

REVIEW 3 major objections 4 minor 82 references

ScribbleLight: Single Image Indoor Relighting with Scribbles

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A scribble-based generative model can relight a single indoor photo locally, turning lamps on or off and adding shadows, while preserving the original colors and textures.

desk verdict First scribble-driven indoor relighting with plausible results, but self-reconstruction training leaves the scribbles' causal role unproven without a control. read the letter →

arxiv 2411.17696 v1 pith:MQODD54A submitted 2024-11-26 cs.CV

Jun Myeong Choi , Annie Wang , Pieter Peers , Anand Bhattad , Roni Sengupta This is my paper · ORCID
classification cs.CV
keywords singleimagerelightingindoorscenesscribble-basededitingalbedoconditioningdiffusionmodelsControlNetintrinsicdecompositionlocallighting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a single indoor photograph can be relit in a local, user-controlled way from scribble strokes alone. A user marks regions to brighten or darken, and the model can turn lamps on or off, add or move cast shadows, introduce highlights, or change window light, while keeping the room's original colors and material textures. The method, called ScribbleLight, is built by conditioning a Stable Diffusion model on the image's estimated albedo and steering the diffusion with an encoder-decoder ControlNet that takes scribbles and surface normals. The authors show quantitative and qualitative results against adapted baselines, and demonstrate iterative refinement where small scribble changes produce proportional lighting changes.

What carries the argument

The load-bearing mechanism is an albedo-conditioned latent diffusion model paired with a scribble-and-normal ControlNet. The albedo image, produced by intrinsic image decomposition, is encoded to a latent and concatenated with the noisy image latent, with a fixed noise level added to the albedo latent during training so the model trusts the image prior and tolerates albedo errors. The ControlNet's encoder compresses the scribble map and normal map into a lighting feature latent, and a decoder is trained to recover shading and normals from that latent, forcing the latent to retain geometric and shading information. A binary scribble encodes brighten versus darken intent, and unmarked regions are left to the model to fill plausibly.

What would settle it

Take a photo with a visible lamp and draw a scribble that darkens a shadow side opposite the lamp; if the output only darkens the marked pixels and leaves the lamp's highlight and adjacent shading unchanged, the method is following scribbles as a filter, not relighting. A second check is to rerun the BigTime evaluation with scribbles derived from the target image's true shading and compare against brightness-only baselines; if the gap collapses, the reported improvement may be simple hint-following rather than physically grounded relighting.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that scribbles are a sufficient control signal for single-image indoor relighting, provided the generative model is constrained to preserve the image's intrinsic albedo. The paper argues that a naive diffusion-based relighting model follows the scribble but drifts in color and texture, and that the drift is fixed by conditioning the diffusion denoiser on an albedo map with deliberately added noise, which makes the model robust to imperfect albedo estimates. It further argues that guiding the denoiser with a latent code jointly decoded to shading and normals keeps geometry intact and lets sparse scribbles produce physically plausible local effects such as soft highlights and layered shadows.

Load-bearing premise

The riskiest premise is that training on unpaired photos, where the target is the input image itself and the scribbles are generated from that same image's shading, teaches genuine relighting behavior rather than local brightening or darkening in response to the scribbles.

Editorial extensions

If this is right

  • Users can iteratively relight one photograph: turn lights on or off, move window light, add highlights and shadows, and refine with new scribbles.
  • Because multiple diffusion seeds give multiple valid results from the same scribbles, the method can act as a proposal tool for interior design and virtual staging.
  • Local lighting control is decoupled from materials: the albedo conditioning keeps wall, floor, and furniture colors stable across edits.
  • The approach does not need paired images of the same room under different lighting, only single photos with automatically generated scribble labels, so it can train on large real-image collections.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Implicit in the design is a testable extension to colored scribbles, which would let users control the color of the added light; the paper lists this as future work but does not implement it.
  • If the self-reconstruction training assumption is the main risk, one could stress-test it by evaluating on paired captures of the same room under controlled lighting changes and separating simple brighten-or-darken compliance from physically consistent shading transfer.
  • The noise-injected albedo conditioning suggests the same architecture could serve as a generic intrinsic-preserving image editor beyond relighting, for example local brightness adjustment that keeps material colors fixed, though the paper does not make that claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces ScribbleLight, a diffusion-based method for relighting a single indoor image from sparse user scribbles that indicate brighten/darken regions. The method fine-tunes Stable Diffusion with an albedo condition (with noise injected into the albedo latent to tolerate imperfect IID estimates) and a ControlNet that ingests a latent encoding of scribbles and surface normals, regularized by a decoder that reconstructs shading and normals. Training is performed on unpaired LSUN Bedroom images, with the input image itself as the denoising target; scribbles are automatically generated from the same image's estimated shading. The paper reports quantitative improvements over two adapted baselines on the BigTime dataset and qualitative demonstrations of turning lights on/off, adding highlights, cast shadows, and progressive editing.

Significance. If the central claim is valid, ScribbleLight is a practically useful interactive relighting tool, and the design choices (albedo-conditioned diffusion with noise-injected albedo, and the control encoder–decoder) are interesting technical contributions that could inform follow-up work. The paper provides ablations (Tables 2 and 3) that support the value of albedo conditioning and the normal/decoder components, and it includes qualitative evidence of user-controllable effects. However, the significance of the central claim is currently weakened by the absence of evidence that the scribbles are causally responsible for the observed relighting behavior, as detailed in the major comments.

major comments (3)
  1. [§3.3, Eq. (3)] The training objective for the ControlNet is self-reconstruction: the denoising target in Eq. (3) is the input image I itself, while the scribble condition M is generated from the same image's estimated shading Smono (Sec. 3.3). The model is therefore never trained on a single example where the scribble specifies a lighting different from the input. Consequently, any relighting at test time is an emergent behavior, and the paper provides no control experiment (e.g., a neutral all-0.5 scribble or a scribble mismatched to the input) to show that the scribble content, rather than the diffusion prior and albedo conditioning alone, drives the results. This is load-bearing for the paper's central claim of scribble-based local control; please add such controls and, if possible, training on paired relighting data (e.g., synthetic multi-light renderings) to demonstrate that the model learns a genuine mapping from scribble changes to lighting changes.
  2. [§4.1, §4.2, Table 1] The auto-generated test scribbles are derived from the target image's shading (Sec. 4.1), so the model is given an oracle hint derived from the ground-truth relit image. Combined with the self-reconstruction training objective, the reported gains in Table 1 may partly reflect the model performing local brightness adjustment in response to bright/dark hints rather than physically consistent relighting. Please report additional quantitative results with scribbles derived only from the source image (no target information) and with neutral/randomized scribbles, and explicitly compare these against the current target-derived scribble results to establish that the scribbles are the causal factor.
  3. [§3.3] The scribble generation rule states "M(x) = 1 when I(x) > µ + σ" and similar for the other thresholds, but the surrounding text says the scribbles are generated from the shading Smono, and I is a color image while µ and σ are described as intensity statistics of the training data. This appears to be a technical typo (presumably Smono(x) should be used), and it must be corrected because the exact preprocessing is necessary for reproducibility.
minor comments (4)
  1. [§3.2] Please specify the dimensions/channels of the lighting feature map f and the architecture of the control encoder EC more precisely, as these details are relevant to the claim that the latent contains sufficient shading and geometry information.
  2. [§4.1] The phrase "Lower RMSE and higher PSNR indicate better per-pixel similarity to the reference" is slightly redundant; it may be clearer to simply state that RMSE is lower-is-better and PSNR is higher-is-better.
  3. [§4.1, Table 1] The paper reports mean and best performance over 5 seeds but does not report standard deviations or significance tests; adding these would strengthen the quantitative claims.
  4. [Appendix B] In Table 4, the description of the RGB↔X baseline says the irradiance field is derived from the target image; please clarify whether the same target shading is provided to all methods, since the comparison is meant to be under monochromatic shading input.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: ScribbleLight's self-reconstruction training is a supervision limitation, not a derivation that reduces to its inputs.

full rationale

ScribbleLight is an empirical generative-modeling paper; it does not claim to derive relighting from first principles, so the circularity patterns that require equation-level reduction (self-definitional losses, imported uniqueness theorems, ansatz-by-citation, renaming known results) do not apply. The central training loss (Eq. 3) is a self-reconstruction objective: the ControlNet is trained to denoise the latent of the input image I while conditioned on scribbles M and normals N, where M is thresholded from the same image's monochromatic shading Smono (Sec. 3.3). Thus at training time the scribbles describe the input's existing shading rather than a lighting change, and the model is never shown paired images of the same room under two different lighting conditions. As a result, the relighting behavior at test time is an emergent generalization, and the BigTime evaluation would be more conclusive with a neutral-scribble or mismatched-scribble control. However, this is a training-supervision and evaluation-design limitation, not circularity under the stated rubric: the quantitative targets (BigTime pairs) are independent of the training set, no parameter is fitted to the test targets, and the paper's claims are empirical demonstrations rather than reductions of outputs to inputs. The only self-citations (StylitGAN [6] and latent-intrinsics relighting [78]) appear in related-work discussion and do not carry the method's load-bearing argument. The paper is also transparent about limitations, including physically inconsistent scribbles and bias toward common light colors. Overall, no specific step in the claimed derivation chain is equivalent to its own inputs by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method rests on pretrained models (Stable Diffusion, IID, DSINE) and on a training scheme that lacks paired relighting examples. The free parameters are mostly training-time choices, especially the empirically selected albedo noise level and the scribble generation thresholds. No new physical entities are introduced.

free parameters (3)
  • Albedo latent noise level T = 200
    Set to 200 as 'an optimal value we observed empirically' (Sec. 3.1); controls the uncertainty injected into the albedo condition and affects albedo preservation versus lighting variation.
  • Scribble threshold parameters = mu +/- sigma of training pixel intensities
    Automatic scribbles are computed as M=1 for I>mu+sigma and M=0 for I<mu-sigma (Sec. 3.3); these thresholds are hand-chosen and not validated against real user scribbles.
  • Dilation/erosion kernel size = random integer in [3,19]
    Applied to automatic scribbles to reduce edge alignment with image content; sampled per-example (Sec. 3.3), a hand-designed augmentation.
assumptions (5)
  • domain assumption Stable Diffusion v2, pretrained on LAION-5B, provides a sufficiently strong image prior for indoor relighting.
    The method relies on this prior to resolve ambiguities from sparse scribbles; invoked in Sec. 3.1 and throughout.
  • domain assumption The IID methods of Careaga and Aksoy provide albedo and shading estimates accurate enough for both conditioning and training scribble generation.
    Albedo maps A and shading Smono come from these models; errors propagate into training and evaluation; Sec. 3.1 and 3.3.
  • domain assumption DSINE normal estimates accurately capture scene geometry for the ControlNet.
    Normals N are a control input; Table 3 shows they are important, so their accuracy is load-bearing; Sec. 3.2.
  • ad hoc to paper Training on unpaired LSUN Bedrooms images, with the input image itself as the denoising target, is sufficient to learn relighting behavior.
    Eq. 3 denoises the input image latent conditioned on scribbles derived from that same image's shading; no paired different-lighting examples are used in training, so generalization to true relighting is assumed.
  • domain assumption Binary scribble annotations (1 brighten, 0 darken, 0.5 unlabeled) are a sufficient interface for lighting control.
    This is the interaction model in Sec. 3; it excludes colored lighting and intensity variation, a limitation the paper acknowledges.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ScribbleLight: Single Image Indoor Relighting with Scribbles." pith.science (2026). https://pith.science/paper/MQODD54A

@misc{pith2026241117696,
  author       = {Pith},
  title        = {Pith review of: ScribbleLight: Single Image Indoor Relighting with Scribbles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MQODD54A}},
  note         = {Machine review of arXiv:2411.17696}
}
read the original abstract

Image-based relighting of indoor rooms creates an immersive virtual understanding of the space, which is useful for interior design, virtual staging, and real estate. Relighting indoor rooms from a single image is especially challenging due to complex illumination interactions between multiple lights and cluttered objects featuring a large variety in geometrical and material complexity. Recently, generative models have been successfully applied to image-based relighting conditioned on a target image or a latent code, albeit without detailed local lighting control. In this paper, we introduce ScribbleLight, a generative model that supports local fine-grained control of lighting effects through scribbles that describe changes in lighting. Our key technical novelty is an Albedo-conditioned Stable Image Diffusion model that preserves the intrinsic color and texture of the original image after relighting and an encoder-decoder-based ControlNet architecture that enables geometry-preserving lighting effects with normal map and scribble annotations. We demonstrate ScribbleLight's ability to create different lighting effects (e.g., turning lights on/off, adding highlights, cast shadows, or indirect lighting from unseen lights) from sparse scribble annotations.

Figures

Figures reproduced from arXiv: 2411.17696 by the authors.

Figure 1
Figure 1. We introduce ScribbleLight, a generative model designed for indoor scene relighting from a single RGB image, that allows users [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. ScribbleLight consists of an Albedo-conditioned Stable image Diffusion model (trained in Stage 1), and a ControlNet (trained [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison of relighting quality between LightIt* [ [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of relighting quality of LightIt* [ [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Our method consistently follows the scribble, even with different random seeds, allowing users to select their preferred result. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Demonstration of ScribbleLight’s ability to generate different plausible relit images by turning on and off different lights while [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Minor changes to the scribbles yield proportional changes in the relit results, enabling a user to iteratively refine the scribbles to [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Given a physically incorrect scribble, i.e., the location of shadow does not match the light source, ScribbleLight often cre￾ates implausible lighting effects that best match the user scribbles. 5. Conclusion In this paper, we introduce ScribbleLight, a generative mode…
Figure 9
Figure 9. Figure 9: We demonstrate the target lighting representation used by each method—IIDiffusion [ [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

82 extracted references · 68 canonical work pages

  1. [1]

    Gwangbin Bae and Andrew J. Davison. Rethinking inductive biases for surface normal estimation. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) ,

  2. [2]

    Intrinsic scene prop- erties from a single rgb-d image

    Jonathan T Barron and Jitendra Malik. Intrinsic scene prop- erties from a single rgb-d image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 17–24, 2013. 3

    2013

  3. [3]

    Shape, illumination, and reflectance from shading

    Jonathan T Barron and Jitendra Malik. Shape, illumination, and reflectance from shading. IEEE transactions on pattern analysis and machine intelligence , 37(8):1670–1687, 2014. 3

    2014

  4. [4]

    Lasagna: Layered score distillation for disentangled object relighting, 2023

    Dina Bashkirova, Arijit Ray, Rupayan Mallick, Sarah Adel Bargal, Jianming Zhang, Ranjay Krishna, and Kate Saenko. Lasagna: Layered score distillation for disentangled object relighting, 2023. 3

    work page 2023

  5. [5]

    Joint learn- ing of intrinsic images and semantic segmentation

    Anil S Baslamisli, Thomas T Groenestege, Partha Das, Hoang-An Le, Sezer Karaoglu, and Theo Gevers. Joint learn- ing of intrinsic images and semantic segmentation. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pages 286–302, 2018. 3

    work page 2018

  6. [6]

    Anand Bhattad, James Soole, and D.A. Forsyth. Stylitgan: Image-based relighting via latent control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    work page 2024

  7. [7]

    Intrinsic image decomposi- tion via ordinal shading

    Chris Careaga and Ya ˘gız Aksoy. Intrinsic image decomposi- tion via ordinal shading. ACM Transactions on Graphics, 43 (1):1–24, 2023. 3, 4

    work page 2023

  8. [8]

    Colorful diffuse intrinsic image decomposition in the wild

    Chris Careaga and Ya ˘gız Aksoy. Colorful diffuse intrinsic image decomposition in the wild. ACM Trans. Graph., 43 (6), 2024. 2, 4

    work page 2024

Show all 82 references
  1. [9]

    A simple model for intrin- sic image decomposition with depth cues

    Qifeng Chen and Vladlen Koltun. A simple model for intrin- sic image decomposition with depth cues. In Proceedings of the IEEE international conference on computer vision, pages 241–248, 2013. 3

    2013

  2. [10]

    Sketch2photo: internet image montage

    Tao Chen, Ming-Ming Cheng, Ping Tan, Ariel Shamir, and Shi-Min Hu. Sketch2photo: internet image montage. ACM Trans. Graph., 28(5):1–10, 2009. 2

    2009

  3. [11]

    Sketchygan: Towards di- verse and realistic sketch to image synthesis, 2018

    Wengling Chen and James Hays. Sketchygan: Towards di- verse and realistic sketch to image synthesis, 2018. 2

    2018

  4. [12]

    Relighting4d: Neural re- lightable human from videos, 2022

    Zhaoxi Chen and Ziwei Liu. Relighting4d: Neural re- lightable human from videos, 2022. 2

    2022

  5. [13]

    Deepscribble: Interactive pathology image segmen- tation using deep neural networks with scribbles

    Sungduk Cho, Hyungjoon Jang, Jing Wei Tan, and Won-Ki Jeong. Deepscribble: Interactive pathology image segmen- tation using deep neural networks with scribbles. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 761–765, 2021. 2

    2021

  6. [14]

    Per- sonalized video relighting with an at-home light stage, 2024

    Jun Myeong Choi, Max Christman, and Roni Sengupta. Per- sonalized video relighting with an at-home light stage, 2024. 2

    2024

  7. [15]

    Scenetrilogy: On human scene-sketch and its complemen- tarity with photo and text, 2023

    Pinaki Nath Chowdhury, Ayan Kumar Bhunia, Aneeshan Sain, Subhadeep Koley, Tao Xiang, and Yi-Zhe Song. Scenetrilogy: On human scene-sketch and its complemen- tarity with photo and text, 2023. 3

    2023

  8. [16]

    Pie-net: Pho- tometric invariant edge guided network for intrinsic image decomposition

    Partha Das, Sezer Karaoglu, and Theo Gevers. Pie-net: Pho- tometric invariant edge guided network for intrinsic image decomposition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19790– 19799, 2022. 3

    2022

  9. [17]

    Tali Dekel, Chuang Gan, Dilip Krishnan, Ce Liu, and William T. Freeman. Smart, sparse contours to represent and edit images, 2018. 2

    2018

  10. [18]

    Flashtex: Fast relightable mesh texturing with lightcontrolnet, 2024

    Kangle Deng, Timothy Omernick, Alexander Weiss, Deva Ramanan, Jun-Yan Zhu, Tinghui Zhou, and Maneesh Agrawala. Flashtex: Fast relightable mesh texturing with lightcontrolnet, 2024. 2

    2024

  11. [19]

    Diffusionrig: Learning personalized priors for facial appearance editing

    Zheng Ding, Xuaner Zhang, Zhihao Xia, Lars Jebe, Zhuowen Tu, and Xiuming Zhang. Diffusionrig: Learning personalized priors for facial appearance editing. In CVPR, pages 12736–12746, 2023. 3

    2023

  12. [20]

    Sketchycoco: Image gener- ation from freehand scene sketches

    Chengying Gao, Qi Liu, Qi Xu, Limin Wang, Jianzhuang Liu, and Changqing Zou. Sketchycoco: Image gener- ation from freehand scene sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 2

    2020

  13. [21]

    Relightable 3d gaussians: Re- alistic point cloud relighting with brdf decomposition and ray tracing, 2024

    Jian Gao, Chun Gu, Youtian Lin, Zhihao Li, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussians: Re- alistic point cloud relighting with brdf decomposition and ray tracing, 2024. 2

    2024

  14. [22]

    Intrinsic images by clustering

    Elena Garces, Adolfo Munoz, Jorge Lopez-Moreno, and Diego Gutierrez. Intrinsic images by clustering. In Com- puter graphics forum , pages 1415–1424. Wiley Online Li- brary, 2012. 3

    2012

  15. [23]

    James A. D. Gardner, Evgenii Kashin, Bernhard Egger, and William A. P. Smith. The sky’s the limit: Re-lightable out- door scenes via a sky-pixel constrained illumination prior and outside-in visibility, 2024. 2

    2024

  16. [24]

    Dokania, Oliver Wang, Alexei A

    Arnab Ghosh, Richard Zhang, Puneet K. Dokania, Oliver Wang, Alexei A. Efros, Philip H. S. Torr, and Eli Shecht- man. Interactive sketch & fill: Multiclass sketch-to-image translation, 2019. 2

    2019

  17. [25]

    Ground truth dataset and baseline eval- uations for intrinsic image algorithms

    Roger Grosse, Micah K Johnson, Edward H Adelson, and William T Freeman. Ground truth dataset and baseline eval- uations for intrinsic image algorithms. In 2009 IEEE 12th International Conference on Computer Vision, pages 2335–

    2009

  18. [26]

    Face relighting with geometrically consistent shadows

    Andrew Hou, Michel Sarkis, Ning Bi, Yiying Tong, and Xi- aoming Liu. Face relighting with geometrically consistent shadows. In In Proceeding of IEEE Computer Vision and Pattern Recognition, New Orleans, LA, 2022. 2

    2022

  19. [27]

    Scribble-supervised video object segmenta- tion

    Peiliang Huang, Junwei Han, Nian Liu, Jun Ren, and Ding- wen Zhang. Scribble-supervised video object segmenta- tion. IEEE/CAA Journal of Automatica Sinica , 9(2):339– 353, 2022. 2

    2022

  20. [28]

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adver- sarial networks, 2018. 2

    2018

  21. [29]

    Self-supervised intrinsic image de- composition

    Michael Janner, Jiajun Wu, Tejas D Kulkarni, Ilker Yildirim, and Josh Tenenbaum. Self-supervised intrinsic image de- composition. Advances in neural information processing systems, 30, 2017. 3

    2017

  22. [30]

    Neural gaffer: Relighting any object via diffusion, 2024

    Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neural gaffer: Relighting any object via diffusion, 2024. 2

    2024

  23. [31]

    Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting, 2024

    Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting, 2024. 2

    2024

  24. [32]

    Lightit: Illumination modeling and control for diffusion models, 2024

    Peter Kocsis, Julien Philip, Kalyan Sunkavalli, Matthias Nießner, and Yannick Hold-Geoffroy. Lightit: Illumination modeling and control for diffusion models, 2024. 2, 3, 4, 5, 6, 1

    2024

  25. [33]

    In- trinsic image diffusion for indoor single-view material es- timation

    Peter Kocsis, Vincent Sitzmann, and Matthias Nießner. In- trinsic image diffusion for indoor single-view material es- timation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5198– 5208, 2024. 2, 3, 1

    2024

  26. [34]

    Lightness and retinex theory

    Edwin H Land and John J McCann. Lightness and retinex theory. Josa, 61(1):1–11, 1971. 3

    1971

  27. [35]

    Colorization using optimization

    Anat Levin, Dani Lischinski, and Yair Weiss. Colorization using optimization. ACM Trans. Graph. , 23(3):689–694,

  28. [36]

    Lighting, reflectance and geometry estimation from 360 ◦ panoramic stereo, 2021

    Junxuan Li, Hongdong Li, and Yasuyuki Matsushita. Lighting, reflectance and geometry estimation from 360 ◦ panoramic stereo, 2021. 2

    2021

  29. [37]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023. 4

    2023

  30. [38]

    Learning intrinsic image de- composition from watching the world

    Zhengqi Li and Noah Snavely. Learning intrinsic image de- composition from watching the world. In Computer Vision and Pattern Recognition (CVPR), 2018. 3, 4

    2018

  31. [39]

    Cgintrinsics: Better intrinsic image decomposition through physically-based rendering

    Zhengqi Li and Noah Snavely. Cgintrinsics: Better intrinsic image decomposition through physically-based rendering. In Proceedings of the European conference on computer vision (ECCV), pages 371–387, 2018

    2018

  32. [40]

    Inverse ren- dering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image

    Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Inverse ren- dering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2020

  33. [41]

    Openrooms: An end-to-end open framework for photorealistic indoor scene datasets, 2021

    Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song, Yuhan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gun- davarapu, Jia Shi, Sai Bi, Zexiang Xu, Hong-Xing Yu, Kalyan Sunkavalli, Milo ˇs Ha ˇsan, Ravi Ramamoorthi, and Manmohan Chandraker. Openrooms: An end-to-end open framework ...

    2021

  34. [42]

    Physically-based editing of indoor scene lighting from a single image, 2022

    Zhengqin Li, Jia Shi, Sai Bi, Rui Zhu, Kalyan Sunkavalli, Miloˇs Haˇsan, Zexiang Xu, Ravi Ramamoorthi, and Manmo- han Chandraker. Physically-based editing of indoor scene lighting from a single image, 2022. 2

    2022

  35. [43]

    Multi-view inverse rendering for large-scale real- world indoor scenes, 2023

    Zhen Li, Lingli Wang, Mofang Cheng, Cihui Pan, and Ji- aqi Yang. Multi-view inverse rendering for large-scale real- world indoor scenes, 2023. 2

    2023

  36. [44]

    Scribblesup: Scribble-supervised convolutional networks for semantic segmentation

    Di Lin, Jifeng Dai, Jiaya Jia, Kaiming He, and Jian Sun. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) ,

  37. [45]

    Intrinsicdiffusion: Joint in- trinsic layers from latent diffusion models

    Jundan Luo, Duygu Ceylan, Jae Shin Yoon, Nanxuan Zhao, Julien Philip, Anna Fr ¨uhst¨uck, Wenbin Li, Christian Richardt, and Tuanfeng Wang. Intrinsicdiffusion: Joint in- trinsic layers from latent diffusion models. In ACM SIG- GRAPH 2024 Conference Papers , New York, NY , USA,

    2024

  38. [46]

    Relightable neu- ral actor with intrinsic decomposition and pose control,

    Diogo Luvizon, Vladislav Golyanik, Adam Kortylewski, Marc Habermann, and Christian Theobalt. Relightable neu- ral actor with intrinsic decomposition and pose control,

  39. [47]

    Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang, Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, and Vishal M. Patel. Lightpainter: Interactive portrait relighting with freehand scribble, 2023. 2, 3

    2023

  40. [48]

    Qureshi, and Mehran Ebrahimi

    Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Z. Qureshi, and Mehran Ebrahimi. Edgeconnect: Generative image in- painting with adversarial edge learning, 2019. 2

    2019

  41. [49]

    Lehrmann

    Thomas Nestmeyer, Jean-Franc ¸ois Lalonde, Iain Matthews, and Andreas M. Lehrmann. Learning physics-guided face relighting under directional light, 2020. 2

    2020

  42. [50]

    Intuitive, interactive beard and hair synthesis with generative models, 2020

    Kyle Olszewski, Duygu Ceylan, Jun Xing, Jose Echevarria, Zhili Chen, Weikai Chen, and Hao Li. Intuitive, interactive beard and hair synthesis with generative models, 2020. 2

    2020

  43. [51]

    Poirier-Ginter, A

    Y . Poirier-Ginter, A. Gauthier, J. Phillip, J.-F. Lalonde, and G. Drettakis. A diffusion approach to radiance field relight- ing using multi-illumination synthesis. Computer Graphics Forum, 43(4), 2024. 2

    2024

  44. [52]

    Difareli: Diffusion face relighting,

    Puntawat Ponglertnapakorn, Nontawat Tritrong, and Supa- sorn Suwajanakorn. Difareli: Diffusion face relighting,

  45. [53]

    Faceshop: deep sketch-based face image editing

    Tiziano Portenier, Qiyang Hu, Attila Szab ´o, Siavash Ar- jomand Bigdeli, Paolo Favaro, and Matthias Zwicker. Faceshop: deep sketch-based face image editing. ACM Trans. Graph., 37(4), 2018. 2

    2018

  46. [54]

    Relightful harmonization: Lighting-aware portrait background replacement, 2024

    Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, and He Zhang. Relightful harmonization: Lighting-aware portrait background replacement, 2024. 2

    2024

  47. [55]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022. 2, 3, 1

    2022

  48. [56]

    Nerf for outdoor scene relighting

    Viktor Rudnev, Mohamed Elgharib, William Smith, Lingjie Liu, Vladislav Golyanik, and Christian Theobalt. Nerf for outdoor scene relighting. In European Conference on Com- puter Vision, pages 615–631. Springer, 2022. 2

    2022

  49. [57]

    Laion-5b: An open large-scale dataset for training next generation image-text models, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Lai...

    2022

  50. [58]

    Jacobs, and Jan Kautz

    Soumyadip Sengupta, Jinwei Gu, Kihwan Kim, Guilin Liu, David W. Jacobs, and Jan Kautz. Neural inverse rendering of an indoor scene from a single image. In International Conference on Computer Vision (ICCV), 2019. 3

    2019

  51. [59]

    Soumyadip Sengupta, Brian Curless, Ira Kemelmacher- Shlizerman, and Steven M. Seitz. A light stage on every desk. CoRR, abs/2105.08051, 2021. 2

    2021 arXiv

  52. [60]

    Sketch-guided image inpainting with partial discrete diffusion process, 2024

    Nakul Sharma, Aditay Tripathi, Anirban Chakraborty, and Anand Mishra. Sketch-guided image inpainting with partial discrete diffusion process, 2024. 2

    2024

  53. [61]

    Scribble- based 3d shape segmentation via weakly-supervised learn- ing

    Zhenyu Shu, Xiaoyong Shen, Shiqing Xin, Qingjun Chang, Jieqing Feng, Ladislav Kavan, and Ligang Liu. Scribble- based 3d shape segmentation via weakly-supervised learn- ing. IEEE Transactions on Visualization and Computer Graphics, 26(8):2671–2682, 2020. 2

    2020

  54. [62]

    Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan T

    Pratul P. Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan T. Barron. Nerv: Neural reflectance and visibility fields for relighting and view synthesis, 2020. 2

    2020

  55. [63]

    Re- lighting humans in the wild: Monocular full-body human re- lighting with domain adaptation, 2021

    Daichi Tajima, Yoshihiro Kanamori, and Yuki Endo. Re- lighting humans in the wild: Monocular full-body human re- lighting with domain adaptation, 2021. 2

    2021

  56. [64]

    Merlin: Single-shot material estimation and relighting for photometric stereo, 2024

    Ashish Tiwari, Satoshi Ikehata, and Shanmuganathan Ra- man. Merlin: Single-shot material estimation and relighting for photometric stereo, 2024. 2

    2024

  57. [65]

    Sketch-based 3d shape retrieval using convolutional neural networks

    Fang Wang, Le Kang, and Yi Li. Sketch-based 3d shape retrieval using convolutional neural networks. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. 3

    2015

  58. [66]

    De-nerf: Decoupled neural radiance fields for view-consistent appear- ance editing and high-frequency environmental relighting

    Tong Wu, Jia-Mu Sun, Yu-Kun Lai, and Lin Gao. De-nerf: Decoupled neural radiance fields for view-consistent appear- ance editing and high-frequency environmental relighting. In ACM SIGGRAPH 2023 Conference Proceedings, New York, NY , USA, 2023. Association for Computing Machinery. 2

    2023

  59. [67]

    Sketchhairsalon: Deep sketch-based hair image synthesis, 2021

    Chufeng Xiao, Deng Yu, Xiaoguang Han, Youyi Zheng, and Hongbo Fu. Sketchhairsalon: Deep sketch-based hair image synthesis, 2021. 2

    2021

  60. [68]

    Holistic inverse rendering of complex facade via aerial 3d scanning, 2024

    Zixuan Xie, Rengan Xie, Rong Li, Kai Huang, Pengju Qiao, Jingsen Zhu, Xu Yin, Qi Ye, Wei Hua, Yuchi Huo, and Hujun Bao. Holistic inverse rendering of complex facade via aerial 3d scanning, 2024. 2

    2024

  61. [69]

    Deep plastic surgery: Robust and controllable image editing with human-drawn sketches, 2020

    Shuai Yang, Zhangyang Wang, Jiaying Liu, and Zongming Guo. Deep plastic surgery: Robust and controllable image editing with human-drawn sketches, 2020. 2

    2020

  62. [70]

    Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop

    Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianx- iong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. 4, 1

    2015 arXiv

  63. [71]

    Self- supervised outdoor scene relighting

    Ye Yu, Abhimitra Meka, Mohamed Elgharib, Hans-Peter Seidel, Christian Theobalt, and William AP Smith. Self- supervised outdoor scene relighting. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part XXII 16, pages 84–101....

    2020

  64. [72]

    Dilightnet: Fine-grained light- ing control for diffusion-based image generation

    Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained light- ing control for diffusion-based image generation. In Special Interest Group on Computer Graphics and Interactive Tech- niques Conference Conference Papers ’24, page 1–12. ACM,

  65. [73]

    Yu Zeng, Zhe Lin, and Vishal M. Patel. Sketchedit: Mask- free local image manipulation with partial sketches, 2021. 2

    2021

  66. [74]

    Rgb ↔x: Image decomposition and synthesis using material- and lighting-aware diffusion models

    Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, and Miloˇs Haˇsan. Rgb ↔x: Image decomposition and synthesis using material- and lighting-aware diffusion models. In Spe- cial Interest Group on Computer Graphics and I...

    2024

  67. [75]

    Cohen, and Brian Curless

    Edward Zhang, Michael F. Cohen, and Brian Curless. Emp- tying, refurnishing, and relighting indoor spaces. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2016), 35(6), 2016. 2

    2016

  68. [76]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 2, 4

    2023

  69. [77]

    Zhang, P

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a percep- tual metric. In 2018 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR) , pages 586–595, Los Alamitos, CA, USA, 2018. IEEE Computer Society. 5

    2018

  70. [78]

    Xiao Zhang, William Gao, Seemandhar Jain, Michael Maire, David. A. Forsyth, and Anand Bhattad. Latent intrinsics emerge from training to relight, 2024. 2

    2024

  71. [79]

    Srinivasan, Dor Verbin, Keunhong Park, Ricardo Martin Brualla, and Philipp Henzler

    Xiaoming Zhao, Pratul P. Srinivasan, Dor Verbin, Keunhong Park, Ricardo Martin Brualla, and Philipp Henzler. IllumiN- eRF: 3D Relighting Without Inverse Rendering. InNeruIPS,

  72. [80]

    Learning-based inverse rendering of complex indoor scenes with differentiable monte carlo raytracing

    Jingsen Zhu, Fujun Luan, Yuchi Huo, Zihao Lin, Zhihua Zhong, Dianbing Xi, Rui Wang, Hujun Bao, Jiaxiang Zheng, and Rui Tang. Learning-based inverse rendering of complex indoor scenes with differentiable monte carlo raytracing. In SIGGRAPH Asia 2022 Conference Papers, pages 1–8...

    2022

  73. [81]

    I 2-sdf: Intrinsic indoor scene reconstruction and editing via raytracing in neural sdfs, 2023

    Jingsen Zhu, Yuchi Huo, Qi Ye, Fujun Luan, Jifan Li, Dian- bing Xi, Lisha Wang, Rui Tang, Wei Hua, Hujun Bao, and Rui Wang. I 2-sdf: Intrinsic indoor scene reconstruction and editing via raytracing in neural sdfs, 2023. 2 ScribbleLight: Single Image Indoor Relighting with Scri...

    2023

  74. [2024]

    Association for Computing Machinery. 2

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.