Pith. sign in

REVIEW 3 major objections 5 minor 50 references

Text2Relight: Creative Portrait Relighting with Text Guidance

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single portrait photo and a text prompt can relight both the person and the background while preserving identity, using a diffusion model trained on 1.5 million synthetic text-relighting pairs.

desk verdict Novel text-driven relighting pipeline with impressive qualitative results, but the quantitative evaluation is partly circular and needs firmer external validation. read the letter →

arxiv 2412.13734 v1 pith:XMWRI5YH submitted 2024-12-18 cs.CV

Junuk Cha , Mengwei Ren , Krishna Kumar Singh , He Zhang , Yannick Hold-Geoffroy , Seunghyun Yoon , HyunJoon Jung , Jae Shin Yoon
show 1 more author
Seungryul Baek
This is my paper · ORCID
classification cs.CV
keywords portraitrelightingtext-guidedimageeditingdiffusionmodelsdatasynthesispipelinepointlightoptimizationhierarchicaltextgenerationshadowremovalpositioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that natural-language text can serve as a direct control signal for portrait lighting, including lighting described by emotion, time, smell, or mood rather than by physical parameters. It introduces Text2Relight, a diffusion model that takes one portrait image, a foreground mask, and a text prompt, and outputs a version of the same scene in which both the person and the background are relighted to match the prompt while identity and content are preserved. Because no scalable dataset of text-to-relighting pairs existed, the paper's central contribution is a synthesis pipeline that manufactures such pairs: a large language model generates diverse light-aware prompts under a hand-crafted category hierarchy, a text-to-image model renders a lighting image for each prompt, and image-based relighting transfers that lighting to the foreground and background separately. Training on 1.5 million pairs, the model reports better structural, perceptual, identity, and text-alignment scores than general text-guided editing baselines, and users prefer it in roughly two-thirds of choices. If the claim holds, creative relighting becomes possible from a single photo with no lightstage equipment.

What carries the argument

The load-bearing mechanism is the synthetic data pipeline, because without it there is no ground truth for the text-to-lighting mapping. Its first stage is hierarchical prompt generation: the authors define 19 high-level categories (atmosphere, color, temperature, directionality, emotions, intensity, light location, smell, sound, taste, time, weather, universe, and others), an LLM expands each into at least 30 sub-words, and two to six randomly selected words are fed back to the LLM as constraints for composing a scene description. The second stage converts the prompt into a lighting image, either an RGB image produced by a fast latent consistency model or an HDR panorama produced by a fine-tuned text-guided diffusion model. The third stage transfers this lighting onto portrait data: for the foreground, an image-based relighting model operates on a single image, and when one-light-at-a-time (OLAT) captures from a lightstage are available, HDR rendering from the panorama is used instead; for the background, the lighting image is decomposed into about 20 point lights whose 3D position, color, intensity, ellipsoid ratio, and diffusion parameter are optimized to minimize photometric error, then those lights are re-rendered against the target background's albedo, depth, and normals. The final model is a diffusion UNet initialized from a text-guided image-editing model, trained with a foreground-mask condition and jointly on auxiliary delighting and light-positioning tasks.

What would settle it

Take portraits captured under known physical lighting, for example lightstage OLAT data, generate prompts that accurately describe that lighting and also prompts that describe incorrect lighting, run the model, and compare the shadow directions, shadow softness, intensity, and color casts in the outputs against the measured ground truth. If the model scores well on its own synthetic test set but cannot reproduce the correct physical lighting on these held-out captures, or cannot distinguish correct from incorrect descriptions, the claim that it has learned a genuine text-to-lighting mapping is falsified.

Watch

Extended reading notes

Core claim

The central discovery claim is that the text-to-lighting mapping can be learned rather than hand-designed, provided the training signal is generated with lighting fully decomposed from content. The paper's formulation is $\tilde{I} = f_\theta(I, M, T)$: a single function that ingests a source image, a foreground mask, and a text prompt and produces a relighted image. The training targets come from three pipeline stages: hierarchical LLM prompt generation; text-conditioned lighting image generation as an RGB image or an HDR panorama; and separate image-based relighting of the foreground and background, where the background lighting is represented as twenty optimized point lights and transferred via $\tilde{I} = \hat{A} * \sum_i s(L_i, \{\hat{D}, \hat{N}\})$. On synthetic test data built by the same pipeline, the model outperforms general text-guided editors on SSIM, LPIPS, CLIP vision similarity, face identity similarity, and a fine-tuned vision-language score, and in a 30-participant user study it is preferred 66.17% of the time. The paper also claims that the hand-crafted hierarchy substantially broadens prompt diversity relative to vision-language captioning, and that auxiliary tasks of shadow removal and light positioning improve the geometry and intrinsic-appearance modeling that relighting requires.

Load-bearing premise

The synthetic targets are trustworthy: the text-to-image model really renders lighting that matches the prompt, and the foreground and background relighting steps really transfer that lighting onto real portraits; if any of these components mislabels the training data, the model learns the wrong text-to-lighting mapping and the gains measured on synthetic test data will not transfer to real relighting.

Editorial extensions

If this is right

  • A single in-the-wild portrait photo is enough: users can specify lighting by mood, time of day, weather, color, or even smell and taste, and get a relighted scene with identity preserved.
  • The same model can be prompted for shadow removal and light positioning, making it a multi-purpose lighting editor rather than a single-task relighter.
  • Because the model is initialized from a general editing model and trained on diverse synthetic pairs, it also handles background harmonization of composited portraits and, per the supplementary results, generalizes to non-portrait objects and frame-by-frame video.
  • Any improvement to the three pipeline components, better text-to-image lighting rendering, better intrinsic decomposition, or better point-light optimization, should improve the downstream relighting model, since the pipeline produces its training data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the training targets are themselves generated by a text-to-image model, the text-to-lighting mapping can only be as faithful as that generator; a direct check would be to run the pipeline on lightstage captures with known physical lighting and compare shadow directions and color casts against ground truth.
  • The background representation factorizes light into point sources, so the same optimized point-light transfer could be reused for other editing tasks such as object insertion, multi-image lighting transfer, or relighting arbitrary scenes, not just portraits.
  • The model's apparent ability to interpret prompts about smell, taste, and emotion suggests it learns a culturally shared visual association rather than a physical model of light; a cross-cultural user study on such prompts would show how much of the mapping is learned convention.
  • The reported limitation that the model sometimes places strong unnatural point lights in the background and confuses text-specified light positions points to a concrete extension: conditioning the network on explicit 3D light-position tokens alongside the text could give users direct positional control.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Text2Relight, a diffusion-model-based method for single-image portrait relighting guided by a text prompt. To address the lack of paired text-relighting data, the authors build a synthetic dataset of 1.5M pairs by (i) generating diverse text prompts with an LLM under a hand-crafted category hierarchy, (ii) generating a lighting image (RGB or HDR panorama) from the prompt, and (iii) transferring the lighting to portrait foregrounds and backgrounds using image-based relighting and point-light optimization. A pretrained InstructPix2Pix model is fine-tuned on this data with auxiliary tasks (shadow removal, light positioning). Experiments compare the model with IP2P, GLIDE, and MGIE on synthetic test pairs and a user study.

Significance. If validated, the paper would be a significant step toward open-vocabulary, text-controllable relighting of both foreground and background in arbitrary portraits. The data synthesis pipeline is a substantial engineering contribution, and the qualitative results, particularly on real images, are visually convincing. The user study provides independent evidence that users prefer the method over three text-editing baselines. The work also demonstrates useful applications such as shadow removal, light positioning, and background harmonization. However, the quantitative claims are weakened by evaluation on targets generated with the same pipeline used for training, and the text-to-lighting alignment of the synthetic targets is not verified.

major comments (3)
  1. [Experiments/Datasets and Table 1] The ground-truth relighting targets are synthesized with the same foreground and background relighting components used to create the training data, so Table 1's SSIM, LPIPS, CVS, and FIS measure how well the model reproduces the outputs of those components rather than how well it performs true text-contingent relighting; because the baselines were not trained on this distribution, the comparison is biased. The user study on real images (Table 2) is a useful independent check, but preference scores can reflect appealing content changes rather than lighting correctness, so it does not fully resolve the circularity.
  2. [Lighting Image Generation] The RGB lighting image produced by a four-step latent consistency model is used without validation as the conditioning target for foreground relighting and as the source of the point lights optimized in Eq. (S1); for abstract prompts such as "joyful lighting" or "sweet cotton candy", this image is likely a stylized scene rather than a physically interpretable illumination field, so the text-to-lighting mapping in the training targets can be systematically wrong. The limitation paragraph's admission that the model is "sometimes confused to localize the text-specified lighting" is consistent with this concern. I recommend a validation study that checks whether the optimized light positions and colors match the prompt's stated attributes on a held-out set.
  3. [Baselines and Experiments] The comparison to IP2P, GLIDE, and MGIE is not controlled: the baselines are used off-the-shelf with an instruction template, while the proposed model is fine-tuned on 1.5M synthetic relighting pairs and additionally benefits from auxiliary-task data; thus the performance gaps in Table 1 may largely reflect the advantage of in-domain training data. To support the claim of "outperforms existing text-guided image generation models", the authors should either fine-tune the baselines on a matched subset of the synthetic data or compare against the same architecture trained without the relighting-specific data, since an ablation is shown for the proposed model but not for the baselines.
minor comments (5)
  1. [Table 1] The FIS value for GLIDE (-0.969) is far below zero and should be explained; also, no error bars or statistical significance tests are provided for any metric, so it is unclear whether the differences in Table 1 and Table 3 are meaningful.
  2. [Metrics] The LLaVA score (LS) is obtained with a model fine-tuned on ChatGPT-generated ratings and a single subjective "Lighting Adjustment" question; the calibration and variance of this metric should be reported, and it should be interpreted cautiously.
  3. [Supplementary, Point Light Reconstruction and Transfer] The point-light transfer uses a "relative distance" heuristic to account for depth differences between the lighting image and the target background, but this heuristic is not validated against physical relighting ground truth; a brief analysis of its effect on the synthesized targets would strengthen the pipeline's credibility.
  4. [Dataset Summary] The dataset statistics report only the total number of pairs (1.5M); a breakdown by source type (OLAT vs. single image), number of identities, number of unique text prompts, and background diversity would help assess the risk of overfitting and the actual text diversity.
  5. [Throughout] There are minor typographical and formatting issues, such as "models does not generalize" in the abstract and inconsistent typesetting of model names like LLaVA; these should be cleaned up.

Circularity Check

2 steps flagged · score 5.0 of 10

Quantitative evaluation is self-referential: Table 1 ground truths are generated by the same synthesis pipeline that made the training data, and the learned text-match metric is trained on ChatGPT outputs while the prompts come from ChatGPT; the user study provides partial independent grounding.

  1. fitted input called prediction [Experiments, Datasets (Table 1)]
    "For quantitative evaluation, we use our data simulation pipeline to synthesize the ground-truth data for text-guided portrait relighting."

    The model is trained on 1.5M pairs produced by the same pipeline (LLM prompts, LCM lighting images, internal foreground relighter, and point-light background transfer of Eq. S1). The Table 1 'ground truths' are generated by that same pipeline, so the reported SSIM/LPIPS/CVS/FIS/LS numbers measure how well the model imitates its own training-data generator rather than whether the relighting is physically correct. Any systematic artifact of the synthesis components is present in both training and test targets, so high scores can be achieved by reproducing those artifacts. The real-image user study is independent evidence, but the headline quantitative comparison is self-referential.

  2. other [Experiments, Metrics (LLaVA score)]
    "To measure the score of how well the text prompt matches an image, we fine-tune the LLaVA (Liu et al. 2024) using the data obtained from ChatGPT (OpenAI 2022): a question template, a concatenated image, and an answer of ChatGPT."

    The text prompts used to create the training pairs are also generated by ChatGPT under the crafted hierarchy. Fine-tuning the semantic text-match metric on ChatGPT's own judgments means the LS column in Table 1 mainly measures agreement with the same LLM family that invented the prompts, not an independent human-verified text-lighting correspondence. This is a secondary circularity because the user study provides a human preference check for the central claim.

full rationale

The central training pipeline (LLM prompt generation, lighting-image synthesis, foreground relighting, background point-light transfer) is not itself circular: the diffusion model is genuinely trained on generated pairs, and the user study on real images supplies independent evidence of preference. However, the paper's quantitative evaluation is substantially self-referential. Test ground truths are created with the same data simulation pipeline that generated the training set, so Table 1 can reward reproducing pipeline-specific artifacts rather than correct text-contingent relighting. The LLaVA metric is fine-tuned on ChatGPT judgments even though the prompts come from ChatGPT, making the text-match score a measure of agreement with the prompt generator. The user study and qualitative real-image results prevent this from being a fully circular derivation, but the headline 'outperforms' claim leans on self-synthesized targets. No uniqueness theorem or ansatz is smuggled in via self-citation; the self-citations to Ren et al. are component reuse rather than load-bearing proof.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests on a synthetic data pipeline whose components (LLM prompts, text-to-image lighting images, point-light relighting) are themselves built on prior models and approximations. The main free choices are the number of point lights, the prompt category weights, and the initialization values. The paper introduces no new physical entities.

free parameters (3)
  • Number of point lights for background relighting = 20
    Chosen based on experiments in Figure S1 as the best trade-off between reconstruction error and optimization time. This hand-set count affects background relighting quality.
  • Sub-category selection weights in text generation = higher weights for physical categories like position and color
    Hand-crafted in the supplementary to bias generation toward physically relevant lighting words. This affects diversity and physical correctness of the training text.
  • Point light initialization values = color (0.5,0.5,0.5), intensity 1/#L, ellipsoid ratio 1, diffuse parameter 1
    Stated in the supplementary as initialization for the optimization. These choices influence convergence and the final reconstructed lighting.
assumptions (6)
  • domain assumption Lambertian reflectance model describes the shading of background surfaces
    Used in Eq. S2 for point-light shading; assumes Lambertian surfaces, which is a simplification for many real materials.
  • domain assumption Intrinsic decomposition I = A*S separates albedo and shading for backgrounds
    Used in the Background Relighting section to transfer lighting between images.
  • domain assumption Text-to-image and text-to-panorama models produce lighting images that faithfully match the text prompt
    The entire training set relies on these generated lighting images being accurate visualizations of the text descriptions.
  • domain assumption LLM-based hierarchical prompt generation produces diverse and semantically correct lighting descriptions
    Used in Scalable Prompt Generation; the paper provides T-SNE evidence of diversity, but semantic correctness is assumed.
  • domain assumption The pre-trained InstructPix2Pix UNet provides a strong base for relighting-specific fine-tuning
    The model is initialized with IP2P weights, assuming its general image editing prior transfers to relighting.
  • ad hoc to paper Relative distance transfer for point lights handles depth differences between source and target backgrounds
    Introduced in the supplementary to avoid invisible lighting after transfer; it is a heuristic without formal justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Text2Relight: Creative Portrait Relighting with Text Guidance." pith.science (2026). https://pith.science/paper/XMWRI5YH

@misc{pith2026241213734,
  author       = {Pith},
  title        = {Pith review of: Text2Relight: Creative Portrait Relighting with Text Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XMWRI5YH}},
  note         = {Machine review of arXiv:2412.13734}
}
read the original abstract

We present a lighting-aware image editing pipeline that, given a portrait image and a text prompt, performs single image relighting. Our model modifies the lighting and color of both the foreground and background to align with the provided text description. The unbounded nature in creativeness of a text allows us to describe the lighting of a scene with any sensory features including temperature, emotion, smell, time, and so on. However, the modeling of such mapping between the unbounded text and lighting is extremely challenging due to the lack of dataset where there exists no scalable data that provides large pairs of text and relighting, and therefore, current text-driven image editing models does not generalize to lighting-specific use cases. We overcome this problem by introducing a novel data synthesis pipeline: First, diverse and creative text prompts that describe the scenes with various lighting are automatically generated under a crafted hierarchy using a large language model (*e.g.,* ChatGPT). A text-guided image generation model creates a lighting image that best matches the text. As a condition of the lighting images, we perform image-based relighting for both foreground and background using a single portrait image or a set of OLAT (One-Light-at-A-Time) images captured from lightstage system. Particularly for the background relighting, we represent the lighting image as a set of point lights and transfer them to other background images. A generative diffusion model learns the synthesized large-scale data with auxiliary task augmentation (*e.g.,* portrait delighting and light positioning) to correlate the latent text and lighting distribution for text-guided portrait relighting.

Figures

Figures reproduced from arXiv: 2412.13734 by the authors.

Figure 1
Figure 1. Text2Relight generates the image of a relighted portrait (right) as a condition of a text prompt while keeping original contents in an input image (left). Abstract We present a lighting-aware image editing pipeline that, given a portrait image and a text prompt, performs single image re￾lighting. Our model modifies the lighting and color of both the foreground and background to align with the provided text descripti… view at source ↗
Figure 2
Figure 2. Results from existing text-guided image editing [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of the data synthesis pipeline. We first generate a text prompt with a language hierarchy from which we generate a lighting image. Subsequently, we trans￾fer the lighting from the lighting image to a portrait image captured from either lightstage or real world (with back￾ground inpainting). These form the training dataset for our Text2Relight model. Method Problem Formulation Given a portrait image, our goa… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Pipeline for text generation with hierarchy. For sub-categories generation, we ask ‘Generate words related to {category}. Write 30 or more words on a single line, separated by commas.’; and for sentence generation, we ask ‘could you describe the lighting property of a …
Figure 6
Figure 6. Figure 6: Pipeline for background relighting. multiple layers of intrinsic values: I = A ∗ S where I, A, and S represent the background image, its albedo, and its shading, respectively. Furthermore, the shading can be de￾scribed as a function of geometry and lighting: S = s(L, G…
Figure 7
Figure 7. Figure 7: Examples of our simulated data. Source, lighting [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 9
Figure 9. Figure 9: Performance comparison according to 10 text [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison with text prompts : ‘The [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 12
Figure 12. Figure 12: Examples of shadow removal and lighting posi [PITH_FULL_IMAGE:figures/full_fig_p007_12.png]
Figure 10
Figure 10. Figure 10: Ablation study on mask condition and auxiliary [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: The T-SNE distribution of the texts predicted [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 30 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Avrahami, O.; Lischinski, D.; and Fried, O. 2022. Blended diffusion for text-driven editing of natural images. In CVPR

    work page 2022

  5. [5]

    Brooks, T.; Holynski, A.; and Efros, A. A. 2023. Instructpix2pix: Learning to follow image editing instructions. In CVPR

    work page 2023

  6. [6]

    Careaga, C.; and Aksoy, Y. 2023. Intrinsic Image Decomposition via Ordinal Shading. ACM Transactions on Graphics

    work page 2023

  7. [7]

    Deng, J.; Guo, J.; Xue, N.; and Zafeiriou, S. 2019. Arcface: Additive angular margin loss for deep face recognition. In CVPR

    work page 2019

  8. [8]

    Y.; Yang, Y.; and Gan, Z

    Fu, T.-J.; Hu, W.; Du, X.; Wang, W. Y.; Yang, Y.; and Gan, Z. 2023. Guiding instruction-based image editing via multimodal large language models. arXiv preprint arXiv:2309.17102

    arXiv 2023

Show all 50 references
  1. [9]

    Futschik, D.; Ritland, K.; Vecore, J.; Fanello, S.; Orts-Escolano, S.; Curless, B.; S \`y kora, D.; and Pandey, R. 2023. Controllable light diffusion for portraits. In CVPR

    2023

  2. [10]

    Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. NIPS

    2020

  3. [11]

    Hou, A.; Sarkis, M.; Bi, N.; Tong, Y.; and Liu, X. 2022. Face relighting with geometrically consistent shadows. In CVPR

    2022

  4. [12]

    Hou, A.; Zhang, Z.; Sarkis, M.; Bi, N.; Tong, Y.; and Liu, X. 2021. Towards high fidelity face relighting with realistic shadows. In CVPR

    2021

  5. [13]

    Iwahori, Y.; Sugie, H.; and Ishii, N. 1990. Reconstructing shape from shading images under point light source illumination. In [1990] Proceedings. 10th International Conference on Pattern Recognition

    1990

  6. [14]

    Ji, C.; Yu, T.; Guo, K.; Liu, J.; and Liu, Y. 2022. Geometry-aware single-image full-body human relighting. In ECCV

    2022

  7. [15]

    Kawar, B.; Zada, S.; Lang, O.; Tov, O.; Chang, H.; Dekel, T.; Mosseri, I.; and Irani, M. 2023. Imagic: Text-based real image editing with diffusion models. In CVPR

    2023

  8. [16]

    Kim, H.; Jang, M.; Yoon, W.; Lee, J.; Na, D.; and Woo, S. 2024. SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting. arXiv preprint arXiv:2402.18848

    2024 arXiv

  9. [17]

    Kocsis, P.; Sitzmann, V.; and Nie ner, M. 2023. Intrinsic Image Diffusion for Single-view Material Estimation. arXiv preprint arXiv:2312.12274

    2023 arXiv

  10. [18]

    Koppal, S. J. 2020. Lambertian reflectance. Computer vision: a reference guide

    2020

  11. [19]

    Kvanchiani, K.; Petrova, E.; Efremyan, K.; Sautin, A.; and Kapitanov, A. 2023. EasyPortrait--Face Parsing and Portrait Segmentation Dataset. arXiv preprint arXiv:2304.13509

    2023 arXiv

  12. [20]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2024. Visual instruction tuning. NIPS

    2024

  13. [21]

    Liu, Y.; Iter, D.; Xu, Y.; Wang, S.; Xu, R.; and Zhu, C. 2023. Gpteval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634

    2023 arXiv

  14. [22]

    Luo, S.; Tan, Y.; Huang, L.; Li, J.; and Zhao, H. 2023. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378

    2023 arXiv

  15. [23]

    Mei, Y.; Zeng, Y.; Zhang, H.; Shu, Z.; Zhang, X.; Bi, S.; Zhang, J.; Jung, H.; and Patel, V. M. 2024. Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image. arXiv preprint arXiv:2403.09632

    2024 arXiv

  16. [24]

    Mei, Y.; Zhang, H.; Zhang, X.; Zhang, J.; Shu, Z.; Wang, Y.; Wei, Z.; Yan, S.; Jung, H.; and Patel, V. M. 2023. LightPainter: interactive portrait relighting with freehand scribble. In CVPR

    2023

  17. [25]

    Nestmeyer, T.; Lalonde, J.-F.; Matthews, I.; and Lehrmann, A. 2020. Learning physics-guided face relighting under directional light. In CVPR

    2020

  18. [26]

    Nichol, A.; Dhariwal, P.; Ramesh, A.; Shyam, P.; Mishkin, P.; McGrew, B.; Sutskever, I.; and Chen, M. 2021. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741

    2021 arXiv

  19. [27]

    OpenAI. 2022. Introducing chatgpt

    2022

  20. [28]

    E.; and Fanello, S

    Pandey, R.; Orts-Escolano, S.; Legendre, C.; Haene, C.; Bouaziz, S.; Rhemann, C.; Debevec, P. E.; and Fanello, S. R. 2021. Total relighting: learning to relight portraits for background replacement. ACM Trans. Graph

    2021

  21. [29]

    Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. 2019. Pytorch: An imperative style, high-performance deep learning library. NIPS

    2019

  22. [30]

    Ponglertnapakorn, P.; Tritrong, N.; and Suwajanakorn, S. 2023. DiFaReli: Diffusion face relighting. In ICCV

    2023

  23. [31]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In ICML

    2021

  24. [32]

    Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I.; et al. 2019. Language models are unsupervised multitask learners. OpenAI blog

    2019

  25. [33]

    Ranftl, R.; Lasinger, K.; Hafner, D.; Schindler, K.; and Koltun, V. 2020. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on pattern analysis and machine intelligence

    2020

  26. [34]

    S.; Shu, Z.; Zhang, J.; Jung, H.; Gerig, G.; and Zhang, H

    Ren, M.; Xiong, W.; Yoon, J. S.; Shu, Z.; Zhang, J.; Jung, H.; Gerig, G.; and Zhang, H. 2023. Relightful Harmonization: Lighting-aware Portrait Background Replacement. arXiv preprint arXiv:2312.06886

    2023 arXiv

  27. [35]

    Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022. High-resolution image synthesis with latent diffusion models. In CVPR

    2022

  28. [36]

    Saito, S.; Schwartz, G.; Simon, T.; Li, J.; and Nam, G. 2023. Relightable gaussian codec avatars. arXiv preprint arXiv:2312.03704

    2023 arXiv

  29. [37]

    Simonyan, K.; and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

    2014 arXiv

  30. [38]

    Song, G.; Cham, T.-J.; Cai, J.; and Zheng, J. 2021. Half-body Portrait Relighting with Overcomplete Lighting Representation. In Computer Graphics Forum

    2021

  31. [39]

    Song, J.; Meng, C.; and Ermon, S. 2020. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502

    2020 arXiv

  32. [40]

    T.; Tsai, Y.-T.; Xu, Z.; Yu, X.; Fyffe, G.; Rhemann, C.; Busch, J.; Debevec, P.; and Ramamoorthi, R

    Sun, T.; Barron, J. T.; Tsai, Y.-T.; Xu, Z.; Yu, X.; Fyffe, G.; Rhemann, C.; Busch, J.; Debevec, P.; and Ramamoorthi, R. 2019. Single image portrait relighting. ACM Transactions on Graphics (TOG)

    2019

  33. [41]

    Tajima, D.; Kanamori, Y.; and Endo, Y. 2021. Relighting Humans in the Wild: Monocular Full-Body Human Relighting with Domain Adaptation. In Computer Graphics Forum

    2021

  34. [42]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  35. [43]

    Wang, W.; Xie, E.; Li, X.; Fan, D.-P.; Song, K.; Liang, D.; Lu, T.; Luo, P.; and Shao, L. 2022. Pvt v2: Improved baselines with pyramid vision transformer. Computational Visual Media

    2022

  36. [44]

    Wang, Y.; Holynski, A.; Zhang, X.; and Zhang, X. 2023. Sunstage: Portrait reconstruction and relighting using the sun as a light stage. In CVPR

    2023

  37. [45]

    C.; Sheikh, H

    Wang, Z.; Bovik, A. C.; Sheikh, H. R.; and Simoncelli, E. P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing

    2004

  38. [46]

    Yeh, Y.-Y.; Nagano, K.; Khamis, S.; Kautz, J.; Liu, M.-Y.; and Wang, T.-C. 2022. Learning to relight portrait images via a virtual light stage and synthetic-to-real adaptation. ACM Transactions on Graphics (TOG)

    2022

  39. [47]

    S.; Shu, Z.; Ren, M.; Zhang, C.; Hold-Geoffroy, Y.; Singh, K

    Yoon, J. S.; Shu, Z.; Ren, M.; Zhang, C.; Hold-Geoffroy, Y.; Singh, K. k.; and Zhang, H. 2024. Generative Portrait Shadow Removal. ACM Transactions on Graphics (TOG)

    2024

  40. [48]

    Zhang, L.; Zhang, Q.; Wu, M.; Yu, J.; and Xu, L. 2021. Neural video portrait relighting in real-time via consistency modeling. In ICCV

    2021

  41. [49]

    A.; Shechtman, E.; and Wang, O

    Zhang, R.; Isola, P.; Efros, A. A.; Shechtman, E.; and Wang, O. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR

    2018

  42. [50]

    Zhou, H.; Hadap, S.; Sunkavalli, K.; and Jacobs, D. W. 2019. Deep single-image portrait relighting. In ICCV

    2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.