Pith. sign in

REVIEW 4 major objections 5 minor 60 references

MatSwap: Light-aware material transfers in images

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MatSwap replaces a surface's material in a single photograph using only a flat texture sample and a mask, while preserving the scene's lighting and geometry.

desk verdict MatSwap is a solid, well-engineered material transfer paper with strong qualitative results and honest limitations; the real-image irradiance metric is partially circular, but the synthetic ablation and user study carry the central claim. read the letter →

arxiv 2502.07784 v2 pith:V7RO4S3F submitted 2025-02-11 cs.CV cs.GR

Ivan Lopes , Valentin Deschaintre , Yannick Hold-Geoffroy , Raoul de Charette This is my paper · ORCID
classification cs.CVcs.GR
keywords materialtransferdiffusionmodelsimageeditingirradianceconditioningintrinsicdecompositionexemplar-basedtexturesyntheticdatasetlight-awareinpainting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MatSwap is a method for replacing the material on a designated surface in a single photograph, using only an exemplar texture image of the material on a flat surface and a user mask. The paper argues that by fine-tuning a large pre-trained diffusion model on paired synthetic renders and conditioning it on estimated scene normals, diffuse irradiance, and a CLIP embedding of the exemplar, the model learns to transfer materials while preserving the scene's lighting and geometry, without UV mapping or 3D scene knowledge. The authors report that MatSwap outperforms existing exemplar-based material transfer methods (ZeST, MaterialFusion, RGB↔X) and generic inpainting on PSNR, LPIPS, and CLIP-I metrics, and that human raters find its results more realistic. If true, this gives artists a practical single-image material editing tool.

What carries the argument

The load-bearing mechanism is the conditioning stack $z_X = [\mathcal{E}(x), \mathcal{E}(N), S_\downarrow(E), S_\downarrow(M)]$ concatenated with the noisy latent $z_t$. Here $\mathcal{E}$ is a pre-trained latent encoder, $S_\downarrow$ downsamples the irradiance map and mask, $N$ is the normal map, and $E$ is the diffuse irradiance map. Combined with IP-Adapter injection of the CLIP embedding of the exemplar texture into the cross-attention layers, this stack tells the UNet where the surface is, how it faces, how it is lit, and what material to paint, enabling perspective-aware transfer without explicit UV mapping.

What would settle it

Render a synthetic scene with known ground-truth irradiance and a known material swap; run MatSwap on it and compare the irradiance of the output region against the ground-truth irradiance of the target material, rather than against φE of the input. If MatSwap's lighting preservation scores are driven by the estimator copying its own input, the ground-truth-based error will be much larger than the reported φE-vs-φE error. Also, replace φE with a different irradiance estimator (e.g., Lotus) and check whether the transfer's shading and stability change materially.

Watch

Extended reading notes

Core claim

The central claim is that material transfer in a photograph can be treated as a light- and geometry-aware conditional generation task rather than a UV-mapping or per-pixel PBR editing problem. The method builds a scene descriptor stack by concatenating the encoded input image, encoded normals, downsampled diffuse irradiance, and the mask to the noisy latent at every diffusion step, and injects the exemplar's CLIP visual embedding through IP-Adapter cross-attention layers. Trained on the PBRand dataset of 250,000 paired renders with swapped materials, the fine-tuned Stable Diffusion model learns to align the transferred texture with the scene's shading cues. The paper reports that the irradiance conditioning is the key component: ablations show that removing it flattens shading, and the model with irradiance preserves illumination better than all baselines on the φE-based metric.

Load-bearing premise

The accuracy and material-independence of the off-the-shelf irradiance estimator φE (RGB↔X) for real images, which is used both as a conditioning input at inference and as the metric for lighting preservation, so if the estimator is biased by the original material the reported irradiance scores may partly measure the estimator copying itself.

Editorial extensions

If this is right

  • A user can replace a surface material in a single photo with one flat texture sample and a mask, with no 3D scene reconstruction or UV unwrapping, and get shading consistent with the scene's lights.
  • The method's reliance on estimated irradiance means its real-image performance is bounded by the quality of single-image intrinsic estimation; better estimators should directly improve transfer quality.
  • Because the model generates the entire image rather than only the masked region, mask-edge artifacts are reduced relative to latent-blending inpainting.
  • The PBRand dataset of 250,000 paired renders, with ground-truth normals and irradiance, can serve as a training resource for other material-aware image editing tasks.
  • Classifier-free guidance on the exemplar embedding lets users trade fidelity to the material sample against realism by tuning $\gamma$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the irradiance-estimator dependency is confirmed as a bottleneck, a testable extension is to train MatSwap end-to-end with a differentiable intrinsics estimator, or to use multiple estimators at inference and measure variance, to decouple lighting preservation from metric circularity.
  • The ablations that mask the target region during training suggest the model could be adapted for guided relighting or object insertion, not just material transfer, by providing irradiance as a control.
  • The scale-control result (cropping the exemplar changes texture scale) implies the CLIP embedding encodes scale information; an explicit scale parameter could give artists finer control than cropping.
  • Downward-facing normals and thin objects are cited as failure cases; a dataset augmented with ceiling views, overhangs, and thin structures would likely close most of the current gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. MatSwap proposes an exemplar-based material transfer method that inpaints a user-selected region of a photograph with a desired material, conditioned on pixel-aligned irradiance and normal maps. The method fine-tunes a Stable Diffusion model with IP-Adapter conditioning on a new synthetic dataset, PBRand, which provides thousands of paired Blender renders with ground-truth geometry and lighting buffers. At inference, irradiance and normals are estimated from the input image with off-the-shelf estimators (RGB↔X and Lotus). Quantitative evaluations on synthetic and real images report improved PSNR/LPIPS over baselines, higher CLIP-I on real data, and a user study favoring the method's realism and fidelity. The paper also ablate the contributions of the irradiance map, normals, full-UNet fine-tuning, and CFG guidance.

Significance. If the claims hold, MatSwap offers a practical, controllable material-transfer tool that does not require manual UV mapping or 3D scene understanding, and the release of PBRand is a useful resource. The strongest evidence is the user study and the qualitative results, which show plausible perspective and lighting behavior in several real photographs. However, the central light-awareness claim on real images rests on a partially circular evaluation, and the reported quantitative gains over strong baselines are small and do not report statistical significance. The contribution is therefore significant but needs additional validation before the headline claim can be accepted.

major comments (4)
  1. [§4, Table 2] The real-image 'adherence to irradiance' metric is partially circular. The conditioning at inference uses E = φE(x) (Eq. 1, Sec. 3.1), and the metric compares φE(Î) against φE(x). A model that reproduces the estimated irradiance pattern of x in the edited region, without physically correct lighting, would score well under this metric. The paper should validate φE against ground-truth irradiance on the synthetic test set where E_GT is available, or use an independent estimator, a multi-illumination capture, or a perceptual user study on lighting consistency, to confirm that the reported advantage reflects genuine light-aware transfer rather than estimator self-consistency.
  2. [§3.1–3.2, §4.1] There is a train/inference gap that is not quantified: the model is trained with Blender ground-truth irradiance maps (material-independent), but deployed with φE(x) from a learned single-image estimator. The manuscript does not measure the discrepancy between φE(x) and E_GT on the synthetic evaluation set, nor whether this error correlates with the material being swapped. Since RGB↔X is a learned decomposition that may leak material information, the authors should report a synthetic experiment that conditions the trained model with φE(x) instead of E_GT and compares the resulting images against the ground-truth render, which would directly test the robustness of the method to estimator error.
  3. [§4.1, Table 1] The quantitative gains on synthetic data are small (PSNR 20.62 vs 19.92 for FLUX.1 and LPIPS 0.1783 vs 0.1825) and no error bars, confidence intervals, or significance tests are provided. Given that diffusion sampling is stochastic and the baselines may have variable outputs, the differences may not be statistically significant. The authors should report mean and variance over multiple seeds/samples and apply a paired significance test (e.g., Wilcoxon signed-rank) for PSNR and LPIPS over the 300 synthetic images.
  4. [§4.2, User study] The 2AFC user study is a positive element, but the reported percentages (e.g., 78% more realistic, 70% more faithful) are not accompanied by confidence intervals or a test of whether they differ from chance per baseline. Reporting the number of participants who preferred the method and a binomial confidence interval would strengthen the claim, especially since the fidelity comparison between ZeST and ours is described as 'similar.'
minor comments (5)
  1. [§1, Abstract] The abstract says 'custom light- and geometry-aware diffusion model,' but the lighting and geometry conditions at inference come from off-the-shelf estimators (RGB↔X, Lotus). This distinction should be made explicit in the abstract or introduction to avoid overstating the novelty.
  2. [§3.1, Eq. (2)] The diffusion loss in Eq. (2) is written for a single timestep without the usual expected value over t and noise; consider writing it as E_{t,ε} for completeness, though this is a minor clarity issue.
  3. [§3.3, Implementation Details] The text says 'we train the full UNet' but the ablation description states 'keeping the IP-Adapter layers frozen with pretrained weights from [YZL*23].' Clarify which components are fine-tuned in the final model, since the ablation (A8) appears to train both the IP-Adapter and the UNet.
  4. [§4, Metrics] The description of the CLIP-I metric should state the exact crop/extent used for the cosine similarity computation (e.g., the masked region only, or the entire image) and how the exemplar is resized, to allow reproducibility.
  5. [§5, Limitations] The limitations section is candid, but it could also mention that the method does not control the absolute scale of the transferred material relative to the real world, only relative to the exemplar crop, as shown in Fig. 10.

Circularity Check

1 steps flagged · score 4.0 of 10

Irradiance-adherence metric is partially circular: MatSwap conditions on φE(x) and is scored against φE(x).

  1. fitted input called prediction [Section 3.1 (Eq. 1) and Section 4, Table 2]
    "Thus, for real images we define E = φE(x) and N = φN(x), with φN and φE the normal and an irradiance estimators, respectively. ... We measure the shading error by estimating the irradiance map of the model output, i.e., φE(ˆI). We then compute its quality against the irradiance of the original image, i.e., φE(x)."

    The headline 'adherence to irradiance' result (Table 2) is computed as a distance between φE(ˆI) and φE(x). But the MatSwap conditioning stack in Eq. (1) includes E = φE(x) as a downsampled input channel at every denoising step, so the model can score well by preserving or echoing its own conditioning map, independent of whether the physical illumination is correct. The evaluation oracle is also the same estimator used to produce the conditioning, and no synthetic experiment reports how far φE(x) is from ground-truth irradiance or whether that error is material-dependent.

full rationale

The core material-transfer contribution is not circular: the method is trained with Blender ground-truth irradiance/normals on the PBRand dataset, and the main synthetic metrics (PSNR/LPIPS on 300 ground-truth renders), CLIP-I, and a 40-participant 2AFC user study are external to the model's conditioning. The diffusion loss (Eq. 2) is an L2 latent denoising objective, not a re-statement of any evaluation metric. The only load-bearing circularity I can exhibit is the irradiance-adherence evaluation on real images: the model is fed φE(x) as input and then graded by comparing φE(ˆI) to φE(x). This inflates the apparent lighting advantage and relies on an estimator from the authors' own prior RGB↔X work, but it does not by itself force the central transfer result. The use of RGB↔X/Lotus as off-the-shelf estimators is an implementation choice, not an imported uniqueness theorem. Accordingly, the paper gets a moderate circularity score: one supporting metric reduces by construction, while the central claim retains independent synthetic and human-evaluation content.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the accuracy of off-the-shelf intrinsic estimators (RGB↔X) for real images, the sufficiency of CLIP embeddings to convey material appearance, and the generalization of a synthetic primitive-based dataset to real photographs. These are not proven by the paper; they are supported by qualitative results and ablations. The method itself introduces no new physical entity.

free parameters (1)
  • CFG guidance scale γ = 3
    Chosen by qualitative inspection of Fig. 12 as a trade-off between realism and fidelity to the conditioning texture; not predicted by theory.
assumptions (4)
  • domain assumption RGB↔X (φN, φE) provides sufficiently accurate normals and irradiance for real images.
    The method's real-image inference depends on off-the-shelf estimators; Section 3.1, Eq. (1).
  • domain assumption CLIP image embedding τ(p) captures material appearance including scale and hue for transfer.
    The visual CLIP features are used as the sole material conditioning; Section 3.1, after Eq. (2).
  • domain assumption PBRand's procedurally generated primitive scenes generalize to real-world photographs.
    Training on synthetic data is claimed to transfer to real images; Sections 3.2 and 4.
  • domain assumption Stable Diffusion contains strong priors about object appearance that are preserved after fine-tuning.
    The method relies on the pre-trained model's priors for generalization; Section 1 and 3.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MatSwap: Light-aware material transfers in images." pith.science (2026). https://pith.science/paper/V7RO4S3F

@misc{pith2026250207784,
  author       = {Pith},
  title        = {Pith review of: MatSwap: Light-aware material transfers in images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V7RO4S3F}},
  note         = {Machine review of arXiv:2502.07784}
}
read the original abstract

We present MatSwap, a method to transfer materials to designated surfaces in an image photorealistically. Such a task is non-trivial due to the large entanglement of material appearance, geometry, and lighting in a photograph. In the literature, material editing methods typically rely on either cumbersome text engineering or extensive manual annotations requiring artist knowledge and 3D scene properties that are impractical to obtain. In contrast, we propose to directly learn the relationship between the input material -- as observed on a flat surface -- and its appearance within the scene, without the need for explicit UV mapping. To achieve this, we rely on a custom light- and geometry-aware diffusion model. We fine-tune a large-scale pre-trained text-to-image model for material transfer using our synthetic dataset, preserving its strong priors to ensure effective generalization to real images. As a result, our method seamlessly integrates a desired material into the target location in the photograph while retaining the identity of the scene. We evaluate our method on synthetic and real images and show that it compares favorably to recent work both qualitatively and quantitatively. We release our code and data on https://github.com/astra-vision/MatSwap

Figures

Figures reproduced from arXiv: 2502.07784 by the authors.

Figure 1
Figure 1. MatSwap allows realistic material transfer in images. From an input image (left), our method seamlessly integrates an exemplar material (top left inset) into the user-specified region (red mask, bottom left inset). We can plausibly replace the wall’s surface (top) with tapestry (first result) or bricks (second), and also alter the wood type on the floor (third) and the mat (rightmost). Similarly, we present two dist… view at source ↗
Figure 2
Figure 2. Overview of MatSwap. We learn to transfer the texture p on a given region M of an input image x by training a light- and geometry-aware diffusion model, leveraging irradiance E and normal N maps. Once encoded (E) or downsampled (S↓), the image, mask, and maps are concatenated into a scene descriptor zX which, together with the noise latent zt , serve as input to the denoising UNet, ϵθ. To integrate the exemplar cond… view at source ↗
Figure 3
Figure 3. Procedural dataset. We show examples of our PBRand dataset, which we use for training. It consists of primitive ob￾jects (spheres, cubes, cylinders, and tori) with random placements, orientations, and materials enclosed within four walls of varying heights. A total of 50,000 3D scenes were created in Blender, each rendered under 5 light variations, with image-based lighting to achieve realistic occlusions and cast s… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Samples from synthetic evaluation dataset. Scenes show a wide diversity in appearance and illumination. 4. Experiments We now compare our method against state-of-the-art inpainting and material transfer methods. Baselines. We consider two types of inpainting baselines.…
Figure 5
Figure 5. Figure 5: Comparison to baselines. We compare against text-based (Blended LD, FLUX) and image-based methods (SD2-ControlNet￾Normal, RGB↔X, ZeST, Material Fusion). Each method uses the input image and mask information either via latent masking or as explicit conditioning. Our app…
Figure 7
Figure 7. Figure 7: Non-planar surfaces. We provide results on non-planar surfaces, demonstrating that our method is capable of handling surfaces with more complex geometry. Zoom in for details. well. Our method, guided by the irradiance map E, understandably outperforms all compared meth…
Figure 8
Figure 8. Figure 8: Adherence to irradiance. We compare the irradiance of the input image x directly against the irradiance estimated from the images ˆI edited by ZeST and our model (with or without training using the irradiance map E). Column “ours w/o E” corresponds to ablation (A5) fro…
Figure 9
Figure 9. Figure 9: Adherence to texture conditioning. We provide different hue variations of the exemplar material as input and observe that our method correctly adapts to it, maintaining realism in the generated image. This shows the robustness of our texture conditioning approach. © 20…
Figure 10
Figure 10. Figure 10: Impact of exemplar scale. We can control the scale of the transfered material by cropping p. We show results using the full material (×1), half-sized crop (×2), and a quarter-sized crop (×4) to observe its effect on resulting image. Our model has learned to properly i…
Figure 11
Figure 11. Figure 11: Ablation on lighting cues. When deprived of lighting cues by masking out the target image x (i.e., providing x·(1-M) as target) and removing the irradiance map E, our method pro￾duces results with flat, implausible shading (leftmost). Reintroduc￾ing either the irradia…
Figure 12
Figure 12. Figure 12: Ablation of classifier-free guidance. We experimentaly found γ = 3 to be a good trade-off between texture realism and fi￾delity to the conditioning image. Not using CFG leads to misalign￾ments for structured textures as well as artifacts. The γ parameter can be change…
Figure 13
Figure 13. Figure 13: Limitations. We illustrate the limitations of our method, where some geometry can be lost during transfer (left column) and when dealing with downward-facing normals (right column). are obtained when both the full target image and its irradiance are provided which val…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 59 canonical work pages

  1. [1]

    https://evermotion.org/

    [24a] Evermotion. https://evermotion.org/. 2024

    work page 2024

  2. [2]

    SpotLight: Shadow-Guided Object Relighting via Diffusion

    [FZM*24] F ORTIER -CHOUINARD , F RÉDÉRIC , Z HANG , Z ITIAN , MESSIER , L OUIS -ETIENNE , et al. “SpotLight: Shadow-Guided Object Relighting via Diffusion”. 2024

    work page 2024

  3. [3]

    Improving image generation with better captions

    [BGJ*23] B ETKER , JAMES , G OH, G ABRIEL , J ING, L I, et al. “Improving image generation with better captions”. Computer Science. 2023

    work page 2023

  4. [4]

    Blended Latent Diffusion

    [AFL23] A VRAHAMI , O MRI, F RIED , O HAD, and L ISCHINSKI , D ANI. “Blended Latent Diffusion”. ACM TOG. 2023 2, 5, 7,

    work page 2023

  5. [5]

    https : / / huggingface

    [24b] FLUX.1. https : / / huggingface . co / black - forest - labs/FLUX.1-dev. 2024 2, 5, 7,

    work page 2024

  6. [7]

    Zest: Zero-shot material transfer from a single image

    [CSM*24] C HENG , T A-Y ING, S HARMA , P RAFULL , M ARKHAM , A N- DREW , et al. “Zest: Zero-shot material transfer from a single image”. ECCV. 2024 2–5, 7, 8,

    work page 2024

  7. [8]

    Appprop: all-pairs appearance-space edit propagation

    [AP08] A N, X IAOBO and P ELLACINI , F ABIO . “Appprop: all-pairs appearance-space edit propagation”. SIGGRAPH. 2008

    work page 2008

  8. [9]

    Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography

    [Deb08] D EBEVEC , PAUL. “Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography”. SIGGRAPH. 2008

    work page 2008

Show all 60 references
  1. [10]

    In- ternvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    [CWW*24] C HEN, Z HE, W U, J IANNAN , W ANG, W ENHAI , et al. “In- ternvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks”. CVPR. 2024

    2024

  2. [11]

    A Generative Framework for Image-based Editing of Ma- terial Appearance using Perceptual Attributes

    [DLC*22] D ELANOY , J OHANNA , L AGUNAS , M ANUEL , C ONDOR , JORGE , et al. “A Generative Framework for Image-based Editing of Ma- terial Appearance using Perceptual Attributes”. CGF. 2022

    2022

  3. [12]

    Diffusion self-guidance for controllable image generation

    [EJP*23] E PSTEIN , DAVE, JABRI , ALLAN , POOLE , BEN, et al. “Diffusion self-guidance for controllable image generation”. NeurIPS. 2023 2,

    2023

  4. [13]

    Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

    [EKB*24] E SSER , P ATRICK , K ULAL , S UMITH , B LATTMANN , A N- DREAS , et al. “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis”. ICML. 2024

    2024

  5. [14]

    Texsliders: Diffusion-based texture editing in clip space

    [GHR*24] G UERRERO -V IU, J ULIA , H ASAN , M ILOS , R OULLIER , ARTHUR , et al. “Texsliders: Diffusion-based texture editing in clip space”. SIGGRAPH. 2024 2–4. [GHS*19] G ARDNER , M ARC -A NDRÉ , H OLD -GEOFFROY , Y ANNICK , SUNKAVALLI , KALYAN, et al. “Deep parametric indoo...

    2024

  6. [15]

    Proceedings published by Eurographics - The European Association for Computer Graphics

    © 2025 The Author(s). Proceedings published by Eurographics - The European Association for Computer Graphics. 12 of 12 I. Lopes & V . Deschaintre & Y. Hold-Geoffroy & R. de Charette / MatSwap: Light-aware material transfers in images [GNKA25] G ARIFULLIN , K AMIL , N IKOLAEV ,...

    2025

  7. [17]

    Generative adversarial networks

    [GPM*20] G OODFELLOW , I AN, P OUGET -A BADIE , J EAN, M IRZA , MEHDI , et al. “Generative adversarial networks”. Commun. ACM . 2020

    2020

  8. [18]

    OutCast: Outdoor Single-image Relighting with Cast Shadows

    [GRP22] G RIFFITHS , D AVID, R ITSCHEL , T OBIAS , and P HILIP , J ULIEN . “OutCast: Outdoor Single-image Relighting with Cast Shadows”. CGF. 2022

    2022

  9. [19]

    Learning to predict indoor illumination from a single im- age

    [GSY*17] G ARDNER , MARC -ANDRÉ , SUNKAVALLI , KALYAN, YUMER , ERSIN , et al. “Learning to predict indoor illumination from a single im- age”. ACM TOG. 2017

    2017

  10. [20]

    Denoising diffusion probabilistic models

    [HJA20] H O, JONATHAN , JAIN, AJAY, and ABBEEL , PIETER . “Denoising diffusion probabilistic models”. NeurIPS. 2020

    2020

  11. [21]

    Lotus: Diffusion- based Visual Foundation Model for High-quality Dense Prediction

    [HLY*25] H E, J ING, L I, H AODONG , Y IN, W EI, et al. “Lotus: Diffusion- based Visual Foundation Model for High-quality Dense Prediction”. ICLR. 2025 2, 4,

    2025

  12. [22]

    Prompt-to-prompt image editing with cross attention control

    [HMT*22] H ERTZ, A MIR, M OKADY , R ON, T ENENBAUM , J AY, et al. “Prompt-to-prompt image editing with cross attention control”. 2022

    2022

  13. [23]

    Classifier-Free Diffusion Guidance

    [HS21] H O, J ONATHAN and S ALIMANS , T IM. “Classifier-Free Diffusion Guidance”. NeurIPS Workshop. 2021 2, 4,

    2021

  14. [24]

    Aesthetics and emotions in images

    [JDF*11] J OSHI , D HIRAJ , D ATTA, R ITENDRA , F EDOROVSKAYA , ELENA , et al. “Aesthetics and emotions in images”. Signal Processing Magazine. 2011

    2011

  15. [25]

    https://polyhaven.com/

    Poly Haven. https://polyhaven.com/. 2025 4,

    2025

  16. [26]

    Adam: A method for stochastic opti- mization

    [Kin15] K INGMA , D IEDERIK P. “Adam: A method for stochastic opti- mization”. ICLR. 2015

    2015

  17. [27]

    Analyzing and Improving the Image Quality of StyleGAN

    [KLA*20] K ARRAS , T ERO, L AINE , S AMULI , A ITTALA , M IIKA , et al. “Analyzing and Improving the Image Quality of StyleGAN”. CVPR. 2020

    2020

  18. [28]

    Segment anything

    [KMR*23] K IRILLOV , A LEXANDER , M INTUN , E RIC, R AVI, N IKHILA , et al. “Segment anything”. ICCV. 2023

    2023

  19. [29]

    Exposing photo manipulation with inconsistent shadows

    [KOF13] K EE, ERIC, O’B RIEN , JAMES F, and FARID , HANY. “Exposing photo manipulation with inconsistent shadows”. ACM TOG. 2013

    2013

  20. [30]

    Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

    [KOH*24] K E, BINGXIN , OBUKHOV , ANTON , HUANG , SHENGYU , et al. “Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation”. CVPR. 2024

    2024

  21. [31]

    Lightit: Illumination modeling and control for diffusion models

    [KPS*24] K OCSIS , P ETER , P HILIP , J ULIEN , S UNKAVALLI , K ALYAN, et al. “Lightit: Illumination modeling and control for diffusion models”. CVPR. 2024

    2024

  22. [32]

    Image-based material editing

    [KRFB06] K HAN, E RUM ARIF, R EINHARD , E RIK, F LEMING , R OLAND W, and BÜLTHOFF , HEINRICH H. “Image-based material editing”.ACM TOG. 2006

    2006

  23. [33]

    Intrinsic Image Diffusion for Indoor Single-view Material Estimation

    [KSN24] K OCSIS , P ETER , S ITZMANN , V INCENT , and N IESSNER , MATTHIAS . “Intrinsic Image Diffusion for Indoor Single-view Material Estimation”. CVPR. 2024

    2024

  24. [34]

    RePaint: Inpainting using Denoising Diffusion Probabilistic Models

    [LDR*22] L UGMAYR , ANDREAS , DANELLJAN , MARTIN , ROMERO , AN- DRES , et al. “RePaint: Inpainting using Denoising Diffusion Probabilistic Models”. CVPR. 2022

    2022

  25. [35]

    Fixing weight de- cay regularization in adam

    [LH*19] L OSHCHILOV , ILYA, HUTTER , FRANK , et al. “Fixing weight de- cay regularization in adam”. ICLR. 2019

    2019

  26. [36]

    Material Palette: Extraction of Materials from a Single Image

    [LPdC24] L OPES , I VAN, P IZZATI , F ABIO , and de C HARETTE , R AOUL . “Material Palette: Extraction of Materials from a Single Image”. CVPR. 2024

    2024

  27. [37]

    MaterialPicker: Multi-Modal Material Generation with Diffusion Transformers

    [MDH*24] M A, X IAOHE , D ESCHAINTRE , VALENTIN , H AŠAN , M ILOŠ , et al. “MaterialPicker: Multi-Modal Material Generation with Diffusion Transformers”. 2024

    2024

  28. [38]

    SDEdit: Guided Image Synthesis and Editing with Stochastic Differ- ential Equations

    [MHS*22] M ENG, C HENLIN , H E, Y UTONG , S ONG, Y ANG, et al. “SDEdit: Guided Image Synthesis and Editing with Stochastic Differ- ential Equations”. ICLR. 2022

    2022

  29. [39]

    Sdxl: Improving latent diffusion models for high-resolution image syn- thesis

    [PEL*24] P ODELL , D USTIN , E NGLISH , Z ION, L ACEY , K YLE, et al. “Sdxl: Improving latent diffusion models for high-resolution image syn- thesis”. ICLR. 2024 2, 5, 7,

    2024

  30. [40]

    A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis

    [PGP*24] P OIRIER -GINTER , Y OHAN , G AUTHIER , A LBAN , P HILLIP , JULIEN , et al. “A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis”. Computer Graphics Forum. 2024

    2024

  31. [41]

    Zero-shot image-to-image translation

    [PKZ*23] P ARMAR , G AURAV, K UMAR SINGH , K RISHNA , Z HANG , RICHARD , et al. “Zero-shot image-to-image translation”. SIGGRAPH. 2023

    2023

  32. [42]

    Total relighting: learning to relight portraits for back- ground replacement

    [POL*21] P ANDEY , R OHIT , O RTS-E SCOLANO , S ERGIO , L EGENDRE , CHLOE , et al. “Total relighting: learning to relight portraits for back- ground replacement.” ACM TOG. 2021

    2021

  33. [43]

    DiFaReli: Diffusion face relight- ing

    [PTS23] P ONGLERTNAPAKORN , P UNTAWAT, T RITRONG , N ONTAWAT, and S UWAJANAKORN , S UPASORN . “DiFaReli: Diffusion face relight- ing”. ICCV. 2023

    2023

  34. [44]

    Scalable diffusion mod- els with transformers

    [PX23] P EEBLES , WILLIAM and XIE, SAINING . “Scalable diffusion mod- els with transformers”. ICCV. 2023

    2023

  35. [45]

    High-Resolution Image Synthesis with Latent Diffusion Models

    [RBL*22] R OMBACH , R OBIN , B LATTMANN , A NDREAS , L ORENZ , D O- MINIK , et al. “High-Resolution Image Synthesis with Latent Diffusion Models”. CVPR. 2022 2, 4, 5, 7,

    2022

  36. [46]

    Learning transferable visual models from natural language supervi- sion

    [RKH*21] R ADFORD , A LEC, K IM, J ONG WOOK, H ALLACY , C HRIS , et al. “Learning transferable visual models from natural language supervi- sion”. ICML. 2021 2, 4,

    2021

  37. [47]

    LAION-5B: An open large-scale dataset for training next generation image-text models

    [SBV*22] S CHUHMANN , C HRISTOPH , B EAUMONT , R OMAIN , V ENCU , RICHARD , et al. “LAION-5B: An open large-scale dataset for training next generation image-text models”. NeurIPS. 2022

    2022

  38. [48]

    Alchemist: Parametric control of material properties with diffusion models

    [SJL*24] S HARMA , P RAFULL , J AMPANI , V ARUN , L I, Y UANZHEN , et al. “Alchemist: Parametric control of material properties with diffusion models”. CVPR. 2024 3,

    2024

  39. [49]

    Materialistic: Selecting similar materials in images

    [SPG*23] S HARMA , P RAFULL , P HILIP , J ULIEN , G HARBI , M ICHAËL , et al. “Materialistic: Selecting similar materials in images”. ACM TOG. 2023 3,

    2023

  40. [50]

    Deep unsupervised learning using nonequilibrium thermodynamics

    [SWMG15] S OHL -DICKSTEIN , J ASCHA , W EISS , E RIC, M AH- ESWARANATHAN , N IRU, and G ANGULI , S URYA. “Deep unsupervised learning using nonequilibrium thermodynamics”. ICML. 2015

    2015

  41. [51]

    Mat- Synth: A Modern PBR Materials Dataset

    [VD24] V ECCHIO , G IUSEPPE and D ESCHAINTRE , V ALENTIN . “Mat- Synth: A Modern PBR Materials Dataset”. CVPR. 2024

    2024

  42. [52]

    Controlmat: a controlled generative approach to ma- terial capture

    [VMR*24] V ECCHIO , G IUSEPPE , M ARTIN , R OSALIE , R OULLIER , ARTHUR , et al. “Controlmat: a controlled generative approach to ma- terial capture”. ACM TOG. 2024

    2024

  43. [53]

    PSDR-Room: Sin- gle Photo to Scene using Differentiable Rendering

    [YLH*23] Y AN, K., L UAN, F., H AŠAN , M., et al. “PSDR-Room: Sin- gle Photo to Scene using Differentiable Rendering”. SIGGRAPH Asia. 2023

    2023

  44. [54]

    Self-supervised outdoor scene relighting

    [YME*20] Y U, Y E, M EKA, A BHIMITRA , E LGHARIB , M OHAMED , et al. “Self-supervised outdoor scene relighting”. ECCV. 2020

    2020

  45. [55]

    IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Mod- els

    [YZL*23] Y E, H U, Z HANG , J UN, L IU, S IBO, et al. “IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Mod- els”. 2023 2–5,

    2023

  46. [56]

    Diffusion normaliz- ing flow

    [ZC21] Z HANG , QINSHENG and CHEN, YONGXIN . “Diffusion normaliz- ing flow”. NeurIPS. 2021

    2021

  47. [57]

    RGBX: Image decomposition and synthesis using material- and lighting-aware diffusion models

    [ZDG*24] Z ENG, Z HENG , D ESCHAINTRE , V ALENTIN , G EORGIEV , ILIYAN, et al. “RGBX: Image decomposition and synthesis using material- and lighting-aware diffusion models”. SIGGRAPH. 2024 2–5, 7, 8,

    2024

  48. [58]

    Zerocomp: Zero-shot object compositing from image intrinsics via diffusion

    [ZFG*25] Z HANG , ZITIAN , FORTIER -CHOUINARD , FRÉDÉRIC , GARON , MATHIEU , et al. “Zerocomp: Zero-shot object compositing from image intrinsics via diffusion”. WACV. 2025

    2025

  49. [59]

    The unreasonable effectiveness of deep features as a perceptual metric

    [ZIE*18] Z HANG , R ICHARD , I SOLA , P HILLIP , E FROS , A LEXEI A, et al. “The unreasonable effectiveness of deep features as a perceptual metric”. CVPR. 2018

    2018

  50. [60]

    Adding Conditional Control to Text-to-Image Diffusion Models

    [ZRA23] Z HANG , L VMIN , R AO, A NYI, and A GRAWALA , M ANEESH . “Adding Conditional Control to Text-to-Image Diffusion Models”. ICCV. 2023 2, 3, 5, 7,

    2023

  51. [61]

    Proceedings published by Eurographics - The European Association for Computer Graphics

    © 2025 The Author(s). Proceedings published by Eurographics - The European Association for Computer Graphics

    2025

  52. [2025]

    arXiv: 2502.06606 2, 3, 5, 7, 8,

    arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.