Pith. sign in

REVIEW 4 major objections 5 minor 64 references

MARBLE: Material Recomposition and Blending in CLIP-Space

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pre-trained diffusion model can edit, blend, and parametrically control materials in images by manipulating only CLIP image features, with no fine-tuning of the model.

desk verdict A useful empirical contribution with a real inconsistency in the parametric-control formulation; needs major revision before it's publishable. read the letter →

arxiv 2506.05313 v1 pith:UIN73DF5 submitted 2025-06-05 cs.CV

Ta-Ying Cheng , Prafull Sharma , Mark Boss , Varun Jampani This is my paper
classification cs.CV
keywords materialeditingCLIP-spaceembeddingsdiffusionmodelsblendingparametriccontrolzero-shotlow-rankattributedirectionsexemplar-basedtransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that most material editing can be done in CLIP embedding space rather than by retraining or adjusting the generative model. It shows that injecting a CLIP material embedding into a single, identified block of the denoising UNet preserves object geometry while transferring materials, and that interpolating two material embeddings produces a controlled material blend. A small network trained on synthetic renders predicts CLIP-space directions for roughness, metallic, transparency, and glow, giving parametric sliders. Since the base diffusion model is untouched, edits compose in one forward pass and transfer to paintings and stylized images. The central bet is that CLIP embeddings carry a low-rank, linear subspace for material attributes.

What carries the argument

The load-bearing mechanism is targeted material-block injection: instead of injecting the CLIP material embedding into all attention layers as in the base ZeST architecture, MARBLE injects it only into the material block, the U-Net attention layer near the bottleneck that the authors identify as responsible for material attribution. This preserves geometry and lighting while transferring material. A second mechanism is the low-rank material-attribute direction: the paper stacks CLIP-space differences between image pairs with an attribute changed, applies SVD, keeps enough components to explain 67 to 80 percent of the variance, and trains a shallow MLP $p_\theta$ to predict that low-rank direction as a function of image and editing strength $\delta$. Blending uses linear interpolation $\alpha z_{m_1} + (1-\alpha) z_{m_2}$ of the two material embeddings.

What would settle it

Render a physical ground-truth sequence of an object with a material parameter moving continuously between two exemplars, then check whether MARBLE's CLIP-space interpolation and MLP-predicted directions reproduce the rendered intermediate materials at every strength; if the edited images diverge systematically from the ground-truth intermediates, the central claim of a linear material subspace in CLIP space is refuted.

Watch

Extended reading notes

Core claim

MARBLE's central claim is that a surprising amount of material editing is possible with CLIP image features alone. Given a material exemplar image $I_m$, the method extracts its CLIP embedding $z_m$ and injects it only into the U-Net attention block responsible for material attribution, identified by exhaustively visualizing per-block injections. For blending, the edited image is generated with $f(\alpha z_{m_1} + (1-\alpha)z_{m_2})$; for parametric control, the new embedding is $z_{m_{a+\delta}} = \mathrm{CLIP}(I_m) + p_\theta(I_m,\delta)$, where $p_\theta$ is a two-layer MLP trained to predict the low-rank SVD-approximated direction for increasing attribute $a$ by strength $\delta$. All this is done without fine-tuning the pre-trained diffusion model, and the paper shows the controls generalize to real photos, paintings, and multi-attribute edits in a single pass.

Load-bearing premise

The load-bearing premise is that CLIP image embeddings contain a low-rank, linearly interpolatable subspace for material attributes, learned from a few hundred synthetic Blender renders, that transfers to real photographs and paintings even though the SVD step discards 20 to 33 percent of the variance and the material block is chosen by visual inspection.

Editorial extensions

If this is right

  • Material transfer, blending, and parametric attribute control can be combined in a single forward pass without retraining the diffusion model.
  • The approach works on stylized content such as paintings because CLIP space is shared across styles, so the rendered-data-trained controls transfer.
  • Dataset requirements are small: attribute control learns from as few as 16 synthetic objects, and the paper shows quality plateaus beyond that.
  • Because attributes are encoded as independent CLIP-space directions, multiple controls such as roughness and metallic can be applied at once and remain disentangled.
  • Targeted block injection preserves geometry better than full-block injection, preventing artifacts such as hallucinated hands during material transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If CLIP embeddings really contain a linearly interpolatable low-rank subspace for materials, the same targeted-injection recipe may extend to other low-level appearance factors such as lighting direction, camera blur, or aging, with the same tiny-data training regime.
  • The shared block for style and material attribution suggests material transfer and style transfer may be the same operation at the bottleneck; a direct comparison of the material block and the style block could clarify whether one block controls both or whether the overlap is coincidental.
  • A stricter falsification setup would render a physical ground truth for intermediate blend weights and check whether the CLIP-space interpolation tracks the shader parameter continuously; the paper currently validates on rendered validation sets and user preference, not on exact intermediate-material ground truth.
  • The discarded 20 to 33 percent of CLIP variance may carry object identity and environment information; conditioning the SVD or learning per-attribute subspaces separately could reduce the texture-pattern artifacts noted as a limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces MARBLE, a method for material editing that operates by manipulating CLIP image embeddings and injecting them into a selected U-Net block of a pre-trained diffusion inpainting model, without fine-tuning the diffusion model. It claims three capabilities: exemplar-based material transfer with improved geometry preservation via targeted material-block injection (Section 3.1), material blending by linear interpolation of CLIP features of two exemplars (Section 3.2, Eq. 2), and parametric control over roughness, metallic, transparency, and glow using a small MLP trained on synthetic Blender renders to predict a direction in CLIP-space (Section 3.3, Eqs. 3-4). The method builds on the authors' ZeST architecture, and is evaluated with qualitative comparisons to InstructPix2Pix and Concept Sliders, a quantitative comparison to image-trained Concept Sliders on a rendered validation set (Table 1), a small user study, and ablations of dataset size and multi-attribute control.

Significance. If the central claims hold, MARBLE would be a useful result: it demonstrates that a surprising amount of material control can be achieved by CLIP-space manipulation alone, with no diffusion-model fine-tuning, while supporting blending and multiple simultaneous parametric edits in a single forward pass. The paper also provides a data-efficiency ablation showing that as few as 16 objects suffice for training the attribute network, and shows qualitative generalization to painted and stylized images. These are strengths worth acknowledging. However, the parametric-control formulation as written contains an internal inconsistency between the training objective and the inference equation, and the quantitative evaluation is narrow, so the current manuscript does not fully establish its claims.

major comments (4)
  1. [Section 3.3, Eqs. (3) and (4)] The training objective and the inference equation are inconsistent, and this is load-bearing for the parametric-control claim. In Eq. (3), the loss minimises cosine-similarity and MSE between s_{ma+δ} and p_θ(I_ma, δ), so p_θ is trained to output the target low-rank CLIP feature directly. The text confirms this: 'p_θ learns to predict the low-rank approximated CLIP feature of the same original image with one attribute α increased by δ.' However, Eq. (4) computes z_{ma+δ} = CLIP(I_m) + p_θ(I_m, δ), adding the predicted target feature to the original embedding. This double-counts the target: if p_θ already equals the desired edited feature, then the sum is roughly the source plus the target, not a shift toward the target. If p_θ is instead intended to be an offset, then Eq. (3)'s target should be s_{ma+δ} − s_{ma} (or its low-rank projection), not s_{ma+δ}. Since the resulting z is injected into the UNet and determines the material appearance, this is not a cosmetic notation issue; the paper must clarify the intended semantics and make Eqs. (3) and (4) consistent, and if the current training was used, the reported parametric-control results need to be re-examined.
  2. [Section 4.2, Table 1 and user study] The quantitative comparison is too narrow to support the claim of superiority across 'all metrics for all attributes'. Table 1 reports only one baseline (image-trained Concept Slider) and gives no standard deviations, confidence intervals, or significance tests, despite the fact that the validation set is generated from 50 objects and the metrics are likely averaged over images; without variance or per-object statistics, the large gaps could still be informative, but the reader cannot assess stability or whether the result is driven by a few outliers. The user study uses only 16 participants and reports only that '87.5% participants chose images generated by our method' without defining how a participant's choice aggregates over the 20 real-world images, how the three image sets per user were constructed, or whether the comparison was paired and balanced. These reporting gaps should be addressed for the quantitative claims to be convincing.
  3. [Section 3.1 and Section 3.3] The central generalization premise—that a low-rank CLIP subspace learned from 250 Blender-rendered objects transfers to real photographs and paintings—is supported only by qualitative results. The paper states that the SVD retains only 67%–80% of the variance for the four attributes, and the material U-Net block is selected by 'exhaustively visualizing' outputs. These choices are reasonable as heuristics, but they are load-bearing for the method's applicability, and the manuscript presents no quantitative evaluation on real images or on held-out render configurations that vary object class, lighting, and background in a controlled way. A concrete test would be to report attribute-edit success rates or metric comparisons on a real-image set with human or pseudo-ground-truth judgements, and to validate the block choice quantitatively by ablating across a random sample of objects rather than relying on visual inspection.
  4. [Section 3.2, Eq. (2)] Material blending is a core contribution but is evaluated only qualitatively (Figure 5). The paper does not provide quantitative metrics for blending, such as whether the interpolated material at α=0.5 matches a ground-truth render with intermediate shader parameters, nor does it analyse how the interpolation behaves for the three stated configurations. Since blending is one of the paper's headline capabilities, adding a quantitative blending study on the rendered validation set would strengthen the claim substantially. Also, Eq. (2) states 'α > 0' but the interpolation formula α z_{m1} + (1−α) z_{m2} only makes sense for α in [0,1]; the intended range should be stated explicitly.
minor comments (5)
  1. [Section 3.3] The text says 'one attribute α increased by δ' but α is elsewhere used as a blending weight; the attribute name should be 'a' consistently.
  2. [Section 3.3] The description of the dataset says 300 synthetic objects are used but then states '250 for training and 50 for validation', which sums to 300; the text should clarify the split and whether the 250 training objects are the same as the 'as few as 8 objects' ablation set.
  3. [Section 4.3, Figure 11] The data-efficiency plot reports PSNR and DreamSim but the axes and the error bars (if any) are not described; adding error bars or shaded regions over repeated training runs would make the claim about 16 objects more robust.
  4. [Section 4.2] The user study result '87.5% participants chose images generated by our method' is presented without the raw counts or the exact question format; please report the number of participants, the number of comparisons, and the distribution of choices.
  5. [Section 3.1] The paper says the material block is identified following InstantStyle, but does not provide a table or figure showing the effect of injecting into each block; a supplementary figure or a quantitative ablation would help the reader reproduce the selection.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the quantitative claims are tested on held-out rendered objects and against external baselines; the main flagged issue is an internal Eq. 3/Eq. 4 inconsistency, which is a correctness concern rather than an input-output circularity.

full rationale

MARBLE's core pipeline is not circular. The material-block choice is found by exhaustive visualization rather than defined by the evaluation metric; the blending operation in Eq. 2 is a proposed interpolation whose plausibility is shown qualitatively; and the parametric controller is a supervised MLP trained with Eq. 3 on a 250-object rendered set and evaluated on a separate 50-object validation set against Concept Sliders, so the quantitative result is not a restatement of the training objective. The reliance on ZeST [15] is a self-citation to the authors' prior architecture, but it is used as a substrate with modifications and is not the sole justification for the central CLIP-space manipulation claim. The paper also acknowledges concrete limitations, which further indicates that the claims are not being forced by definition. The strongest caveat is that generalization to real photos and paintings rests on qualitative examples, and the implementation as written has a consistency issue: Eq. 3 makes p_theta approximate the absolute target feature s_{ma+delta}, while Eq. 4 adds p_theta to CLIP(I_m) as though it were an offset, which would double-count; this is an internal inconsistency and a correctness risk, not a definitional equivalence between input and output, so it does not constitute circularity under the provided rubric.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on four domain assumptions about CLIP geometry, layer attribution, frozen-model priors, and synthetic-data transfer, plus three fitted quantities: SVD ranks, the selected layer index, and the attribute MLP weights. No new physical entities are introduced.

free parameters (3)
  • SVD rank per attribute = not reported (chosen by elbow; variance explained 67-80%)
    Ranks for the low-rank editing directions are selected per attribute by the elbow method, affecting how much material signal is retained.
  • Material U-Net block index = not stated numerically
    The specific attention block used for injection is identified by exhaustive visualization; it is central to geometry preservation.
  • MLP parameters p_theta = learned on synthetic data
    The attribute editing network weights are the actual fitted editing directions and are the core of parametric control.
assumptions (5)
  • domain assumption CLIP image embeddings encode material attributes in approximately linear, interpolatable subspaces
    Used for blending (Eq. 2) and direction prediction (Eq. 3); supported by qualitative results, not proven.
  • domain assumption A single U-Net block near the bottleneck is responsible for material attribution
    Found via exhaustive block-wise visualization (Section 3.1); the foundation of the base architecture.
  • domain assumption Pre-trained SDXL and IP-Adapter features can be repurposed for material editing without fine-tuning
    The whole method assumes the implicit priors of the frozen diffusion model suffice.
  • domain assumption Synthetic Blender renders with Objaverse objects and HDRIs are representative for learning directions that generalize to real images and paintings
    The MLP is trained only on synthetic data (Section 3.3); generalization beyond this distribution is assumed.
  • domain assumption Low-rank approximation retains sufficient attribute information (67-80% variance)
    SVD truncation is used to denoise CLIP features; the discarded variance is assumed to be unimportant.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MARBLE: Material Recomposition and Blending in CLIP-Space." pith.science (2026). https://pith.science/paper/UIN73DF5

@misc{pith2026250605313,
  author       = {Pith},
  title        = {Pith review of: MARBLE: Material Recomposition and Blending in CLIP-Space},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UIN73DF5}},
  note         = {Machine review of arXiv:2506.05313}
}
read the original abstract

Editing materials of objects in images based on exemplar images is an active area of research in computer vision and graphics. We propose MARBLE, a method for performing material blending and recomposing fine-grained material properties by finding material embeddings in CLIP-space and using that to control pre-trained text-to-image models. We improve exemplar-based material editing by finding a block in the denoising UNet responsible for material attribution. Given two material exemplar-images, we find directions in the CLIP-space for blending the materials. Further, we can achieve parametric control over fine-grained material attributes such as roughness, metallic, transparency, and glow using a shallow network to predict the direction for the desired material attribute change. We perform qualitative and quantitative analysis to demonstrate the efficacy of our proposed method. We also present the ability of our method to perform multiple edits in a single forward pass and applicability to painting. Project Page: https://marblecontrol.github.io/

Figures

Figures reproduced from arXiv: 2506.05313 by the authors.

Figure 1
Figure 1. Overview. We present MARBLE, a method for performing various material editing in images such as material blending (top row) and parametric control of material properties (bottom row) leveraging CLIP-space and pre-trained generative models. Given two material exemplar images, we can achieve a controllable blend of materials on the object by blending the material representation in CLIP-space. For parametric material a… view at source ↗
Figure 2
Figure 2. Comparison of material block injection vs. all Blocks injection. We present examples of using the same input and ma￾terial exemplar. Given the same depth condition, injecting only into the material block allows much better geometry preservation compared to injecting to all blocks in the UNet. in the images [7]. These methods are trained and hence lim￾ited to domains of high-level semantic changes, failing to edit lo… view at source ↗
Figure 3
Figure 3. Method overview for parametric material attribute control. During training, we aim to learn pθ, a shallow MLP that predicts the editing direction in CLIP space given an image Ima . During inference, we can use pθ to predict the offset that can be added to the CLIP embedding for parametric control. Note that Itest* during test time can be separated into two images, one for the context information (background, shading… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Examples of Dataset. We show samples from the ren￾dered dataset for varying roughness, metallic, transparency, and glow. To this end, we propose to alter f(·) to inject the ma￾terial embedding zm only in that specific block of denois￾ing UNet [PITH_FULL_IMAGE:figures/…
Figure 5
Figure 5. Figure 5: Material blending results. By interpolating the CLIP features of the material exemplars, MARBLE can transfer the in￾termediate blended features to the input image, creating material blending effect. Blending can work with exemplar images with following configurations: …
Figure 6
Figure 6. Figure 6: Parametric control results. We present four sets of results controlling roughness, transparency, metallic, and glow. For each set of results, we present one example directly using the reference image for context and material, and another set where we change to a new ma…
Figure 7
Figure 7. Figure 7: Qualitative comparisons. We compare against InstructPix2Pix and 2 versions of Concept Sliders. (T) and (I) denote text and image trained versions, respectively. All baselines either fail to capture the parametric control (Concept Sliders/chair/transparency), or result …
Figure 10
Figure 10. Figure 10: Parametric control with different styles. By lever￾aging the generalization capability of CLIP, our parametric con￾trols can be also be adopted for images with various styles. We present parametric control over two styles of paintings generated by SDXL. Despite being …
Figure 8
Figure 8. Figure 8: Multiple controls at once. With minimal tuning on the pre-trained components, MARBLE can perform material trans￾fer and incorporate multiple attribute controls all in a single pass on real-world images. We present a grid of results of increasing roughness and metallic …
Figure 9
Figure 9. Figure 9: Additional Results for Attribute Control. We present 12 pairs of results on increasing attribute value (From left to right). As In-the-wild editing by Subias et al. also support metallic, we show the qualitative comparisons for the three examples. Zoom in for details. …
Figure 12
Figure 12. Figure 12: Limitations. Our method has two primary limitations. (1) Sometimes performing parametric control also changes the tex￾ture patterns of the object such as the pattern on side of the leather backpack changes as roughness increases (left). (2) Sometimes the effects of th…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 34 canonical work pages

  1. [1]

    Practical svbrdf capture in the frequency domain.ACM Trans

    Miika Aittala, Tim Weyrich, and Jaakko Lehtinen. Practical svbrdf capture in the frequency domain.ACM Trans. Graph., 32(4):110–1, 2013. 3

    work page 2013

  2. [2]

    Two- shot svbrdf capture for stationary materials.ACM Trans

    Miika Aittala, Tim Weyrich, Jaakko Lehtinen, et al. Two- shot svbrdf capture for stationary materials.ACM Trans. Graph., 34(4):110–1, 2015. 3

    work page 2015

  3. [3]

    Multidiffusion: Fusing diffusion paths for controlled image generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. 2023. 2

    2023

  4. [4]

    Continuous, Subject-Specific Attribute Control in T2I Models by Identifying Semantic Directions

    Stefan Andreas Baumann, Felix Krause, Michael Neumayr, Nick Stracke, Vincent Tao Hu, and Björn Ommer. Continu- ous, subject-specific attribute control in t2i models by identi- fying semantic directions.arXiv preprint arXiv:2403.17064,

    work page Pith review arXiv

  5. [5]

    Material recognition in the wild with the materials in con- text database

    Sean Bell, Paul Upchurch, Noah Snavely, and Kavita Bala. Material recognition in the wild with the materials in con- text database. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3479–3487,

    work page

  6. [6]

    Loosecontrol: Lifting controlnet for generalized depth con- ditioning.arXiv preprint arXiv:2312.03079, 2023

    Shariq Farooq Bhat, Niloy J Mitra, and Peter Wonka. Loosecontrol: Lifting controlnet for generalized depth con- ditioning.arXiv preprint arXiv:2312.03079, 2023. 3

    arXiv 2023

  7. [7]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 2, 3, 6

    work page 2023

  8. [8]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing.arXiv preprint arXiv:2304.08465, 2023

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing.arXiv preprint arXiv:2304.08465, 2023. 2

    arXiv 2023

Show all 64 references
  1. [9]

    Texfusion: Synthesizing 3d textures with text-guided image diffusion models

    Tianshi Cao, Karsten Kreis, Sanja Fidler, Nicholas Sharp, and Kangxue Yin. Texfusion: Synthesizing 3d textures with text-guided image diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4169–4181, 2023. 3

    2023

  2. [10]

    Matatlas: Text- driven consistent geometry texturing and material assign- ment.arXiv preprint arXiv:2404.02899, 2024

    Duygu Ceylan, Valentin Deschaintre, Thibault Groueix, Rosalie Martin, Chun-Hao Huang, Romain Rouffet, Vladimir Kim, and Gaëtan Lassagne. Matatlas: Text- driven consistent geometry texturing and material assign- ment.arXiv preprint arXiv:2404.02899, 2024

    2024 arXiv

  3. [11]

    Text2tex: Text-driven texture synthesis via diffusion models.arXiv preprint arXiv:2303.11396, 2023

    Dave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner. Text2tex: Text-driven texture synthesis via diffusion models.arXiv preprint arXiv:2303.11396, 2023. 3

    2023 arXiv

  4. [12]

    Training-free layout control with cross-attention guidance

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. InProceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5343–5353, 2024. 2

    2024

  5. [13]

    Subject-driven text-to-image generation via apprenticeship learning.arXiv preprint arXiv:2304.00186, 2023

    Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Rui, Xuhui Jia, Ming-Wei Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning.arXiv preprint arXiv:2304.00186, 2023. 2

    2023 arXiv

  6. [14]

    Subject-driven text-to-image generation via apprenticeship learning.Advances in Neural Information Processing Sys- tems, 36, 2024

    Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming-Wei Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning.Advances in Neural Information Processing Sys- tems, 36, 2024. 2

    2024

  7. [15]

    Zest: Zero-shot material trans- fer from a single image

    Ta-Ying Cheng, Prafull Sharma, Andrew Markham, Niki Trigoni, and Varun Jampani. Zest: Zero-shot material trans- fer from a single image. InEuropean Conference on Com- puter Vision, pages 370–386. Springer, 2025. 2, 3

    2025

  8. [16]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2023

  9. [17]

    A generative framework for image- based editing of material appearance using perceptual at- tributes

    Johanna Delanoy, Manuel Lagunas, J Condor, Diego Gutier- rez, and Belén Masia. A generative framework for image- based editing of material appearance using perceptual at- tributes. InComputer Graphics Forum, pages 453–464. Wi- ley Online Library, 2022. 3

    2022

  10. [18]

    Flexible svbrdf capture with a multi-image deep network

    Valentin Deschaintre, Miika Aittala, Frédo Durand, George Drettakis, and Adrien Bousseau. Flexible svbrdf capture with a multi-image deep network. InComputer graphics fo- rum, pages 1–13. Wiley Online Library, 2019. 3

    2019

  11. [19]

    Diffusion models beat gans on image synthesis.Advances in neural informa- tion processing systems, 34:8780–8794, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 2

    2021

  12. [20]

    Dream- sim: Learning new dimensions of human visual similar- ity using synthetic data.arXiv preprint arXiv:2306.09344,

    Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dream- sim: Learning new dimensions of human visual similar- ity using synthetic data.arXiv preprint arXiv:2306.09344,

    arXiv

  13. [21]

    Concept sliders: Lora adap- tors for precise control in diffusion models.arXiv preprint arXiv:2311.12092, 2023

    Rohit Gandikota, Joanna Materzynska, Tingrui Zhou, Anto- nio Torralba, and David Bau. Concept sliders: Lora adap- tors for precise control in diffusion models.arXiv preprint arXiv:2311.12092, 2023. 3, 6

    2023 arXiv

  14. [22]

    Expressive text-to-image generation with rich text

    Songwei Ge, Taesung Park, Jun-Yan Zhu, and Jia-Bin Huang. Expressive text-to-image generation with rich text. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7545–7556, 2023. 2

    2023

  15. [23]

    Texsliders: Diffusion- based texture editing in clip space

    Julia Guerrero-Viu, Milos Hasan, Arthur Roullier, Midhun Harikumar, Yiwei Hu, Paul Guerrero, Diego Gutierrez, Be- len Masia, and Valentin Deschaintre. Texsliders: Diffusion- based texture editing in clip space. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 3

    2024

  16. [24]

    Ganspace: Discovering interpretable gan con- trols.Advances in neural information processing systems, 33:9841–9850, 2020

    Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. Ganspace: Discovering interpretable gan con- trols.Advances in neural information processing systems, 33:9841–9850, 2020. 4

    2020

  17. [25]

    Prompt-to-prompt im- age editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022. 2

    2022 arXiv

  18. [26]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 2

    2022 arXiv

  19. [27]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  20. [28]

    Cascaded diffusion models for high fidelity image generation.The Journal of Machine Learning Research, 23(1):2249–2281, 2022

    Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation.The Journal of Machine Learning Research, 23(1):2249–2281, 2022

    2022

  21. [29]

    Scal- ing up gans for text-to-image synthesis

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scal- ing up gans for text-to-image synthesis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10124–10134, 2023

    2023

  22. [30]

    Elucidating the design space of diffusion-based generative models.Advances in Neural Information Processing Sys- tems, 35:26565–26577, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.Advances in Neural Information Processing Sys- tems, 35:26565–26577, 2022. 2

    2022

  23. [31]

    Image-based material editing.ACM Transactions on Graphics (TOG), 25(3):654–663, 2006

    Erum Arif Khan, Erik Reinhard, Roland W Fleming, and Heinrich H Bülthoff. Image-based material editing.ACM Transactions on Graphics (TOG), 25(3):654–663, 2006. 2, 3

    2006

  24. [32]

    In- trinsic image diffusion for indoor single-view material es- timation

    Peter Kocsis, Vincent Sitzmann, and Matthias Nießner. In- trinsic image diffusion for indoor single-view material es- timation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5198– 5208, 2024. 3

    2024

  25. [33]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1931–1941, 2023. 2

    1931

  26. [34]

    Clearclip: Decom- posing clip representations for dense vision-language infer- ence.arXiv preprint arXiv:2407.12442, 2024

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Clearclip: Decom- posing clip representations for dense vision-language infer- ence.arXiv preprint arXiv:2407.12442, 2024. 5

    2024 arXiv

  27. [35]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22511–22521, 2023. 2

    2023

  28. [36]

    Multimodal material segmentation

    Yupeng Liang, Ryosuke Wakaki, Shohei Nobuhara, and Ko Nishino. Multimodal material segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 19800–19808, 2022. 3

    2022

  29. [37]

    Material palette: Extraction of materials from a single image

    Ivan Lopes, Fabio Pizzati, and Raoul de Charette. Material palette: Extraction of materials from a single image. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4379–4388, 2024. 3

    2024

  30. [38]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.arXiv preprint arXiv:2302.08453, 2023

    Chong Mou, Xintao Wang, Liangbin Xie, Jian Zhang, Zhon- gang Qi, Ying Shan, and Xiaohu Qie. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.arXiv preprint arXiv:2302.08453, 2023. 3

    2023 arXiv

  31. [39]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  32. [40]

    Drag your gan: Interactive point-based manipulation on the generative image manifold

    Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. Drag your gan: Interactive point-based manipulation on the generative image manifold. InACM SIGGRAPH 2023 Conference Pro- ceedings, pages 1–11, 2023. 4

    2023

  33. [41]

    Karran Pandey, Paul Guerrero, Metheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, and Niloy J. Mitra. Diffusion handles: Enabling 3d edits for diffusion models by lifting activations to 3d.CVPR, 2024. 2

    2024

  34. [42]

    Precisecontrol: En- hancing text-to-image diffusion models with fine-grained at- tribute control

    Rishubh Parihar, VS Sachidanand, Sabariswaran Mani, Te- jan Karmali, and R Venkatesh Babu. Precisecontrol: En- hancing text-to-image diffusion models with fine-grained at- tribute control. InEuropean Conference on Computer Vision, pages 469–487. Springer, 2025. 3

    2025

  35. [43]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  36. [44]

    Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  37. [45]

    Texture: Text-guided texturing of 3d shapes.arXiv preprint arXiv:2302.01721, 2023

    Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or. Texture: Text-guided texturing of 3d shapes.arXiv preprint arXiv:2302.01721, 2023. 3

    2023 arXiv

  38. [46]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. 2022 ieee. InCVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, 2021. 2, 3

    2022

  39. [47]

    Rb-modulation: Training-free personalization of diffu- sion models using stochastic optimal control.arXiv preprint arXiv:2405.17401, 2024

    Litu Rout, Yujia Chen, Nataniel Ruiz, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Rb-modulation: Training-free personalization of diffu- sion models using stochastic optimal control.arXiv preprint arXiv:2405.17401, 2024. 2

    2024 arXiv

  40. [48]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.arXiv preprint arXiv:2208.12242, 2022

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.arXiv preprint arXiv:2208.12242, 2022. 2

    2022 arXiv

  41. [49]

    Photorealistic text-to-image diffusion models with deep language understanding.Advances in Neural Information Processing Systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.Advances in Neural Information ...

    2022

  42. [50]

    Ziplora: Any subject in any style by effectively merging loras

    Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svet- lana Lazebnik, Yuanzhen Li, and Varun Jampani. Ziplora: Any subject in any style by effectively merging loras. In European Conference on Computer Vision, pages 422–438. Springer, 2025. 2

    2025

  43. [51]

    Alchemist: Parametric control of material proper- ties with diffusion models.arXiv preprint arXiv:2312.02970,

    Prafull Sharma, Varun Jampani, Yuanzhen Li, Xuhui Jia, Dmitry Lagun, Fredo Durand, William T Freeman, and Mark Matthews. Alchemist: Parametric control of material proper- ties with diffusion models.arXiv preprint arXiv:2312.02970,

    arXiv

  44. [52]

    Materialistic: Se- lecting similar materials in images.ACM Transactions on Graphics (TOG), 42(4):1–14, 2023

    Prafull Sharma, Julien Philip, Michaël Gharbi, Bill Freeman, Fredo Durand, and Valentin Deschaintre. Materialistic: Se- lecting similar materials in images.ACM Transactions on Graphics (TOG), 42(4):1–14, 2023. 3

    2023

  45. [53]

    Generative modeling by esti- mating gradients of the data distribution.Advances in neural information processing systems, 32, 2019

    Yang Song and Stefano Ermon. Generative modeling by esti- mating gradients of the data distribution.Advances in neural information processing systems, 32, 2019. 2

    2019

  46. [54]

    In-the-wild mate- rial appearance editing using perceptual attributes

    J Daniel Subias and Manuel Lagunas. In-the-wild mate- rial appearance editing using perceptual attributes. InCom- puter Graphics Forum, pages 333–345. Wiley Online Li- brary, 2023. 3

    2023

  47. [55]

    A dense material seg- mentation dataset for indoor and outdoor scene parsing

    Paul Upchurch and Ransen Niu. A dense material seg- mentation dataset for indoor and outdoor scene parsing. In European Conference on Computer Vision, pages 450–466. Springer, 2022. 3

    2022

  48. [56]

    Andrey V oynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman.p+: Extended textual conditioning in text-to- image generation.arXiv preprint arXiv:2303.09522, 2023. 2

    2023 arXiv

  49. [57]

    Instantstyle: Free lunch towards style-preserving in text-to-image generation.arXiv preprint arXiv:2404.02733, 2024

    Haofan Wang, Matteo Spinelli, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen. Instantstyle: Free lunch towards style-preserving in text-to-image generation.arXiv preprint arXiv:2404.02733, 2024. 2, 3

    2024 arXiv

  50. [58]

    Instancediffusion: Instance- level control for image generation.arXiv preprint arXiv:2402.03290, 2024

    Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Ro- hit Girdhar, and Ishan Misra. Instancediffusion: Instance- level control for image generation.arXiv preprint arXiv:2402.03290, 2024. 2

    2024 arXiv

  51. [59]

    Reco: Region-controlled text-to-image genera- tion

    Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, et al. Reco: Region-controlled text-to-image genera- tion. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 14246–14255,

  52. [60]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models.arXiv preprint arXiv:2308.06721,

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models.arXiv preprint arXiv:2308.06721,

    arXiv

  53. [61]

    Texture- dreamer: Image-guided texture synthesis through geometry- aware diffusion.arXiv preprint arXiv:2401.09416, 2024

    Yu-Ying Yeh, Jia-Bin Huang, Changil Kim, Lei Xiao, Thu Nguyen-Phuoc, Numair Khan, Cheng Zhang, Manmohan Chandraker, Carl S Marshall, Zhao Dong, et al. Texture- dreamer: Image-guided texture synthesis through geometry- aware diffusion.arXiv preprint arXiv:2401.09416, 2024. 3

    2024 arXiv

  54. [62]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 3

    2023

  55. [63]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6

    2018

  56. [64]

    Uni-controlnet: All-in-one control to text-to-image diffusion models.Advances in Neural Information Processing Sys- tems, 36, 2024

    Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan-Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models.Advances in Neural Information Processing Sys- tems, 36, 2024. 3

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.