REVIEW 5 major objections 5 minor 98 references
LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Model
T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read LineArt transfers complex materials and lighting onto detailed design drawings with a training-free diffusion pipeline that outperforms current state-of-the-art methods in accuracy, realism, and material precision.
desk verdict A useful training-free appearance transfer pipeline with a valuable new dataset, but the evaluation hygiene and a circular soft-edge bootstrap keep the central claim from being fully supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The framework's two engines are the multi-frequency lines fusion module and the two-part painting process. The first decomposes the input line drawing into three levels of edge assertions — a single-line outer contour from the mask, double lines from morphological dilation-erosion, and soft edges extracted via Haar filtering of an initial depth-guided generation — which are fused and fed to ControlNet as geometric conditioning. The second splits appearance transfer: Base Layer Shaping computes a multi-scale Retinex illumination image, extracts its brightness mean $L_{\text{mean}}$, and shifts the initial latent noise by $x'_0 = L_{\text{mean}} + (x_0 - L_{\text{mean}}) \times 0.5$ to pre-bias lighting, while Surface Layer Coloring removes the reference background, reassembles the remaining pixels into a synthetic texture image, and injects its embedding into specific U-net attention layers to control material and color.
What would settle it
Run a controlled sweep: fix one line drawing and one seed, vary only the reference photo's brightness (or its Retinex-derived $L_{\text{mean}}$) across a wide range, and measure whether the generated image's mean illumination tracks the reference's $L_{\text{mean}}$ monotonically. If output brightness is insensitive to $L_{\text{mean}}$, or is non-monotonic, the core mechanism of Base Layer Shaping is falsified; a complementary check replaces the shifted latent $x'_0$ by the unshifted $x_0$ and compares lighting-aware metrics.
Extended reading notes
Core claim
The central discovery is that a knowledge-guided, fully training-free pipeline built on ControlNet, IP-Adapter, and Stable Diffusion can outperform state-of-the-art appearance-transfer and sketch-to-image methods specifically on detailed, professional design drawings. By decomposing the input drawing into single lines, double lines, and soft edges, and by separating the colorization into Base Layer Shaping (driving the initial latent with a brightness analysis of the reference's Retinex illumination) and Surface Layer Coloring (injecting a reassembled texture map into chosen U-net attention layers), the method claims high fidelity to both the drawing's structure and the reference's material, with quantitative gains on eight metrics and user-study wins across edge fidelity, appearance, and overall perception.
Load-bearing premise
The method assumes that shifting the initial latent noise by a pixel-domain brightness statistic of the reference photo transfers the reference's lighting to the final image; this mapping is asserted rather than derived, and the 0.5 scale is chosen by hand, so if that linkage is wrong the Base Layer Shaping stage does not do its claimed work.
Editorial extensions
If this is right
- Designers can preview materials on technical drawings without any training or 3D modeling, needing only a line drawing and a reference photo.
- The three-level line decomposition gives control over pattern thickness and texture density, so the same framework adapts to different drawing complexities.
- Separating lighting from texture avoids the attribute degradation that appears when a single text prompt or image prompt tries to convey both at once.
- The ProLines dataset provides a professional-drawing benchmark for future sketch-to-image and appearance-transfer research.
- Because the pipeline is training-free, it can be rebuilt on any future diffusion backbone or adapter with the same structure and appearance guidance modules.
Reading between the lines
- The pixel-space-to-latent-space brightness transfer used in Base Layer Shaping suggests a testable extension: replacing the hand-set 0.5 factor with a calibrated or per-image adaptive scale could make lighting control more precise and less dependent on the reference photo's dynamic range.
- Because the method is assembled from modular components, an ablation that swaps the Retinex brightness analysis for a simple average brightness, or the patch reassembly for plain cropping, would reveal how much of the measured gain comes from the knowledge-guided decomposition versus the base model itself.
- The same hierarchical edge decomposition could serve neighboring tasks, such as line-art colorization, sketch-based editing, or material preview in product design, wherever preserving fine structural detail is the bottleneck.
- A direct stress test would apply the method to drawings with heavy pre-existing shading or dense cross-hatching, since the paper states these are failure cases and the framework's structure-preservation claims are most at risk there.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. LineArt proposes a training-free pipeline for transferring appearance from a reference photo to a detailed line drawing. It builds on Stable Diffusion, ControlNet, and IP-Adapter, and consists of two stages: a multi-frequency line fusion module (single lines, double lines, and soft edges) for structure guidance, and a two-part painting process (Base Layer Shaping via Retinex-based brightness initialization, and Surface Layer Coloring via texture synthesis and selective attention-layer injection). The paper also introduces the ProLines dataset of 5,101 filtered design drawings and reports quantitative, qualitative, and user-study comparisons against several state-of-the-art methods. The central claim is that LineArt outperforms existing methods in accuracy, realism, and material precision while preserving structural details of the input line drawing.
Significance. Training-free appearance transfer for professional line drawings is practically valuable, and the paper's analogy to classical underpainting/glazing is a creative way to structure the pipeline. The proposed ProLines dataset is a potentially useful resource for the sketch-to-image community, and the use of standard components with no training makes the method easy to adopt if the claims hold. However, the current evidence is not yet convincing: the soft-edge construction is circular, the latent brightness initialization is heuristic, key ablations are deferred to an absent supplementary document, and the quantitative evaluation lacks uncertainty estimates. The paper also does not mention code release or dataset availability in the main text. Strengths include a multi-metric comparison, a user study, and a method description detailed enough to be reimplemented; the weaknesses are concentrated in the validation of the central claims.
major comments (5)
- [Section 4.1] The soft-edge construction is circular. The paper computes S_soft = Haar(G_initial), where G_initial is itself generated using ControlNet-depth conditioned on both the line drawing and the appearance image. Therefore Condition_geometry = Fusion(L_double, L_single, S_soft) contains high-frequency information derived from the reference appearance, not only from the original line drawing. This conflates texture with structure and directly threatens the central claim of structure preservation. Please ablate S_soft by replacing it with an edge/high-frequency map computed solely from L_original, and report structure metrics (SSIM/CD) for the variant without S_soft and for the appearance-free soft-edge variant. Without this ablation, the contribution of multi-frequency line fusion to structural fidelity is not established.
- [Section 4.2] The Base Layer Shaping initialization x'_0 = Lmean + (x0 - Lmean) * 0.5 lacks a derivation or controlled validation. The paper does not explain why a brightness statistic of the pixel-space Retinex illumination can be additively transplanted into the VAE latent x0, nor what Lmean and x0 are precisely (a scalar, a per-channel vector, or a latent tensor), and the factor 0.5 appears hand-picked. The claim that this creates a 'soft link' between the lighting of the reference and the generated image is not supported by theory or by quantitative ablation; Figure 7(b) is only qualitative. Please clarify the notation, provide a derivation or a rigorous empirical study, and report quantitative ablations over Lmean values and the 0.5 factor to justify this key design choice.
- [Table 1 / Section 6.1] The quantitative evaluation reports single values without error bars, statistical significance, or the number of images evaluated per dataset. Several comparisons are close or even reverse: for Bronze, DreamBooth achieves higher PSNR (27.15) than LineArt (25.20), and the FID gap between LineArt and the baseline is small in several rows. The claim that LineArt 'performs better in accuracy, realism, and material precision compared to SOTAs' is therefore not statistically substantiated. Please report mean and standard deviation over multiple seeds or test splits, run significance tests, and specify the evaluation set size and protocol for each metric.
- [Section 6.3] The ablation study, which addresses four central design choices (double lines, soft edges, surface-layer coloring, and patch size), is only described as being in the supplementary materials; the main text contains no ablation table or figure, and the arXiv version does not appear to include the supplement. The user study details are also deferred. Since these ablations directly support the method's core contributions, the main text should include at least a compact ablation table and the essential user-study protocol (number of participants, number of stimuli, ranking scale, and statistical analysis).
- [Section 5] The ProLines dataset is filtered to hand-chosen IC intervals (e.g., Bronze 0.2576–0.2903, ImageNet-Sketch 0.2500–0.2650) followed by manual screening by the authors. The paper does not state what fraction of images was removed at each stage, whether annotators agreed, or whether the thresholds were selected before or after observing model performance. This creates a risk that the evaluation is conducted on a self-selected subset favorable to the proposed method. Please provide a sensitivity analysis across IC thresholds and report results on a held-out random sample of the unfiltered source data.
minor comments (5)
- [Abstract and Section 4.2] There are several typos and grammatical errors: the section title 'Knowleged-guided Structure Preservation' should be 'Knowledge-guided', 'Accoding' should be 'According', and Figure 7's caption contains 'Origianl' instead of 'Original'.
- [Section 4.1 and Section 4.2] The notation is inconsistent: 'Conditiongeometry' appears in Section 4.1 while 'Condition_geometry' appears elsewhere; the operators Fusion, TS, and moveBackground are used without precise definitions. Please define all operators and use consistent notation throughout.
- [Section 6.1] The metrics GLCM and CH are not defined in the main text. Please specify the exact distance or statistic used for the gray-level co-occurrence matrix and the color histogram loss so that the reported numbers are reproducible.
- [Section 5] The 'Bronze' dataset source is not cited in the reference list, and no dataset URL or release statement is provided for ProLines. Please add the missing citation and state how the dataset will be distributed.
- [Figures 1 and 8] The list of compared methods is not consistent between the qualitative figures and Table 1: Figure 1 shows T2I-Adapter-SDXL while Table 1 includes DreamBooth and InstructPix2Pix but not T2I-Adapter-SDXL. Please align the comparison methods across all experiments.
Circularity Check
The soft-edge structure channel in Section 4.1 is bootstrapped from a first-pass generation that already uses the appearance reference, making the claimed structure preservation partially self-referential.
-
self definitional
[Section 4.1, Knowledge-guided Structure Preservation (S_soft = Haar(G_initial) and Condition_geometry = Fusion(...))]
"Next, we use [91] to predict the depth information of the line drawing and generate it for the first time using ControlNet-depth, in combination with the specified appearance image. The initial generation result Ginitial is then used to extract soft edges Ssoft that capture spatial gradients and texture characteristics as Ssoft = Haar(Ginitial). These soft edges act as high-frequency constraints in subsequent appearance generation, ensuring accuracy and naturalness of the results in terms of texture primitives."
The high-frequency constraints that are fused into Condition_geometry are not derived from the original line drawing L_original. They are extracted from G_initial, a first-pass image produced by ControlNet-depth 'in combination with the specified appearance image.' Thus the same reference appearance that is later injected through IP-Adapter also determines the soft-edge conditioning, so structure and appearance are not cleanly separated. The paper even describes S_soft as capturing 'texture characteristics' and 'texture primitives,' which means the 'geometry' channel partly carries appearance content rather than independent structural evidence from the drawing.
full rationale
LineArt's central claim is high-fidelity appearance transfer with structural accuracy, and most of its pipeline (depth-based ControlNet, L_double/L_single morphology, IP-Adapter material injection, Retinex-based latent mean shift, and the ProLines evaluation against external SOTAs) is independent and empirically testable. The one load-bearing self-reference is the soft-edge term: S_soft = Haar(G_initial), where G_initial is itself generated using the appearance image. This makes the final ControlNet geometry conditioning partly a function of the appearance input, so the method's structural channel is not purely derived from the line drawing. However, the final output is not identical to G_initial, the single- and double-line channels remain tied to L_original, and the quantitative evaluation compares against external baselines, so the circularity is partial rather than total. I found no self-citation chains, uniqueness arguments, or fitted-parameter-as-prediction constructions. The hand-chosen 0.5 factor in x'_0 = Lmean + (x0 - Lmean)*0.5 and the dataset IC thresholds are heuristic choices, not circularities, though they are correctness risks.
Assumptions & free parameters
free parameters (8)
- Morphological kernel sizes for erode/dilate =
not specified
- Retinex multi-scale Gaussian scales and weights =
not specified
- Texture synthesis patch size =
not specified
- Latent brightness adjustment factor =
0.5
- IP-Adapter projection feature count =
4
- Selected attention layers for material injection =
up blocks.0.attentions.1; down blocks.2.attentions.1
- Fusion weights for Conditiongeometry =
not specified
- ProLines IC threshold ranges =
e.g., Bronze 0.2576-0.2903, DifferSketching 0.0461-0.2165
assumptions (6)
- domain assumption Visual representations of sketches can be decomposed into single lines, double lines, and soft edges (based on Marr's assertions).
- domain assumption The Imprimatura painting technique (underpainting and glazing) maps to two separate diffusion guidance stages.
- domain assumption Multi-scale Retinex extracts an illumination map that corresponds to perceived brightness.
- ad hoc to paper The initial latent shift x'_0 = Lmean + (x0 - Lmean) * 0.5 provides a soft brightness link.
- ad hoc to paper The selected attention layers are the right places to inject material features.
- domain assumption Depth Anything's depth estimates on line drawings are reliable enough for ControlNet-depth guidance.
Cite this review
Pith. "Pith review of LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Model." pith.science (2026). https://pith.science/paper/VAYYLS4W
@misc{pith2026241211519,
author = {Pith},
title = {Pith review of: LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Model},
year = {2026},
howpublished = {\url{https://pith.science/paper/VAYYLS4W}},
note = {Machine review of arXiv:2412.11519}
}
read the original abstract
Image rendering from line drawings is vital in design and image generation technologies reduce costs, yet professional line drawings demand preserving complex details. Text prompts struggle with accuracy, and image translation struggles with consistency and fine-grained control. We present LineArt, a framework that transfers complex appearance onto detailed design drawings, facilitating design and artistic creation. It generates high-fidelity appearance while preserving structural accuracy by simulating hierarchical visual cognition and integrating human artistic experience to guide the diffusion process. LineArt overcomes the limitations of current methods in terms of difficulty in fine-grained control and style degradation in design drawings. It requires no precise 3D modeling, physical property specs, or network training, making it more convenient for design tasks. LineArt consists of two stages: a multi-frequency lines fusion module to supplement the input design drawing with detailed structural information and a two-part painting process for Base Layer Shaping and Surface Layer Coloring. We also present a new design drawing dataset ProLines for evaluation. The experiments show that LineArt performs better in accuracy, realism, and material precision compared to SOTAs.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Cross-image attention for zero- shot appearance transfer
Yuval Alaluf, Daniel Garibi, Or Patashnik, Hadar Averbuch- Elor, and Daniel Cohen-Or. Cross-image attention for zero- shot appearance transfer. In ACM SIGGRAPH 2024 Confer- ence Papers, pages 1–12, 2024. 1, 4, 8
2024
-
[2]
Sketchinr: A first look into sketches as implicit neural representations
Hmrishav Bandyopadhyay, Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Aneeshan Sain, Tao Xiang, Timothy Hospedales, and Yi-Zhe Song. Sketchinr: A first look into sketches as implicit neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12565–12574, 2024. 3
2024
-
[3]
What sketch explainability really means for downstream tasks? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10997–11008, 2024
Hmrishav Bandyopadhyay, Pinaki Nath Chowdhury, Ayan Kumar Bhunia, Aneeshan Sain, Tao Xiang, and Yi-Zhe Song. What sketch explainability really means for downstream tasks? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10997–11008, 2024. 3, 4
2024
-
[4]
Masksketch: Unpaired structure-guided masked image generation
Dina Bashkirova, Jose Lezama, Kihyuk Sohn, Kate Saenko, and Irfan Essa. Masksketch: Unpaired structure-guided masked image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1879–1889, 2023. 2, 3
2023
-
[5]
Style and abstraction in por- trait sketching
Itamar Berger, Ariel Shamir, Moshe Mahler, Elizabeth Carter, and Jessica Hodgins. Style and abstraction in por- trait sketching. ACM Transactions on Graphics (TOG) , 32 (4):1–12, 2013. 3
2013
-
[6]
Pixelor: A competitive sketching ai agent
Ayan Kumar Bhunia, Ayan Das, Umar Riaz Muhammad, Yongxin Yang, Timothy M Hospedales, Tao Xiang, Yulia Gryaditskaya, and Yi-Zhe Song. Pixelor: A competitive sketching ai agent. so you think you can sketch?ACM Trans- actions on Graphics (TOG), 39(6):1–15, 2020. 3
2020
-
[7]
Sketch less for more: On-the-fly fine-grained sketch-based image retrieval
Ayan Kumar Bhunia, Yongxin Yang, Timothy M Hospedales, Tao Xiang, and Yi-Zhe Song. Sketch less for more: On-the-fly fine-grained sketch-based image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9779–9788, 2020. 2
2020
-
[8]
B-cos net- works: Alignment is all we need for interpretability
Moritz B ¨ohle, Mario Fritz, and Bernt Schiele. B-cos net- works: Alignment is all we need for interpretability. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10329–10338, 2022. 4
2022
Show all 98 references
-
[9]
In- structpix2pix: Learning to follow image editing instructions
Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer 9 Vision and Pattern Recognition, pages 18392–18402, 2023. 8
2023
-
[10]
Painting materials and working methods.A Corpus of Rembrandt Paintings: 1625–1631 , pages 11–33,
J Bruyn, B Haak, SH Levie, PJJ Van Thiel, E Van De Weter- ing, and E vd W. Painting materials and working methods.A Corpus of Rembrandt Paintings: 1625–1631 , pages 11–33,
-
[11]
Diffusart: Enhancing line art coloriza- tion with conditional diffusion models
Hernan Carrillo, Micha ¨el Cl ´ement, Aur ´elie Bugeau, and Edgar Simo-Serra. Diffusart: Enhancing line art coloriza- tion with conditional diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3486–3490, 2023. 2
2023
-
[12]
Fast image pro- cessing with fully-convolutional networks
Qifeng Chen, Jia Xu, and Vladlen Koltun. Fast image pro- cessing with fully-convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision , pages 2497–2506, 2017. 4
2017
-
[13]
Deepfacedrawing: Deep generation of face im- ages from sketches
Shu Yu Chen, Wanchao Su, Lin Gao, Shihong Xia, and Hongbo Fu. Deepfacedrawing: Deep generation of face im- ages from sketches. ACM Transactions on Graphics, 39(4): 72, 2020. 2, 3
2020
-
[14]
Sketchygan: Towards di- verse and realistic sketch to image synthesis
Wengling Chen and James Hays. Sketchygan: Towards di- verse and realistic sketch to image synthesis. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 9416–9425, 2018. 2
2018
-
[15]
Adaptively-realistic image gen- eration from stroke and sketch with diffusion model
Shin-I Cheng, Yu-Jie Chen, Wei-Chen Chiu, Hung-Yu Tseng, and Hsin-Ying Lee. Adaptively-realistic image gen- eration from stroke and sketch with diffusion model. In Pro- ceedings of the IEEE/CVF winter conference on applications of computer vision, pages 4054–4062, 2023. 3
2023
-
[16]
Zest: Zero-shot material trans- fer from a single image
Ta-Ying Cheng, Prafull Sharma, Andrew Markham, Niki Trigoni, and Varun Jampani. Zest: Zero-shot material trans- fer from a single image. In European Conference on Com- puter Vision, pages 370–386. Springer, 2025. 1, 8
2025
-
[17]
Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer
Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8795–8805, 2024. 1, 3, 4, 8
2024
-
[18]
Cross- modal latent space alignment for image to avatar translation
Manuel Ladron De Guevara, Jose Echevarria, Yijun Li, Yan- nick Hold-Geoffroy, Cameron Smith, and Daichi Ito. Cross- modal latent space alignment for image to avatar translation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 520–529, 2023. 2
2023
-
[19]
Stytr2: Im- age style transfer with transformers
Yingying Deng, Fan Tang, Weiming Dong, Chongyang Ma, Xingjia Pan, Lei Wang, and Changsheng Xu. Stytr2: Im- age style transfer with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11326–11336, 2022. 2, 3
2022
-
[20]
How do humans sketch objects? ACM Transactions on graphics (TOG) , 31 (4):1–10, 2012
Mathias Eitz, James Hays, and Marc Alexa. How do humans sketch objects? ACM Transactions on graphics (TOG) , 31 (4):1–10, 2012. 6
2012
-
[21]
Image smoothing via unsupervised learning
Qingnan Fan, Jiaolong Yang, David Wipf, Baoquan Chen, and Xin Tong. Image smoothing via unsupervised learning. ACM Transactions on Graphics (TOG), 37(6):1–14, 2018. 4
2018
-
[22]
Faragasso
J. Faragasso. Student’s Guide to Painting . Dover Publica- tions, 2020. 4
2020
-
[23]
Ic9600: a benchmark dataset for automatic image complexity assess- ment
Tinglei Feng, Yingjie Zhai, Jufeng Yang, Jie Liang, Deng- Ping Fan, Jing Zhang, Ling Shao, and Dacheng Tao. Ic9600: a benchmark dataset for automatic image complexity assess- ment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):8577–8593, 2022. 6
2022
-
[24]
Sketchycoco: Image gener- ation from freehand scene sketches
Chengying Gao, Qi Liu, Qi Xu, Limin Wang, Jianzhuang Liu, and Changqing Zou. Sketchycoco: Image gener- ation from freehand scene sketches. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5174–5183, 2020. 2, 3
2020
-
[25]
Neural image com- pression via attentional multi-scale back projection and fre- quency decomposition
Ge Gao, Pei You, Rong Pan, Shunyuan Han, Yuanyuan Zhang, Yuchao Dai, and Hojae Lee. Neural image com- pression via attentional multi-scale back projection and fre- quency decomposition. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 14677–...
2021
-
[26]
Get3d: A generative model of high quality 3d tex- tured shapes learned from images
Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. Get3d: A generative model of high quality 3d tex- tured shapes learned from images. In Advances in Neural Information Processing Systems, pages 31841–31854. Cur...
2022
-
[27]
A neural algorithm of artistic style
Leon A Gatys. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015. 3
2015 arXiv
-
[28]
Exploring the structure of a real-time, arbitrary neural artistic stylization network
Golnaz Ghiasi, Honglak Lee, Manjunath Kudlur, Vincent Dumoulin, and Jonathon Shlens. Exploring the structure of a real-time, arbitrary neural artistic stylization network. arXiv preprint arXiv:1705.06830, 2017. 3
2017 arXiv
-
[29]
Interactive sketch & fill: Multiclass sketch-to-image transla- tion
Arnab Ghosh, Richard Zhang, Puneet K Dokania, Oliver Wang, Alexei A Efros, Philip HS Torr, and Eli Shechtman. Interactive sketch & fill: Multiclass sketch-to-image transla- tion. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 1171–1180, 2019. 3
2019
-
[30]
Opens- ketch: A richly-annotated dataset of product design sketches
Yulia Gryaditskaya, Mark Sypesteyn, Jan Willem Hoftijzer, Sylvia Pont, Fr ´edo Durand, and Adrien Bousseau. Opens- ketch: A richly-annotated dataset of product design sketches. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) , 38, 2019. 6
2019
-
[31]
Sketch2mesh: Reconstructing and editing 3d shapes from sketches
Benoit Guillard, Edoardo Remelli, Pierre Yvernay, and Pas- cal Fua. Sketch2mesh: Reconstructing and editing 3d shapes from sketches. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 13023–13032,
-
[32]
Cogs: Controllable generation and search from sketch and style
Cusuh Ham, Gemma Canet Tarres, Tu Bui, James Hays, Zhe Lin, and John Collomosse. Cogs: Controllable generation and search from sketch and style. In European Conference on Computer Vision, pages 632–650. Springer, 2022. 2, 3, 6
2022
-
[33]
Diffusion- enhanced patchmatch: A framework for arbitrary style trans- fer with diffusion models
Mark Hamazaspyan and Shant Navasardyan. Diffusion- enhanced patchmatch: A framework for arbitrary style trans- fer with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 797–805, 2023. 3
2023
-
[34]
Guided image fil- tering
Kaiming He, Jian Sun, and Xiaoou Tang. Guided image fil- tering. IEEE transactions on pattern analysis and machine intelligence, 35(6):1397–1409, 2012. 4
2012
-
[35]
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 5 10
2022
-
[36]
Why do line drawings work? a realism hypothesis
Aaron Hertzmann. Why do line drawings work? a realism hypothesis. Perception, 49(4):439–451, 2020. 2, 3, 4
2020
-
[37]
Aespa-net: Aesthetic pattern-aware style transfer networks
Kibeom Hong, Seogkyu Jeon, Junsoo Lee, Namhyuk Ahn, Kunhee Kim, Pilhyeon Lee, Daesik Kim, Youngjung Uh, and Hyeran Byun. Aespa-net: Aesthetic pattern-aware style transfer networks. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 22758–22767,
-
[38]
Arbitrary style transfer in real-time with adaptive instance normalization
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceed- ings of the IEEE international conference on computer vi- sion, pages 1501–1510, 2017. 2
2017
-
[39]
Sc-fegan: Face editing gen- erative adversarial network with user’s sketch and color
Youngjoo Jo and Jongyoul Park. Sc-fegan: Face editing gen- erative adversarial network with user’s sketch and color. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1745–1753, 2019. 2, 3
2019
-
[40]
The quick, draw!-ai experi- ment
Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jong- min Kim, and Nick Fox-Gieg. The quick, draw!-ai experi- ment. Mount View, CA, accessed Feb, 17(2018):4, 2016. 6
2018
-
[41]
Giampietrino, boltraffio, and the influence of leonardo
Larry Keith and Ashok Roy. Giampietrino, boltraffio, and the influence of leonardo. National Gallery Technical Bulletin, 17:4–19, 1996. 4
1996
-
[42]
Pic- ture that sketch: Photorealistic image generation from ab- stract sketches
Subhadeep Koley, Ayan Kumar Bhunia, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, and Yi-Zhe Song. Pic- ture that sketch: Photorealistic image generation from ab- stract sketches. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 6850–6861,
-
[43]
It’s all about your sketch: Democratising sketch control in diffusion models
Subhadeep Koley, Ayan Kumar Bhunia, Deeptanshu Sekhri, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, and Yi- Zhe Song. It’s all about your sketch: Democratising sketch control in diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...
2024
-
[44]
Deeppatent: Large scale patent drawing recognition and re- trieval
Michal Kucer, Diane Oyen, Juan Castorena, and Jian Wu. Deeppatent: Large scale patent drawing recognition and re- trieval. In Proceedings of the IEEE/CVF Winter Confer- ence on Applications of Computer Vision, pages 2309–2318,
-
[45]
Photo-sketching: Inferring contour draw- ings from images
Mengtian Li, Zhe Lin, Radomir Mech, Ersin Yumer, and Deva Ramanan. Photo-sketching: Inferring contour draw- ings from images. In 2019 IEEE Winter Conference on Ap- plications of Computer Vision (WACV) , pages 1403–1412. IEEE, 2019. 4
2019
-
[46]
Gligen: Open-set grounded text-to-image generation
Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22511–22521, 2023. 2
2023
-
[47]
Visual attribute transfer through deep image analogy
Jing Liao, Yuan Yao, Lu Yuan, Gang Hua, and Sing Bing Kang. Visual attribute transfer through deep image analogy. arXiv preprint arXiv:1705.01088, 2017. 4
2017 arXiv
-
[48]
Sketchdream: Sketch-based text-to-3d generation and edit- ing
Feng-Lin Liu, Hongbo Fu, Yu-Kun Lai, and Lin Gao. Sketchdream: Sketch-based text-to-3d generation and edit- ing. ACM Transactions on Graphics (TOG) , 43(4):1–13,
-
[49]
Deflocnet: Deep im- age editing via flexible low-level controls
Hongyu Liu, Ziyu Wan, Wei Huang, Yibing Song, Xintong Han, Jing Liao, Bin Jiang, and Wei Liu. Deflocnet: Deep im- age editing via flexible low-level controls. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10765–10774, 2021. 3
2021
-
[50]
Unsupervised sketch to photo synthesis
Runtao Liu, Qian Yu, and Stella X Yu. Unsupervised sketch to photo synthesis. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 36–52. Springer, 2020. 2
2020
-
[51]
Learning re- cursive filters for low-level vision via a hybrid neural net- work
Sifei Liu, Jinshan Pan, and Ming-Hsuan Yang. Learning re- cursive filters for low-level vision via a hybrid neural net- work. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 , pages 560–576. Springer,
2016
-
[52]
Image generation from sketch constraint using con- textual gan
Yongyi Lu, Shangzhe Wu, Yu-Wing Tai, and Chi-Keung Tang. Image generation from sketch constraint using con- textual gan. In Proceedings of the European conference on computer vision (ECCV), pages 205–220, 2018. 3
2018
-
[53]
Early processing of visual information
David Marr. Early processing of visual information. Philo- sophical Transactions of the Royal Society of London. B, Bi- ological Sciences, 275(942):483–519, 1976. 2, 3
1976
-
[54]
Visual information processing: The structure and creation of visual representations
David Marr. Visual information processing: The structure and creation of visual representations. Philosophical Trans- actions of the Royal Society of London. B, Biological Sci- ences, 290(1038):199–218, 1980. 2, 3
1980
-
[55]
Sdedit: Guided image synthesis and editing with stochastic differential equa- tions
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. arXiv preprint arXiv:2108.01073, 2021. 3
2021 arXiv
-
[56]
Sked: Sketch-guided text-based 3d editing
Aryan Mikaeili, Or Perel, Mehdi Safaee, Daniel Cohen-Or, and Ali Mahdavi-Amiri. Sked: Sketch-guided text-based 3d editing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 14607–14619, 2023. 2
2023
-
[57]
T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models
Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 2024. 1, 3
2024
-
[58]
John J. Newman. Glazing and the use of casein procedure methods in a controversial art media. Design, 53(9):217– 217, 1952. 4
1952
-
[59]
Unsupervised learning of visual representations by solving jigsaw puzzles
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In Euro- pean conference on computer vision, pages 69–84. Springer,
-
[60]
H. Osborne. The Oxford Companion to Art. Clarendon Press,
-
[61]
Discourse on the practice of painting and its main processes: Underpainting, overpainting and re- touching
Jean-Baptiste Oudry. Discourse on the practice of painting and its main processes: Underpainting, overpainting and re- touching. Forelsening holdt ved det franske Acad´emie royale, 2, 1752. 4
-
[62]
Zero-shot image-to-image translation
Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceed- ings, pages 1–11, 2023. 2 11
2023
-
[63]
Dead- iff: An efficient stylization diffusion model with disentan- gled representations
Tianhao Qi, Shancheng Fang, Yanze Wu, Hongtao Xie, Ji- awei Liu, Lang Chen, Qian He, and Yongdong Zhang. Dead- iff: An efficient stylization diffusion model with disentan- gled representations. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognit...
2024
-
[64]
Sketchxai: A first look at explain- ability for human sketches
Zhiyu Qu, Yulia Gryaditskaya, Ke Li, Kaiyue Pang, Tao Xi- ang, and Yi-Zhe Song. Sketchxai: A first look at explain- ability for human sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 23327–23337, 2023. 3
2023
-
[65]
Kornia: an open source differentiable computer vision library for pytorch
Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee, and Gary Bradski. Kornia: an open source differentiable computer vision library for pytorch. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3674–3683, 2020. 6
2020
-
[66]
Encoding in style: a stylegan encoder for image-to-image translation
Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-Or. Encoding in style: a stylegan encoder for image-to-image translation. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 2287–22...
2021
-
[67]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 4
2022
-
[68]
Nonlinear total variation based noise removal algorithms
Leonid I Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena, 60(1-4):259–268, 1992. 4
1992
-
[69]
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...
2023
-
[70]
The sketchy database: learning to retrieve badly drawn bunnies
Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. The sketchy database: learning to retrieve badly drawn bunnies. ACM Transactions on Graphics (TOG), 35(4):1–12,
-
[71]
Grad-cam: Visual explanations from deep networks via gradient-based localization
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE in- ternational conference on computer vision , pages 618–626,
-
[72]
Alchemist: Parametric control of material proper- ties with diffusion models
Prafull Sharma, Varun Jampani, Yuanzhen Li, Xuhui Jia, Dmitry Lagun, Fredo Durand, Bill Freeman, and Mark Matthews. Alchemist: Parametric control of material proper- ties with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...
2024
-
[73]
Learning to sketch with shortcut cycle consistency
Jifei Song, Kaiyue Pang, Yi-Zhe Song, Tao Xiang, and Tim- othy M Hospedales. Learning to sketch with shortcut cycle consistency. In Proceedings of the IEEE conference on com- puter vision and pattern recognition , pages 801–810, 2018. 6
2018
-
[74]
Bilateral filtering for gray and color images
Carlo Tomasi and Roberto Manduchi. Bilateral filtering for gray and color images. In Sixth international conference on computer vision (IEEE Cat. No. 98CH36271) , pages 839–
-
[75]
Sketch-guided object localization in nat- ural images
Aditay Tripathi, Rajath R Dani, Anand Mishra, and Anir- ban Chakraborty. Sketch-guided object localization in nat- ural images. In Computer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Pro- ceedings, Part VI 16, pages 532–547. Springer, 2020. 3
2020
-
[76]
Texture networks: Feed-forward syn- thesis of textures and stylized images
Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor Lempitsky. Texture networks: Feed-forward syn- thesis of textures and stylized images. arXiv preprint arXiv:1603.03417, 2016. 3
2016 arXiv
-
[77]
Clipasso: Semantically-aware object sketching
Yael Vinker, Ehsan Pajouheshgar, Jessica Y Bo, Ro- man Christian Bachmann, Amit Haim Bermano, Daniel Cohen-Or, Amir Zamir, and Ariel Shamir. Clipasso: Semantically-aware object sketching. ACM Transactions on Graphics (TOG), 41(4):1–11, 2022. 3
2022
-
[78]
Clipascene: Scene sketching with different types and levels of abstraction
Yael Vinker, Yuval Alaluf, Daniel Cohen-Or, and Ariel Shamir. Clipascene: Scene sketching with different types and levels of abstraction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4146– 4156, 2023. 3
2023
-
[79]
Sketch-guided text-to-image diffusion models
Andrey V oynov, Kfir Aberman, and Daniel Cohen-Or. Sketch-guided text-to-image diffusion models. In ACM SIG- GRAPH 2023 Conference Proceedings, pages 1–11, 2023. 2, 3
2023
-
[80]
Learning robust global representations by penalizing local predictive power.Advances in Neural Information Pro- cessing Systems, 32, 2019
Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power.Advances in Neural Information Pro- cessing Systems, 32, 2019. 6
2019
-
[81]
Instantstyle: Free lunch towards style- preserving in text-to-image generation
Haofan Wang, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen. Instantstyle: Free lunch towards style- preserving in text-to-image generation. arXiv preprint arXiv:2404.02733, 2024. 6
2024 arXiv
-
[82]
Unsupervised scene sketch to photo synthesis
Jiayun Wang, Sangryul Jeon, Stella X Yu, Xi Zhang, Hi- manshu Arora, and Yu Lou. Unsupervised scene sketch to photo synthesis. In European Conference on Computer Vi- sion, pages 273–289. Springer, 2022. 2
2022
-
[83]
Sketch your own gan
Sheng-Yu Wang, David Bau, and Jun-Yan Zhu. Sketch your own gan. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14050–14060, 2021. 2, 3
2021
-
[84]
Pretraining is all you need for image-to-image translation
Tengfei Wang, Ting Zhang, Bo Zhang, Hao Ouyang, Dong Chen, Qifeng Chen, and Fang Wen. Pretraining is all you need for image-to-image translation. arXiv preprint arXiv:2205.12952, 2022. 3
2022 arXiv
-
[85]
Stylediffusion: Controllable disentangled style transfer via diffusion models
Zhizhong Wang, Lei Zhao, and Wei Xing. Stylediffusion: Controllable disentangled style transfer via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7677–7689, 2023. 2, 3
2023
-
[86]
Fine-grained image analysis with deep learning: A survey
Xiu-Shen Wei, Yi-Zhe Song, Oisin Mac Aodha, Jianxin Wu, Yuxin Peng, Jinhui Tang, Jian Yang, and Serge Belongie. Fine-grained image analysis with deep learning: A survey. IEEE transactions on pattern analysis and machine intelli- gence, 44(12):8927–8948, 2021. 2
2021
-
[87]
Style- former: Real-time arbitrary style transfer via parametric style composition
Xiaolei Wu, Zhihao Hu, Lu Sheng, and Dong Xu. Style- former: Real-time arbitrary style transfer via parametric style composition. In Proceedings of the IEEE/CVF Interna- 12 tional Conference on Computer Vision, pages 14618–14627,
-
[88]
Sketch-based model- ing with a differentiable renderer
Nan Xiang, Ruibin Wang, Tao Jiang, Li Wang, Yanran Li, Xiaosong Yang, and Jianjun Zhang. Sketch-based model- ing with a differentiable renderer. Computer Animation and Virtual Worlds, 31(4-5):e1939, 2020. 2
2020
-
[89]
Adversarial open domain adap- tation for sketch-to-photo synthesis
Xiaoyu Xiang, Ding Liu, Xiao Yang, Yiheng Zhu, Xiaohui Shen, and Jan P Allebach. Adversarial open domain adap- tation for sketch-to-photo synthesis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1434–1444, 2022. 2
2022
-
[90]
Differsketching: How differently do people sketch 3d objects?ACM Transactions on Graphics (TOG), 41(6):1–16, 2022
Chufeng Xiao, Wanchao Su, Jing Liao, Zhouhui Lian, Yi- Zhe Song, and Hongbo Fu. Differsketching: How differently do people sketch 3d objects?ACM Transactions on Graphics (TOG), 41(6):1–16, 2022. 6
2022
-
[91]
Depth anything: Unleashing the power of large-scale unlabeled data
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 5
2024
-
[92]
Controllable sketch-to-image translation for robust face synthesis
Shuai Yang, Zhangyang Wang, Jiaying Liu, and Zongming Guo. Controllable sketch-to-image translation for robust face synthesis. IEEE Transactions on Image Processing, 30: 8797–8810, 2021. 2
2021
-
[93]
Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,
-
[94]
Texture- dreamer: Image-guided texture synthesis through geometry- aware diffusion
Yu-Ying Yeh, Jia-Bin Huang, Changil Kim, Lei Xiao, Thu Nguyen-Phuoc, Numair Khan, Cheng Zhang, Manmohan Chandraker, Carl S Marshall, Zhao Dong, et al. Texture- dreamer: Image-guided texture synthesis through geometry- aware diffusion. In Proceedings of the IEEE/CVF Confer- enc...
2024
-
[95]
Adding conditional control to text-to-image diffusion models
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2, 3, 4, 8
2023
-
[96]
The unreasonable effectiveness of deep features as a perceptual metric
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 7
2018
-
[97]
Prospect: Prompt spectrum for attribute-aware personalization of diffusion models.ACM Transactions on Graphics (TOG), 42(6):1–14, 2023
Yuxin Zhang, Weiming Dong, Fan Tang, Nisha Huang, Haibin Huang, Chongyang Ma, Tong-Yee Lee, Oliver Deussen, and Changsheng Xu. Prospect: Prompt spectrum for attribute-aware personalization of diffusion models.ACM Transactions on Graphics (TOG), 42(6):1–14, 2023. 2
2023
-
[98]
Inversion-based style transfer with diffusion models
Yuxin Zhang, Nisha Huang, Fan Tang, Haibin Huang, Chongyang Ma, Weiming Dong, and Changsheng Xu. Inversion-based style transfer with diffusion models. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10146–10156, 2023. 3 13
2023
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.