REVIEW 3 major objections 6 minor 48 references
This paper claims that artists can learn to predict a text-to-image diffusion model's visual behavior by bending its internal layers, turning a black box into a creative material.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 04:44 UTC pith:AFIY3IO6
load-bearing objection A genuinely useful toolkit and an honest preliminary mapping, but the abstract's 'consistent families' claim outruns the statistics; worth refereeing with revisions. the 3 major comments →
Unboxing Diffusion Models for the Arts: Interactive Model Bending and Practice-Based Explainability
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is that intervening on specific components of the SD1.5 UNet produces consistent families of visual effects, enabling practice-based explainability. Quantitatively, the input region of the UNet has a stronger impact on latent divergence than the output region (mean cosine distance 0.1776 vs 0.1088), Residual Blocks are more impactful than Cross-Attention layers (0.1333 vs 0.1070), and normalization layers are as impactful as convolutional layers. Layer type exerts a stronger influence than spatial location, contrasting with StyleGAN findings, and bending during early denoising steps predominantly shapes global structure while later steps yield fine-grained changes. Thes
What carries the argument
The key mechanism is the bending operator: a training-free intervention that hooks into a specified layer's output during inference and transforms the tensor using operations like scalar multiplication, ablation, rotation, or noise. Layer paths are specified via the UNet's hierarchical structure (e.g., 'diffusion model.middle block.0.in layers'), and the system copies the model definition so bends affect only downstream parts of the workflow. The interactive interface adds an SVG layer explorer and sliders, letting artists select and bend layers without scripting.
Load-bearing premise
The hand-picked layers in the study are representative of the UNet regions and module types they illustrate, so the reported hierarchy reflects architectural properties rather than a biased selection of layers.
What would settle it
Run a systematic ablation sweep across all UNet layers (not a hand-picked subset) with multiple seeds and a fixed prompt, then group by layer type and location. If the variance in latent distance within a layer type exceeds the variance between types, or if input-block and output-block means overlap substantially, the claimed hierarchy collapses.
If this is right
- Artists can build model-specific cheat sheets mapping layers, magnitudes, and timestep ranges to visual outcomes, making bending reproducible and shareable.
- Bending effects that are consistent across seeds and prompts allow artists to achieve intended stylistic shifts without prompting tricks.
- The finding that early timesteps dominate global structure lets artists control coarse vs fine changes by limiting the denoising step range.
- The tools lower the barrier for non-technical artists to engage with model internals, supporting AI literacy and critical engagement.
- The approach can extend beyond the UNet to other pipeline components, such as VAE, CLIP embeddings, and LoRA matrices, broadening creative control.
Where Pith is reading between the lines
- The layer-type-over-location hierarchy may reflect a general property of iterative denoising, suggesting similar patterns could emerge in newer diffusion transformers, though this remains untested.
- A systematic sweep across all layers of each type, rather than hand-picked examples, could confirm whether the quantitative hierarchy holds or is an artifact of selection.
- The consistency of bending effects raises the possibility of using bending as a probe for model robustness or safety guardrails, as the ethics statement itself hints.
- Perceptual metrics like LPIPS or DINOv2, which the paper suggests but does not run, might reveal whether latent cosine distance aligns with human-perceived visual impact.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper argues that large text-to-image diffusion models can be treated as creative materials when their internal structure is exposed and manipulable, and that practice-based explainability through 'model bending' can serve artists. The authors contribute a ComfyUI plugin and an interactive web interface for inspecting Stable Diffusion 1.5's UNet and applying training-free interventions at selected layers and timesteps. They report a systematic study that enumerates bending locations, seeds, prompts, and timestep ranges, using qualitative image grids and latent-space cosine distance from the unbent baseline. The paper's central claim is that bending specific components yields relatively consistent families of visual effects, enabling artists to develop layer-level heuristics for what to bend, when, and by how much.
Significance. If the central claim holds, the paper offers a concrete path toward 'doing-based' XAI for the arts and a practical toolkit for artists. The system contribution is genuine: a reproducible, open-source implementation integrated into a widely used workflow, with interactive layer selection and caching of experimental results. The study design is transparent, and the authors' limitation section candidly acknowledges the lack of perceptual and human evaluation. The main weakness is that the consistency claim—the core of the paper—is not supported by the quantitative analysis as currently presented. Because that claim is testable with existing data, the manuscript is promising but needs additional evidence.
major comments (3)
- [Section 4.4 / Abstract] The paper's central claim—that bending specific components produces 'relatively consistent families of visual effects'—is not directly tested. The only quantitative evidence in Section 4.3 is cosine distance from the unbent latent, which measures the magnitude of the intervention, not whether the same layer produces a consistent visual character across seeds/prompts or whether different layers produce distinguishable effects. The qualitative grids (Figs. 8–17) are suggestive but are not a substitute. I ask the authors to add a pairwise similarity analysis: at matched bending magnitude, compare same-layer outputs across seeds/prompts against different-layer outputs, using a perceptual or latent metric (e.g., LPIPS, cosine distance), with a permutation or bootstrap test. This would substantiate or refine the hierarchy claimed in Section 4.4.
- [Section 4.3] The text states 'significant disparities in impact' and 'a clear hierarchy where Input>Output and ResBlocks>Cross-Attention,' but no inferential statistics are reported. The descriptive means and standard deviations overlap widely (e.g., LayerNorm 0.1688±0.1562 vs Conv2d 0.1685±0.1658; ResBlocks 0.1333±0.1541 vs Cross-Attention 0.1070±0.1110). Please provide confidence intervals, effect sizes, and tests (permutation or bootstrap) for the stated differences, or revise the language to 'observed' rather than 'significant.'
- [Section 4.2.2] The cross-seed and cross-prompt experiments use 'select layers, hand-picked to span input, middle and output blocks' (five layers). No criterion for representativeness is given, and the quantitative hierarchy in Section 4.3 is computed over these and other selected layers. This selection could bias the conclusions about layer type and location. Please justify the selection or sample layers systematically; alternatively, explicitly restrict the generalization claims to the selected layers.
minor comments (6)
- [Page 9] Typo: 'the scripts used to generate the results in the in Section 4' should read 'in Section 4.'
- [Section 4.2.3] The opening sentence says 'Using the same prompt' but the experiment varies prompts; this should say 'Using the same seed.'
- [Section 3.3] The phrase 'We address goals' is missing 'these'; consider reformulating.
- [Section 4.2.2/Figures 8–12] Layer path labels are inconsistent (e.g., 'time embed.2' vs 'time embed.2' in text; 'input blocks.1.0 .in layers.0' with spaces). Please standardize the notation.
- [Section 2.2] T5 is described as a contrastive text encoder; T5 is not trained contrastively. Please clarify or correct.
- [Figure 1] Color coding ('gold' vs 'purple') may be difficult for color-blind readers; add labels or patterns.
Circularity Check
No circularity: the 'relatively consistent families' claim is an empirical summary of generated outputs, not a consequence of fitted inputs or self-cited assumptions.
full rationale
The paper's central claim is that bending specific components of SD1.5 produces relatively consistent visual families. This is an empirical summary of generated grids (Figs. 6–17) and a post-hoc cosine-distance metric, not a result derived from equations that already contain the conclusion. No parameters are fitted and then re-predicted; the bending operators, layer paths, and baseline latents are all inputs, and the distance-to-baseline is an independent measurement. The quantitative analysis only summarizes distances by architectural grouping, so the hierarchy (Input>Output, ResBlocks>Cross-Attention) is descriptive rather than circular. The self-citations ([9], [13], [14]) support background framing and prior tools, but are not used as a uniqueness theorem or as the source of the empirical hierarchy. The paper itself restricts its scope: 'our intent is not to establish findings that generalize across different UNet architectures' (Sec. 5.1) and 'observations reported here should not be assumed to generalize' (Sec. 6). The main concerns—hand-picked layers and the absence of a statistical test of cross-seed consistency—are validity/selection issues, not circularity. Score 1 reflects only the presence of minor, non-load-bearing self-citations.
Axiom & Free-Parameter Ledger
axioms (3)
- domain assumption Cosine distance between final latent representations is a meaningful proxy for the visual impact of a bending intervention.
- ad hoc to paper The hand-picked layers in the cross-seed and cross-prompt experiments are representative of the UNet regions and module types they are chosen to span.
- domain assumption Qualitative visual comparison of image grids reliably detects consistent families of visual effects.
read the original abstract
Explainable AI (XAI) in creative practice can be less about technocentric explanation and more about enabling artists to inspect modify and debug models as part of making Yet largescale texttoimage diffusion systems are typically presented as opaque endtoend tools limiting this kind of material engagement We argue that even large models can function as creative materials when their internal structure is made visible and manipulable To support this we propose a handson approach to explainability centred on experimentation and intervention We instantiate this approach with a model bending and an interactive (inspection) interface integrated into ComfyUIs nodebased workflow including interactive layer selection and intervention controls Through qualitative and quantitative analysis of bending interventions in Stable Diffusion 15 we show how manipulating specific components of a diffusion pipeline produces relatively consistent families of visual effects allowing artists to build practical layerlevel intuition about how different parts of the model shape generated images
Figures
Reference graph
Works this paper leans on
-
[1]
XAI—Explainable artificial intelligence
David Gunning et al. “XAI—Explainable artificial intelligence”. In:Science robotics4.37 (2019)
2019
-
[2]
Exploring XAI for the Arts: Explaining Latent Space in Generative Music
Nick Bryan-Kinns et al. “Exploring XAI for the Arts: Explaining Latent Space in Generative Music”. In:1st Workshop on eXplainable AI approaches for debugging and diagnosis. 2023
2023
-
[3]
Explainability Paths for Sustained Artistic Practice with AI
Austin Tecks, Thomas Peschlow, and Gabriel Vigliensoni. “Explainability Paths for Sustained Artistic Practice with AI”. In:In Proceedings of Ex- plainable AI for the Arts Workshop 2024 (XAIxArts 2024). arXiv, July 21, 2024.doi:10.48550/arXiv.2407.15216. arXiv:2407.15216 [cs].url: http://arxiv.org/abs/2407.15216(visited on 04/08/2025). 28 Ahmed M. Abuzurai...
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2407.15216 2024
-
[4]
XAIxArts Manifesto: Explainable AI for the Arts
Nick Bryan-Kinns et al. “XAIxArts Manifesto: Explainable AI for the Arts”. In:Extended Abstracts of the CHI Conference on Human Factors in Com- puting Systems. CHI. Yokohama, Japan, 2025.doi:10 . 1145 / 3706599 . 3716227. arXiv:2502.21220 [cs].url:http://arxiv.org/abs/2502. 21220(visited on 04/13/2025)
Pith/arXiv arXiv 2025
-
[5]
A Small-Data Mindset for Generative AI Creative Work
Gabriel Vigliensoni, Phoenix Perry, and Rebecca Fiebrink. “A Small-Data Mindset for Generative AI Creative Work”. In:In Generative AI in HCI Workshop, CHI ’22. New York, NY, USA, 2022, p. 5
2022
-
[6]
”Explorers of Unknown Planets
Baptiste Caramiaux and Sarah Fdili Alaoui. “”Explorers of Unknown Planets” Practices and Politics of Artificial Intelligence in Visual Arts”. In:Proceedings of the ACM on Human-Computer Interaction6.CSCW2 (2022), pp. 1–24
2022
-
[7]
Using Generative AI as an Artistic Material: A Hacker’s Guide
Terence Broad. “Using Generative AI as an Artistic Material: A Hacker’s Guide”. In:In Proceedings of Explainable AI for the Arts Workshop 2024. 2024
2024
-
[8]
How Artists Use AI as a Responsive Material for Art Creation
Sonja Rozental, Michel Van Dartel, and Alwin De Rooij. “How Artists Use AI as a Responsive Material for Art Creation”. In: ISEA 2025: 30th International Symposium on Electronic/Emerging Art. Korea, 2025.doi:10.31234/osf. io/gjdnw.url:https://osf.io/gjdnw(visited on 06/10/2025)
doi:10.31234/osf 2025
-
[9]
Seizing the Means of Produc- tion: Exploring the Landscape of Crafting, Adapting and Navigating Gener- ative AI Models
Ahmed M. Abuzuraiq and Philippe Pasquier. “Seizing the Means of Produc- tion: Exploring the Landscape of Crafting, Adapting and Navigating Gener- ative AI Models”. In:the 3rd Generative AI and HCI Workshop. CHI ’24. Hawaii, USA, 2024
2024
-
[10]
Network Bend- ing: Expressive Manipulation of Deep Generative Models
Terence Broad, Frederic Fol Leymarie, and Mick Grierson. “Network Bend- ing: Expressive Manipulation of Deep Generative Models”. In:Artificial In- telligence in Music, Sound, Art and Design: 10th International Conference, EvoMUSART 2021, Held as Part of EvoStar 2021, Virtual Event, April 7–9, 2021, Proceedings 10. Springer, 2021
2021
-
[11]
com / comfyanonymous / ComfyUI
comfyanonymous.https : / / github . com / comfyanonymous / ComfyUI
-
[12]
John Wiley & Sons, 2005
Reed Ghazala.Circuit-Bending: Build Your Own Alien Instruments. John Wiley & Sons, 2005
2005
-
[13]
Autolume-Live: Turning GANs into a Live VJing Tool
Jonas Kraasch and Philippe Pasquier. “Autolume-Live: Turning GANs into a Live VJing Tool”. In:Proceedings of the 10th Conference on Computation, Communication, Aesthetics & X. Coimbra, Portugal, 2022, pp. 152–169
2022
-
[14]
https : / / www
Metacreation Lab.Autolume: A Neural-network Based Visual Synthesizer. https : / / www . metacreation . net / autolume. 2024. (Visited on 02/13/2024)
2024
-
[15]
Controlling the Image Generation Process with Parametric Acti- vation Functions
Ilia Pavlov. “Controlling the Image Generation Process with Parametric Acti- vation Functions”. In:International Conference on Computational Creativity ICCC25. Brazil, 2025
2025
-
[16]
Network Bend- ing of Diffusion Models for Audio-Visual Generation
Luke Dzwonczyk, Carmine Emanuele Cella, and David Ban. “Network Bend- ing of Diffusion Models for Audio-Visual Generation”. In: 27th International Conference on Digital Audio Effects (DAFx24). arXiv, 2024-06-28.doi: 10.48550/arXiv.2406.19589. arXiv:2406.19589 [cs].url:http: //arxiv.org/abs/2406.19589(visited on 02/20/2025). Unboxing Diffusion Models for t...
-
[17]
2025.url: https://garincurtis.com/projects/beyond-prompts
Garin Curtis.Beyond Prompts: Developing An Expressive Tool for Real-Time Manipulation of Diffusion Models through Active Divergence. 2025.url: https://garincurtis.com/projects/beyond-prompts
2025
-
[18]
Patch Explorer: Interpreting Diffusion Models through Interaction
Imke Grabe et al. “Patch Explorer: Interpreting Diffusion Models through Interaction”. In:Mechanistic Interpretability for Vision at CVPR 2025 (Non- proceedings Track). Mar. 31, 2025.url:https : / / openreview . net / forum?id=0n9wqVyHas(visited on 07/02/2025)
2025
-
[19]
Mingi Kwon, Jaeseok Jeong, and Youngjung Uh.Diffusion Models Already Have a Semantic Latent Space. Mar. 29, 2023.doi:10.48550/arXiv.2210. 10960. arXiv:2210.10960 [cs].url:http://arxiv.org/abs/2210. 10960(visited on 05/07/2025). Pre-published
-
[20]
Unstable Audio: Code Bending Text-to-Music Generation
Nick Collins. “Unstable Audio: Code Bending Text-to-Music Generation”. In:Proceedings of the AES International Conference on Machine Learn- ing and Artificial Intelligence for Audio. 2025.url:https : / / durham - repository.worktribe.com/output/4280622(visited on 11/20/2025)
arXiv 2025
-
[21]
Studio Report: Sound Synthe- sis with DDSP and Network Bending Techniques
Matthew Yee-King and Louis McCallum. “Studio Report: Sound Synthe- sis with DDSP and Network Bending Techniques”. In:Proceedings of 2nd Conference on AI Music Creativity. AIMC. 2021
2021
-
[22]
Jaden Fiotto-Kaufman et al.NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals. Apr. 1, 2025.doi:10 . 48550 / arXiv.2407.14561. arXiv:2407.14561 [cs].url:http://arxiv.org/ abs/2407.14561(visited on 02/12/2026). Pre-published
-
[23]
Pyvene: A Library for Understanding and Improving PyTorch Models via Interventions
Zhengxuan Wu et al. “Pyvene: A Library for Understanding and Improving PyTorch Models via Interventions”. In:Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies (Volume 3: System Demonstrations). Ed. by Kai-Wei Chang, Annie Lee, and Nazneen Rajani. Mexico City, Me...
-
[24]
2022.url:https://github.com/davidbau/baukit
David Bau.Baukit. 2022.url:https://github.com/davidbau/baukit
2022
-
[25]
Hacking Generative Models with Differentiable Network Bending
Giacomo Aldegheri et al. “Hacking Generative Models with Differentiable Network Bending”. 2023. arXiv:2310 . 04816.url:https : / / github . com/GAldegheri/gan-bending
2023
-
[26]
Autolume: A GAN-based No-Coding Small Data and Model Crafting Visual Synthesizer for Artistic Creation
Arshia Sobhan et al. “Autolume: A GAN-based No-Coding Small Data and Model Crafting Visual Synthesizer for Artistic Creation”. In:Proceedings of the Conference on Animation and Interactive Art. Expanded ’25. New York, NY, USA: Association for Computing Machinery, Nov. 5, 2025, pp. 165– 172.isbn: 979-8-4007-1532-7.doi:10 . 1145 / 3749893 . 3749968.url: htt...
2025
-
[27]
Hidden Layer Interaction: A Technique to Explore the Material of Generative AI
Imke Grabe and Tom Jenkins. “Hidden Layer Interaction: A Technique to Explore the Material of Generative AI”. In:Proceedings of the 2025 ACM Designing Interactive Systems Conference. DIS ’25. New York, NY, USA: Association for Computing Machinery, July 4, 2025, pp. 1913–1927.isbn: 30 Ahmed M. Abuzuraiq and Philippe Pasquier 979-8-4007-1485-6.doi:10.1145/3...
arXiv 2025
-
[28]
High-Resolution Image Synthesis With Latent Dif- fusion Models
Robin Rombach et al. “High-Resolution Image Synthesis With Latent Dif- fusion Models”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, pp. 10684–10695.url:https : / / openaccess.thecvf.com/content/CVPR2022/html/Rombach_High- Resolution_Image_ Synthesis_With_Latent_Diffusion _Models_ CVPR_2022_paper(visited on 0...
2022
-
[29]
Automatic1111 WebUI.https://github.com/AUTOMATIC1111/stable- diffusion-webui. 2024. (Visited on 02/13/2024)
2024
-
[30]
Streamdiffusion: A pipeline-level solution for real-time interactive generation
Akio Kodaira et al. “Streamdiffusion: A pipeline-level solution for real-time interactive generation”. In:Proceedings of the IEEE/CVF International Con- ference on Computer Vision. 2025, pp. 12371–12380
2025
-
[31]
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang et al. “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric”. In:CVPR. 2018
2018
-
[32]
Dinov2: Learning robust visual features without super- vision
Maxime Oquab et al. “Dinov2: Learning robust visual features without super- vision”. In:arXiv preprint arXiv:2304.07193(2023)
Pith/arXiv arXiv 2023
-
[33]
Learning transferable visual models from natural lan- guage supervision
Alec Radford et al. “Learning transferable visual models from natural lan- guage supervision”. In:International conference on machine learning. PmLR. 2021, pp. 8748–8763
2021
-
[34]
Visual Parameter Space Exploration for AI Art Design
Julian Hummel. “Visual Parameter Space Exploration for AI Art Design”. B.S. thesis. 2023
2023
-
[35]
Visual Parameter Space Analysis: A Conceptual Framework
Michael Sedlmair et al. “Visual Parameter Space Analysis: A Conceptual Framework”. In:IEEE Transactions on Visualization and Computer Graphics 20.12 (Dec. 2014), pp. 2161–2170.issn: 1077-2626.doi:10.1109/TVCG. 2014.2346321
arXiv 2014
-
[36]
Richard Lee Davis et al. “Fashioning Creative Expertise with Generative AI: Graphical Interfaces for Design Space Exploration Better Support Ideation than Text Prompts”. In:Proceedings of the 2024 CHI Conference on Hu- man Factors in Computing Systems. Chi ’24. New York, NY, USA: As- sociation for Computing Machinery, 2024.isbn: 979-8-4007-0330-0.doi: 10 ...
arXiv 2024
-
[37]
Generative Rotoscoping: A First-Person Autobiographical Ex- ploration on Generative Video-to-Video Practices
David Ledo. “Generative Rotoscoping: A First-Person Autobiographical Ex- ploration on Generative Video-to-Video Practices”. In:Proceedings of the 2025 Conference on Creativity and Cognition. C&C ’25. New York, NY, USA: Association for Computing Machinery, June 22, 2025, pp. 931– 948.isbn: 979-8-4007-1289-0.doi:10 . 1145 / 3698061 . 3726926.url: https ...
2025
-
[40]
Lora: Low-rank Adaptation of Large Language Models
Edward J Hu et al. “Lora: Low-rank Adaptation of Large Language Models”
-
[41]
StabilityAI.Stable Diffusion.https://stability.ai/stable- image
-
[42]
(Visited on 08/30/2024)
2024
-
[43]
On the commoditization of artificial intelligence
Abdullah A Abonamah, Muhammad Usman Tariq, and Samar Shilbayeh. “On the commoditization of artificial intelligence”. In:Frontiers in psychology12 (2021)
2021
-
[44]
A Taxonomy of the Biases of the Images created by Generative Artificial Intelligence
Adriana Fern ´andez de Caleya V ´azquez and Eduardo C Garrido-Merch ´an. “A Taxonomy of the Biases of the Images created by Generative Artificial Intelligence”. In:Current Trends in Business Management 3(2024)
2024
-
[45]
The Impact of Generative AI on Artists
Reishiro Kawakami and Sukrit Venkatagiri. “The Impact of Generative AI on Artists”. In:Proceedings of the 16th Conference on Creativity & Cognition. 2024, pp. 79–82
2024
-
[46]
Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions
Aditya Bhattacharya. “Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions”. In:Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 2024, pp. 1–6
2024
-
[47]
Active Divergence with Generative Deep Learning–A Survey and Taxonomy
Terence Broad et al. “Active Divergence with Generative Deep Learning–A Survey and Taxonomy”. In:Proceedings of the 12th International Conference on Computational Creativity (ICCC ’21). 2021.isbn: 978-989-54-1603-5. doi:arXivpreprintarXiv:2107.05599
Pith/arXiv arXiv 2021
-
[48]
AI in the public eye: Investigating public AI literacy through AI art
Drew Hemment et al. “AI in the public eye: Investigating public AI literacy through AI art”. In:Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 2023, pp. 931–942
2023
-
[49]
Long Ouyang et al.Training Language Models to Follow Instructions with Human Feedback. Mar. 4, 2022.doi:10.48550/arXiv.2203.02155. arXiv: 2203.02155 [cs].url:http://arxiv.org/abs/2203.02155(visited on 11/28/2025). Pre-published
-
[2024]
(Visited on 02/13/2024)
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.