Real-Time AttentionBender: Granular Interactive Network Bending of Video Diffusion Transformers
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-04 17:21 UTCglm-5.2pith:5IQ7H73Yrecord.jsonopen to challenge →
The pith
Bending every layer of a video transformer in real time
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the full depth of a video diffusion transformer — self-attention, cross-attention, and feed-forward layers, down to individual neurons — can be exposed as real-time interactive control surfaces without breaking the generative loop, and that this exposure simultaneously enables novel aesthetic discovery and mechanistic understanding of how transformer internals map to visual outputs. The mechanism carrying this claim is a monkeypatching architecture that injects parameter-fetching listeners into WanSelfAttention, WanT2VCrossAttention, and the Feed-Forward Sequential module, allowing user-defined modulations to alter network behavior at inference time without halting.
What carries the argument
The load-bearing machinery is a set of runtime patches injected into three Wan DiT classes — WanSelfAttention, WanT2VCrossAttention, and the Feed-Forward Sequential module. These patches install global listeners that fetch user-defined modulation parameters during inference, enabling amplitude scaling, noise injection, geometric reshaping of attention maps, and hidden-neuron gain/threshold/noise — all targetable by diffusion step range, DiT layer range, prompt token, or neuron index band. The system wraps real-time Wan-based pipelines (LongLive, Krea) and operates within the DayDream Scope plugin ecosystem.
If this is right
- If real-time network bending is practical at full DiT depth, then the boundary between using a generative model and inspecting it dissolves: every creative manipulation becomes simultaneously an interpretive probe into what that layer or neuron does.
- Granular neuron-level targeting in the feed-forward layers could, if systematically explored, produce functional maps of hidden-neuron bands to visual outcomes — a form of mechanistic interpretability driven by artistic exploration rather than automated probing.
- The KV-cache resistance problem the authors identify suggests a fundamental tension in real-time bending: models optimized for temporal coherence will fight structural interventions, meaning the most responsive manipulation targets may be those that operate before cache state dominates.
Where Pith is reading between the lines
- The claim of ~15 FPS is stated without latency variance, frame-time distribution, or performance-under-load measurements; if multiple simultaneous modulations across layers and neurons cause frame-time spikes, the material-intimacy value proposition could degrade in practice even if average throughput holds.
- The paper's examples focus on single-layer-type or single-neuron-band interventions; the combinatorial space of simultaneous multi-layer, multi-step, multi-token modulations is largely uncharted, and its expressive range — or its tendency to collapse into incoherence — remains an open empirical question.
- If the monkeypatching approach generalizes beyond Wan to other DiT architectures (Sora-class, Hunyuan, etc.), it could become a standard layer for creative-AI tooling, but the paper does not test cross-architecture portability.
Load-bearing premise
The paper assumes that monkeypatching three DiT module classes to fetch and apply user parameters at every inference step can sustain interactive frame rates without destabilizing the real-time diffusion loop. The entire value proposition of material intimacy depends on the feedback loop between manipulation and visual response remaining tight, but the paper provides no latency measurements, no frame-time variance, and no evidence of how performance degrades as multiple modul
What would settle it
If the system cannot sustain interactive frame rates (roughly 10+ FPS) when multiple simultaneous modulations are applied across different layers and neurons — causing noticeable lag between slider movement and visual response — then the material intimacy claim collapses, because the felt understanding depends on immediacy. Alternatively, if targeted neuron-level or layer-level interventions produce visually indistinguishable results from global modifications, the granular targeting architecture adds complexity without explanatory or expressive value.
Figures
read the original abstract
Generative video models have achieved remarkable visual fidelity, yet their prompt-only interface offers thin creative agency and obscures the model's material process from the artists working with it. We present Real-Time AttentionBender, a tool that extends the practice of network bending across the full depth of the video diffusion transformer (DiT) and brings it into live, interactive generation. Built as a plugin within the DayDream Scope ecosystem and wrapping open-source real-time Wan pipelines, the tool exposes self-attention, cross-attention, and the feed-forward network as independently manipulable surfaces, with targeting down to individual diffusion steps, DiT layers, prompt tokens, and hidden neurons. The immediacy of live manipulation affords what we call "material intimacy" with the model: a responsive, near-mechanistic feel for how specific layers and neurons shape generated video. We position the tool as simultaneously an XAIxArts probe into transformer internals and an expressive instrument for discovering aesthetics outside the model's default representational space.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Real-Time AttentionBender, a tool that extends network bending to the full depth of video diffusion transformers (DiTs) — exposing self-attention, cross-attention, and feed-forward network layers as independently manipulable surfaces with targeting down to individual diffusion steps, DiT layers, prompt tokens, and hidden neurons. The system is implemented as a plugin within the DayDream Scope ecosystem, wrapping open-source real-time Wan pipelines (LongLive, Krea) via monkeypatching of WanSelfAttention, WanT2VCrossAttention, and the FFN module. The authors claim ~15 FPS on a single RTX A6000 Pro GPU at 320×576 resolution. The paper positions the tool as both an XAIxArts probe and a creative instrument, introducing the concept of 'material intimacy' — a responsive feedback loop between parameter changes and visual output.
Significance. The paper addresses a genuine gap: while network bending has been applied to GANs and U-Nets, and the authors' prior AttentionBender explored offline cross-attention manipulation in DiTs, no existing framework provides live, full-depth DiT manipulation. The granular targeting scheme (composing step, layer, token, and neuron dimensions) is a meaningful design contribution. The system wraps externally validated open-source pipelines (Wan, LongLive, Krea), which is a strength for reproducibility. The concept of 'material intimacy' as an XAIxArts outcome is interesting but currently under-supported by evidence.
major comments (4)
- §3.1: The performance claim of '~15 frames per second on a single NVIDIA RTX A6000 Pro GPU' is load-bearing for the paper's central claim of real-time interaction, yet no latency measurements, frame-time variance, or performance degradation data under multiple simultaneous modulations are provided. At minimum, a table or plot showing frame rate as a function of (a) number of active modulations, (b) targeting granularity (global vs. per-neuron), and (c) cache state (enabled/disabled) is needed to substantiate 'real-time' and 'responsive.' Without this, the reader cannot assess whether the system actually delivers interactive frame rates under the usage conditions described in §3.2–3.3.
- §4.3 and §4.2: The KV cache limitation creates a structural tension with the core 'material intimacy' claim. The paper acknowledges that 'if a bending parameter is altered mid-generation, the cached visual history can mute or override the new modulation, reducing responsiveness' (§4.3). Yet §4.2 claims that 'the ability to instantly see the impact of shifting a slider' produces 'material intimacy.' If mid-generation parameter changes — the most natural interactive use case — are suppressed by caching, the feedback loop that constitutes 'material intimacy' may be broken precisely when it matters most. The mitigations (manual reset/disable cache) either break temporal coherence or eliminate the computational advantage enabling real-time performance. The paper should explicitly characterize under what conditions the feedback loop is actually responsive (e.g., parameter changes between frame
- §4.1, §4.2: The claims about 'expressive potential,' 'material intimacy,' and the system functioning as an 'XAIxArts probe' are supported only by the authors' own observations and figures. The paper acknowledges in §4.3 that 'rigorous evaluation remains' and that user studies are planned, but the current manuscript contains no evidence from external users, no structured self-report, and no comparison to the prior offline AttentionBender workflow. For a tool paper whose central contributions include the experiential concept of 'material intimacy,' at least a small-scale informal user study (3–5 artists, structured tasks, observed workflows) or a comparative walkthrough showing what the real-time loop enables that the offline version could not would substantially strengthen the contribution.
- §1: The novelty claims — 'first application of network bending to the full depth of the video transformer architecture' and 'first time such manipulations can be explored in real-time' — should be more carefully situated against [1] (Abuzuraiq & Pasquier, 2025), which describes 'Explainability-in-Action' enabling 'expressive manipulation and tacit understanding by bending diffusion models in ComfyUI.' While [1] appears to target U-Net diffusion models rather than video DiTs, the paper should explicitly distinguish its scope (video DiT, full-depth including FFN, real-time) from this prior work to clarify what is genuinely novel versus what overlaps with these claims.
minor comments (7)
- §3.1: The monkeypatching approach is described at a high level but no code, pseudocode, or link to a repository is provided. Given that the system wraps specific open-source pipelines with named classes (WanSelfAttention, etc.), a code snippet or repo link would aid reproducibility.
- Figure 1: The caption states 'Each row shows modulations targeting a single DiT layer type,' but the specific modulation parameters used for each sample are not listed. Adding parameter values (or at least modulation type and intensity) would increase the figure's evidentiary value.
- Figure 4: The caption mentions 'early (0–10), middle (10–20), and late (20–30) layers' but the Wan 1.3B model has 30 blocks and the 14B has 40 — the model used for this figure should be specified.
- §3.2: The FFN controls (gain, thresholding, noise injection on hidden layers) are described qualitatively. A brief mathematical formulation (e.g., h' = clip(g·h + ε, ...) ) would clarify the manipulation space.
- §2.2: The paper lists CausVid, Self-Forcing, LongLive, and Krea as real-time Wan-based pipelines but does not explain why LongLive and Krea were chosen as integration targets over the others.
- The term 'material intimacy' is introduced in the abstract and used throughout but is only defined loosely as 'a responsive, near-mechanistic feel for how specific layers and neurons shape generated video' (§1). A more precise operational definition, or at least a concrete description of what user behaviors or experiences constitute it, would help readers engage with the concept.
- Reference [6] (DayDream Scope) is cited as a GitHub URL with no author list, title, or date beyond '2026.' A proper bibliographic entry would help readers locate the ecosystem.
Simulated Author's Rebuttal
We thank the referee for a careful and constructive reading. All four major comments are well-taken and actionable. We agree that (1) performance claims need quantitative substantiation, (2) the KV cache tension with 'material intimacy' requires explicit characterization, (3) experiential claims need external evidence beyond author observations, and (4) novelty claims need sharper situating against Abuzuraiq & Pasquier (2025). We will revise the manuscript accordingly.
read point-by-point responses
-
Referee: §3.1: The performance claim of '~15 frames per second on a single NVIDIA RTX A6000 Pro GPU' is load-bearing for the paper's central claim of real-time interaction, yet no latency measurements, frame-time variance, or performance degradation data under multiple simultaneous modulations are provided. At minimum, a table or plot showing frame rate as a function of (a) number of active modulations, (b) targeting granularity (global vs. per-neuron), and (c) cache state (enabled/disabled) is needed to substantiate 'real-time' and 'responsive.'
Authors: The referee is correct. The ~15 FPS claim is load-bearing and currently unsubstantiated by systematic measurement. We will add a performance evaluation table reporting mean frame rate, frame-time variance (std dev and p95), and latency under the three axes the referee specifies: (a) number of simultaneously active modulations (0, 1, 5, 10, 20), (b) targeting granularity (global layer-level vs. per-token vs. per-neuron), and (c) cache state (enabled vs. disabled vs. reset-on-change). We will also report the baseline (unmodulated) frame rate as a reference point. This will be added to §3.1 as a new subsection or table. We agree that without this data the reader cannot assess whether the system delivers interactive frame rates under realistic usage conditions. revision: yes
-
Referee: §4.3 and §4.2: The KV cache limitation creates a structural tension with the core 'material intimacy' claim. If mid-generation parameter changes are suppressed by caching, the feedback loop that constitutes 'material intimacy' may be broken precisely when it matters most. The paper should explicitly characterize under what conditions the feedback loop is actually responsive.
Authors: This is a fair and incisive observation. We agree that the tension between KV caching and material intimacy needs explicit treatment rather than being acknowledged only in the limitations section. In the revision, we will add a dedicated discussion (likely in §4.2 or as a bridge between §4.2 and §4.3) that characterizes the conditions under which the feedback loop is genuinely responsive: (1) parameter changes between diffusion steps respond immediately, as each new step computes fresh activations; (2) parameter changes within a step that affect the current frame's computation (e.g., self-attention modulation of the current latent) respond within the current frame; (3) parameter changes that rely on recomputing cached cross-frame attention states are the case where muting occurs, and we will specify this boundary clearly. We will also note that the 'disable cache' mode, while reducing temporal coherence, does restore full responsiveness and is the mode we recommend for exploratory XAIxArts use, while cache-enabled mode is better suited to sustained creative generation. This reframing should make clear that material intimacy holds under specific, characterizable conditions rather than universally. revision: yes
-
Referee: §4.1, §4.2: The claims about 'expressive potential,' 'material intimacy,' and the system functioning as an 'XAIxArts probe' are supported only by the authors' own observations and figures. For a tool paper whose central contributions include the experiential concept of 'material intimacy,' at least a small-scale informal user study (3–5 artists, structured tasks, observed workflows) or a comparative walkthrough showing what the real-time loop enables that the offline version could not would substantially strengthen the contribution.
Authors: We agree that the experiential claims are currently under-supported by external evidence. We will conduct a small-scale informal user study with 4–5 media artists, using structured tasks (e.g., 'achieve a specific visual effect using only attention modulation,' 'explore FFN neuron bands and describe what you observe') and semi-structured interviews about their experience. We will report observed workflows, qualitative responses, and any convergence or divergence with our own characterizations of material intimacy. We will also add a comparative walkthrough showing a concrete creative task performed in both the real-time and offline AttentionBender workflows, documenting what the real-time loop enables (e.g., rapid iterative exploration, building up layered modulations through live trial-and-error) that the offline version's parameter-tweak-then-wait cycle did not. If the user study cannot be completed before the revision deadline, we will at minimum add the comparative walkthrough and explicitly scope down the material intimacy claim from a general assertion to a hypothesis grounded in our own experience, clearly flagged as preliminary. We are committed to not overstating the evidence available. revision: yes
-
Referee: §1: The novelty claims — 'first application of network bending to the full depth of the video transformer architecture' and 'first time such manipulations can be explored in real-time' — should be more carefully situated against [1] (Abuzuraiq & Pasquier, 2025), which describes 'Explainability-in-Action' enabling 'expressive manipulation and tacit understanding by bending diffusion models in ComfyUI.'
Authors: The referee is right that we should explicitly distinguish our scope from Abuzuraiq & Pasquier (2025). We will revise §1 and §2.2 to clarify the distinction along three axes: (1) architecture — their work targets U-Net diffusion models, while ours targets video DiTs; (2) depth — their work operates on attention layers within the U-Net, while ours extends across all three DiT block components including the FFN, with neuron-level targeting; (3) interactivity — their ComfyUI-based workflow operates in an offline or near-offline render cycle, while ours wraps real-time pipelines achieving interactive frame rates. We will adjust the novelty claims to be precisely scoped: 'first application of network bending to the full depth of the video DiT architecture (including FFN and neuron-level targeting)' and 'first framework enabling such manipulations at interactive frame rates.' This avoids overstating overlap with their contributions while accurately characterizing what is genuinely new. revision: yes
Circularity Check
No significant circularity: the paper is a systems/tool contribution whose claims rest on wrapping external pipelines, not on self-citation chains.
full rationale
This is a tool/systems paper, not a derivation paper. There is no chain of equations or fitted parameters being presented as predictions. The central claims—(1) real-time interaction with DiT internals, (2) full-depth block modulation, (3) granular targeting—are architectural contributions verified by the existence of the running system, not by a derivation that could be circular. The self-citations to prior AttentionBender [2] and DayDream Scope [6] are used to establish lineage and context (e.g., 'We previously introduced AttentionBender [2], a tool that allowed artists to apply 2D transforms... to the cross-attention maps'), not to import a uniqueness theorem or an unverified ansatz that would make the present result forced by definition. The cross-attention spatial modulation methodology is explicitly attributed to the prior work ('building on the methodology of the original AttentionBender [2]'), but this is a design choice, not a load-bearing derivation step. The KV cache limitation (§4.3) is a genuine acknowledged constraint, not a circularity. No step in the paper reduces to its inputs by construction.
Axiom & Free-Parameter Ledger
axioms (3)
- domain assumption Real-time Wan pipelines (LongLive, Krea) provide a stable enough inference loop that monkeypatched modulation hooks can be injected without breaking generation.
- domain assumption KV cache limitations can be sufficiently mitigated by manual reset/disable controls to maintain a usable interactive experience.
- domain assumption The Wan DiT block structure (self-attention, cross-attention, FFN) is a consistent target architecture across the real-time video ecosystem.
invented entities (1)
-
Material intimacy
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Abuzuraiq and Philippe Pasquier
Ahmed M. Abuzuraiq and Philippe Pasquier. 2025. Explainability-in-Action: En- abling Expressive Manipulation and Tacit Understanding by Bending Diffusion Models in ComfyUI. (Aug. 10, 2025). arXiv: 2508.07183[cs]. Pre-published
-
[2]
Adam Cole and Mick Grierson. 2026. AttentionBender: Manipulating Cross- Attention in Video Diffusion Transformers as a Creative Probe. InProceedings of the 2026 ACM Conference on Creativity and Cognition. ACM Creativity and Cognition 2026. London, United Kingdom, (July 13, 2026). arXiv: 2604.20936 [cs]. doi:10.1145/3803784.3807565
work page internal anchor Pith review Pith/arXiv arXiv doi:10.1145/3803784.3807565 2026
-
[3]
Andreas Blattmann, Robin Rombach, et al. 2023. Align Your Latents: High- Resolution Video Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 22563–22575
work page 2023
-
[4]
Terence Broad, Frederic Fol Leymarie, and Mick Grierson. 2021. Network Bending: Expressive Manipulation of Deep Generative Models. InArtificial Intelligence in Music, Sound, Art and Design: 10th International Conference, EvoMUSART 2021, Held as Part of EvoStar 2021, Virtual Event, April 7–9, 2021, Proceedings. Springer-Verlag, Berlin, Heidelberg, (Apr. 7,...
work page 2021
- [5]
-
[6]
Erwann Millon. 2025. Krea Realtime 14B: Real-Time, Long-Form AI Video Generation. Krea Blog. (Oct. 15, 2025). https://www.krea.ai/blog/krea-realtime -14b
work page 2025
-
[7]
Jonathan Ho, Tim Salimans, et al. 2022. Video Diffusion Models. (June 22, 2022). arXiv: 2204.03458[cs]. Pre-published
work page internal anchor Pith review Pith/arXiv arXiv 2022
-
[8]
Xun Huang, Zhengqi Li, et al. 2025. Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion. (Nov. 10, 2025). arXiv: 2506.08009 [cs]. Pre-published
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[9]
Tom Hume. 2025. Meet Flow: AI-powered filmmaking with Veo 3. Google: The Keyword. (May 20, 2025). https://blog.google/innovation-and-ai/products/goo gle-flow-veo-ai-filmmaking-tool/. Real-Time AttentionBender
work page 2025
-
[10]
Weijie Kong, Qi Tian, et al. 2025. HunyuanVideo: A Systematic Framework For Large Video Generative Models. (Jan. 17, 2025). arXiv: 2412.03603 [cs]. Pre-published
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[11]
OpenAI. 2024. Sora (Blogpost). (Feb. 15, 2024). https://openai.com/index/sora/
work page 2024
-
[12]
William Peebles and Saining Xie. 2023. Scalable Diffusion Models with Trans- formers (preprint). In Proceedings of the IEEE/CVF International Conference on Computer Vision. (Mar. 2, 2023), 4195–4205
work page 2023
-
[13]
Robin Rombach, Andreas Blattmann, et al. 2022. High-Resolution Image Syn- thesis With Latent Diffusion Models (Stable Diffusion). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. (Apr. 13, 2022), 10684–10695
work page 2022
-
[14]
WanTeam, Ang Wang, et al. 2025. Wan: Open and Advanced Large-Scale Video Generative Models. (Mar. 26, 2025). arXiv: 2503.20314[cs]. Pre-published
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[15]
Shuai Yang, Wei Huang, et al. 2025. LongLive: Real-time Interactive Long Video Generation. (Sept. 26, 2025). arXiv: 2509.22622[cs]. Pre-published
work page internal anchor Pith review Pith/arXiv arXiv 2025
- [16]
-
[17]
Tianwei Yin, Qiang Zhang, et al. 2025. From Slow Bidirectional to Fast Autore- gressive Video Diffusion Models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition. (July 11, 2025). arXiv: 2412.07772 [cs]. doi:10.48550/a rXiv.2412.07772
work page doi:10.48550/a 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.