Pith. sign in

REVIEW 4 major objections 6 minor 4 cited by

Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Diffusion Transformer redundancy is model-specific: the same caching shortcut does not transfer across models, even though each model's redundancy pattern is stable across prompts, step counts, and schedulers.

desk verdict A useful systematic measurement of DiT redundancy, with a solid descriptive core and a prescriptive conclusion that outruns the evidence. read the letter →

arxiv 2411.13588 v1 pith:QWZW4NBY submitted 2024-11-18 cs.CV cs.AI

Xibo Sun , Jiarui Fang , Aoyu Li , Jinzhe Pan This is my paper · ORCID
classification cs.CVcs.AI
keywords DiffusionTransformerredundancyactivationcachinginferencelatencymodelscacheschedulingDiTanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the 'redundancy' exploited by diffusion-transformer caching methods, the near-identical behavior of activations at neighboring denoising steps, is not a universal phenomenon. Measuring L1 distances between consecutive-step key, value, and attention-output activations across seven mainstream DiT models, it finds that each model has its own redundancy fingerprint, with different trends across layers and steps, including conflicting shapes such as early spikes, U-curves, and later-stage surges. At the same time, the fingerprint of a given model is stable when prompts, total step counts, or schedulers change. The authors conclude that caching strategies tuned on one DiT or one small family cannot be assumed to transfer, and that each model merits its own analysis; they release a tool for that per-model analysis. If true, this redirects the field from seeking a universal DiT acceleration recipe toward model-specific or adaptively learned caching.

What carries the argument

The central mechanism is the per-step, per-layer L1 distance between the input key (K), value (V), and attention output (A) activations at consecutive diffusion steps, which is the paper's operational definition of 'input temporal redundancy.' The coefficient of variation of these distances across prompts quantifies prompt sensitivity, while plots of the distances over steps and layers are what the paper compares across models and ablations. This signal is the one a caching schedule would exploit to decide which activations can be reused, and the released tool automates producing it for an arbitrary DiT.

What would settle it

A controlled counterexample in the paper's own protocol, for instance two DiT architectures with distinct conditioning mechanisms whose normalized L1-distance curves over layers and steps coincide, or a scheduler switch that flips a model's curve shape, would refute the claims of model-specific and scheduler-independent redundancy.

Watch

Extended reading notes

Core claim

The central discovery is stated in the paper's own conclusions: 'the redundancy distribution within each DiT model exhibits distinct trends,' while 'within a singular model, the redundancy distribution demonstrates stability irrespective of alterations in input prompts, step counts, or scheduling strategies.' Empirically, prompt effects are small, with the coefficient of variation of L1 distances across 1000 prompts roughly 10 to 15 percent, and changing FLUX.1-dev from 28 to 56 or 14 steps, or swapping in Stable-Diffusion-3's scheduler, preserves the qualitative curve shape. Because the seven studied models, FLUX.1-dev, PixArt-Alpha, Stable-Diffusion-3, CogVideoX-5B, Open-Sora, Latte-1, and Mochi-1-preview, display mutually inconsistent patterns, the authors conclude that prior single-model observations do not generalize and that caching designs must be tailored per architecture, supported by their released analysis tool.

Load-bearing premise

The study's load-bearing premise is that the L1 distance between consecutive-step key, value, and attention-output activations fully captures what makes a diffusion step cacheable, and that other possible redundancy signals such as feedforward activations or attention scores would not change the conclusions.

Editorial extensions

If this is right

  • A caching schedule validated on one DiT family should be re-validated before use on another; transfer without measurement is unsafe.
  • Redundancy curves obtained from a single set of prompts can characterize a model, since prompt variation changes L1 distances by only about 10 to 15 percent.
  • Changing the number of sampling steps or the scheduler for a given model does not require redesigning the caching strategy, only rescaling step indices.
  • Researchers can use the released tool to map a new DiT's redundancy distribution and route caching to the high-redundancy layers and stages.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The stability across prompts and steps suggests that a single calibration pass per model could precompute a caching schedule before deployment, a step the paper leaves implicit.
  • Model-to-model variation may track known architectural axes such as conditioning mechanism, depth, image versus video generation, and backbone design, but the paper does not perform a controlled ablation holding those factors fixed.
  • Because the K, V, and A proxy excludes feedforward activations and attention scores, an expanded proxy might reveal cross-model regularities that this study misses; that remains speculative.
  • The non-transferability result implies that benchmark comparisons of DiT caching methods should control for model architecture, reporting per-model speedups rather than a single average.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents an empirical study of 'input temporal redundancy'—defined as the L1 distance between K, V, and attention-output activations at consecutive diffusion steps—across seven DiT-based image and video generation models (FLUX.1-dev, Pixart-Alpha, Stable-Diffusion-3, CogVideoX-5B, Open-Sora, Latte-1, Mochi-1-preview). The authors report that redundancy profiles differ substantially across models, while within a single model the profiles are stable across prompts, step counts, and scheduler choices. From this they conclude that caching strategies developed for one group of DiT models may not transfer to others, and they release an analysis tool (DiTCacheAnalysis) to help researchers design model-specific caching strategies.

Significance. If the central claims hold, the paper would be a useful negative result for the DiT acceleration community: it would caution against assuming that observations from one or two DiT models generalize, and it would motivate model-specific caching analysis. The breadth of models covered is a genuine strength, as is the release of the analysis tool and the candid reporting of conflicting prior observations. However, the significance is currently conditional because the operationalization of 'redundancy' via L1 distances of K, V, and A is never validated against actual caching outcomes, and the cross-model and within-model claims rest on visual inspection rather than quantitative comparison or statistical testing.

major comments (4)
  1. [Section 4.1 and Section 4.6] The paper defines redundancy exclusively as the L1 distance between consecutive-step K, V, and A activations, then uses this operationalization to draw the headline conclusion that caching strategies do not transfer across DiT models. This is load-bearing: cacheability depends not only on the magnitude of activation drift but also on which activations are skipped, how long reused activations remain accurate, and how sensitive downstream generation quality is to induced error. No experiment connects the measured L1 distances to actual caching performance, reuse-window accuracy, or end-to-end generation quality. The authors should either validate the proxy by comparing L1-profile predictions with real caching experiments on at least two models, or substantially soften the non-transferability conclusion to a statement about the measured activation-distance distributions rather than about caching strategies.
  2. [Section 4.2, Table 2] The claim that prompts affect redundancy by only 10-15% rests on a single coefficient-of-variation number averaged over all layers and diffusion steps. This averaging can hide high-variability regions (e.g., early steps or deep layers where absolute L1 distances are large). No confidence intervals, per-layer/per-step breakdowns, or statistical tests are provided, and the cross-model comparison that would justify 'the DiT model predominantly determines the redundancy distribution' is absent. At minimum, report the range and distribution of per-layer/per-step coefficients of variation and show that within-prompt variability is small relative to between-model variability.
  3. [Section 4.3] The central cross-model finding—that redundancy distributions differ qualitatively across models—is supported only by visual inspection of multi-panel figures. There is no quantitative summary of the curves (e.g., normalized L1-distance profiles, summary statistics such as peak location or area under curve) and no statistical comparison across models. Given that the paper's main message is that caching strategies are model-specific, the authors should provide a numeric characterization of the profiles and a quantitative measure of between-model divergence, rather than relying on the reader's interpretation of the figures.
  4. [Sections 4.4 and 4.5] The ablations on step count and scheduler are performed only on FLUX.1-dev. The conclusion that redundancy is stable within a single model 'irrespective of alterations in input prompts, step counts, or scheduling strategies' is therefore demonstrated for only one architecture. This is especially problematic because the cross-model section shows that models differ dramatically (e.g., Stable-Diffusion-3 has a U-shaped profile, Pixart-Alpha has a late-step rise, Mochi-1 has abrupt early fluctuations). At least one additional model with a qualitatively different profile should be tested under varying step counts and schedulers before the within-model stability claim is generalized.
minor comments (6)
  1. [Section 4.1, Table 1 and Table 2] The caption for Table 1 says 'Coefficient of variation in L1 distance of K, V, and A over various prompts,' but the table lists model parameters, attention layers, and diffusion steps; the coefficient-of-variation table appears later as Table 2. This mismatch is confusing and should be fixed.
  2. [Table 2] Table 2 appears to omit Mochi-1-preview values for K, V, and A, even though the header includes seven models. If the values are absent because only 100 prompts were used, this should be stated in the caption or table note.
  3. [Section 4.1] The sentence 'Note that, in the diffusion process, Step i + 1 appear before Step i' is grammatically awkward and also unclear about the direction of the L1 comparison; please rephrase to clarify whether the distance is computed between the current step and the previous or subsequent step.
  4. [Section 4.1] Minor typos include '12 billion parameter' (should be 'parameters'), 'comprises pure transformer blocks' (should be 'comprise'), and the repeated phrase 'differences in L1 distance in L1 distance' in Section 4.3's Stable-Diffusion-3 paragraph.
  5. [References] Reference [14] (Megascale) appears mis-cited as an example of DiT architecture transitioning from U-Net; Megascale is about large-scale LLM training, not DiT architecture. Please verify this citation or replace it with a more appropriate reference.
  6. [Figures 2-11] The multi-panel figures are difficult to read because axes labels and legends are small and many subplots are compressed. Adding representative single-panel examples or a summary figure with aggregated curves would improve readability substantially.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports direct empirical measurements and its conclusions summarize those measurements without fitting or self-referential construction.

full rationale

This paper makes no fitted predictions and does not derive any quantity from its own conclusions. Its central claims are direct empirical measurements: it records L1 distances between K, V, and A activations at consecutive diffusion steps across seven DiT models and reports the observed distributions. Section 4.1 defines the measured quantity explicitly as the L1 distance between X^{i,j}_M(p) and X^{i+1,j}_M(p), and Sections 4.2 through 4.5 present those measurements under varied prompts, models, step counts, and schedulers. The conclusion in Section 4.6 that redundancy distributions are model-specific and stable within a model is a summary of those measurements, not a construction that presumes the conclusion. The released analysis tool is an instrumentation utility, not an input to the measurements, and no parameter is fitted to a subset of data and then 'predicted' for a closely related quantity. The two self-citations, PipeFusion and xDiT, appear only in background discussion of parallelization and are not load-bearing for the redundancy analysis. The proxy validity concern, namely whether L1 distance of K, V, and A fully captures cacheability, is a correctness or external-validity question rather than a circularity, because the paper's derived statements are explicitly about this measured quantity and do not secretly redefine a different target in terms of the same metric. Therefore no circular step is present.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The paper introduces no fitted parameters and no new theoretical entities. Its conclusions rest on the measurement proxy (L1 distance on K, V, A) and on the representativeness of the tested configurations.

assumptions (2)
  • domain assumption L1 distance between K, V, and A activations at consecutive diffusion steps is a valid proxy for input temporal redundancy relevant to caching.
    Introduced in Section 4.1 where redundancy is defined solely via L1 distances of K, V, and A; other potential redundancy signals (e.g., feedforward activations, attention scores) are not considered.
  • domain assumption The tested set of models, prompts, and schedulers is representative of the DiT landscape.
    The study uses seven models and 1000 MS-COCO prompts; step-count and scheduler ablations are only on FLUX.1-dev, limiting the generality of the stability claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study." pith.science (2026). https://pith.science/paper/QWZW4NBY

@misc{pith2026241113588,
  author       = {Pith},
  title        = {Pith review of: Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QWZW4NBY}},
  note         = {Machine review of arXiv:2411.13588}
}
read the original abstract

The increased model capacity of Diffusion Transformers (DiTs) and the demand for generating higher resolutions of images and videos have led to a significant rise in inference latency, impacting real-time performance adversely. While prior research has highlighted the presence of high similarity in activation values between adjacent diffusion steps (referred to as redundancy) and proposed various caching mechanisms to mitigate computational overhead, the exploration of redundancy in existing literature remains limited, with findings often not generalizable across different DiT models. This study aims to address this gap by conducting a comprehensive investigation into redundancy across a broad spectrum of mainstream DiT models. Our experimental analysis reveals substantial variations in the distribution of redundancy across diffusion steps among different DiT models. Interestingly, within a single model, the redundancy distribution remains stable regardless of variations in input prompts, step counts, or scheduling strategies. Given the lack of a consistent pattern across diverse models, caching strategies designed for a specific group of models may not easily transfer to others. To overcome this challenge, we introduce a tool for analyzing the redundancy of individual models, enabling subsequent research to develop tailored caching strategies for specific model architectures. The project is publicly available at https://github.com/xdit-project/DiTCacheAnalysis.

Figures

Figures reproduced from arXiv: 2411.13588 by the authors.

Figure 1
Figure 1. shows the results from these papers. (a) L2C. (b) TGATE. (c) PAB [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Distribution of redundancy in FLUX.1-dev. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Distribution of redundancy in Pixart-Alpha. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Distribution of redundancy in Stable-Diffusion-3. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Distribution of redundancy in CogVideoX-5B. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Distribution of redundancy in Latte-1. 10 2 10 1 10 0 L1 Distance 10 2 10 1 10 0 L1 Distance 20 10 0 Step 10 2 10 1 10 0 L1 Distance 20 10 0 Step 20 10 0 Step 20 10 0 Step 20 10 0 Step 20 10 0 Step 20 10 0 Step 20 10 0 Step 20 10 0 Step 20 10 0 Step 20 10 0 Step 20 10 …
Figure 7
Figure 7. Figure 7: Distribution of redundancy in Open-Sora. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Distribution of redundancy in Mochi-1-preview. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Distribution of redundancy in FLUX.1-dev with 56 diffusion steps. [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Distribution of redundancy in FLUX.1-dev with 14 diffusion steps. [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Distribution of redundancy in FLUX.1-dev with Stable-Diffusion-3’s scheduler. [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DiTReducio: A Training-Free Acceleration for DiT-Based TTS via Progressive Calibration

    cs.SD 2025-09 conditional novelty 6.0 of 10

    DiTReducio is a training-free, pattern-guided layer and branch skipping method that accelerates DiT-based TTS, reporting significant FLOP and RTF reductions with modest quality loss at tuned thresholds.

  2. Chipmunk: Training-Free Acceleration of Diffusion Transformers with Dynamic Column-Sparse Deltas

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Chipmunk speeds up diffusion transformer inference by recomputing, at each step, only the top percent of attention and MLP activation columns that change most between steps, caching the rest in column-sparse GPU kernels.

  3. TokenRing: An Efficient Parallelism Framework for Infinite-Context LLMs via Bidirectional Communication

    cs.DC 2024-12 conditional novelty 4.0 of 10

    TokenRing claims faster long-sequence attention on multiple GPUs by overlapping forward query transfers with backward partial-output transfers, supported by one 4-GPU profiling comparison.

  4. Efficient Diffusion Models: A Survey

    cs.LG 2025-02 conditional novelty 2.0 of 10

    The paper organizes research on efficient diffusion models into a taxonomy spanning algorithms, systems, and frameworks, and provides a curated reference list.

Reference graph

Works this paper leans on

28 extracted references · 10 canonical work pages · cited by 4 Pith papers

  1. [1]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer, 2015

    2015

  2. [2]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023

    2023

  3. [3]

    Ld-pruner: Efficient pruning of latent diffusion models using task-agnostic insights

    Thibault Castells, Hyoung-Kyu Song, Bo-Kyeong Kim, and Shinkook Choi. Ld-pruner: Efficient pruning of latent diffusion models using task-agnostic insights. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 821–830, 2024

    work page 2024

  4. [4]

    One-step diffusion distillation via deep equilibrium models

    Zhengyang Geng, Ashwini Pokle, and J Zico Kolter. One-step diffusion distillation via deep equilibrium models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  5. [5]

    Q-diffusion: Quantizing diffusion models

    Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17535–17545, 2023

    2023

  6. [6]

    Pipefusion: Patch-level pipeline parallelism for diffusion transformers inference, 2024

    Jiarui Fang, Jinzhe Pan, Jiannan Wang, Aoyu Li, and Xibo Sun. Pipefusion: Patch-level pipeline parallelism for diffusion transformers inference, 2024

    work page 2024

  7. [7]

    xdit: an inference engine for diffusion transformers (dits) with massive parallelism, 2024

    Jiarui Fang, Jinzhe Pan, Xibo Sun, Aoyu Li, and Jiannan Wang. xdit: an inference engine for diffusion transformers (dits) with massive parallelism, 2024

    work page 2024

  8. [8]

    Learning-to-cache: Accelerating diffusion transformer via layer caching

    Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinchao Wang. Learning-to-cache: Accelerating diffusion transformer via layer caching. arXiv preprint arXiv:2406.01733, 2024

    arXiv 2024

Show all 28 references
  1. [9]

    Cross- attention makes inference cumbersome in text-to-image diffusion models

    Wentian Zhang, Haozhe Liu, Jinheng Xie, Francesco Faccio, Mike Zheng Shou, and Jürgen Schmidhuber. Cross- attention makes inference cumbersome in text-to-image diffusion models. arXiv preprint arXiv:2404.02747, 2024

    2024 arXiv

  2. [10]

    Real-time video generation with pyramid attention broadcast

    Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You. Real-time video generation with pyramid attention broadcast. arXiv preprint arXiv:2408.12588, 2024. 8 Sun et al. Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study

    2024 arXiv

  3. [11]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  4. [12]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    2022

  5. [13]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  6. [14]

    Megascale: Scaling large language model training to more than 10,000 gpus

    Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, et al. Megascale: Scaling large language model training to more than 10,000 gpus. arXiv preprint arXiv:2402.15627, 2024

    2024 arXiv

  7. [15]

    Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023

    arXiv

  8. [16]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024

    2024 arXiv

  9. [17]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine ...

    2024

  10. [18]

    Frdiff: Feature reuse for exquisite zero-shot acceleration of diffusion models

    Junhyuk So, Jungwon Lee, and Eunhyeok Park. Frdiff: Feature reuse for exquisite zero-shot acceleration of diffusion models. CoRR, abs/2312.03517, 2023

    2023 arXiv

  11. [19]

    Deepcache: Accelerating diffusion models for free

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15762–15772, 2024

    2024

  12. [20]

    \delta-dit: A training-free acceleration method tailored for diffusion transformers

    Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Savvas Bouganis, Yiren Zhao, and Tao Chen. \delta-dit: A training-free acceleration method tailored for diffusion transformers. arXiv preprint arXiv:2406.01125, 2024

    2024 arXiv

  13. [21]

    Ditfastattn: Attention compression for diffusion transformer models

    Zhihang Yuan, Pu Lu, Hanling Zhang, Xuefei Ning, Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang. Ditfastattn: Attention compression for diffusion transformer models. arXiv preprint arXiv:2406.08552, 2024

    2024 arXiv

  14. [22]

    Flux.1-dev

    Black Forest Lab. Flux.1-dev. https://huggingface.co/black-forest-labs/FLUX.1-dev , 2024

    2024

  15. [23]

    Pixart-xl-2-1024-ms

    PixArt Alpha. Pixart-xl-2-1024-ms. https://huggingface.co/PixArt-alpha/PixArt-XL-2-1024-MS , 2024

    2024

  16. [24]

    stable-diffusion-3-medium

    Stabilityai. stable-diffusion-3-medium. https://huggingface.co/stabilityai/ stable-diffusion-3-medium , 2024

    2024

  17. [25]

    Cogvideo-5b

    THUDM. Cogvideo-5b. https://huggingface.co/THUDM/CogVideoX-5b, 2024

    2024

  18. [26]

    Open-sora

    hpcaitech. Open-sora. https://github.com/hpcaitech/Open-Sora, 2024

    2024

  19. [27]

    maxin cn. Latte-1. https://huggingface.co/maxin-cn/Latte-1 , 2024

    2024

  20. [28]

    mochi-1-preview

    Genmo. mochi-1-preview. https://huggingface.co/genmo/mochi-1-preview , 2024. 9

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.