Pith. sign in

REVIEW 4 major objections 5 minor 69 references

FlashDiff claims that diffusion serving can be sped up by refining only latent patches that still need work, cutting latency 30-97% and raising throughput 1.2-2.2x without quality loss across image, video, and audio workloads.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-02 06:39 UTC pith:ODQ6JKCN

load-bearing objection Efficiency results are solid and novel; the no-quality-loss claim is under-supported—needs a no-skip baseline and a repair or downgrade of the formal guarantee. the 4 major comments →

arxiv 2607.12121 v2 pith:ODQ6JKCN submitted 2026-07-13 cs.DC cs.LG

FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving

classification cs.DC cs.LG
keywords diffusion servingsemantic patch parallelismadaptive gatingpatch schedulinglatency optimizationthroughputattention saliencyquality guarantee
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Diffusion generation wastes much of its compute on latent regions that have already stabilized. FlashDiff's central claim is that treating the latent as a set of semantically coherent patches, estimating which patches still need refinement from attention signals, and scheduling only those active patches across GPUs yields large efficiency gains with no perceptual quality loss. The paper shows that across image, video, and audio workloads, this semantic patch parallelism reduces end-to-end serving latency by 30-97%, improves throughput by 1.2-2.2x, and eliminates 24-66% of computation while matching full-execution generation quality. A sympathetic reader would see the work as a new execution principle for diffusion serving: decoupling where and when refinement happens instead of running every step uniformly on the whole latent.

Core claim

The paper's discovery is that diffusion refinement is spatially and temporally heterogeneous: different latent regions stabilize at different rates, and adjacent denoising steps are strongly correlated. FlashDiff converts this into serving efficiency through three coordinated mechanisms: a partitioner that groups latent elements into contiguous, semantically coherent patches using early cross-attention saliency; a gate that skips a patch when the relative change in its self-attention importance is below a threshold, reusing a cached prediction; and a scheduler that packs active patches onto GPUs with a locality-aware least-maximum-load rule while respecting request priorities. The paper prov

What carries the argument

Semantic patch parallelism: the latent grid is partitioned into contiguous, semantically coherent patches via a recursive Otsu-style thresholding of a prompt-driven cross-attention saliency map. Each patch is then gated independently—skipped when its normalized self-attention change falls below a threshold, reactivated when dependencies re-emerge—and active patches are dispatched as preemptible monotasks to workers using a load-balancing, affinity-aware assignment. The gating rule carries the paper's main quality guarantee: terminal error is bounded by ε times cumulative refinement activity plus a staleness penalty, with the penalty shrinking as the maximum consecutive skip count decreases.

Load-bearing premise

The whole quality guarantee rests on one assumption: a patch whose self-attention importance has stopped changing can safely reuse a stale prediction, with the drift error it introduces bounded by that attention change—a bound the paper states with an unspecified model-specific constant rather than proves.

What would settle it

For a diverse set of prompts, measure each skipped patch's true drift error (the normed difference between a freshly computed noise prediction and the cached one) and compare it against the attention-change signal used by the gate; if any patch shows large drift error with near-zero attention change, or if the implied constant C0 grows with resolution or prompt complexity, the gating proxy fails. A simpler check is to drive the forced-reactivation bound K to 1 and verify the predicted error collapse; failure would indicate the staleness argument is incorrect.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the central claim holds, interactive image, video, and audio generation can be served with far fewer GPUs, since 24-66% of patch-steps are provably skippable without perceptible quality loss.
  • The efficiency gains compose with existing step-reduction and model-compression techniques, adding a complementary axis of savings rather than replacing them.
  • The explicit error bound gives operators a smooth, principled knob: lowering the gating threshold ε trades compute for quality, enabling quality-targeted serving.
  • Because freed compute is reallocated to other concurrent requests, the design absorbs bursty request arrivals without the latency spikes seen in monolithic serving.
  • The partitioning and gating machinery is architecture- and modality-agnostic, applying to any diffusion model built on cross- and self-attention.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same attention-change proxy could enable cross-request caching: patches with stable self-attention across similar prompts could share cached predictions, extending savings beyond temporal skipping within a single generation.
  • The spatial-temporal heterogeneity FlashDiff exploits is likely present in other iterative generative processes, such as flow-matching and consistency-model refinement, suggesting the execution pattern may transfer beyond diffusion.
  • A testable extension is to combine patch gating with step-distillation: gating removes spatial redundancy while distillation removes temporal redundancy, potentially compounding the two savings.
  • The quality guarantee is comparative (against full execution of the same model), not absolute; a natural next step is a bound relative to the true data distribution, which would require assumptions about score approximation error.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. FlashDiff is a diffusion serving engine that partitions latent tensors into semantic patches, selectively skips low-activity patches during denoising, and schedules the resulting monotasks across GPUs. The paper claims system-level gains on T2I, T2V, and T2A workloads: 30–97% latency reduction, 1.2–2.2× throughput improvement, and 24–66% reduction in executed computation without degrading generation quality. The core technical pieces are a cross-attention-driven recursive partitioner (Section 4.1), a self-attention-based gating rule (Section 4.2), a priority-based patch scheduler (Section 4.3), and a formal error bound (Appendix A). The evaluation is extensive: four models, three modalities, online and offline scenarios, sensitivity analyses, and an overhead ablation (0.4–3.7%). The formal appendix attempts to bound the quality gap between FlashDiff and no-skip execution in terms of the gating threshold and a staleness penalty.

Significance. If the central claims hold, FlashDiff would be a valuable system contribution: it addresses a real bottleneck in diffusion serving, generalizes across modalities, and is implemented on a production-grade stack (TensorRT). The paper gives creditably detailed measurements, including a clean component breakdown (Figure 14), a hardware-scale study (Figure 17), and multiple ablations. The overhead ablation in Figure 13 is particularly informative. The formal error-bound attempt, if made rigorous, would be a useful template for other approximate-execution systems. However, the manuscript's headline 'without degrading quality' claim is not currently established: the empirical comparisons are against other approximate engines, and the formal proof rests on an unvalidated coherence assumption whose proof conflates scalar attention-weight changes with vector-valued attention-output changes. These are load-bearing issues for the paper's main claim, not merely presentation concerns.

major comments (4)
  1. [§6.2, Table 2] The claim that FlashDiff 'does not degrade generation quality' is not tested against the relevant baseline. Table 2 compares FlashDiff only to xDiT, DistriFusion, and NaivePatch, all of which are approximate execution engines that introduce their own errors (DistriFusion explicitly reuses stale states). To support the stated claim, the authors should compare FlashDiff against the unmodified model with the same initial noise, the same sampler, and full no-skip execution, reporting the same quality metrics (e.g., PSNR, LPIPS, HPSv3, VBench, CLAP) and the same seeds. Without this baseline, the quality-preservation claim is unsupported, and the slightly 'higher' quality scores in Table 2 cannot be interpreted.
  2. [Appendix A.2, Eq. (8)–(10) and Lemma A.4] Assumption A.3 is the load-bearing assumption for Theorem A.6, but it is not established. Eq. (8) defines Δ_i as the change in the scalar patch-level refinement importance R_i, which is the mean of self-attention weights in the patch. Lemma A.4's proof, however, treats Δ_i as proportional to the vector norm ‖SA^{(i)}(x̂_n,t_n) − SA^{(i)}(x̂_{n_i},t_{n_i})‖. These are different objects: the mean attention weight in a patch can stay nearly constant while the value-weighted attention output changes substantially. The 'Justification' paragraph after Assumption A.3 asserts that 'Δ_i measures the right-hand side up to the per-patch normalisation factor', but this is exactly the point that must be proved or empirically validated. The proof of Lemma A.4 also contains an inconsistency: the stated β = L_SA(L M_f + L_t) does not match the derivation, which yields L_SA(M_f + 1) K h. Because Theorem
  3. [Appendix A.3, Assumption A.10 and Proposition A.11] The optimality claim for the saliency-based partitioner depends on Assumption A.10, which asserts a monotone relationship between saliency and skip perturbation. This assumption is plausible but is not validated in the paper, and the proof of Proposition A.11(ii) invokes a 'variance-preserving property of monotone transformations' with a Lipschitz constant κ of the saliency-to-perturbation mapping. Monotone transformations do not in general preserve variance bounds without additional quantitative conditions, and the existence of κ is simply assumed. The proposition is also limited to binary threshold splits, as acknowledged in the limitations paragraph, so the recursive R-way partitioning is not actually covered. This does not invalidate the engineering contributions, but it means the partitioner's theoretical grounding is much weaker than stated.
  4. [Theorem A.6 and Corollary A.14] Even if Assumption A.3 held, the theorem's error bound contains the factor e^{LT}√R with model-specific uncalibrated constants L, C0, β, and L_SA. The paper does not estimate or bound these constants for any of the evaluated models (FLUX, SD3, Wan2.1, StableAudioOpen). As a result, the statement that the quality gap is 'O(ε·ΣD_n + γ)' gives no numerically meaningful guarantee for the specific systems in Section 6. The vanishing as ε→0 is true by construction, but the paper claims a practical quality guarantee at the operating ε values used in the experiments; that claim requires either calibrated constants or a direct empirical validation of the coherence bound. I recommend adding such a validation (e.g., scatter plots of ‖δ_n^(i)‖ against Δ_n^(i) for representative models and steps) or softening the formal claim.
minor comments (5)
  1. [Appendix A.2, Remark A.16(c)] The text says 'cf. Figure 13' when discussing prompt-dependent skip rates; the relevant figure appears to be Figure 15 (statistical efficiency breakdown). Please correct the cross-reference.
  2. [§6.1] The methodology states that 'all results are averaged over five independent runs' but Table 2 and Figures 11–21 do not report error bars, confidence intervals, or statistical significance. For a serving-system paper, reporting variance across runs would strengthen the comparisons, especially where FlashDiff and DistriFusion quality scores are very close (e.g., Table 2, FLUX PSNR 20.98 vs 20.85).
  3. [§4.2, Eq. (8)] The gating signal R_i^{(t)} is the mean over |A_i|^2 self-attention entries. For large patches this quantity may be noisy or saturated; the definition and the normalization in Eq. (9) should be stated more precisely (e.g., whether SA is post-softmax, and how the diagonal is handled).
  4. [§6.4, Figure 19] The caption says the left axis is 'Offline Throughput' and the right axis PSNR, but the legend lists 'DistriFusion Throughput', 'FlashDiff Throughput', 'DistriFusion PSNR', 'FlashDiff PSNR'. The figure would be easier to read if line styles for PSNR were distinguished and the axis labels were matched to the legend.
  5. [Algorithm 1] The PatchGate pseudocode recomputes R_i from self-attention for all patches, but in the implementation Section 5 skipped patches reuse cached states and do not compute fresh self-attention. The pseudocode should clarify that only active patches are freshly evaluated.

Circularity Check

0 steps flagged

No significant circularity: efficiency gains are empirical; the quality bound is conditional on an explicit unvalidated assumption, which is a rigor gap rather than a circular reduction.

full rationale

FlashDiff's efficiency claims (latency reduction, throughput gain, skipped computation) are supported by direct measurements against external baselines (xDiT, DistriFusion, NaivePatch) and are not derived from a fitted parameter that is then renamed as a prediction. The theoretical quality guarantee (Theorem 4.1 / Theorem A.6) is an explicitly conditional statement: Appendix A.2 introduces Assumption A.3, which asserts that the fresh attention-change signal bounds the drift perturbation. This is an unvalidated modeling assumption, not an equation identical to the theorem's conclusion; the Gronwall-style argument legitimately aggregates the assumption into a terminal error bound. The paper itself notes the limitation: 'The constants L, L_t, C0, and beta are model-specific: while the form of the bound is universal, its numerical tightness varies across architectures and should be validated empirically on each target model.' The main-text proof sketch omits this assumption, but the appendix states it explicitly, so the logic is transparent rather than circular. The absence of a direct no-skip baseline in Table 2 is a genuine empirical weakness of the 'without degrading quality' claim, but that is a benchmarking gap, not a circular derivation. There is no load-bearing self-citation, no imported uniqueness theorem from the authors' prior work, and no ansatz smuggled in via citation to the authors' own results. I therefore find no significant circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 7 axioms · 0 invented entities

FlashDiff's formal quality guarantees rest on Assumptions A.1–A.3, A.10, A.13, and A.17. A.3 and A.10 are ad hoc properties that essentially assert the gate and partitioner are safe. Several operational knobs (T_w, ε, K, R, η) are chosen by the designers; R is adapted online. No new physical entities are introduced.

free parameters (6)
  • warm-up steps T_w = 5 (default; 1–20 ablated in Fig. 18)
    Controls quality of the saliency map versus lost skipping opportunity; chosen by operator.
  • gating threshold ε = not fixed in text; sweep 0–0.54 in Fig. 20
    Sets the quality-efficiency tradeoff; no stated default operating point.
  • maximum consecutive skip bound K = K ≈ 3–5 (Remark A.7)
    Controls the staleness penalty γ in the error bound; not derived.
  • patch count R = adapted online via binary search (§4.1)
    Tuned from observed latency and quality signals; no a priori optimal value.
  • smoothing constant η = unspecified
    Appears in Eq. (9) to normalize the gating weights; value not reported.
  • coherence constants C0, β, L, L_t, L_SA = unspecified; 'model-specific' (Appendix A.2)
    Introduced to make the error bound close; the paper never estimates them.
axioms (7)
  • standard math Assumption A.1: drift f_θ(·,t) is L-Lipschitz uniformly in t
    Standard in diffusion theory; plausible for Lipschitz neural networks but unverified for DiT.
  • domain assumption Assumption A.2: drift is L_t-Lipschitz in time
    Smoothness of the denoising trajectory; standard but unmeasured.
  • ad hoc to paper Assumption A.3: drift error from skipping is bounded by fresh attention-delta, ∥δ_n^{(i)}∥ ≤ C0·Δ_n^{(i),★}
    This is the gate's core correctness property, not independently verified; C0 is vague and model-specific.
  • ad hoc to paper Assumption A.10: higher-saliency tokens incur larger skip perturbation (monotonicity)
    Needed for the Otsu partition optimality proof; asserted as natural but not tested.
  • domain assumption Assumption A.13: quality metrics are K_Q-Lipschitz in the latent
    Plausible for PSNR/SSIM/LPIPS on a bounded latent range.
  • standard math Assumption A.17: data distribution satisfies log-Sobolev inequality with constant α>0
    Used for the tightened contractivity bound; plausible but unverified for image/video/audio latents.
  • domain assumption Cross-attention maps stabilize in early steps and self-attention reflects refinement activity
    The partitioner and gate rely on this empirical property, citing [29,31]; no direct measurement is provided in this paper.

pith-pipeline@v1.3.0-alltime-deepseek · 28934 in / 12284 out tokens · 131853 ms · 2026-08-02T06:39:52.268751+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving." pith.science (2026). https://pith.science/paper/ODQ6JKCN

@misc{pith2026260712121,
  author       = {Pith},
  title        = {Pith review of: FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ODQ6JKCN}},
  note         = {Machine review of arXiv:2607.12121}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Diffusion models have become the central backbone for modern image, video, and audio generation, but their efficient service remains a challenge. Unlike autoregressive decoding, diffusion inference repeatedly updates high-dimensional spatial or temporal latents over many denoising steps. This all-region execution pattern makes generation latency high and limits serving throughput. Existing multi-GPU parallelization methods can reduce per-step computation, but often introduce substantial activation exchange overhead, causing communication to offset or even outweigh the benefits of parallel execution. This paper presents FlashDiff, a diffusion serving system that improves inference efficiency through adaptive regional execution and scheduling. FlashDiff is based on the observation that diffusion refinement is not uniform across latent regions or denoising steps: different regions often stabilize at different rates, while neighboring steps exhibit strong temporal correlation. FlashDiff leverages these properties to selectively execute only regions that require further refinement and to reallocate the resulting compute slack across concurrent serving requests. FlashDiff consists of three mechanisms. First, it decomposes the latent representation into coherent execution regions using early-stage attention signals, preserving semantic structure while exposing fine-grained parallelism. Second, it uses a lightweight runtime controller to estimate region activity and bypass low-impact updates when further refinement is unlikely to affect output quality. Third, it applies an affinity-aware online scheduler that co-locates dependent regions, balances residual load across GPUs, and reuses reclaimed compute capacity to improve serving efficiency. Across real-world image, video, and audio workloads, FlashDiff reduces end-to-end serving latency by 30-97% and improves throughput by 1.2-2.2x.

Figures

Figures reproduced from arXiv: 2607.12121 by Ayush Barik, Chensong Zhang, Fan Lai, Ping He, Songrun Xie, Yaqi Qiao, Zhengzhong Tu.

Figure 1
Figure 1. Figure 1: Diffusion models generate images, videos, and audio through many iterative steps. Each step refines all latent elements, making inference compute-intensive. 2 Background and Motivation 2.1 Diffusion Model Serving Diffusion models generate high-fidelity outputs via a reverse diffusion process, which gradually denoises an initial Gauss￾ian latent tensor x𝑇 into a coherent latent representation xˆ 0 over many… view at source ↗
Figure 4
Figure 4. Figure 4: Latent regions ex￾hibit heterogeneous refine￾ment needs. 10 2 10 3 10 4 10 5 10 6 Patch Size (Pixels) 0.0 0.2 0.4 0.6 0.8 1.0 CDF across Patches FLUX.1-dev Wan2.1 SD3 Medium [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 7
Figure 7. Figure 7: Semantic-aware latent partitioning workflow. gating, and (iii) systems efficiency, ensuring that the parti￾tioning logic itself does not introduce a prohibitive runtime tax or execution bubbles. However, meeting these needs reveals multifold chal￾lenges. Fine-grained partitioning (numerous micro-patches) maximizes the theoretical parallelism and skipping potential but imposes a substantial cross-patch sync… view at source ↗
Figure 8
Figure 8. Figure 8: Our partitioning outperforms the uniform par￾titioning method. 36 37 38 39 40 41 42 43 Number of Equivalent Steps 0 5 10 15 SSIM Gain (%) 0 10 20 30 40 LPIPS Reduction (%) [PITH_FULL_IMAGE:figures/full_fig_p005_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Patch monotask scheduler. Workers fetch tasks from a priority queue, where activated patches can preempt lower-priority ones for low latency and high throughput. scheduler [2]) while extracting maximal efficiency from avail￾able hardware. With the resource freed by patch skipping, we next introduce how the Patch Scheduler (1) enables patch switching efficiently, while (2) adapting to online serving dynami… view at source ↗
Figure 11
Figure 11. Figure 11: FlashDiff achieves higher serving throughput across a one-hour online deployment, generalizing across model architectures and modalities, whereas existing baselines are limited to specific tasks. 0 15 30 45 Time(minutes) 0 200 400 Request Completion Time (s) FlashDiff DistriFusion xDiT (a) FLUX.1-dev (T2I) 0 15 30 45 Time(minutes) 50 100 Request Completion Time (s) FlashDiff DistriFusion xDiT (b) SD3 Medi… view at source ↗
Figure 12
Figure 12. Figure 12: FlashDiff reduces end-to-end per-request latency. By reducing effective computation, FlashDiff lowers both raw generation time and queueing delays, especially under high load. We add ablation studies on arrival rates in Section 6.4. FlashDiff incurs little to no degradation in generation fi￾delity and often achieves slightly higher quality. By preserv￾ing cross-patch interactions through bidirectional att… view at source ↗
Figure 13
Figure 13. Figure 13: , FlashDiff introduces very marginal overhead, 0.4–3.7%, across all models and modalities. This overhead comes primarily from patch partitioning, gating, and mono￾task scheduling, which involve lightweight control logic and irregular memory accesses (e.g., for loading differ￾ent patches). By exploiting patch-level skipping and cross￾request backfilling, FlashDiff substantially improves genera￾tion latency… view at source ↗
Figure 15
Figure 15. Figure 15: FlashDiff performance breakdown by statistical efficiency, showing differences in skipped computation. yielding 45 and 95 effective gated steps, respectively. We include ablation studies on different denosing steps in Sec￾tion 6.4 [PITH_FULL_IMAGE:figures/full_fig_p010_15.png] view at source ↗
Figure 17
Figure 17. Figure 17: Impact of GPU numbers. latency and throughput, since requests can be served im￾mediately. FlashDiff continues to increase throughput and keeps latency bounded even under heavy load. At ×1.5 load, FlashDiff achieves up to 25× faster request completion. Impact of Number of GPUs [PITH_FULL_IMAGE:figures/full_fig_p010_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Impact of warm-up steps. 10 20 30 40 50 Total Inference Steps 0 2 4 6 8 10 Offline Throughput (req/min) 6.8 4.4 3.4 2.5 2.2 9.8 6.1 4.6 4.0 3.4 DistriFusion Throughput DistriFusion PSNR FlashDiff Throughput FlashDiff PSNR 18.5 19.0 19.5 20.0 20.5 21.0 PSNR (dB) [PITH_FULL_IMAGE:figures/full_fig_p011_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Impact of total denoising steps. 0.0 0.18 0.36 0.54 Epsilon ( ) 12 16 20 24 28 PSNR (dB) PSNR (dB) Skip Rate 0.0 0.2 0.4 0.6 0.8 1.0 Skip Rate [PITH_FULL_IMAGE:figures/full_fig_p011_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Quality under dif￾ferent 𝜖 thresholds. 5122 10242 15362 20482 Image Size 0 5 10 Offline Throughput (req/min) 8.1 6.5 3.7 2.2 11.2 9.4 8.1 3.4 DistriFusion FlashDiff [PITH_FULL_IMAGE:figures/full_fig_p011_20.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 1 canonical work pages

  1. [1]

    Adobe Firefly Team. 2025. Adobe Firefly: The next evolution of creative AI is here. Adobe Blog.https://blog.adobe.com/en/publish/2025/04/ 24/adobe-firefly-next-evolution-creative-ai-is-here

  2. [2]

    Shubham Agarwal, Subrata Mitra, Sarthak Chakraborty, Srikrishna Karanam, Koyel Mukherjee, and Shiv Kumar Saini. 2024. Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models. In21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). USENIX Association, Santa Clara, CA, 1173–1189.https:// www.usenix.org/conference...

  3. [3]

    Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, and Ramachandran Ram- jee. 2024. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve. In18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, Santa Clara, CA, 117–134.https://www.usenix...

  4. [4]

    Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligian- nidis. 2023. Nearly𝑑-linear convergence bounds for diffusion models via stochastic localization.arXiv preprint arXiv:2308.03686(2023)

  5. [5]

    Junsong Chen, Yuyang Zhao, Jincheng Yu, Ruihang Chu, Junyu Chen, Shuai Yang, Xianbang Wang, Yicheng Pan, Daquan Zhou, Huan Ling, Haozhe Liu, Hongwei Yi, Hao Zhang, Muyang Li, Yukang Chen, Han Cai, Sanja Fidler, Ping Luo, Song Han, and Enze Xie. 2025. SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer. arXiv:2509.24695 [cs.CV] d...

  6. [6]

    Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru R Zhang. 2022. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions.arXiv preprint arXiv:2209.11215(2022)

  7. [7]

    Zigeng Chen, Xinyin Ma, Gongfan Fang, Zhenxiong Tan, and Xinchao Wang. 2024. AsyncDiff: Parallelizing Diffusion Models by Asynchro- nous Denoising. InAdvances in Neural Information Processing Systems, Amir Globerson, Lester Mackey, Danielle Belgrave, Afra Fan, Ugo Paquet, Jakub Tomczak, and Cheng Zhang (Eds.), Vol. 37. Curran As- sociates, Inc.https://pro...

  8. [8]

    Jason Cramer, Ho-Hsiang Wu, Justin Salamon, and Juan Pablo Bello

  9. [9]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Fred- eric Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. 2024. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. arXiv:2403.03206 [cs.CV] doi...

  10. [10]

    Parker, C

    Zach Evans, Julian D. Parker, C. J. Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons. 2024. Stable Audio Open. arXiv:2407.14358 [cs.SD] doi:10.48550/arXiv.2407.14358

  11. [11]

    Jiarui Fang, Jinzhe Pan, Aoyu Li, Xibo Sun, and Jiannan Wang. 2025. PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transform- ers Inference. InAdvances in Neural Information Processing Systems (NeurIPS 2025). to appear.https://neurips.cc/virtual/2025/loc/san- diego/poster/119821NeurIPS 2025 poster. arXiv:2405.14430

  12. [12]

    Jiarui Fang, Jinzhe Pan, Xibo Sun, Aoyu Li, and Jiannan Wang. 2024. xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism.arXiv preprint arXiv:2411.01738(2024).https: //arxiv.org/abs/2411.01738

  13. [13]

    Deepanway Ghosal, Navonil Majumder, Ambuj Mehrish, and Sou- janya Poria. 2023. Text-to-Audio Generation using Instruction Guided Latent Diffusion Model. InProceedings of the 31st ACM International Conference on Multimedia (MM ’23). ACM, 3590–3598. doi:10.1145/ 3581783.3612348

  14. [14]

    Martin Gonzalez, Nelson Fernandez, Thuy Tran, Elies Gherbi, Hatem Hajri, and Nader Masmoudi. 2023. SEEDS: Exponen- tial SDE Solvers for Fast High-Quality Sampling from Diffu- sion Models. InAdvances in Neural Information Processing Sys- tems, Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.), Vol. 36. Curran As...

  15. [15]

    Susung Hong, Gyuseong Lee, Wooseok Jang, and Seungryong Kim

  16. [16]

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chan- paisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. 2024. VBench: Comprehensive Benchmark Suite for Video Generative Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

  17. [17]

    Vlado Keselj. 2009. Speech and Language Processing (second edition) Daniel Jurafsky and James H. Martin (Stanford University and Uni- versity of Colorado at Boulder) Pearson Prentice Hall, 2009, xxxi+988 pp; hardbound, ISBN 978-0-13-187321-6, $115.00.Computational Lin- guistics35, 3 (09 2009), 463–466. doi:10.1162/coli.B09-001

  18. [18]

    Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gun- hee Kim. 2019. AudioCaps: Generating Captions for Audios in The Wild. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minn...

  19. [19]

    Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexan- der C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. 2023. Seg- ment Anything. InProceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV). 4015–4026. arXiv:2304.02643 doi:10.48550/arXiv.2304.02643

  20. [20]

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Hongmei Wang, Jacob Song, Ji- awang Bai, Jianbing Wu, Jinbao Xue, Joey Wang, Kai Wang, Mengyang Liu, Pengyu Li, Shuai Li...

  21. [21]

    Khaled Koutini, Jan Schlueter, Hamid Eghbal-zadeh, and Gerhard Widmer. 2022. Efficient Training of Audio Transformers with Patchout. InProc. Interspeech 2022. 2753–2757. doi:10.21437/Interspeech.2022- 227

  22. [22]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica

  23. [23]

    Black Forest Labs. 2024. FLUX.https://github.com/black-forest-labs/ flux

  24. [24]

    Junyoung Lee, Seohyun Kim, Shinhyoung Jang, Jongho Park, and Yeseong Kim. 2025. Diffusion-Based Generative System Surrogates for Scalable Learning-Driven Optimization in Virtual Playgrounds. ACM SIGMETRICS Performance Evaluation Review53, 1 (2025), 43–45. doi:10.1145/3744970.3727282

  25. [25]

    InProceedings of the 29th Symposium on Operating Systems Principles (SOSP ’23)

    Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the 29th Symposium on Operating Systems Principles (SOSP ’23). ACM, Koblenz, Germany, 611–626. doi:10.1145/3600006.3613165 FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving

  26. [26]

    Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. 2023. Q-Diffusion: Quan- tizing Diffusion Models. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 17535–17545. doi:10.1109/ ICCV51070.2023.01608

  27. [27]

    Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. 2023. SnapFusion: Text- to-Image Diffusion Model on Mobile Devices within Two Seconds. In Advances in Neural Information Processing Systems. arXiv:2306.00980 doi:10.48550/arXiv.2306.00980

  28. [28]

    Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Kai Li, and Song Han. 2024. DistriFusion: Dis- tributed Parallel Inference for High-Resolution Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 7183–7193. doi:10.1109/CVPR52733.2024.00686

  29. [29]

    Bingyan Liu, Chengyu Wang, Tingfeng Cao, Kui Jia, and Jun Huang

  30. [30]

    Plumbley

    Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D. Plumbley. 2023. AudioLDM: Text-to-Audio Generation with Latent Diffusion Models. InProceed- ings of the 40th International Conference on Machine Learning (Pro- ceedings of Machine Learning Research, Vol. 202). PMLR, 21450–21474. arXiv:2301.12503 doi:10.48550/arXiv...

  31. [31]

    Gon- zalez, and Ion Stoica

    Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E. Gon- zalez, and Ion Stoica. 2023. AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving. In17th USENIX Sym- posium on Operating Systems Design and Implementation (OSDI 23). USENIX Association, Boston, ...

  32. [32]

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. 2022. DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. InAdvances in Neural Information Processing Systems, Sanmi Koyejo, Sidahmed Mohamed, Alekh Agarwal, Danielle Belgrave, Kyunghyun Cho, and Alice Oh (Eds.), Vol. 35. Curran Associates, Inc...

  33. [33]

    Ma, Ang Chen, and Mosharaf Chowdhury

    Runyu Lu, Shiqi He, Wenxuan Tan, Shenggui Li, Ruofan Wu, Jeff J. Ma, Ang Chen, and Mosharaf Chowdhury. 2025. TetriS- erve: Efficient DiT Serving for Heterogeneous Image Generation. arXiv:2510.01565 [cs.LG] doi:10.48550/arXiv.2510.01565arXiv v2 (last revised 13 Oct 2025)

  34. [34]

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. 2023. Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference. arXiv:2310.04378 [cs.CV]

  35. [35]

    Haozhe Liu, Wentian Zhang, Jinheng Xie, Francesco Faccio, Meng- meng Xu, Tao Xiang, Mike Zheng Shou, Juan-Manuel Perez-Rua, and Jürgen Schmidhuber. 2025. Faster Diffusion Through Temporal Atten- tion Decomposition.Transactions on Machine Learning Research(Feb. 2025).https://openreview.net/forum?id=xXs2GKXPnHPublished: 2025-02-26

  36. [36]

    2023.Get creative with generative AI in Performance Max

    Pallavi Naresh. 2023.Get creative with generative AI in Performance Max. Google Ads & Commerce Blog. https://blog.google/products/ads-commerce/get-creative-with- generative-ai-in-performance-max/

  37. [37]

    2025.NVIDIA TensorRT.https://github.com/NVIDIA/ TensorRTGitHub repository (tag v10.14, commit 3b4ddc1)

    NVIDIA. 2025.NVIDIA TensorRT.https://github.com/NVIDIA/ TensorRTGitHub repository (tag v10.14, commit 3b4ddc1). Accessed 2026-01-13

  38. [38]

    2026.TensorRT-LLM.https://github.com/NVIDIA/TensorRT- LLMGitHub repository (tag v1.2.0rc6.post1, commit e4a6c99)

    NVIDIA. 2026.TensorRT-LLM.https://github.com/NVIDIA/TensorRT- LLMGitHub repository (tag v1.2.0rc6.post1, commit e4a6c99). Ac- cessed 2026-01-13

  39. [39]

    Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li

  40. [40]

    Nobuyuki Otsu. 1979. A Threshold Selection Method from Gray-Level Histograms.IEEE Transactions on Systems, Man, and Cybernetics9, 1 (1979), 62–66. doi:10.1109/TSMC.1979.4310076

  41. [41]

    William Peebles and Saining Xie. 2023. Scalable Diffusion Models with Transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 4195–4205. arXiv:2212.09748 doi:10.48550/ arXiv.2212.09748

  42. [42]

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2024. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthe- sis. InThe Twelfth International Conference on Learning Representations (ICLR). arXiv:2307.01952 doi:10.48550/arXiv.2307.01952

  43. [43]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis With Latent Diffusion Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10684–10695. arXiv:2112.10752 doi:10.1109/CVPR52688.2022.01042

  44. [44]

    2024.Sora System Card

    OpenAI. 2024.Sora System Card. Technical Report. OpenAI.https: //openai.com/index/sora-system-card/System card for the Sora video generation model

  45. [45]

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. arXiv preprint arXiv:1909.08053(2019)

  46. [46]

    Jiaming Song, Chenlin Meng, and Stefano Ermon. 2021. Denoising Diffusion Implicit Models. InInternational Conference on Learning Rep- resentations (ICLR). OpenReview.net. arXiv:2010.02502 doi:10.48550/ arXiv.2010.02502

  47. [47]

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. 2023. Consistency Models. InProceedings of the 40th International Confer- ence on Machine Learning (Proceedings of Machine Learning Research, Vol. 202). PMLR, 32211–32252.https://proceedings.mlr.press/v202/ song23a.html

  48. [48]

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Ku- mar, Stefano Ermon, and Ben Poole. 2020. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456(2020)

  49. [49]

    Silvestru Sever and Silvestru Dragomir. 2002. Some Gronwall type inequalities and applications. (12 2002)

  50. [50]

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, T...

  51. [51]

    Yuqing Wang, Ye He, and Molei Tao. 2024. Evaluating the Design Space of Diffusion-Based Generative Models. InAd- vances in Neural Information Processing Systems 37 (NeurIPS 2024).https://proceedings.neurips.cc/paper_files/paper/2024/file/ 227404a13d20898dec2018ebe368b202-Paper-Conference.pdf

  52. [52]

    Bovik, Hamid R

    Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli

  53. [53]

    Wang, Evan Montoya, David Munechika, Haoyang Yang, Ben- jamin Hoover, and Duen Horng Chau

    Zijie J. Wang, Evan Montoya, David Munechika, Haoyang Yang, Ben- jamin Hoover, and Duen Horng Chau. 2023. DiffusionDB: A Large- scale Prompt Gallery Dataset for Text-to-Image Generative Models. In Proceedings of the 61st Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), Anna Rogers, Jordan Boyd- Graber, and Naoaki ...

  54. [54]

    Suno. 2024. Suno has raised $125 million to build a future where any- one can make music. Suno Blog.https://suno.com/blog/fundraising- announcement-may-2024

  55. [55]

    Yuchen Xia, Divyam Sharma, Yichao Yuan, Souvik Kundu, and Nishil Talati. 2026. MoDM: Efficient Serving for Image Generation via Mixture-of-Diffusion Models. InProceedings of the 31st ACM Interna- tional Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’26). to appear.https://arxiv.org/abs/ 2503.11972To appear in ...

  56. [56]

    Freeman, and Taesung Park

    Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Du- rand, William T. Freeman, and Taesung Park. 2024. One-step Diffusion with Distribution Matching Distillation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6613–

  57. [57]

    Yifan Yu, Yu Gan, Nikhil Sarda, Lillian Tsai, Jiaming Shen, Yanqi Zhou, Arvind Krishnamurthy, Fan Lai, Hank Levy, and David E. Culler. 2025. IC-Cache: Efficient Large Language Model Serving via In-context Caching. InProceedings of the ACM SIGOPS 31st Sym- posium on Operating Systems Principles (SOSP 2025). ACM, 375–398. doi:10.1145/3731569.3764829

  58. [58]

    Efros, Eli Shechtman, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 586–595. doi:10.1109/CVPR. 2018.00068

  59. [59]

    Wei Zhang, Zhiyu Wu, Yi Mu, Rui Ning, Banruo Liu, Nikhil Sarda, Myungjin Lee, and Fan Lai. 2026. JITServe: SLO-aware LLM Serving with Imprecise Request Information. In23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI ’26). Renton, WA, USA, to appear.https://arxiv.org/abs/2504.20068Accepted to NSDI

  60. [60]

    Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg- Kirkpatrick, and Shlomo Dubnov. 2023. Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to- Caption Augmentation. InICASSP 2023 – 2023 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5. doi:10.1109/ICASSP49357.2023.10095969

  61. [61]

    Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Azizzadenesheli, and Anima Anandkumar. 2023. Fast Sampling of Diffusion Models via Operator Learning. InProceedings of the 40th International Confer- ence on Machine Learning (Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskill, Kyunghyun Cho, Bar- bara Engelhardt, Sivan Sabato...

  62. [62]

    ideal coherence

    Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xu- anzhe Liu, Xin Jin, and Hao Zhang. 2024. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving. In18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, Santa Clara, CA, 193– 210.https://www.usenix.org/co...

  63. [67]

    Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li. 2023. Py- Torch FSDP: Experiences on Scaling Fully Sharded Data Parallel. Proceedings of the VLDB Endowment16, ...

  64. [2004]

    doi:10.1109/TIP.2003.819861

    Image Quality Assessment: From Error Visibility to Structural Similarity.IEEE Transactions on Image Processing13, 4 (April 2004), 600–612. doi:10.1109/TIP.2003.819861

  65. [2019]

    InICASSP 2019 – 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Look, Listen, and Learn More: Design Choices for Deep Audio Embeddings. InICASSP 2019 – 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, Brighton, UK, 3852–3856. doi:10.1109/ICASSP.2019.8682475

  66. [2023]

    InProceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV)

    Improving Sample Quality of Diffusion Models Using Self- Attention Guidance. InProceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV). 7462–7471. doi:10.1109/ICCV51070. 2023.00686

  67. [2024]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Towards Understanding Cross and Self-Attention in Stable Dif- fusion for Text-Guided Image Editing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 7817–7826.https://openaccess.thecvf.com/content/CVPR2024/ papers/Liu_Towards_Understanding_Cross_and_Self-Attention_in_ Stable_Diffusion_for_Text-Guided_CVPR_2024_paper.pdf

  68. [2025]

    InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    HPSv3: Towards Wide-Spectrum Human Preference Score. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 15086–15095.https://openaccess. thecvf.com/content/ICCV2025/html/Ma_HPSv3_Towards_Wide- Spectrum_Human_Preference_Score_ICCV_2025_paper.html

  69. [6623]

    arXiv:2311.18828 doi:10.1109/CVPR52733.2024.00632