Pith. sign in

REVIEW 4 major objections 6 minor 39 references

SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By replacing SiLU with ReLU, quantizing most weights and activations to 4 bits, and exploiting temporal per-channel activation sparsity, this paper reports diffusion image generation at 6.91x speed-up with 51.5% energy saving and better…

desk verdict A genuinely interesting temporal-sparsity observation, but the headline quality claim needs the full-precision ReLU baseline to actually hold. read the letter →

arxiv 2501.15448 v1 pith:6SLAUFJF submitted 2025-01-26 cs.CV cs.AIcs.ARcs.LG

Zichen Fan , Steve Dai , Rangharajan Venkatesan , Dennis Sylvester , Brucek Khailany This is my paper · ORCID
classification cs.CVcs.AIcs.ARcs.LG
keywords diffusionmodels4-bitquantizationactivationsparsitytemporalmixed-precisionReLUhardwareacceleratorimagegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that diffusion image generators can be pushed down to 4-bit weights and activations without losing quality, if the activation function is also changed. The authors replace SiLU with ReLU and finetune briefly, which makes activations non-negative so unsigned 4-bit uses all its levels, and they keep only the most sensitive outer blocks at 8-bit. Across four image datasets, the resulting image-quality scores beat previously published 4-bit formats. The same ReLU swap also creates a temporal per-channel sparsity pattern, and the paper's accelerator exploits it by sending sparse channels to sparse hardware and dense channels to dense hardware, reaching a 6.91x speed-up and 51.5% energy saving over a dense FP16 baseline in simulation. If true, this is a practical recipe for fast diffusion inference without high-precision side branches.

What carries the argument

The load-bearing mechanism is the pairing of ReLU with unsigned 4-bit quantization plus the temporal per-channel sparsity pattern it induces. ReLU clamps negative activations to zero, which both eliminates the signed-format quantization under-utilization (only 10 of 16 INT4 levels are used under SiLU) and creates an average 65% activation sparsity. The sparsity is not random: for a given layer, each channel tends to be either sparse or dense at a given time step, and that label flips over the denoising schedule. The accelerator stores activations channel-last, maintains a per-channel dense/sparse index updated by a time-step-aware detector at every step, and computes sparse channels on sparse processing elements while dense channels run on dense elements, with weights kept dense throughout.

What would settle it

A reader could settle the quality claim by computing FID for the finetuned ReLU-based EDM at FP16/FP32 on CIFAR-10 and comparing it to the SiLU FP16 FID of 1.85; if the ReLU model is materially worse, the 4-bit FID scores in Table II are measured against a degraded baseline and the central quality claim does not hold.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that diffusion models do not need high-precision side branches or activation smoothing to survive 4-bit quantization: replacing SiLU with a finetuned ReLU makes activations non-negative, so unsigned 4-bit uses all quantization levels, and keeping only the small fraction of quantization-sensitive first and last blocks at 8-bit restores most of the quality loss. Across EDM1 on CIFAR-10, AFHQv2, FFHQ and EDM2 on ImageNet, the resulting FID scores (e.g., 2.12 versus 15.60 for INT4-VSQ on CIFAR-10) are the best among compared 4-bit formats. The same ReLU activation also pushes average activation sparsity from about 10% to 65%, and the observed sparsity is temporally per-channel: channels alternate between dense and sparse across denoising time steps. The paper's accelerator groups channels by this label, routes them to dense or sparse processing elements, updates the label every time step, and reports a 6.91x total speed-up and 51.5% energy saving over a dense FP16 baseline.

Load-bearing premise

The argument assumes the finetuned ReLU-based EDM matches the original SiLU-based model's image quality, but no full-precision FID for the ReLU model is reported in the paper.

Editorial extensions

If this is right

  • A 4-bit diffusion model becomes practical without low-rank FP16 branches or activation smoothing, since the paper reports quality better than prior 4-bit methods while quantizing the original weights and activations directly.
  • Only a small fraction of compute (about 5%) needs to stay at 8-bit, so the memory and compute savings remain close to what 4-bit precision promises.
  • The 3.78x speed-up from quantization and the 1.83x speed-up from temporal sparsity multiply to the reported 6.91x total, showing the two optimizations combine rather than overlap.
  • The dense/sparse partitioning can be refreshed at every denoising time step without meaningful overhead, so the hardware stays matched to the evolving sparsity distribution.
  • Since the paper treats EDM as representative of convolution-based diffusion backbones, the same combination should transfer to text-to-image and weather-downscaling models built on EDM.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension left implicit: because the channel sparsity label drifts predictably along the noise schedule, a compiler could pre-classify channels from the time step instead of detecting them on chip, saving the detection logic entirely.
  • The quantization-level argument for ReLU is not specific to U-Nets, so the same finetune-and-quantize recipe could be tried on SiLU-based transformer or latent diffusion backbones, though the per-channel sparsity statistics would need to be re-measured.
  • If the simulated 51.5% energy saving holds in silicon, it implies roughly doubling the number of denoising steps that fit in a fixed energy budget, which could be spent on higher-resolution or higher-quality generation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes SQ-DM, a software-hardware co-design for accelerating EDM/EDM2 diffusion models. On the model side, it replaces SiLU with ReLU and finetunes, applies mixed-precision quantization (4-bit with FP8 scale factors for most blocks, MXINT8 for sensitive blocks), and exploits ReLU-induced temporal per-channel activation sparsity. On the hardware side, it presents a dense/sparse heterogeneous PE array with channel-last address mapping and a time-step-aware sparsity detector, evaluated with Stonne at 28nm. The reported results are FID comparisons in Tables I-II and speedup/energy numbers in Figure 12, culminating in a 6.91x speedup and 51.5% energy saving over a dense FP16 baseline.

Significance. If the claims hold, the paper would be a useful co-design recipe: it identifies that ReLU promotes hardware-friendly structured activation sparsity and that only a small fraction of blocks need higher precision, and it proposes concrete architectural mechanisms (channel-last mapping, temporal sparsity detector). The evaluation is a mix of real FID measurements and cycle-level simulation, so the central claims are not circularly derived. However, the current evidence is incomplete: the quality comparison lacks a full-precision ReLU control and the leading 4-bit baseline SVDquant is absent, and the hardware numbers are from a single simulated configuration. The paper is therefore a promising work-in-progress rather than a fully supported state-of-the-art claim.

major comments (4)
  1. [Section III-B, Table II] The claim in Section III-B that the finetuned ReLU-based model 'achieves similar image quality to the original SiLU-based model' is never backed by a reported FID for the full-precision ReLU model. Table II compares Ours(MP+ReLU) only against SiLU-based baselines; the gap between Ours(MP+ReLU) (e.g., 2.12 on CIFAR-10) and FP32 SiLU (1.85) cannot be decomposed into degradation from the ReLU finetune versus degradation from 4-bit/mixed-precision quantization. Since the abstract's 'superior generation quality' and the overall quality comparison rest on this point, the paper must provide the FP16/FP32 FID of the ReLU model after the same finetuning procedure and report the quantization-only drop relative to that control.
  2. [Section III-A, Table II, Abstract] The headline '4-bit quantization technique' is not what Table II evaluates. Ours(MP+ReLU) keeps quantization-sensitive blocks in MXINT8 (Section III-A), so it is a mixed-precision scheme; comparing it to uniform INT4-VSQ conflates bit-width with architecture and precision assignment. Please add a uniform INT4+ReLU row (or report Ours(MP-only) and Ours(MP+ReLU) against a uniform-4-bit ReLU model) and adjust the abstract and contributions accordingly. In addition, SVDquant [13], which is cited in Section II-A as an existing 4-bit method, is missing from Table II; without it, the claim of 'superior generation quality compared to existing 4-bit methods' is not fully evidenced.
  3. [Section IV.D, Figure 12] The 6.91x and 51.5% figures are based on Stonne simulations with exactly one DPE and one SPE (128 multipliers each) and a baseline of two DPEs. The paper reports no sensitivity to the number of PEs, the dense/sparse partition, memory bandwidth, or interconnect, even though the speedup depends on balancing dense and sparse workloads (Section IV-C). The 51.5% energy saving is attributed solely to temporal sparsity in Figure 12, yet the abstract presents it as the accelerator's total energy reduction; the total energy saving including quantization effects should be reported. The 3.78x quantization speedup and 1.83x sparsity speedup are multiplied to get 6.91x without explicitly validating that the two effects overlap without penalty; a combined simulation with the full architecture should be reported.
  4. [Section II-A, Section III-A] The paper states in Section II-A that it 'directly quantize[s] both the original weights and activations to 4-bit ... without smoothing or using any high-precision floating-point components,' but the proposed scheme keeps roughly 5% of computation in MXINT8. This is internally consistent only if MXINT8 is not counted as high precision; please clarify the precision terminology and quantify the overhead of these blocks in the speedup and energy results.
minor comments (6)
  1. [References] Reference [13] has a typo in the title: 'Svdqunat' should be 'SVDquant'.
  2. [Table I] The INT4 rows show FID values above 100 (e.g., 136.5, 289.3, 244.8, 346.2). Consider discussing whether FID remains meaningful at these values or presenting the comparison on a log scale.
  3. [Section IV.C] The 30% sparsity threshold is selected based on a single analysis plot (Figure 11, left) without a sensitivity study. Reporting FID and speedup for a small range of thresholds (e.g., 20%, 40%) would strengthen the robustness of the design choice.
  4. [Figure 12] The figure lacks axis labels and units. Please specify whether the reported energy includes memory, control logic, and sparsity-detection overhead, and state the exact baseline configuration used for the energy comparison.
  5. [General] No code or model release is mentioned. Adding a reproducibility statement or releasing the finetuned ReLU models and simulation scripts would help verify the FID and speedup claims.
  6. [Table II] The EDM2 row appears only for ImageNet; clarify the training and sampling configuration for EDM2 and whether the same ReLU finetuning procedure was applied.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: FID and speedup are measured/simulated; minor self-citations are not load-bearing.

full rationale

The paper's central claims — 4-bit quantization quality and the 6.91x speedup / 51.5% energy reduction — are empirical measurements or Stonne simulations, not quantities derived from their own inputs by construction. Table II reports FID scores from generated images, and Figure 12 reports cycle/energy simulation results against a fixed dense FP16 baseline. The 30% sparsity threshold and the every-timestep update schedule are tuned design choices; tuning a parameter and then reporting the outcome of that configuration is a design optimization, not a circular prediction. The self-citations to VS-Quant [6] and MAGNet [27] are used as a comparison baseline and an architecture generator respectively, and neither is invoked to prove the central quality or speedup claims. The ReLU-finetune premise — 'The resulting ReLU-based model achieves similar image quality to the original SiLU-based model' — is asserted without a full-precision ReLU FID table, which is a genuine evidence gap for attributing the quality improvement specifically to quantization, but it is not circular: no equation defines the 4-bit FID in terms of the SiLU FID, and no fitted parameter is renamed as a prediction. Overall, no circular step meeting the quoted-reduction standard is present; the only ground for a nonzero score is the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

Central claims depend on several design choices and simulation assumptions: a 30% sparsity threshold, the choice of which blocks stay at 8-bit, a per-timestep sparsity update interval, and FP8 scale factors. The hardware results rest on Stonne simulation rather than measurement. No new physical entities are introduced.

free parameters (4)
  • dense/sparse channel threshold = 30%
    Channels with activation sparsity above 30% are treated as sparse; threshold chosen from Figure 11 to balance DPE/SPE workloads, not derived.
  • high-precision block set = ~5% of total cost (first and last blocks at MXINT8)
    Selected from the block-wise sensitivity experiment in Figure 3; a design choice that trades quality for speedup.
  • sparsity update interval = 1 time step
    Chosen to maximize categorization accuracy; paper states overhead is negligible and hidden behind computation.
  • INT4 data format scale factors = FP8 per-vector scales
    Chosen to improve dynamic range versus INT4-VSQ format; a design decision, not derived from first principles.
assumptions (4)
  • domain assumption Computational equivalence of 1 FP16 multiplication to 2 INT8 to 4 INT4 multiplications (based on [23]) is used to compute computation and memory savings.
    This equivalence underpins the 73% computation saving and speedup figures; if the memory/compute ratio changes, the speedup changes.
  • domain assumption Stonne cycle-level simulation at 28nm faithfully models latency and energy of the proposed DPE/SPE and the dense baseline.
    All speedup and energy numbers come from simulation, not silicon; Stonne's accuracy for this specific architecture is not validated in the paper.
  • domain assumption ReLU finetuned model retains original image quality (no FID table provided).
    The FID comparison implicitly assumes the activation change alone does not degrade quality; this premise is load-bearing and unquantified.
  • domain assumption The temporal per-channel sparsity pattern observed in one layer (Figure 7) holds across layers, datasets, and time steps sufficiently to deliver 1.83x speedup.
    The hardware speedup depends on the measured distribution of sparse/dense channels; no ablation across all layers and datasets is shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity." pith.science (2026). https://pith.science/paper/6SLAUFJF

@misc{pith2026250115448,
  author       = {Pith},
  title        = {Pith review of: SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6SLAUFJF}},
  note         = {Machine review of arXiv:2501.15448}
}
read the original abstract

Diffusion models have gained significant popularity in image generation tasks. However, generating high-quality content remains notably slow because it requires running model inference over many time steps. To accelerate these models, we propose to aggressively quantize both weights and activations, while simultaneously promoting significant activation sparsity. We further observe that the stated sparsity pattern varies among different channels and evolves across time steps. To support this quantization and sparsity scheme, we present a novel diffusion model accelerator featuring a heterogeneous mixed-precision dense-sparse architecture, channel-last address mapping, and a time-step-aware sparsity detector for efficient handling of the sparsity pattern. Our 4-bit quantization technique demonstrates superior generation quality compared to existing 4-bit methods. Our custom accelerator achieves 6.91x speed-up and 51.5% energy reduction compared to traditional dense accelerators.

Figures

Figures reproduced from arXiv: 2501.15448 by the authors.

Figure 1
Figure 1. Generated images and achieved speed-up for different [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Execution process and model architecture of EDM [4], [5]. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Block-wise quantization sensitivity for EDM model. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: EDM model computation and memory breakdown. [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: Comparison of activation data distributions at the output [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: SiLU(x) after INT4 quantization versus ReLU(x) after UINT4 quantization. activation data distribution of one Conv + SiLU layer where SiLU(x) = x/(1 + e −x ) [24]. At the output of the SiLU non￾linear function, the data distribution spans from [−0.278, ∞). The existence…
Figure 7
Figure 7. Figure 7: Temporal per-channel sparsity pattern [PITH_FULL_IMAGE:figures/full_fig_p004_7.png]
Figure 8
Figure 8. Figure 8: New computation scheme for temporal per-channel [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]
Figure 9
Figure 9. Figure 9: Overall architecture of our diffusion model accelerator. [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]
Figure 10
Figure 10. Figure 10: Channel-last data-address mapping. Finally, we propose a temporal sparsity update schedul￾ing mechanism to address the dynamic per-channel sparsity observed across time steps [PITH_FULL_IMAGE:figures/full_fig_p006_10.png]
Figure 12
Figure 12. Figure 12: (bottom) illustrates the total speed-up compared to an FP16 SiLU-based diffusion model. Our 4-bit quantization contributes to 3.78× speed-up, while our temporal per-channel sparsity adds 1.83× speed-up on top of quantization. In combination, our proposed model optimiz…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 21 canonical work pages

  1. [13]

    Svdqunat: Absorbing outliers by low-rank com- ponents for 4-bit diffusion models,

    M. Li, Y . Lin, Z. Zhang, T. Cai, X. Li, J. Guo, E. Xie, C. Meng, J.- Y . Zhu, and S. Han, “Svdqunat: Absorbing outliers by low-rank com- ponents for 4-bit diffusion models,” arXiv preprint arXiv:2411.05007 , 2024

    arXiv 2024

  2. [1]

    ediff-i: Text-to-image diffu- sion models with an ensemble of expert denoisers,

    Y . Balaji, S. Nah, X. Huang, A. Vahdat, J. Song, Q. Zhang, K. Kreis, M. Aittala, T. Aila, S. Laine, et al. , “ediff-i: Text-to-image diffu- sion models with an ensemble of expert denoisers,” arXiv preprint arXiv:2211.01324, 2022

    arXiv 2022

  3. [2]

    Align your latents: High-resolution video synthesis with latent diffusion models,

    A. Blattmann, R. Rombach, H. Ling, T. Dockhorn, S. W. Kim, S. Fidler, and K. Kreis, “Align your latents: High-resolution video synthesis with latent diffusion models,” pp. 22563–22575, 2023

    work page 2023

  4. [3]

    Generative residual diffusion modeling for km-scale atmospheric downscaling,

    M. Mardani, N. Brenowitz, Y . Cohen, J. Pathak, C.-Y . Chen, C.-C. Liu, A. Vahdat, K. Kashinath, J. Kautz, and M. Pritchard, “Generative residual diffusion modeling for km-scale atmospheric downscaling,” arXiv preprint arXiv:2309.15214 , 2023

    arXiv 2023

  5. [4]

    Elucidating the design space of diffusion-based generative models,

    T. Karras, M. Aittala, T. Aila, and S. Laine, “Elucidating the design space of diffusion-based generative models,” Advances in neural infor- mation processing systems , vol. 35, pp. 26565–26577, 2022

    work page 2022

  6. [5]

    Analyzing and improving the training dynamics of diffusion models,

    T. Karras, M. Aittala, J. Lehtinen, J. Hellsten, T. Aila, and S. Laine, “Analyzing and improving the training dynamics of diffusion models,” arXiv preprint arXiv:2312.02696 , 2023

    arXiv 2023

  7. [6]

    Vs-quant: Per-vector scaled quantization for accurate low-precision neural network inference,

    S. Dai, R. Venkatesan, M. Ren, B. Zimmer, W. Dally, and B. Khailany, “Vs-quant: Per-vector scaled quantization for accurate low-precision neural network inference,” Proceedings of Machine Learning and Sys- tems, vol. 3, pp. 873–884, 2021

    2021

  8. [7]

    Post-training quantiza- tion on diffusion models,

    Y . Shang, Z. Yuan, B. Xie, B. Wu, and Y . Yan, “Post-training quantiza- tion on diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 1972–1981, 2023

    work page 1972

Show all 39 references
  1. [8]

    Q-diffusion: Quantizing diffusion models,

    X. Li, Y . Liu, L. Lian, H. Yang, Z. Dong, D. Kang, S. Zhang, and K. Keutzer, “Q-diffusion: Quantizing diffusion models,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , p. 17489–17499, IEEE, Oct. 2023

    2023

  2. [9]

    Ptqd: Accurate post-training quantization for diffusion models,

    Y . He, L. Liu, J. Liu, W. Wu, H. Zhou, and B. Zhuang, “Ptqd: Accurate post-training quantization for diffusion models,” 2023

    2023

  3. [10]

    Efficient quantization strategies for latent diffusion models,

    Y . Yang, X. Dai, J. Wang, P. Zhang, and H. Zhang, “Efficient quantization strategies for latent diffusion models,” arXiv preprint arXiv:2312.05431, 2023

    2023 arXiv

  4. [11]

    Tmpq-dm: Joint timestep reduction and quantization precision selection for efficient diffusion models,

    H. Sun, C. Tang, Z. Wang, Y . Meng, X. Ma, W. Zhu, et al., “Tmpq-dm: Joint timestep reduction and quantization precision selection for efficient diffusion models,” arXiv preprint arXiv:2404.09532 , 2024

    2024 arXiv

  5. [12]

    Tfmq-dm: Temporal feature maintenance quantization for diffusion models,

    Y . Huang, R. Gong, J. Liu, T. Chen, and X. Liu, “Tfmq-dm: Temporal feature maintenance quantization for diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 7362–7371, 2024

    2024

  6. [14]

    Accelerating sparse deep neural networks,

    A. Mishra, J. A. Latorre, J. Pool, D. Stosic, D. Stosic, G. Venkatesh, C. Yu, and P. Micikevicius, “Accelerating sparse deep neural networks,” arXiv preprint arXiv:2104.08378 , 2021

    2021 arXiv

  7. [15]

    Structural pruning for diffusion models,

    G. Fang, X. Ma, and X. Wang, “Structural pruning for diffusion models,” 2023

    2023

  8. [16]

    Sparsedm: Toward sparse efficient diffusion models,

    K. Wang, J. Chen, H. Li, Z. Mi, and J. Zhu, “Sparsedm: Toward sparse efficient diffusion models,” 2024

    2024

  9. [17]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton, et al., “Learning multiple layers of features from tiny images,” 2009

    2009

  10. [18]

    Stargan v2: Diverse image synthesis for multiple domains,

    Y . Choi, Y . Uh, J. Yoo, and J.-W. Ha, “Stargan v2: Diverse image synthesis for multiple domains,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 8188–8197, 2020

    2020

  11. [19]

    A style-based generator architecture for generative adversarial networks,

    T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 4401–4410, 2019

    2019

  12. [20]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition , pp. 248–255, Ieee, 2009

    2009

  13. [21]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems , vol. 30, 2017

    2017

  14. [22]

    Microscaling data formats for deep learning,

    B. D. Rouhani, R. Zhao, A. More, M. Hall, A. Khodamoradi, S. Deng, D. Choudhary, M. Cornea, E. Dellinger, K. Denolf, et al., “Microscaling data formats for deep learning,” arXiv preprint arXiv:2310.10537, 2023

    2023 arXiv

  15. [23]

    Nvidia blackwell platform: Advancing generative ai and accelerated computing,

    A. Tirumala and R. Wong, “Nvidia blackwell platform: Advancing generative ai and accelerated computing,” in 2024 IEEE Hot Chips 36 Symposium (HCS), pp. 1–33, IEEE Computer Society, 2024

    2024

  16. [24]

    Sigmoid-weighted linear units for neural network function approximation in reinforcement learning,

    S. Elfwing, E. Uchibe, and K. Doya, “Sigmoid-weighted linear units for neural network function approximation in reinforcement learning,” Neural networks, vol. 107, pp. 3–11, 2018

    2018

  17. [25]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural informa- tion processing systems , vol. 25, 2012

    2012

  18. [26]

    The sparsity roofline: Understanding the hardware limits of sparse neural networks,

    C. Shinn, C. McCarthy, S. Muralidharan, M. Osama, and J. D. Owens, “The sparsity roofline: Understanding the hardware limits of sparse neural networks,” arXiv preprint arXiv:2310.00496 , 2023

    2023 arXiv

  19. [27]

    Magnet: A modular accelerator generator for neural networks,

    R. Venkatesan, Y . S. Shao, M. Wang, J. Clemons, S. Dai, M. Fojtik, B. Keller, A. Klinefelter, N. Pinckney, P. Raina, et al. , “Magnet: A modular accelerator generator for neural networks,” in 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1–8, IEEE, 2019

    2019

  20. [28]

    In-datacenter performance analysis of a tensor processing unit,

    N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers, et al. , “In-datacenter performance analysis of a tensor processing unit,” in Proceedings of the 44th annual international symposium on computer architecture , pp....

    2017

  21. [29]

    Maeri: Enabling flexible dataflow mapping over dnn accelerators via reconfigurable intercon- nects,

    H. Kwon, A. Samajdar, and T. Krishna, “Maeri: Enabling flexible dataflow mapping over dnn accelerators via reconfigurable intercon- nects,” ACM SIGPLAN Notices , vol. 53, no. 2, pp. 461–475, 2018

    2018

  22. [30]

    Eie: Efficient inference engine on compressed deep neural network,

    S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, “Eie: Efficient inference engine on compressed deep neural network,” ACM SIGARCH Computer Architecture News , vol. 44, no. 3, pp. 243–254, 2016

    2016

  23. [31]

    Cambricon-x: An accelerator for sparse neural networks,

    S. Zhang, Z. Du, L. Zhang, H. Lan, S. Liu, L. Li, Q. Guo, T. Chen, and Y . Chen, “Cambricon-x: An accelerator for sparse neural networks,” in 2016 49th Annual IEEE/ACM International Symposium on Microarchi- tecture (MICRO), pp. 1–12, IEEE, 2016

    2016

  24. [32]

    Sparten: A sparse tensor accelerator for convolutional neural networks,

    A. Gondimalla, N. Chesnut, M. Thottethodi, and T. Vijaykumar, “Sparten: A sparse tensor accelerator for convolutional neural networks,” in Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, pp. 151–165, 2019

    2019

  25. [33]

    Extensor: An accelerator for sparse tensor algebra,

    K. Hegde, H. Asghari-Moghaddam, M. Pellauer, N. Crago, A. Jaleel, E. Solomonik, J. Emer, and C. W. Fletcher, “Extensor: An accelerator for sparse tensor algebra,” in Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture , pp. 319–333, 2019

    2019

  26. [34]

    Sigma: A sparse and irregular gemm ac- celerator with flexible interconnects for dnn training,

    E. Qin, A. Samajdar, H. Kwon, V . Nadella, S. Srinivasan, D. Das, B. Kaul, and T. Krishna, “Sigma: A sparse and irregular gemm ac- celerator with flexible interconnects for dnn training,” in 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp...

    2020

  27. [35]

    Tensaurus: A versatile accelerator for mixed sparse-dense tensor com- putations,

    N. Srivastava, H. Jin, S. Smith, H. Rong, D. Albonesi, and Z. Zhang, “Tensaurus: A versatile accelerator for mixed sparse-dense tensor com- putations,” in 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 689–702, IEEE, 2020

    2020

  28. [36]

    Griffin: Rethinking sparse optimization for deep learning architectures,

    J. H. Shin, A. Shafiee, A. Pedram, H. Abdel-Aziz, L. Li, and J. Hassoun, “Griffin: Rethinking sparse optimization for deep learning architectures,” in 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pp. 861–875, IEEE, 2022

    2022

  29. [37]

    Enabling flexibility for sparse tensor acceleration via heterogeneity,

    E. Qin, R. Garg, A. Bambhaniya, M. Pellauer, A. Parashar, S. Rajaman- ickam, C. Hao, and T. Krishna, “Enabling flexibility for sparse tensor acceleration via heterogeneity,” arXiv preprint arXiv:2201.08916, 2022

    2022 arXiv

  30. [38]

    Stonne: Enabling cycle-level microarchitectural simulation for dnn inference accelerators,

    F. Mu ˜noz-Mart´ınez, J. L. Abell ´an, M. E. Acacio, and T. Krishna, “Stonne: Enabling cycle-level microarchitectural simulation for dnn inference accelerators,” in 2021 IEEE International Symposium on Workload Characterization (IISWC), pp. 201–213, IEEE, 2021

    2021

  31. [39]

    Video diffusion models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,” Advances in Neural Information Processing Systems, vol. 35, pp. 8633–8646, 2022

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.