Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Huff-LLM: End-to-End Lossless Compression for Efficient LLM Inference

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Huffman-coding 16-bit LLM weights field-by-field achieves lossless compression with faster, cheaper inference.

desk verdict Genuinely useful bit-split Huffman compression for LLM weights, but the latency/energy numbers hang on an unproven single-cycle decoder and a missing FPGA prototype. read the letter →

arxiv 2502.00922 v1 pith:7MWEPOGI submitted 2025-02-02 cs.LG cs.AR

Patrick Yubeaton , Tareq Mahmoud , Shehab Naga , Pooria Taheri , Tianhua Xia , Arun George , Yasmein Khalil , Sai Qian Zhang
show 3 more authors
Siddharth Joshi Chinmay Hegde Siddharth Garg
This is my paper · ORCID
classification cs.LGcs.AR
keywords losslesscompressionHuffmancodingLLMinferencehardwareacceleratorsystolicarraycontent-addressablememoryedgedeploymentweight
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Huff-LLM sets out to show that lossless compression can be an end-to-end solution for large language model deployment rather than just a download-time nicety. The authors claim that by Huffman-coding each 16-bit floating-point weight as four separate fields — the sign bit plus three 5-bit groups — and decoding each group with a one-cycle CAM lookup inside the accelerator, weights can stay compressed in disk, DRAM, and on-chip buffers until the instant they multiply with activations. On simulated systolic-array and vector-style accelerators they report model-size reductions of 15–32%, latency improvements up to 31%, and energy savings up to 26%, at under 6% area overhead. The draw is that none of this changes model outputs: decompression is bit-exact, so a compressed model behaves identically to the original. If the single-cycle decoder assumption holds, edge devices get larger effective memory and faster inference at no accuracy cost.

What carries the argument

The load-bearing object is the 1-5-5-5 split Huffman code plus a single-cycle decoder built on a 32-entry content-addressable memory (CAM; a memory that compares an input against all stored entries at once). Each CAM entry stores one codeword, its 5-bit decoded source value, and the codeword length; matching bits in a register selects the entry, outputs the source bits, and updates a start pointer by the length. Three such decoders handle the exponent and two mantissa fields while the sign bit bypasses compression, and the reconstructed 16-bit value is concatenated before the multiply-accumulate units. The work this does is to eliminate variable-length decoding as a source of pipeline bubbles: the decoder is inserted as an extra pipeline stage that consumes compressed weight bits and produces one standard-format weight per clock, so the accelerator's synchronization is preserved.

What would settle it

Measure the decoder's combinational delay on the target process or an FPGA test at 1–2 GHz, and count stall cycles when the codebook or weight matrix changes. If the match-plus-refill path exceeds one cycle, or if codebook switching inserts bubbles, the simulated 26–31% latency savings are eroded by the ratio of stalls to total weight fetches.

Watch

Extended reading notes

Core claim

The paper's central claim is that Huffman coding becomes practical and inexpensive for LLM inference when it is applied per bit-field rather than to whole weights. An FP16 weight is split as 1 sign bit (stored raw), 5 exponent bits, 5 high mantissa bits, and 5 low mantissa bits; each 5-bit field is compressed with its own Huffman codebook built offline from the weights. The entropy table shows why this is nearly free: for Llama-3-8B FP16, the whole 16-bit weight has entropy 10.54 bits per parameter, while the 1-5-5-5 split has total entropy 10.61 bits per parameter, so splitting costs roughly 0.07 bits while reducing each codebook to 32 entries. Because the codebook is that small, a content-addressable memory can match the current codeword, output its 5-bit source symbol, advance the start pointer, and refill the register in one cycle. Huffman decoders placed between the weight buffer and the processing elements therefore emit one decompressed weight per clock cycle with no bubbles or stalls. Across Llama, OPT, Qwen, and Vicuna families, on both systolic arrays and vector-style accelerators, the paper reports 15–32% smaller models, up to 31% latency savings, and up to 26% energy savings, with the largest gains coming from output-stationary arrays and memory-bound bandwidth settings.

Load-bearing premise

The central benefit rests on the 5-bit Huffman decoder completing a CAM match, a start-pointer update, and a register refill within one clock cycle at 1–2 GHz, so the arithmetic pipeline never waits for a weight.

Editorial extensions

If this is right

  • A 128x128 systolic array with the decoders added at under 6% area can stream compressed weights without stalling, so memory-bound inference finishes up to 31% sooner than the uncompressed baseline while producing identical outputs.
  • On a device with fixed on-chip weight storage, the same buffer holds 15–32% more weights, so larger models fit on the same edge hardware or the same model leaves room for longer contexts or batching.
  • In output-stationary systolic arrays, where weights are re-fetched more often, the per-weight byte savings translate into larger latency and energy gains than in weight-stationary designs.
  • At very high DRAM bandwidth the system becomes compute-bound and latency savings shrink, but energy still drops because fewer DRAM bytes are fetched, per the paper's 256 GB/s roofline analysis.
  • BF16 models compress through a 1-4-4-7 split with the seven mantissa bits nearly incompressible; all tested families cluster at similar BF16 ratios, while FP16 ratios differ between Llama/Qwen and OPT/Vicuna families.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because compression is bit-exact, users could stack it after lossy quantization: the shipped model would be exactly the quantized model, giving a precise behavioral guarantee. The paper does not test this two-stage path.
  • The per-field split suggests a general recipe for other numeric formats: split a weight into its bit-fields, Huffman-code the skewed fields, bypass the flat ones, and choose codebook size to fit a single-cycle CAM. INT8 or FP8 weights would be natural next targets.
  • The compression-ratio gap between model families may reflect training dynamics rather than architecture: if training imprints shape the weight distribution, lossless ratio could become a lightweight diagnostic of a model's provenance. This is an interpretation the paper does not make.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes Huff-LLM, a lossless compression scheme for LLM weights. An FP16 weight is split into four fields (sign, exponent, mantissa MSBs, mantissa LSBs); each 5-bit field is Huffman-coded separately with per-matrix codebooks, while the sign bit is stored uncompressed. A CAM-based Huffman decoder is designed to reconstruct one 5-bit symbol per clock cycle, and decoders are inserted between weight buffers and MAC arrays in systolic-array and Simba-like architectures. The paper reports measured compression ratios for Llama, OPT, Qwen, and Vicuna model families, and uses analytical simulations (modified SCALE-Sim) and Timeloop/Accelergy to estimate latency and energy savings, together with GF12 Verilog synthesis of the decoder and PE for area overhead. The central claims are up to 32% model-size reduction, up to 31% latency improvement, and up to 26% energy reduction with less than 6% area overhead, while preserving model behavior exactly.

Significance. The compression-ratio part of the paper is credible and useful: the 1-5-5-5 bit split is justified by an entropy analysis and the ratios in Table 5 are measured on actual model weights, so the model-size reduction claim is on solid ground. The losslessness guarantee is a genuine advantage over quantization and pruning for safety-critical deployments. The main contribution is the hardware-software co-design of a lightweight Huffman decoder, and if the single-cycle, stall-free assumption is validated, the proposed integration could provide meaningful latency and energy savings in edge accelerators. The paper also provides synthesis area data and a reproducible simulation methodology, which are strengths. However, the hardware validation currently stops short of demonstrating that the decoder can operate at the assumed frequency without stalling, and the energy model omits the decoder itself.

major comments (4)
  1. [Section 3.2, Figure 7, Section 5.2] The abstract and introduction claim evaluation 'using simulations and an FPGA prototype,' but the body contains no FPGA measurements, no post-synthesis timing report, and no throughput validation for the Huffman decoder. The design in Figure 7 must perform a CAM match over Lmax bits, select the matching source symbol and length, update the start pointer, and refill the codeword register in a single cycle at 1–2 GHz, yet no timing closure is reported. The Verilog synthesis in Section 5.2 reports only area (PE 484 um^2, HD 1199.3 um^2). Because the latency savings in Tables 6 and 13 and the analytical model in Appendix A.1 (Eq. 3) assume decompression adds zero cycles, the claimed 31% latency improvement rests entirely on an unverified assumption. Please add post-synthesis timing estimates, FPGA frequency and throughput measurements, or at least a cycle-accurate model of the decoder path.
  2. [Section 5.2, Figure 6(b), Tables 7/13] The energy simulations do not include the Huffman decoder's power consumption. The paper reports only area overhead for the HD (1199.3 um^2 vs. 484 um^2 per PE) and does not model decoder dynamic or static energy in the energy savings of Tables 7 and 13 or in the energy breakdown of Figure 6(b). Since the decoder sits on the data path and is comparable in area to a PE, its energy is likely non-negligible; the claimed 26% energy reduction is therefore an upper bound. The authors should either include synthesized decoder power in the energy model or state clearly that decoder energy is excluded and quantify the potential impact.
  3. [Section 3.1, Table 5] The entropy analysis and the measured compression ratios are not reconciled. Section 3.1 reports a 1-5-5-5 entropy of 10.61 bits/parameter for Llama-3-8B FP16, while Table 5 reports an average of 10.96 bits/parameter (ratio 1.46). The 0.35 bits/parameter gap is not explained. If it is due to Huffman codebook overhead or per-matrix codebook storage, the paper should say so explicitly and confirm that the reported model-size reductions (up to 32%) include all storage overheads. If it is due to a different computation (e.g., averaging over matrices), the text should clarify why the entropy estimate differs from the measured bits/parameter.
  4. [Section 3.2] The paper does not address codebook switching. If each weight matrix has its own Huffman codebook, then the CAM contents in each HD must be reconfigured at matrix boundaries, potentially adding stalls or requiring shadow CAMs. The authors should state whether codebooks are per matrix, how often they change during inference, and what the reload cost is. This matters because the latency savings are claimed for layer-level operations, and frequent codebook updates could erode the gains.
minor comments (6)
  1. [Section 3.1] The reference 'Table 3.1' should be 'Table 1'.
  2. [Section 4] The word 'approximxately' should be 'approximately'.
  3. [Table 5 caption] The word 'Highighted' should be 'highlighted'.
  4. [Section 5.2] The sentence 'The HUFF-LLM compression scheme leads to significant latency improvements...' appears twice in slightly different forms; please consolidate.
  5. [Throughout] The paper uses 'Huff-LLM' and 'HUFF-LLM' inconsistently; choose one spelling.
  6. [Section 3.1, Table 1] The split '4-4-4-4' is mentioned in the text but not defined in the figure; please add a sentence explaining that this is a four-way split of the 16 bits.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: compression ratios are measured from real weight distributions, and the latency/energy savings are simulator outputs under an explicitly stated single-cycle decoder assumption; the promised FPGA support is missing, but that is an evidence gap, not circularity.

full rationale

The paper's chain is: (i) choose a bit-split for Huffman coding; (ii) build per-split codebooks from actual LLM weights and measure compressed bits/parameter; (iii) insert CAM-based decoders into systolic-array and Simba-like accelerator models whose memory traffic is reduced by the measured compression; (iv) use SCALE-Sim/Timeloop/Accelergy to estimate latency and energy. Nothing in this chain is fitted to the reported savings. The {1,5,5,5} split is selected from the entropy table (Table 1) and the CAM-area constraint (Figure 3), not from the 31%/26% results; the compression ratios in Table 5 are direct measurements of Huffman code lengths. The accuracy-retention statement in Section 4 is a tautology correctly labeled 'by construction,' not an empirical prediction. The only near-self-citation is STAR-Sim (Sun et al. 2024, which includes co-author S. Q. Zhang), used solely as a simulation-methodology reference and not load-bearing; the hardware estimates also rest on external Timeloop/Accelergy/SCALE-Sim. Separately, the single-cycle decoder and the abstract's promised FPGA prototype lack supporting timing/throughput evidence (Sections 3.2 and 5.2), so the latency claim is conditional on an unproven architectural assumption; that is a correctness/evidence gap, not a circular derivation, and does not change the circularity score.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three hand-chosen or fitted quantities: the bit-split pattern, the longest codeword length, and the per-matrix Huffman codebooks. The codebooks are the actual compression mechanism, so their storage and switching cost should be counted, but the paper does not. The hardware feasibility rests on an unverified single-cycle timing assumption for the CAM decoder.

free parameters (3)
  • Lmax (longest codeword length) = 12 (empirically determined)
    Sets the CAM window and register width and directly affects area overhead; derived from actual Huffman codebooks and not independently verified across all matrices or models.
  • Bit-split configuration = {1,5,5,5} for FP16, {1,4,4,7} for BF16
    Chosen by hand from entropy analysis and the CAM size constraint that only N=4 or 5 is feasible; this choice directly determines compression ratio and hardware cost.
  • Per-matrix Huffman codebooks = Variable, one per weight matrix
    Codebooks are fitted to each weight matrix's distribution; the paper does not account for codebook storage or switching overhead in the reported memory and energy savings.
assumptions (3)
  • standard math Huffman coding is optimal for a fixed distribution and gives lossless reconstruction.
    Implicit in Section 3.1; standard information theory.
  • domain assumption A CAM lookup, codeword match, and pointer update complete in one clock cycle at 1 GHz (and 2 GHz for Simba) with negligible stall impact.
    Section 3.2 and Figure 7 assume single-cycle decoding; no timing closure or FPGA measurements are reported, yet the latency savings depend on it.
  • domain assumption The weight buffer can supply compressed bits fast enough that the decoder's register never empties and no bubbles are introduced.
    Section 3.2 assumes refill without stalling; buffer bandwidth and codebook layout details are not modeled.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Huff-LLM: End-to-End Lossless Compression for Efficient LLM Inference." pith.science (2026). https://pith.science/paper/7MWEPOGI

@misc{pith2026250200922,
  author       = {Pith},
  title        = {Pith review of: Huff-LLM: End-to-End Lossless Compression for Efficient LLM Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7MWEPOGI}},
  note         = {Machine review of arXiv:2502.00922}
}
read the original abstract

As they become more capable, large language models (LLMs) have continued to rapidly increase in size. This has exacerbated the difficulty in running state of the art LLMs on small, edge devices. Standard techniques advocate solving this problem through lossy compression techniques such as quantization or pruning. However, such compression techniques are lossy, and have been shown to change model behavior in unpredictable manners. We propose Huff-LLM, an \emph{end-to-end, lossless} model compression method that lets users store LLM weights in compressed format \emph{everywhere} -- cloud, disk, main memory, and even in on-chip memory/buffers. This allows us to not only load larger models in main memory, but also reduces bandwidth required to load weights on chip, and makes more efficient use of on-chip weight buffers. In addition to the memory savings achieved via compression, we also show latency and energy efficiency improvements when performing inference with the compressed model.

Figures

Figures reproduced from arXiv: 2502.00922 by the authors.

Figure 3
Figure 3. Area overhead of a CAM lookup for a single-cycle N-bit Huffman decoder normalized to a column of 128 FP16 multipliers, both clocked at 1 GhZ. Area overheads of Huffman decoding grow quickly, leaving only N={4,5} as viable options. Split Entropy (Bits/Param) Total Bits/Param 16 10.54 10.54 8-8 5.54, 5.03 10.57 1-5-5-5 1.00, 2.60, 4.97, 2.04 10.61 4-4-4-4 2.14, 3.91, 4.00, 1.34 11.09 [PITH_FULL_IMAGE:figures/full_fig… view at source ↗
Figure 2
Figure 2. Simba-like architecture diagram with a detailed look at the PE. Colors are associated to different operands. Weights are yellow, inputs are light blue, and partial-sum/outputs are pink. to the GPU memory while putting any data that doesn’t fit on the GPU into the buddy memory. Selective Memory Compression (Nihaal & Mutyam, 2024) introduces a mem￾ory compression scheme that aims to reduce page thrashing by gradually … view at source ↗
Figure 4
Figure 4. Our Huffman Compression method follows these steps for every parameter. It breaks a FP16 number into 4 groups of bits. The sign bit remains uncompressed. The exponent, and mantissa bits are sent through a Huffman Coder to be compressed. They are then stored in memory until they are needed for inference. Weight Input Output PE PE PE PE PE PE PE PE PE … … … … … … HD HD HD … 5-bit Huffman Decoder Huffman Code 5-bit Sig… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Systolic array and Huffman Decoder integration. served in [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Each Huffman Decoder module follows the process shown in this figure. Lmax bits are taken from the register and a match is found in the Huffman Table. Afterwards, the decoded source symbol is sent to the PE while the length is sent to update the start position S [PITH…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Lossless Tensor Compression as Program Synthesis

    cs.SE 2026-08 conditional novelty 7.0 of 10

    By expressing each tensor as a synthesized reversible program and storing the shortest one, Brevis losslessly compresses 2.13 TB of model checkpoints to 1.41 TB, beating ZipNN, zstd, gzip, LZ4, and Snappy.

Reference graph

Works this paper leans on

43 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    L., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J

    Ashkboos, S., Mohtashami, A., Croci, M. L., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J. Quarot: Outlier-free 4-bit inference in rotated llms. arXiv preprint arXiv:2404.00456, 2024

    arXiv 2024

  2. [2]

    B., Muralimanohar, N., Shafiee, A., and Srinivas, V

    Balasubramonian, R., Kahng, A. B., Muralimanohar, N., Shafiee, A., and Srinivas, V. Cacti 7: New tools for interconnect exploration in innovative off-chip memories. ACM Transactions on Architecture and Code Optimization (TACO), 14 0 (2): 0 1--25, 2017

    work page 2017

  3. [3]

    S., and Sze, V

    Chen, Y.-H., Krishna, T., Emer, J. S., and Sze, V. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks. IEEE journal of solid-state circuits, 52 0 (1): 0 127--138, 2016

    work page 2016

  4. [4]

    E., Stoica, I., and Xing, E

    Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., and Xing, E. P. Vicuna: An open-source chatbot impressing gpt-4 with 90\ URL https://lmsys.org/blog/2023-03-30-vicuna/

    2023

  5. [5]

    B., O’Connor, M., Erez, M., Pool, J., Nellans, D., and Keckler, S

    Choukse, E., Sullivan, M. B., O’Connor, M., Erez, M., Pool, J., Nellans, D., and Keckler, S. W. Buddy compression: Enabling larger memory for deep learning and hpc workloads on gpus. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), pp.\ 926--939. IEEE, 2020

    work page 2020

  6. [6]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    arXiv 2018

  7. [7]

    Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35: 0 30318--30332, 2022

    2022

  8. [8]

    The llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

Show all 43 references
  1. [9]

    Accuracy is not all you need

    Dutta, A., Krishnan, S., Kwatra, N., and Ramjee, R. Accuracy is not all you need. arXiv preprint arXiv:2407.09141, 2024

    2024 arXiv

  2. [10]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022

    2022 arXiv

  3. [11]

    Does reduced precision hurt? Blog post, 2024

    Guha, E. Does reduced precision hurt? Blog post, 2024. URL https://sambanova.ai/blog/does-reduced-precision-hurt. Accessed:

    2024

  4. [12]

    Han, S., Mao, H., and Dally, W. J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015

    2015 arXiv

  5. [13]

    Neuzip: Memory-efficient training and inference with dynamic compression of neural networks

    Hao, Y., Cao, Y., and Mou, L. Neuzip: Memory-efficient training and inference with dynamic compression of neural networks. arXiv preprint arXiv:2410.20650, 2024

    2024 arXiv

  6. [14]

    Measuring massive multitask language understanding

    Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  7. [15]

    Zipnn: Lossless compression for ai models

    Hershcovitch, M., Wood, A., Choshen, L., Girmonsky, G., Leibovitz, R., Ennmouri, I., Malka, M., Chin, P., Sundararaman, S., and Harnik, D. Zipnn: Lossless compression for ai models. arXiv preprint arXiv:2411.05239, 2024

    2024 arXiv

  8. [16]

    Decoding compressed trust: Scrutinizing the trustworthiness of efficient llms under compression

    Hong, J., Duan, J., Zhang, C., Li, Z., Xie, C., Lieberman, K., Diffenderfer, J., Bartoldson, B., Jaiswal, A., Xu, K., et al. Decoding compressed trust: Scrutinizing the trustworthiness of efficient llms under compression. arXiv preprint arXiv:2403.15447, 2024

    2024 arXiv

  9. [17]

    S., Choi, Y., Kim, C., Kim, Y., Yu, H., Abdel-Aziz, H., Park, J.-S., Lee, H., Lee, D., Kim, M

    Jang, J.-W., Lee, S., Kim, D., Park, H., Ardestani, A. S., Choi, Y., Kim, C., Kim, Y., Yu, H., Abdel-Aziz, H., Park, J.-S., Lee, H., Lee, D., Kim, M. W., Jung, H., Nam, H., Lim, D., Lee, S., Song, J.-H., Kwon, S., Hassoun, J., Lim, S., and Choi, C. Sparsity-aware and re-config...

    2021

  10. [18]

    G., Zimmer, B., Dally, W

    Keller, B., Venkatesan, R., Dai, S., Tell, S. G., Zimmer, B., Dally, W. J., Thomas Gray, C., and Khailany, B. A 17–95.6 tops/w deep learning inference accelerator with per-vector scaled 4-bit quantization for transformers in 5nm. In 2022 IEEE Symposium on VLSI Technology and C...

    2022

  11. [19]

    Bit-plane compression: Transforming data for better compression in many-core architectures

    Kim, J., Sullivan, M., Choukse, E., and Erez, M. Bit-plane compression: Transforming data for better compression in many-core architectures. ACM SIGARCH Computer Architecture News, 44 0 (3): 0 329--340, 2016

    2016

  12. [20]

    Cerebras architecture deep dive: First look inside the hardware/software co-design for deep learning

    Lie, S. Cerebras architecture deep dive: First look inside the hardware/software co-design for deep learning. IEEE Micro, 43 0 (3): 0 18--30, 2023

    2023

  13. [21]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration

    Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang, W.-C., Xiao, G., Dang, X., Gan, C., and Han, S. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6: 0 87--100, 2024

    2024

  14. [22]

    How does quantization affect multilingual llms? arXiv preprint arXiv:2407.03211, 2024

    Marchisio, K., Dash, S., Chen, H., Aumiller, D., \"U st \"u n, A., Hooker, S., and Ruder, S. How does quantization affect multilingual llms? arXiv preprint arXiv:2407.03211, 2024

    2024 arXiv

  15. [23]

    and Mutyam, M

    Nihaal, A. and Mutyam, M. Selective memory compression for gpu memory oversubscription management. In Proceedings of the 53rd International Conference on Parallel Processing, pp.\ 189--198, 2024

    2024

  16. [24]

    S., Chen, Y.-H., Ying, V

    Parashar, A., Raina, P., Shao, Y. S., Chen, Y.-H., Ying, V. A., Mukkara, A., Venkatesan, R., Khailany, B., Keckler, S. W., and Emer, J. Timeloop: A systematic approach to dnn accelerator evaluation. In 2019 IEEE International Symposium on Performance Analysis of Systems and So...

    2019

  17. [25]

    Arrayflex: A systolic array architecture with configurable transparent pipelining

    Peltekis, C., Filippas, D., Dimitrakopoulos, G., Nicopoulos, C., and Pnevmatikatos, D. Arrayflex: A systolic array architecture with configurable transparent pipelining. In 2023 Design, Automation and Test in Europe Conference and Exhibition (DATE), pp.\ 1--6, 2023. doi:10.239...

    2023

  18. [26]

    M., Zhu, Y., Whatmough, P., Mattina, M., and Krishna, T

    Samajdar, A., Joseph, J. M., Zhu, Y., Whatmough, P., Mattina, M., and Krishna, T. A systematic methodology for characterizing scalability of dnn accelerators using scale-sim. In 2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp.\ 58...

    2020

  19. [27]

    S., Reagen, B., Wei, G.-Y., and Brooks, D

    Shao, Y. S., Reagen, B., Wei, G.-Y., and Brooks, D. Aladdin: A pre-rtl, power-performance accelerator simulator enabling large design space exploration of customized architectures. In Proceeding of the 41st Annual International Symposium on Computer Architecuture, ISCA '14, pp...

    2014

  20. [28]

    S., Clemons, J., Venkatesan, R., Zimmer, B., Fojtik, M., Jiang, N., Keller, B., Klinefelter, A., Pinckney, N., Raina, P., Tell, S

    Shao, Y. S., Clemons, J., Venkatesan, R., Zimmer, B., Fojtik, M., Jiang, N., Keller, B., Klinefelter, A., Pinckney, N., Raina, P., Tell, S. G., Zhang, Y., Dally, W. J., Emer, J., Gray, C. T., Khailany, B., and Keckler, S. W. Simba: Scaling deep-learning inference with multi-ch...

    2019

  21. [29]

    The nvidia deep learning accelerator

    Sijstermans, F. The nvidia deep learning accelerator. In Hot Chips, volume 30, pp.\ 19--21, 2018

    2018

  22. [30]

    Q., Gomez, J., Khwa, W.-S., Sarwar, S

    Sun, X., Peng, X., Zhang, S. Q., Gomez, J., Khwa, W.-S., Sarwar, S. S., Li, Z., Cao, W., Wang, Z., Liu, C., Chang, M.-F., De Salvo, B., Akarvardar, K., and Wong, H.-S. P. Estimating power, performance, and area for on-sensor deployment of ar/vr workloads using an analytical fr...

    2024 doi

  23. [31]

    Google coral edge tpu board vs nvidia jetson nano dev board hardware comparison, 2020

    Suryavansh, M. Google coral edge tpu board vs nvidia jetson nano dev board hardware comparison, 2020

    2020

  24. [32]

    Llama3.1 model quality evaluation: Cerebras, groq, sambanova, together, and fireworks

    Thangarasa, V. Llama3.1 model quality evaluation: Cerebras, groq, sambanova, together, and fireworks. Blog post, 2024. URL https://cerebras.ai/blog/. Accessed:

    2024

  25. [33]

    S., Wang, M., Clemons, J., Dai, S., Fojtik, M., Keller, B., Klinefelter, A., Pinckney, N., Raina, P., Zhang, Y., Zimmer, B., Dally, W

    Venkatesan, R., Shao, Y. S., Wang, M., Clemons, J., Dai, S., Fojtik, M., Keller, B., Klinefelter, A., Pinckney, N., Raina, P., Zhang, Y., Zimmer, B., Dally, W. J., Emer, J., Keckler, S. W., and Khailany, B. Magnet: A modular accelerator generator for neural networks. In 2019 I...

    2019

  26. [34]

    Spatten: Efficient sparse attention architecture with cascade token and head pruning

    Wang, H., Zhang, Z., and Han, S. Spatten: Efficient sparse attention architecture with cascade token and head pruning. 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pp.\ 97--110, 2020. URL https://api.semanticscholar.org/CorpusID:229298088

    2021

  27. [35]

    Welch, T. A. A technique for high-performance data compression. Computer, 17 0 (06): 0 8--19, 1984

    1984

  28. [36]

    The roofline model: A pedagogical tool for program analysis and optimization

    Williams, S., Patterson, D., Oliker, L., Shalf, J., and Yelick, K. The roofline model: A pedagogical tool for program analysis and optimization. In 2008 IEEE Hot Chips 20 Symposium (HCS), pp.\ 1--71, 2008. doi:10.1109/HOTCHIPS.2008.7476531

    2008

  29. [37]

    Beyond perplexity: Multi-dimensional safety evaluation of llm compression

    Xu, Z., Gupta, A., Li, T., Bentham, O., and Srikumar, V. Beyond perplexity: Multi-dimensional safety evaluation of llm compression. arXiv preprint arXiv:2407.04965, 2024

    2024 arXiv

  30. [38]

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  31. [39]

    Zeroquant: Efficient and affordable post-training quantization for large-scale transformers

    Yao, Z., Yazdani Aminabadi, R., Zhang, M., Wu, X., Li, C., and He, Y. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 35: 0 27168--27183, 2022

    2022

  32. [40]

    15.1 a 0.795 fj/bit physically-unclonable function-protected tcam for a software-defined networking switch

    Yue, Z., Xiang, X., Tu, F., Wang, Y., Wang, Y., Wei, S., Hu, Y., and Yin, S. 15.1 a 0.795 fj/bit physically-unclonable function-protected tcam for a software-defined networking switch. In 2024 IEEE International Solid-State Circuits Conference (ISSCC), volume 67, pp.\ 276--278...

    2024

  33. [41]

    V., et al

    Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  34. [42]

    Does your llm truly unlearn? an embarrassingly simple approach to recover unlearned knowledge

    Zhang, Z., Wang, F., Li, X., Wu, Z., Tang, X., Liu, H., He, Q., Yin, W., and Wang, S. Does your llm truly unlearn? an embarrassingly simple approach to recover unlearned knowledge. arXiv preprint arXiv:2410.16454, 2024

    2024 arXiv

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.