Pith. sign in

REVIEW 27 cited by

Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2004.09602 v1 pith:4CAQC7FG submitted 2020-04-20 cs.LG stat.ML

Hao Wu , Patrick Judd , Xiaojie Zhang , Mikhail Isaev , Paulius Micikevicius This is my paper · ORCID
classification cs.LGstat.ML
keywords quantizationintegerdeepincludinginferencemodelsnetworksneural
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Quantization techniques can reduce the size of Deep Neural Networks and improve inference latency and throughput by taking advantage of high throughput integer instructions. In this paper we review the mathematical aspects of quantization parameters and evaluate their choices on a wide range of neural network models for different application domains, including vision, speech, and language. We focus on quantization techniques that are amenable to acceleration by processors with high-throughput integer math pipelines. We also present a workflow for 8-bit quantization that is able to maintain accuracy within 1% of the floating-point baseline on all networks studied, including models that are more difficult to quantize, such as MobileNets and BERT-large.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 27 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 218 citations worldwide. Full citation record

  1. Spec Sheets Are Not Kernels: An ISA- and Source-Level Audit of INT8 Availability on NVIDIA Blackwell Ultra

    cs.AR 2026-08 accept novelty 7.0 of 10

    On NVIDIA Blackwell Ultra, INT8 W8A8 is undeployable by default because the PTX ISA, CUTLASS, vLLM, and SGLang all lack a fifth-generation INT8 tensor-core path, despite the datasheet listing INT8 support.

  2. Jack of All Scales: A Versatile FPGA Tensor Block for MXFP Precisions

    cs.AR 2026-07 conditional novelty 7.0 of 10

    Targeted DSP tensor-mode changes enable native MXFP4/MXFP6/E4M3 support on Agilex-5-like FPGAs, with a 36% block-area cost and 4.2x average systolic-array throughput gain over baseline mapping strategies.

  3. CyberSentinel: Efficient Anomaly Detection in Programmable Switch using Knowledge Distillation

    cs.CR 2024-12 conditional novelty 7.0 of 10

    CyberSentinel distills autoencoder knowledge into an Isolation Forest deployed as whitelist rules in a switch data plane, detecting IoT anomalies at line rate without control plane support.

  4. You Only Charge Once 2.0 : A End-to-End Analog Computing-in-Memory Architecture with Reconfigurable Switched Capacitors

    cs.AR 2026-08 conditional novelty 6.0 of 10

    Charge-CIM reuses switched capacitors as combined DAC, analog MAC, shift-add, and SAR ADC to reduce ADC-related energy by 91.7 percent in simulation.

  5. INT8 Quantization Makes ARM Edge Inference Dispatch-Invariant

    cs.ET 2026-07 conditional novelty 6.0 of 10

    INT8 QDQ CNNs are byte-exact across ARM Cortex-A53/A72/A76 under XNNPACK even with different SIMD kernels, unlike FP32 or x86 INT8.

  6. Float8@2bits: Entropy Coding Enables Data-Free Model Compression

    cs.LG 2026-01 conditional novelty 6.0 of 10

    EntQuant stores LLM weights at ~2 bits per parameter by entropy-coding Float8 weights, matching data-dependent compression quality without needing calibration data.

  7. FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models

    cs.LG 2025-08 conditional novelty 6.0 of 10

    FlashSVD fuses low-rank SVD projections into attention and feed-forward GPU kernels so SVD-compressed transformers avoid materializing dense activations, cutting activation memory at a latency cost.

  8. DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantization

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A post-training quantization method that combines learned channel scaling and power-of-two scaling keeps diffusion image quality high at 4-bit weight, 6-bit activation precision.

  9. Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition

    cs.LG 2025-06 conditional novelty 6.0 of 10

    ODLRI initializes the low-rank component using activation-outlier channels, improving low-bit compression of large language models over the CALDERA baseline.

  10. Adaptive Semantic Token Communication for Transformer-based Edge Inference

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A single adaptive deep joint source-channel coding model with budget-conditioned token selection and Lyapunov-based resource allocation achieves better accuracy-compression trade-offs than static DJSCC and digital bas...

  11. Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization

    cs.LG 2025-02 conditional novelty 6.0 of 10

    A discrete hill-climbing search over permutation, scaling, and rotation invariances improves 2-bit quantized OPT models when applied on top of GPTQ, AWQ, and OmniQuant.

  12. Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Sparse, 8-bit quantized S5 linear RNNs match dense model audio denoising accuracy with 2x less compute and 36% less memory, and run 42x faster with 149x lower energy on Loihi 2 than a dense FP32 model on Jetson Orin Nano.

  13. Post-Training Quantization for Vision Mamba with k-Scaled Quantization and Reparameterization

    eess.IV 2025-01 conditional novelty 6.0 of 10

    A post-training quantization recipe using k-scaled token-wise scaling and SSM reparameterization keeps Vision Mamba 8-bit ImageNet accuracy within 0.8-1.2% of FP32.

  14. QMamba: Post-Training Quantization for Vision State Space Models

    cs.CV 2025-01 conditional novelty 6.0 of 10

    QMamba combines a log-based quantizer for skewed SSM parameters with a temporal-group quantizer for hidden states, outperforming prior PTQ methods on Vim and VMamba.

  15. Evaluating Single Event Upsets in Deep Neural Networks for Semantic Segmentation: an embedded system perspective

    cs.CV 2024-12 conditional novelty 6.0 of 10

    In a U-Net for segmentation, bit-flip damage concentrates in batch-norm gamma and bias parameters, and rewriting risky floating-point exponents can reduce error rates with no runtime overhead.

  16. EfQAT: An Efficient Framework for Quantization-Aware Training

    cs.LG 2024-11 conditional novelty 6.0 of 10

    EfQAT fine-tunes only the most important weight rows of a PTQ-initialized model, achieving near-full-precision accuracy with up to 1.64x faster QAT backward passes.

  17. Approximate reservoir computing with a semiconductor laser for reducing energy consumption

    physics.optics 2026-07 conditional novelty 5.0 of 10

    Quantizing laser-based reservoir computing to 5 bits and lowering sampling frequency and drive current cuts estimated energy per sample by 91% without losing prediction accuracy.

  18. Real-Time Analysis of Unstructured Data with Machine Learning on Heterogeneous Architectures

    physics.data-an 2025-08 conditional novelty 5.0 of 10

    A graph neural network (ETX4VELO) reconstructs LHCb VELO tracks with performance comparable to the production 'search by triplet' algorithm while running end to end in the GPU-based first-level trigger, with additiona...

  19. Compress Any Segment Anything Model (SAM)

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Birkhoff compresses 18 SAM variants to about one-fifth their size with less than 1% accuracy loss, data-free, using a trajectory-based codebook and a fused GPU operator.

  20. EQuARX: Efficient Quantized AllReduce in XLA for Distributed Machine Learning Acceleration

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A compiler-native, block-wise int8 quantized AllReduce for TPUs that reaches about 55% of baseline BF16 AllReduce time and accelerates Gemma 3 prefill by up to 1.28x with small quality impact.

  21. StableQuant: Layer Adaptive Post-Training Quantization for Speech Foundation Models

    eess.AS 2025-04 conditional novelty 5.0 of 10

    StableQuant adaptively clips outliers per layer before choosing quantization scales, keeping WER within 0.3% at 8-bit precision on HuBERT and wav2vec2.0.

  22. Effective Method with Compression for Distributed and Federated Cocoercive Variational Inequalities

    math.OC 2024-12 conditional novelty 5.0 of 10

    MARINA, a compression-based distributed method, is adapted to cocoercive strongly monotone variational inequalities and proven to converge linearly with a communication complexity of O((1+δ(ℓ/µ)(1+α/n)) log(1/ε)).

  23. Efficient EEG Seizure Detection Using INT8 Quantization, Channel Pruning, and Spiking Neural Networks

    eess.SP 2026-07 conditional novelty 4.0 of 10

    On a shared 1D-CNN baseline for CHB-MIT seizure detection, INT8 quantization cut model size from 1.63 to 0.44 MB and latency by 2.8x with preserved AUC, while SNN conversion was 288x slower on CPU.

  24. Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Models

    cs.DL 2025-06 reject novelty 4.0 of 10

    Transformer-based models can predict many MeSH Publication Types with high accuracy, but the paper does not fairly compare them to the existing MTI baseline, leaving the central improvement claim unsupported.

  25. Power-of-Two (PoT) Weights in Large Language Models (LLMs)

    eess.SP 2025-05 conditional novelty 4.0 of 10

    Power-of-two weight quantization applied post-training to a 124M GPT-2 model degrades cross-entropy from 3.17 to about 4.1-4.5 at 4-6 bits, while promising memory and bit-shift savings.

  26. Mix-QSAM: Mixed-Precision Quantization of the Segment Anything Model

    cs.CV 2025-05 conditional novelty 4.0 of 10

    A mixed-precision post-training quantization method for SAM that allocates bit-widths via an integer quadratic program guided by KL-divergence importance scores and a cross-layer synergy heuristic.

  27. A Survey on Large Language Model Acceleration based on KV Cache Management

    cs.AI 2024-12 conditional novelty 4.0 of 10

    A survey that classifies KV cache management techniques for faster LLM inference into token-level, model-level, and system-level categories, with benchmark resources.

Pith tools