Pith. sign in

REVIEW 44 cited by

A Survey of Quantization Methods for Efficient Neural Network Inference

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2103.13630 v3 pith:7RPBKFQU submitted 2021-03-25 cs.CV

Amir Gholami , Sehoon Kim , Zhen Dong , Zhewei Yao , Michael W. Mahoney , Kurt Keutzer This is my paper · ORCID
classification cs.CV
keywords computationsneuralproblemquantizationefficientnetworknumericalresearch
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

As soon as abstract mathematical computations were adapted to computation on digital computers, the problem of efficient representation, manipulation, and communication of the numerical values in those computations arose. Strongly related to the problem of numerical representation is the problem of quantization: in what manner should a set of continuous real-valued numbers be distributed over a fixed discrete set of numbers to minimize the number of bits required and also to maximize the accuracy of the attendant computations? This perennial problem of quantization is particularly relevant whenever memory and/or computational resources are severely restricted, and it has come to the forefront in recent years due to the remarkable performance of Neural Network models in computer vision, natural language processing, and related areas. Moving from floating-point representations to low-precision fixed integer values represented in four bits or less holds the potential to reduce the memory footprint and latency by a factor of 16x; and, in fact, reductions of 4x to 8x are often realized in practice in these applications. Thus, it is not surprising that quantization has emerged recently as an important and very active sub-area of research in the efficient implementation of computations associated with Neural Networks. In this article, we survey approaches to the problem of quantizing the numerical values in deep Neural Network computations, covering the advantages/disadvantages of current methods. With this survey and its organization, we hope to have presented a useful snapshot of the current research in quantization for Neural Networks and to have given an intelligent organization to ease the evaluation of future research in this area.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 44 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models

    cs.CV 2026-08 conditional novelty 7.0 of 10

    A temporal-spatial LSB mask over one shared weight buffer lets diffusion models use lower bit precision in less sensitive denoising stages, cutting compute by 25-50% on bit-serial hardware with no loss in image quality.

  2. When Bits Break Recourse: Counterfactual-Faithful Quantization

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Quantization can preserve accuracy while invalidating algorithmic recourse; CFQ trains the quantized model at teacher recourse points and preserves counterfactual validity and recourse cost.

  3. No Certificate, No Categorical Speech Act: A Brouwerian Assertibility Constraint for Public Reason

    cs.CY 2026-03 unverdicted novelty 7.0 of 10

    An AI may assert or deny high-stakes claims only when it can exhibit a publicly contestable certificate; otherwise it is obligated to return Undetermined.

  4. When Do Neural Networks Learn World Models?

    cs.LG 2025-02 conditional novelty 7.0 of 10

    With Boolean variables, a low-degree bias, and a task distribution weighted toward simple functions of the latents, multi-task training provably recovers the latent world model up to permutations and negations.

  5. FlashAttention on a Napkin: A Diagrammatic Approach to Deep Learning IO-Awareness

    cs.LG 2024-12 conditional novelty 7.0 of 10

    A diagrammatic framework for deriving IO-aware GPU algorithms and performance models, applied to matrix multiplication and FlashAttention-style attention.

  6. Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

    cs.LG 2024-11 conditional novelty 7.0 of 10

    A new theorem and method (HIGGS) make per-layer quantization error a reliable predictor of final model perplexity, enabling state-of-the-art data-free and dynamic bit-width LLM compression.

  7. Hardware-Aware Deployment of Joint SAR Compression and Despeckling on FPGA

    eess.IV 2026-08 conditional novelty 6.0 of 10

    A joint SAR despeckling and compression model was deployed on an FPGA, where hardware-friendly changes like ReLU instead of GDN improved performance, residual blocks were not worth their 10x compute cost, and the FPGA...

  8. ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMM

    cs.AR 2026-07 conditional novelty 6.0 of 10

    ExaGEMM co-explores lightweight CPU ISA support and lookup-table GEMM kernels for 1/2/4-bit ML inference, prunes ~99% of candidates, and reports up to 13.3x simulated speedups over software.

  9. Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Monotone Lyapunov projection of layerwise hidden-state energy suppresses two's-complement overflow under wrapping fixed-point QAT/PTQ, recovering 86.55% MNIST accuracy where unconstrained models collapse to chance.

  10. MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM

    cs.LG 2026-02 conditional novelty 6.0 of 10

    MoBiQuant assigns each token its own quantization bit-width via learned routing over recursive residual bit slices, enabling elastic 2–6-bit LLM inference that matches static PTQ accuracy.

  11. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    cs.CL 2025-11 reject novelty 6.0 of 10

    TAQ estimates per-layer importance from hidden representations and output sensitivity on task calibration data to allocate mixed precision in a training-free PTQ setting, outperforming task-agnostic baselines on accur...

  12. BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression

    cs.LG 2025-09 conditional novelty 6.0 of 10

    BALF is a fine-tuning-free method that compresses CNNs and vision transformers by low-rank factorizing layers with activation-aware whitening and automatic per-layer rank allocation under FLOP or parameter budgets.

  13. SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A training-free framework jointly quantizes a VLA model to 4 bits and prunes visual tokens, recovering or exceeding full-precision success rates at 1.93x speedup.

  14. Hardware Acceleration of Kolmogorov-Arnold Network (KAN) in Large-Scale Systems

    cs.AR 2025-09 conditional novelty 6.0 of 10

    A co-design of quantization, lookup-table sharing, and analog in-memory circuits lets large KAN recommendation models (39-63MB) scale with 28-41x area growth for 500K-807Kx parameter growth, with 0.11-0.23% accuracy l...

  15. NeurStore: Efficient In-database Deep Learning Model Management System

    cs.DB 2025-09 conditional novelty 6.0 of 10

    A database model-management system that deduplicates tensors across models and uses delta quantization to cut storage and speed loading.

  16. Automatic mixed precision for optimizing gained time with constrained loss mean-squared-error based on model partition to sequential sub-graphs

    cs.LG 2025-05 conditional novelty 6.0 of 10

    The paper derives an additive loss-MSE sensitivity metric and a hardware-aware time-gain model, then uses integer programming to assign per-layer FP8 or BF16 formats for LLM inference.

  17. Gradual Binary Search and Dimension Expansion : A general method for activation quantization in LLMs

    cs.LG 2025-04 conditional novelty 6.0 of 10

    Gradual Binary Search over per-projection clipping ratios, combined with Hadamard rotations and dimension expansion, enables 3-bit WAKV quantization with better benchmark accuracy than QuaRot.

  18. Forget the Data and Fine-Tuning! Just Fold the Network to Compress

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Model folding compresses a network by k-means clustering similar neurons across adjacent layers and repairing activation statistics without data (Fold-AR, Fold-DIR), surpassing prior data-free methods at high sparsity.

  19. Setup Once, Secure Always: A Single-Setup Secure Federated Learning Aggregation Protocol with Forward and Backward Secrecy for Dynamic Users

    cs.CR 2025-02 conditional novelty 6.0 of 10

    A single-setup secure aggregation protocol for federated learning that achieves forward and backward secrecy, dynamic user participation, and dropout tolerance using fresh random masks with a cyclic key-negation trick.

  20. Column-wise Quantization of Weights and Partial Sums for Accurate and Efficient Compute-In-Memory Accelerators

    cs.AR 2025-02 conditional novelty 6.0 of 10

    Applying quantization-aware training with column-wise scale factors for both weights and partial sums improves CIM accelerator accuracy by 0.99 to 2.69 percentage points over reported baselines.

  21. Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Sparse, 8-bit quantized S5 linear RNNs match dense model audio denoising accuracy with 2x less compute and 36% less memory, and run 42x faster with 149x lower energy on Loihi 2 than a dense FP32 model on Jetson Orin Nano.

  22. Towards Responsible Governing AI Proliferation

    cs.CY 2024-12 conditional novelty 6.0 of 10

    The paper proposes a 'Proliferation' paradigm of AI, where small, hidden, augmented, decentralized, and open-weight models challenge compute-centric governance.

  23. MarketGPT: Developing a Pre-trained transformer (GPT) for Modeling Financial Time Series

    q-fin.TR 2024-11 conditional novelty 6.0 of 10

    A transformer trained on Nasdaq ITCH messages can generate believable order flow, reproducing heavy tails, volatility clustering, and long-range dependence in returns, though with notable quantitative deviations and w...

  24. Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systems

    cs.LG 2024-11 conditional novelty 6.0 of 10

    Stochastically rounding a single pretrained model into several low-precision copies yields a training-free ensemble that improves NLL and calibration on large models.

  25. SmartInv: Multimodal Learning for Smart Contract Invariant Inference

    cs.SE 2024-11 conditional novelty 6.0 of 10

    SmartInv fine-tunes LLaMA on contracts and comments to generate invariants, verifies them with Boogie and CORRAL, and claims to find 119 zero-day smart contract bugs, five confirmed by developers.

  26. Lightweight Image Classification of Raptor Species for Edge Devices: Rare-Species Dataset Expansion via Video Frame Extraction, Knowledge Distillation, and TensorRT Deployment

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A three-student ensemble distilled from DINOv2-L reaches 0.935 macro recall on six raptor species with ~1/9 the teacher's parameters and runs at 313 images/s on an Orin Nano, with gains driven by rare-species data exp...

  27. PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation

    cs.CV 2026-03 conditional novelty 5.0 of 10

    A 1.3M-parameter CNN with ROI-implicit prompting and SAM3 distillation reaches ~65% mIoU on COCO/LVIS and 11.82 ms INT8 inference fully in-sensor on the Sony IMX500.

  28. LCS: An AI-based Low-Complexity Scaler for Power-Efficient Super-Resolution of Game Content

    cs.CV 2025-07 conditional novelty 5.0 of 10

    An efficient neural upscaler trained on natively rendered game image pairs beats traditional upscalers on the LPIPS and NIQE perceptual metrics, at a fraction of the parameter count.

  29. EQuARX: Efficient Quantized AllReduce in XLA for Distributed Machine Learning Acceleration

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A compiler-native, block-wise int8 quantized AllReduce for TPUs that reaches about 55% of baseline BF16 AllReduce time and accelerates Gemma 3 prefill by up to 1.28x with small quality impact.

  30. BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexing

    eess.AS 2025-06 conditional novelty 5.0 of 10

    Quantization-aware training with ternary weights plus base-3 weight indexing reduces a JETS/HiFi-GAN TTS model from 25.66 MB to 4.39 MB while keeping naturalness MOS around 3.1 to 3.3.

  31. Large Language Model Meets Constraint Propagation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Using BERT-like predictions to preview future positions reduces autoregressive LLM calls and increases the count of feasible solutions inside the GenCP constrained text generation framework, on the tasks tested.

  32. Is (Selective) Round-To-Nearest Quantization All You Need?

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Selective round-to-nearest quantization, combined with Marlin-based kernels, matches or beats GPTQ, AWQ, and BitsAndBytes on accuracy and speed in most tested settings.

  33. Efficient Vision-based Vehicle Speed Estimation

    cs.CV 2025-05 conditional novelty 5.0 of 10

    Replacing the detector in a vanishing-point-based speed estimation pipeline with YOLOv6 plus post-training quantization yields comparable or better speed accuracy at substantially higher frame rates on BrnoCompSpeed.

  34. MultiPruner: Balanced Structure Removal in Foundation Models

    cs.LG 2025-01 conditional novelty 5.0 of 10

    Pruning residual blocks, then MLP channels, then attention heads in sequence improves zero-shot accuracy of compressed LLMs compared with block-only pruning.

  35. MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers

    cs.CV 2026-07 conditional novelty 4.5 of 10

    KL-isolation fragility plus MCKP bit allocation yields mixed-precision ViT PTQ that lags recent ImageNet PTQ but reports large COCO AP gains at MP3/MP3.

  36. QScheduler: Adaptive Gradient Sampling for Zeroth-Order On-Device Training on INT8 NPUs

    cs.LG 2026-07 conditional novelty 4.0 of 10

    An adaptive sample-count scheduler for zeroth-order optimization matches fixed-q baselines on EuroSAT and STL-10 while avoiding q hyperparameter sweeps.

  37. Performance Analysis of Post-Training Quantization for CNN-based Conjunctival Pallor Anemia Detection

    eess.IV 2025-07 conditional novelty 4.0 of 10

    A MobileNet classifier reaches about 93% accuracy on the CP-AnemiC conjunctival pallor dataset, and FP16 post-training quantization preserves accuracy while INT8 and INT4 degrade it sharply.

  38. Design of an Edge-based Portable EHR System for Anemia Screening in Remote Health Applications

    cs.ET 2025-07 conditional novelty 4.0 of 10

    A compact edge EHR with a random forest model estimates hemoglobin from fingernail images with RMSE 1.969 g/dL and 79.2% sensitivity on a public 250-image dataset.

  39. TD-MPC-Opt: Distilling Model-Based Multi-Task Reinforcement Learning Agents

    cs.LG 2025-07 conditional novelty 4.0 of 10

    Reward-level distillation and FP16 quantization compress a 317M-parameter TD-MPC2 agent to 1M parameters, reaching 28.45 normalized score on MT30, though most of the gap over the original 18.93 comes from a longer tra...

  40. Can Post-Training Quantization Benefit from an Additional QLoRA Integration?

    cs.CL 2025-02 conditional novelty 4.0 of 10

    Adding QLoRA on top of a 4-bit post-training quantized LLM improves task accuracy over quantization alone, and can match or beat full 16-bit fine-tuning on some tasks.

  41. Light-T2M: A Lightweight and Fast Model for Text-to-motion Generation

    cs.CV 2024-12 conditional novelty 4.0 of 10

    Light-T2M generates 3D human motion from text with 4.48M parameters, reporting FID 0.040 on HumanML3D (vs 0.045 for MoMask) and faster inference.

  42. Fuzzy Norm-Explicit Product Quantization for Recommender Systems

    cs.IR 2024-12 conditional novelty 4.0 of 10

    A fuzzy type-2 clustering codebook construction for NEQ improves recall on Netflix, Audio, and Cifar60k by a few points at similar runtime, though without released code or recall error bars.

  43. QPART: Adaptive Model Quantization and Dynamic Workload Balancing for Accuracy-aware Edge Inference

    cs.DC 2025-06 conditional novelty 3.0 of 10

    QPART chooses, for each edge inference request, a layer split point and per-layer bit widths that minimize time, energy, and server cost subject to an accuracy budget, cutting communication payload by over 80% with me...

  44. Image Segmentation with Large Language Models: A Survey with Perspectives for Intelligent Transportation Systems

    cs.CV 2025-06 reject novelty 3.0 of 10

    A survey that organizes vision-language segmentation methods for intelligent transportation, but its synthesis is undermined by fabricated references and unverifiable benchmarks.

Pith tools