Pith. sign in

REVIEW 4 cited by

Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.06357 v1 pith:7VPTM6P4 submitted 2025-01-10 cs.CV

Navin Ranjan , Andreas Savakis This is my paper
classification cs.CV
keywords layerquantizationmix-qvitimportancemixed-precisionperformanceprecisionsensitivity
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this paper, we propose Mix-QViT, an explainability-driven MPQ framework that systematically allocates bit-widths to each layer based on two criteria: layer importance, assessed via Layer-wise Relevance Propagation (LRP), which identifies how much each layer contributes to the final classification, and quantization sensitivity, determined by evaluating the performance impact of quantizing each layer at various precision levels while keeping others layers at a baseline. Additionally, for post-training quantization (PTQ), we introduce a clipped channel-wise quantization method designed to reduce the effects of extreme outliers in post-LayerNorm activations by removing severe inter-channel variations. We validate our approach by applying Mix-QViT to ViT, DeiT, and Swin Transformer models across multiple datasets. Our experimental results for PTQ demonstrate that both fixed-bit and mixed-bit methods outperform existing techniques, particularly at 3-bit, 4-bit, and 6-bit precision. Furthermore, in quantization-aware training, Mix-QViT achieves superior performance with 2-bit mixed-precision.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    SFMP assigns adjacent integer bit-widths to salient weight blocks via Fisher information without search, and runs mixed-precision GEMM on one-bit lookup tables, beating AMQ on accuracy and speed.

  2. MSQ: Memory-Efficient Bit Sparsification Quantization

    cs.LG 2025-07 conditional novelty 6.0 of 10

    MSQ computes and prunes least significant bits of weights directly from the full-precision parameters, cutting training memory and time for mixed-precision quantization compared with bit-splitting methods.

  3. I-Segmenter: Integer-Only Vision Transformer for Efficient Semantic Segmentation

    cs.CV 2025-09 conditional novelty 5.0 of 10

    I-Segmenter is an integer-only Vision Transformer for semantic segmentation that keeps mIoU within roughly 5 points of the FP32 baseline while cutting model size by up to 3.8x.

  4. MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers

    cs.CV 2026-07 conditional novelty 4.5 of 10

    KL-isolation fragility plus MCKP bit allocation yields mixed-precision ViT PTQ that lags recent ImageNet PTQ but reports large COCO AP gains at MP3/MP3.

Pith tools