Pith. sign in

REVIEW 4 major objections 5 minor 17 references

DQA: An Efficient Method for Deep Quantization of Deep Neural Network Activations

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Sub-6-bit activation quantization can stay accurate if important channels get extra bits, right-shifted back, with the discarded errors stored as Huffman codes.

desk verdict A useful but overclaimed activation quantization paper: the method mostly works, but the abstract's superiority over NoisyQuant is contradicted by its own Table 1. read the letter →

arxiv 2412.09687 v1 pith:GGZP545W submitted 2024-12-12 cs.LG cs.CVstat.ML

classification cs.LGcs.CVstat.ML
keywords activationquantizationdeepHuffmancodingchannelimportancemixed-precisionefficientinferencesub-6-bitCNN
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes DQA, a method for quantizing DNN activations to fewer than 6 bits while keeping accuracy high enough for real use. Instead of expensive per-image searches or noise injection, DQA ranks activation channels once offline, gives important channels extra bits during quantization, right-shifts them back to the target width, and stores the discarded low bits as Huffman-coded side information that is added back at de-quantization. On ResNet-32, MobileNetV2, and U-Net, DQA reports up to 29.28% higher accuracy than direct uniform symmetric quantization, and generally higher accuracy than NoisyQuant at 3, 4, and 5 bits. The online computation is reduced to shifts and table lookups, making deep activation quantization more plausible on resource-constrained devices.

What carries the argument

The mechanism carrying the argument is a right-shift compensation loop. An important channel is first quantized with $m$ extra bits; a right shift by $m$ bits drops those extra bits, and the dropped bits (the shifting error) are saved as a Huffman code. Because the quantization step size for the $n+m$-bit representation is $2^m$ times finer, re-adding the shifting error yields a residual error $2^m$ times smaller than direct quantization of the same channel. An offline greedy channel ranking decides which channels receive this treatment, and that combination of rank, shift, and Huffman side channel produces the accuracy gains without online search or heavy mathematical operations.

What would settle it

Run DQA with ranks computed on clean CIFAR-10 but evaluate on a corrupted or out-of-distribution variant, and compare accuracy against direct quantization; if DQA no longer beats direct quantization once channel importance shifts, the offline-ranking assumption is falsified. Alternatively, implement DQA on a microcontroller and measure end-to-end latency and energy; if Huffman decoding and side-channel overhead outweigh the savings, the resource-constrained motivation collapses.

Watch

Extended reading notes

Core claim

DQA's central claim is that sub-6-bit activation quantization does not have to sacrifice accuracy when important channels are quantized with extra bits and the discarded shifting errors are encoded with Huffman coding and restored during de-quantization. DQA selects important channels by a greedy offline search over calibration data, then at inference quantizes those channels with $n+m$ bits, right-shifts by $m$ bits so every stored value uses the same $n$ bits, and saves the shifted-off bits as Huffman codes. De-quantization decodes the errors and adds them back, reducing the quantization error by a factor of $2^m$. The paper reports that this beats direct quantization and NoisyQuant on the tested models, with the largest gains at 3 bits and for image-classification CNNs.

Load-bearing premise

Offline channel rankings computed from calibration data remain valid for the data seen during inference, because the paper assumes calibration and inference inputs come from sufficiently similar distributions.

Editorial extensions

If this is right

  • If DQA's results hold, activation quantization at 3 to 5 bits can be accurate without per-input optimization, so memory-bound inference on small devices could use deeper quantization than before.
  • The reported accuracy improvements are largest at 3 bits, the regime where direct quantization collapses and NoisyQuant's noise search only partially recovers accuracy.
  • Because all quantized values keep the same bit width, DQA avoids the storage waste of channel-wise mixed precision while still giving important channels effectively more precision.
  • The method moves the accuracy-motivated work offline into channel ranking, leaving only shifts, lookups, and Huffman coding for online inference, which is a compute profile friendly to simple hardware.
  • On U-Net at 3 bits the paper finds NoisyQuant still wins, so DQA's advantage is so far demonstrated mainly for image-classification CNNs rather than all architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper fixes $m=3$ for all target bit widths; a natural extension would be to search $m$ per layer or per channel, trading side-information size against accuracy, which the paper does not explore.
  • The offline rank is computed on a subset of the training set, so the first thing likely to break is out-of-distribution input; one could stress-test DQA by computing ranks on one dataset and evaluating on a shifted version.
  • DQA's measured Huffman compression ratio is modest (up to 1.12x on CIFAR-10), suggesting the real benefit may be accuracy rather than memory; a hardware implementation would need to count the cost of Huffman tables and per-channel rank storage.
  • The shift-error argument is not architecture-specific, so the method could plausibly extend to transformers or large language models, where NoisyQuant originally operated, though the paper does not test that regime.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DQA, a post-training quantization method for DNN activations at 3–5 bits. DQA identifies 'important' activation channels offline via a greedy search on calibration/training data, quantizes those channels with m extra bits and then right-shifts them to the target bit width, Huffman-encodes the shifting errors, and adds the residuals back during dequantization; the remaining channels are quantized directly. The method is evaluated against direct uniform symmetric quantization and NoisyQuant on ResNet-32 and MobileNetV2 (CIFAR-10) and U-Net (Cityscapes). The paper claims significantly better accuracy (up to 29.28%) and suitability for resource-constrained devices, with simplicity coming from shifts, table lookups, and Huffman coding.

Significance. DQA is appealing in its simplicity: the online operations are shifts and adds, no matrix multiplications or online searches are required, and the error analysis in Eq. (4) gives a clean explanation of why the extra m bits reduce quantization error exponentially. The offline channel-ranking idea is also practical and clearly described. If the accuracy gains held uniformly, the method would be a useful contribution to sub-6-bit activation quantization. However, the evidence as presented is mixed: Table 1 shows DQA losing to NoisyQuant in two of the nine configurations, no variance is reported despite five runs, and the efficiency claims are not measured. The central idea is worth pursuing, but the manuscript's claims need to be brought in line with the evidence, and several load-bearing details need clarification.

major comments (4)
  1. [Abstract, §1, §5, Table 1] The abstract, introduction, and conclusion state that DQA shows 'significantly better accuracy (up to 29.28%)' compared with both direct quantization and NoisyQuant. Table 1 does not support this claim. At 3-bit U-Net, NoisyQuant achieves 92.16% versus DQA's 90.38%, and at 5-bit MobileNetV2, NoisyQuant achieves 91.33% versus DQA's 91.28%. The 29.28 percentage-point figure is the ResNet-32 3-bit gap to Direct (82.13−52.85); the corresponding gap to NoisyQuant is 10.91 percentage points. Please revise the central claims to state the per-baseline and per-configuration results and avoid implying uniform superiority over NoisyQuant.
  2. [§4.1] Every experiment is reported as an average over five runs, but no standard deviation, confidence interval, or significance test is reported anywhere. Without this, 'significantly better' is not established, particularly for the 0.05 percentage-point difference at MobileNetV2 5-bit and the small differences at U-Net 4-bit and 5-bit. Please report variances or run a pairwise significance test between DQA and each baseline.
  3. [§1, §3.2.2, §4.1] The title and abstract claim efficiency, but the paper contains no measurement of inference time, throughput, memory footprint, or energy on any device; all experiments run on an RTX 3090 and Section 4.1 explicitly says the goal is accuracy. The only efficiency-related number is the Huffman compression ratio of 1.12 for CIFAR-10, which does not account for the offline rank table, the Huffman code table, or the precomputed shift-error mapping. Please either add efficiency measurements or explicitly limit the contribution to accuracy and soften the efficiency claims.
  4. [§3.1, §4.1, Algorithm 3] The important-channel ranking is a key component of the reported accuracy gains, but its construction has underspecified choices. First, Section 3.1 says the greedy search runs 'inference on evaluation data,' while Section 4.1 says a random subset of training data is used; if the ranking procedure uses test data, there is data leakage and the results are not a valid post-training quantization evaluation. Second, the important-channel ratios (40% for classification, 50% for segmentation) and m=3 are fixed with no sensitivity analysis, so the reported gains may be partly tuned to these choices. Third, the greedy per-channel ranking is not compared with cheaper alternatives such as magnitude-based channel importance. Please clarify the exact data used for ranking and add ablations or a sensitivity discussion.
minor comments (5)
  1. [Eq. (1), Algorithm 1] The denominator notation in Eq. (1) and Algorithm 1 is ambiguous: '2N−1' and '2n+m−1' could be read as 2^N−1 and 2^{n+m}−1 or as 2^{N-1} and 2^{n+m-1}. This matters because Eq. (4) relies on the relation Δ_N = 2^m Δ_{N+m}. Please state the denominator convention explicitly and ensure Algorithm 2 uses the same convention.
  2. [§3.2.2] Please define the 'compression ratio' precisely. If it is the ratio of the raw size of shifting errors to the size of the Huffman-coded errors, clarify whether the Huffman table size is included, and report the ratio for the Cityscapes/U-Net experiments as well as for CIFAR-10.
  3. [Figure 2] The figure lacks axis labels and a legend. The y-axis presumably shows frequency, but this should be stated, and it is unclear whether the distributions are averaged over batches, channels, or both.
  4. [§4.1] The statement 'we set m = 3 in all cases to simplify the evaluation' means that for 3-bit quantization m equals n, while for 4- and 5-bit it does not. Reporting at least one configuration with a different m would strengthen the claim that the method is not tied to this particular choice.
  5. [Abstract, §4.2] The '29.28%' and '0.9%' figures are percentage-point differences, not relative improvements. Please use 'percentage points' or report relative improvements to avoid misleading readers.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation chain; the only formal result is an algebraic error decomposition, and accuracy claims are evaluated on held-out data.

full rationale

The paper's derivation chain does not reduce to its inputs. Section 3.2.1 defines the shifting error se as the information lost by right-shifting a more finely quantized value and then adds se back at dequantization (Eqs. 3-4). The cancellation of se is therefore an algebraic identity, not an empirical prediction; the paper itself notes the error reduction is 'expected since we add m more bits of information.' The offline importance ranking (Section 3.1) is fitted to calibration/training data, but the reported accuracies in Table 1 are on held-out evaluation data (CIFAR-10 test and Cityscapes), so the improvements are not statistically forced. The only self-citation ([6], by co-author Cano) appears in a future-work sentence about hardware co-design and is not load-bearing. The strongest claim in the abstract ('up to 29.28% ... compared to ... NoisyQuant') is not supported by Table 1, where DQA is worse than NoisyQuant in two of nine configurations and the 29.28% figure is relative to Direct only; this is an internal-evidence/overclaiming problem, not circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method's central accuracy claims rest on the offline channel-importance ranking, a rounding-error model borrowed from AWQ, a Huffman codebook fitted to calibration data, and hand-chosen hyperparameters m=3 and channel ratios 40%/50%. No new physical entities are introduced.

free parameters (3)
  • m (extra bits) = 3
    Set to 3 for all quantization levels and models; the paper states exploring other values is future work (Section 4.1), so it is a hand-chosen constant in the evaluation.
  • Important channel ratio = 40% (classification), 50% (segmentation)
    Chosen by hand for all channels in all layers (Section 4.1); no sensitivity analysis is reported.
  • Huffman code table = Derived from calibration batches
    The code lengths depend on the empirical frequency of shifting errors on a training-data subset (Section 3.2.2); it is fitted to calibration data and not part of a parameter-free derivation.
assumptions (5)
  • standard math Huffman coding produces an optimal prefix code for a given symbol distribution.
    Used in Section 3.2.2 to justify compressing shifting errors.
  • domain assumption Training/calibration and inference data are drawn from sufficiently similar distributions.
    Stated in Section 3.1; this is the load-bearing premise for reusing precomputed channel ranks.
  • domain assumption The average rounding error is 0.25 and independent of the activation value, as used in AWQ.
    Used in Section 3.2.1 to claim exponential error reduction; it is an empirical approximation, not a theorem.
  • domain assumption The shifting-error frequencies observed on calibration batches match inference-time frequencies, and codebook storage overhead is negligible.
    Section 3.2.2 reports compression ratios without including codebook size.
  • ad hoc to paper Greedy per-channel ranking and the fixed 40%/50% important-channel ratios are a good proxy for jointly optimal channel selection.
    Algorithm 3 ranks channels by per-channel accuracy impact but the paper does not justify greedy decomposition or the chosen ratios.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DQA: An Efficient Method for Deep Quantization of Deep Neural Network Activations." pith.science (2026). https://pith.science/paper/GGZP545W

@misc{pith2026241209687,
  author       = {Pith},
  title        = {Pith review of: DQA: An Efficient Method for Deep Quantization of Deep Neural Network Activations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GGZP545W}},
  note         = {Machine review of arXiv:2412.09687}
}
read the original abstract

Quantization of Deep Neural Network (DNN) activations is a commonly used technique to reduce compute and memory demands during DNN inference, which can be particularly beneficial on resource-constrained devices. To achieve high accuracy, existing methods for quantizing activations rely on complex mathematical computations or perform extensive searches for the best hyper-parameters. However, these expensive operations are impractical on devices with limited computation capabilities, memory capacities, and energy budgets. Furthermore, many existing methods do not focus on sub-6-bit (or deep) quantization. To fill these gaps, in this paper we propose DQA (Deep Quantization of DNN Activations), a new method that focuses on sub-6-bit quantization of activations and leverages simple shifting-based operations and Huffman coding to be efficient and achieve high accuracy. We evaluate DQA with 3, 4, and 5-bit quantization levels and three different DNN models for two different tasks, image classification and image segmentation, on two different datasets. DQA shows significantly better accuracy (up to 29.28%) compared to the direct quantization method and the state-of-the-art NoisyQuant for sub-6-bit quantization.

Figures

Figures reproduced from arXiv: 2412.09687 by the authors.

Figure 1
Figure 1. DQA overview. 1 offline, rank the activation channels based on importance using training/calibration data and a greedy search algorithm (green circles represent the most important channels for which we skip quantization); 2 during inference, quantize important activation channels with m extra bits and then right-shift them while saving the shifting errors; 3 the shifting errors are Huffman-encoded to reduce the memo… view at source ↗
Figure 2
Figure 2. Average frequency distribution of shifting errors for ResNet-32 and CIFAR-10 with 3, 4, [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 11 canonical work pages

  1. [1]

    arXiv preprint arXiv:2410.13056 (2024)

    Chen, Z., Xie, B., Li, J., Shen, C.: Channel-wise mixed-precision quantization for large language models. arXiv preprint arXiv:2410.13056 (2024)

  2. [2]

    In: Proc

    Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  3. [3]

    In: International Conference on Learning Representations (2021)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on Learning Representations (2021)

  4. [4]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Elhoushi, M., Chen, Z., Shafiq, F., Tian, Y .H., Li, J.Y .: Deepshift: Towards multiplication-less neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 2359–2368 (2021)

  5. [5]

    In: Low-Power Computer Vision, pp

    Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M.W., Keutzer, K.: A survey of quantization methods for efficient neural network inference. In: Low-Power Computer Vision, pp. 291–326. Chapman and Hall/CRC (2022)

  6. [6]

    ACM Transactions on Architecture and Code Optimization (TACO) (2024)

    Gibson, P., Cano, J., Crowley, E.J., Storkey, A., O’Boyle, M.: DLAS: A Conceptual Model for Across-Stack Deep Learning Acceleration. ACM Transactions on Architecture and Code Optimization (TACO) (2024)

  7. [7]

    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 770–778 (2016)

  8. [8]

    arXiv:1704.04861 (Apr 2017)

    Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv:1704.04861 (Apr 2017)

Show all 17 references
  1. [9]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

    Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., Kalenichenko, D.: Quantization and training of neural networks for efficient integer-arithmetic-only inference. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). p...

  2. [10]

    TR (2009)

    Krizhevsky, A.: Learning Multiple Layers of Features from Tiny Images. TR (2009)

  3. [11]

    In: Gibbons, P., Pekhimenko, G., Sa, C.D

    Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.M., Wang, W.C., Xiao, G., Dang, X., Gan, C., Han, S.: Awq: Activation-aware weight quantization for on-device llm compression and acceleration. In: Gibbons, P., Pekhimenko, G., Sa, C.D. (eds.) Proceedings of Machine Learning and S...

  4. [12]

    IEEE Journal of Solid-State Circuits 59(9), 3082–3093 (2024)

    Liu, R., Wei, C., Yang, Y ., Wang, W., Yuan, B., Yang, H., Liu, Y .: A dynamic execution neural network processor for fine-grained mixed-precision model training based on online quantization sensitivity analysis. IEEE Journal of Solid-State Circuits 59(9), 3082–3093 (2024)

  5. [13]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Liu, Y ., Yang, H., Dong, Z., Keutzer, K., Du, L., Zhang, S.: Noisyquant: Noisy bias-enhanced post-training activation quantization for vision transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20321–20330 (2023)

  6. [14]

    In: NeurIPS (2019)

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: Pytorch: an imperativ...

  7. [15]

    arXiv preprint arXiv:2208.06064 (2022)

    Rakka, M., Fouda, M.E., Khargonekar, P., Kurdahi, F.: Mixed-precision neural networks: A survey. arXiv preprint arXiv:2208.06064 (2022)

  8. [16]

    In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. pp....

  9. [17]

    In: ICALP

    Van Leeuwen, J.: On the construction of huffman trees. In: ICALP. pp. 382–410 (1976) 7 Appendix Algorithm 3 Ranking Important Activation Channels Using Greedy Search 1: Input: Model M, Training or Calibration Dataset D 2: Output: All ranks of activation channels R 3: R = {} 4:...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.