REVIEW 4 major objections 5 minor 17 references
DQA: An Efficient Method for Deep Quantization of Deep Neural Network Activations
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Sub-6-bit activation quantization can stay accurate if important channels get extra bits, right-shifted back, with the discarded errors stored as Huffman codes.
desk verdict A useful but overclaimed activation quantization paper: the method mostly works, but the abstract's superiority over NoisyQuant is contradicted by its own Table 1. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism carrying the argument is a right-shift compensation loop. An important channel is first quantized with $m$ extra bits; a right shift by $m$ bits drops those extra bits, and the dropped bits (the shifting error) are saved as a Huffman code. Because the quantization step size for the $n+m$-bit representation is $2^m$ times finer, re-adding the shifting error yields a residual error $2^m$ times smaller than direct quantization of the same channel. An offline greedy channel ranking decides which channels receive this treatment, and that combination of rank, shift, and Huffman side channel produces the accuracy gains without online search or heavy mathematical operations.
What would settle it
Run DQA with ranks computed on clean CIFAR-10 but evaluate on a corrupted or out-of-distribution variant, and compare accuracy against direct quantization; if DQA no longer beats direct quantization once channel importance shifts, the offline-ranking assumption is falsified. Alternatively, implement DQA on a microcontroller and measure end-to-end latency and energy; if Huffman decoding and side-channel overhead outweigh the savings, the resource-constrained motivation collapses.
Extended reading notes
Core claim
DQA's central claim is that sub-6-bit activation quantization does not have to sacrifice accuracy when important channels are quantized with extra bits and the discarded shifting errors are encoded with Huffman coding and restored during de-quantization. DQA selects important channels by a greedy offline search over calibration data, then at inference quantizes those channels with $n+m$ bits, right-shifts by $m$ bits so every stored value uses the same $n$ bits, and saves the shifted-off bits as Huffman codes. De-quantization decodes the errors and adds them back, reducing the quantization error by a factor of $2^m$. The paper reports that this beats direct quantization and NoisyQuant on the tested models, with the largest gains at 3 bits and for image-classification CNNs.
Load-bearing premise
Offline channel rankings computed from calibration data remain valid for the data seen during inference, because the paper assumes calibration and inference inputs come from sufficiently similar distributions.
Editorial extensions
If this is right
- If DQA's results hold, activation quantization at 3 to 5 bits can be accurate without per-input optimization, so memory-bound inference on small devices could use deeper quantization than before.
- The reported accuracy improvements are largest at 3 bits, the regime where direct quantization collapses and NoisyQuant's noise search only partially recovers accuracy.
- Because all quantized values keep the same bit width, DQA avoids the storage waste of channel-wise mixed precision while still giving important channels effectively more precision.
- The method moves the accuracy-motivated work offline into channel ranking, leaving only shifts, lookups, and Huffman coding for online inference, which is a compute profile friendly to simple hardware.
- On U-Net at 3 bits the paper finds NoisyQuant still wins, so DQA's advantage is so far demonstrated mainly for image-classification CNNs rather than all architectures.
Reading between the lines
- The paper fixes $m=3$ for all target bit widths; a natural extension would be to search $m$ per layer or per channel, trading side-information size against accuracy, which the paper does not explore.
- The offline rank is computed on a subset of the training set, so the first thing likely to break is out-of-distribution input; one could stress-test DQA by computing ranks on one dataset and evaluating on a shifted version.
- DQA's measured Huffman compression ratio is modest (up to 1.12x on CIFAR-10), suggesting the real benefit may be accuracy rather than memory; a hardware implementation would need to count the cost of Huffman tables and per-channel rank storage.
- The shift-error argument is not architecture-specific, so the method could plausibly extend to transformers or large language models, where NoisyQuant originally operated, though the paper does not test that regime.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DQA, a post-training quantization method for DNN activations at 3–5 bits. DQA identifies 'important' activation channels offline via a greedy search on calibration/training data, quantizes those channels with m extra bits and then right-shifts them to the target bit width, Huffman-encodes the shifting errors, and adds the residuals back during dequantization; the remaining channels are quantized directly. The method is evaluated against direct uniform symmetric quantization and NoisyQuant on ResNet-32 and MobileNetV2 (CIFAR-10) and U-Net (Cityscapes). The paper claims significantly better accuracy (up to 29.28%) and suitability for resource-constrained devices, with simplicity coming from shifts, table lookups, and Huffman coding.
Significance. DQA is appealing in its simplicity: the online operations are shifts and adds, no matrix multiplications or online searches are required, and the error analysis in Eq. (4) gives a clean explanation of why the extra m bits reduce quantization error exponentially. The offline channel-ranking idea is also practical and clearly described. If the accuracy gains held uniformly, the method would be a useful contribution to sub-6-bit activation quantization. However, the evidence as presented is mixed: Table 1 shows DQA losing to NoisyQuant in two of the nine configurations, no variance is reported despite five runs, and the efficiency claims are not measured. The central idea is worth pursuing, but the manuscript's claims need to be brought in line with the evidence, and several load-bearing details need clarification.
major comments (4)
- [Abstract, §1, §5, Table 1] The abstract, introduction, and conclusion state that DQA shows 'significantly better accuracy (up to 29.28%)' compared with both direct quantization and NoisyQuant. Table 1 does not support this claim. At 3-bit U-Net, NoisyQuant achieves 92.16% versus DQA's 90.38%, and at 5-bit MobileNetV2, NoisyQuant achieves 91.33% versus DQA's 91.28%. The 29.28 percentage-point figure is the ResNet-32 3-bit gap to Direct (82.13−52.85); the corresponding gap to NoisyQuant is 10.91 percentage points. Please revise the central claims to state the per-baseline and per-configuration results and avoid implying uniform superiority over NoisyQuant.
- [§4.1] Every experiment is reported as an average over five runs, but no standard deviation, confidence interval, or significance test is reported anywhere. Without this, 'significantly better' is not established, particularly for the 0.05 percentage-point difference at MobileNetV2 5-bit and the small differences at U-Net 4-bit and 5-bit. Please report variances or run a pairwise significance test between DQA and each baseline.
- [§1, §3.2.2, §4.1] The title and abstract claim efficiency, but the paper contains no measurement of inference time, throughput, memory footprint, or energy on any device; all experiments run on an RTX 3090 and Section 4.1 explicitly says the goal is accuracy. The only efficiency-related number is the Huffman compression ratio of 1.12 for CIFAR-10, which does not account for the offline rank table, the Huffman code table, or the precomputed shift-error mapping. Please either add efficiency measurements or explicitly limit the contribution to accuracy and soften the efficiency claims.
- [§3.1, §4.1, Algorithm 3] The important-channel ranking is a key component of the reported accuracy gains, but its construction has underspecified choices. First, Section 3.1 says the greedy search runs 'inference on evaluation data,' while Section 4.1 says a random subset of training data is used; if the ranking procedure uses test data, there is data leakage and the results are not a valid post-training quantization evaluation. Second, the important-channel ratios (40% for classification, 50% for segmentation) and m=3 are fixed with no sensitivity analysis, so the reported gains may be partly tuned to these choices. Third, the greedy per-channel ranking is not compared with cheaper alternatives such as magnitude-based channel importance. Please clarify the exact data used for ranking and add ablations or a sensitivity discussion.
minor comments (5)
- [Eq. (1), Algorithm 1] The denominator notation in Eq. (1) and Algorithm 1 is ambiguous: '2N−1' and '2n+m−1' could be read as 2^N−1 and 2^{n+m}−1 or as 2^{N-1} and 2^{n+m-1}. This matters because Eq. (4) relies on the relation Δ_N = 2^m Δ_{N+m}. Please state the denominator convention explicitly and ensure Algorithm 2 uses the same convention.
- [§3.2.2] Please define the 'compression ratio' precisely. If it is the ratio of the raw size of shifting errors to the size of the Huffman-coded errors, clarify whether the Huffman table size is included, and report the ratio for the Cityscapes/U-Net experiments as well as for CIFAR-10.
- [Figure 2] The figure lacks axis labels and a legend. The y-axis presumably shows frequency, but this should be stated, and it is unclear whether the distributions are averaged over batches, channels, or both.
- [§4.1] The statement 'we set m = 3 in all cases to simplify the evaluation' means that for 3-bit quantization m equals n, while for 4- and 5-bit it does not. Reporting at least one configuration with a different m would strengthen the claim that the method is not tied to this particular choice.
- [Abstract, §4.2] The '29.28%' and '0.9%' figures are percentage-point differences, not relative improvements. Please use 'percentage points' or report relative improvements to avoid misleading readers.
Circularity Check
No circular derivation chain; the only formal result is an algebraic error decomposition, and accuracy claims are evaluated on held-out data.
full rationale
The paper's derivation chain does not reduce to its inputs. Section 3.2.1 defines the shifting error se as the information lost by right-shifting a more finely quantized value and then adds se back at dequantization (Eqs. 3-4). The cancellation of se is therefore an algebraic identity, not an empirical prediction; the paper itself notes the error reduction is 'expected since we add m more bits of information.' The offline importance ranking (Section 3.1) is fitted to calibration/training data, but the reported accuracies in Table 1 are on held-out evaluation data (CIFAR-10 test and Cityscapes), so the improvements are not statistically forced. The only self-citation ([6], by co-author Cano) appears in a future-work sentence about hardware co-design and is not load-bearing. The strongest claim in the abstract ('up to 29.28% ... compared to ... NoisyQuant') is not supported by Table 1, where DQA is worse than NoisyQuant in two of nine configurations and the 29.28% figure is relative to Direct only; this is an internal-evidence/overclaiming problem, not circularity.
Assumptions & free parameters
free parameters (3)
- m (extra bits) =
3
- Important channel ratio =
40% (classification), 50% (segmentation)
- Huffman code table =
Derived from calibration batches
assumptions (5)
- standard math Huffman coding produces an optimal prefix code for a given symbol distribution.
- domain assumption Training/calibration and inference data are drawn from sufficiently similar distributions.
- domain assumption The average rounding error is 0.25 and independent of the activation value, as used in AWQ.
- domain assumption The shifting-error frequencies observed on calibration batches match inference-time frequencies, and codebook storage overhead is negligible.
- ad hoc to paper Greedy per-channel ranking and the fixed 40%/50% important-channel ratios are a good proxy for jointly optimal channel selection.
Cite this review
Pith. "Pith review of DQA: An Efficient Method for Deep Quantization of Deep Neural Network Activations." pith.science (2026). https://pith.science/paper/GGZP545W
@misc{pith2026241209687,
author = {Pith},
title = {Pith review of: DQA: An Efficient Method for Deep Quantization of Deep Neural Network Activations},
year = {2026},
howpublished = {\url{https://pith.science/paper/GGZP545W}},
note = {Machine review of arXiv:2412.09687}
}
read the original abstract
Quantization of Deep Neural Network (DNN) activations is a commonly used technique to reduce compute and memory demands during DNN inference, which can be particularly beneficial on resource-constrained devices. To achieve high accuracy, existing methods for quantizing activations rely on complex mathematical computations or perform extensive searches for the best hyper-parameters. However, these expensive operations are impractical on devices with limited computation capabilities, memory capacities, and energy budgets. Furthermore, many existing methods do not focus on sub-6-bit (or deep) quantization. To fill these gaps, in this paper we propose DQA (Deep Quantization of DNN Activations), a new method that focuses on sub-6-bit quantization of activations and leverages simple shifting-based operations and Huffman coding to be efficient and achieve high accuracy. We evaluate DQA with 3, 4, and 5-bit quantization levels and three different DNN models for two different tasks, image classification and image segmentation, on two different datasets. DQA shows significantly better accuracy (up to 29.28%) compared to the direct quantization method and the state-of-the-art NoisyQuant for sub-6-bit quantization.
Figures
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2410.13056 (2024)
Chen, Z., Xie, B., Li, J., Shen, C.: Channel-wise mixed-precision quantization for large language models. arXiv preprint arXiv:2410.13056 (2024)
arXiv 2024
-
[2]
In: Proc
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
2016
-
[3]
In: International Conference on Learning Representations (2021)
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on Learning Representations (2021)
2021
-
[4]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Elhoushi, M., Chen, Z., Shafiq, F., Tian, Y .H., Li, J.Y .: Deepshift: Towards multiplication-less neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 2359–2368 (2021)
work page 2021
-
[5]
In: Low-Power Computer Vision, pp
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M.W., Keutzer, K.: A survey of quantization methods for efficient neural network inference. In: Low-Power Computer Vision, pp. 291–326. Chapman and Hall/CRC (2022)
work page 2022
-
[6]
ACM Transactions on Architecture and Code Optimization (TACO) (2024)
Gibson, P., Cano, J., Crowley, E.J., Storkey, A., O’Boyle, M.: DLAS: A Conceptual Model for Across-Stack Deep Learning Acceleration. ACM Transactions on Architecture and Code Optimization (TACO) (2024)
work page 2024
-
[7]
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 770–778 (2016)
work page 2016
-
[8]
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv:1704.04861 (Apr 2017)
arXiv 2017
Show all 17 references
-
[9]
In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., Kalenichenko, D.: Quantization and training of neural networks for efficient integer-arithmetic-only inference. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). p...
2018
-
[10]
TR (2009)
Krizhevsky, A.: Learning Multiple Layers of Features from Tiny Images. TR (2009)
2009
-
[11]
In: Gibbons, P., Pekhimenko, G., Sa, C.D
Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.M., Wang, W.C., Xiao, G., Dang, X., Gan, C., Han, S.: Awq: Activation-aware weight quantization for on-device llm compression and acceleration. In: Gibbons, P., Pekhimenko, G., Sa, C.D. (eds.) Proceedings of Machine Learning and S...
2024
-
[12]
IEEE Journal of Solid-State Circuits 59(9), 3082–3093 (2024)
Liu, R., Wei, C., Yang, Y ., Wang, W., Yuan, B., Yang, H., Liu, Y .: A dynamic execution neural network processor for fine-grained mixed-precision model training based on online quantization sensitivity analysis. IEEE Journal of Solid-State Circuits 59(9), 3082–3093 (2024)
2024
-
[13]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Liu, Y ., Yang, H., Dong, Z., Keutzer, K., Du, L., Zhang, S.: Noisyquant: Noisy bias-enhanced post-training activation quantization for vision transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20321–20330 (2023)
2023
-
[14]
In: NeurIPS (2019)
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: Pytorch: an imperativ...
2019
-
[15]
arXiv preprint arXiv:2208.06064 (2022)
Rakka, M., Fouda, M.E., Khargonekar, P., Kurdahi, F.: Mixed-precision neural networks: A survey. arXiv preprint arXiv:2208.06064 (2022)
2022 arXiv
-
[16]
In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. pp....
2015
-
[17]
In: ICALP
Van Leeuwen, J.: On the construction of huffman trees. In: ICALP. pp. 382–410 (1976) 7 Appendix Algorithm 3 Ranking Important Activation Channels Using Greedy Search 1: Input: Model M, Training or Calibration Dataset D 2: Output: All ranks of activation channels R 3: R = {} 4:...
1976
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.