Pith. sign in

REVIEW 2 major objections 6 minor 61 references

Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpart

T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By training each low-precision block alongside a frozen full-precision successor network, BWRF achieves state-of-the-art top-1 accuracy for 4-, 3-, and 2-bit quantization on ImageNet and CIFAR-10 with no extra inference cost.

desk verdict BWRF is a clever new QAT wrapper with a plausible mechanism, but its SOTA claim rests on comparisons to a stronger teacher; the mechanism itself has solid internal support. read the letter →

arxiv 2412.15846 v1 pith:VS65V4OW submitted 2024-12-20 cs.LG

Chengting Yu , Shu Yang , Fengzhao Zhang , Hanzhi Ma , Aili Wang , Er-Ping Li This is my paper · ORCID
classification cs.LG
keywords quantization-awaretraininglow-precisionnetworkquantizationblock-wisereplacementmixed-precisionmodelsknowledgedistillationuniformstraight-throughestimatorImageNetclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Quantization-aware training of very low-bit networks typically loses accuracy twice: the discrete quantizer cannot represent what the full-precision network computes, and the straight-through estimator feeds back imperfect gradients. This paper proposes BWRF, a general wrapper that keeps a frozen full-precision copy of the network and, during training, builds mixed-precision models by replacing the last low-precision blocks with the corresponding full-precision blocks. Each quantized block is thereby trained while receiving full-precision features downstream and improved gradients from the full-precision branches. The paper reports state-of-the-art results for 4-, 3-, and 2-bit weight-and-activation quantization on ImageNet and CIFAR-10, with gains up to +2.4 top-1 points over its retrained LSQ baseline, and the deployed network is unchanged at inference.

What carries the argument

The mechanism is block-wise replacement on the full-precision counterpart. The network is split into blocks, and for each $k$ a mixed-precision model $M^k$ reuses the computed low-precision features through block $k$ and then continues through fixed full-precision blocks; because the low-precision forward flow is reused, no separate forward passes are needed. The losses combine $L_{\text{target}}$ (cross-entropy for the low-precision output and each mixed-precision output) and $L_{\text{distill}}$ (KL distillation from the full-precision output and from the averaged ensemble of earlier mixed-precision outputs), so each quantized block is pushed to imitate the full-precision representation while still solving the task.

What would settle it

Run the exact BWRF training and the LSQ baseline on ImageNet ResNet-18 at 4, 3, and 2 bits while using the same vanilla pretrained ResNet-18 as both the initialization and the frozen full-precision counterpart; if the top-1 gap over LSQ falls to near zero, the claimed gains are driven by teacher quality, not by the block-replacement mechanism.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a low-precision network can be trained as the backbone of a family of mixed-precision models $M^k = \{Q_1,\dots,Q_k,F_{k+1},\dots,F_n\}$, formed by splicing the first $k$ trainable low-precision blocks onto the frozen remaining full-precision blocks. Training with the combined objective $L = L_{\text{target}} + L_{\text{distill}}$ (Eqs. 8–11) lets each quantized block simulate full-precision representation in the forward pass and receive better gradient estimates in the backward pass. The paper claims this consistently improves over prior QAT methods: for example, BWRF reaches 71.9%, 70.8%, and 67.7% top-1 on ImageNet ResNet-18 at 4, 3, and 2 bits, and 79.0%, 77.8%, and 74.2% on ResNet-50, outperforming both uniform and most non-uniform quantizers.

Load-bearing premise

The load-bearing premise is that the full-precision counterpart is a fair and representative teacher: if the reported gains come mostly from the stronger pretrained full-precision models used in BWRF's experiments rather than from block replacement itself, the advantage over published baselines could shrink in the standard setting.

Editorial extensions

If this is right

  • BWRF acts as a wrapper: attaching it to an existing QAT method such as LSQ raises ImageNet ResNet-18 top-1 by +1.5, +1.5, and +2.4 points at 4, 3, and 2 bits relative to the same baseline.
  • Uniform quantization under BWRF matches or beats non-uniform quantizers in almost every setting, so the inference-friendly uniform format is enough at these bit widths.
  • The mixed-precision branches are dropped at inference, so the deployed model has exactly the same architecture and compute cost as the low-precision backbone.
  • The same framework extends to non-uniform quantizers with a small further gain, but the authors recommend uniform quantization for simplicity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the reported full-precision counterpart for ResNet-18 was itself trained with knowledge distillation and the ResNet-34/50 counterparts come from improved pretrained weights, the absolute numbers may include a teacher-strength effect; the paper does not run its QKD/QFD comparisons with the same teacher, so part of the outperformance could be teacher-driven rather than a
  • Editorial inference: BWRF's block-by-block alignment suggests a general 'progressive grafting' recipe — a compressed or quantized subnet trained against a frozen full-precision parent — that could transfer to pruning, sparsity, or layer-dropping where a full-precision model guides a structurally thinner version.
  • Editorial inference: the authors note the largest supervision gap is between the first mixed-precision model and the full-precision model; a testable extension is to split the first low-precision block into smaller subblocks to supply finer-grained guidance, which they explicitly suggest.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes Block-wise Replacement Framework (BWRF), a wrapper for quantization-aware training. Given a low-precision (LP) network and its full-precision (FP) counterpart, BWRF constructs mixed-precision (MP) models M^k by replacing the last blocks of the LP network with frozen FP blocks. The LP blocks are trained with a combined objective (Eqs. 8--11) that includes task cross-entropy on the LP and MP outputs, knowledge-distillation losses from the FP teacher and from an ensemble of MP outputs, and per-branch loss weights alpha_k. During inference only the original LP network is used, so there is no added inference cost. Experiments on ImageNet (ResNet-18/34/50) and CIFAR-10 (ResNet-20/56) report improved accuracy over published QAT methods at 4-, 3-, and 2-bit precision, and ablations show monotonic gains as MP branches and auxiliary losses are added.

Significance. If the state-of-the-art claim held, BWRF would be a simple, broadly applicable QAT wrapper that improves low-bit accuracy without changing the deployed model. The core mechanism is plausible: frozen FP blocks provide better backward gradients and feature alignment, and the internal ablations (Tables IV and V) consistently show that adding MP branches and auxiliary losses helps. The framework also extends beyond LSQ to at least one non-uniform quantizer (Table VI). However, the external comparisons are confounded by the use of stronger-than-standard FP teachers, so the SOTA claim is not yet established. The paper does not ship code or machine-checked proofs, but the empirical internal controls are a useful strength.

major comments (2)
  1. [Section IV-A and Tables I-II] The comparison against published QKD and QFD results is confounded by teacher strength. The text states, 'For FP counterparts, we trained ResNet18 with knowledge distillation and applied pre-trained ResNet-34 and ResNet-50 from TIMM,' and the resulting ResNet-18 teacher reaches 72.26 top-1, which is well above a vanilla pretrained ResNet-18. Since Eq. (9) directly includes Lkd(yQ, yF) and related distillation losses from this teacher, a stronger teacher can inflate the reported LP accuracy independently of the block-replacement mechanism. No same-teacher baselines for QKD or QFD are provided; the only controlled baseline, LSQ†, shares the teacher but is not a distillation-based method. The reported margins over QKD/QFD (e.g., +0.5 top-1 at ResNet-18 4-bit in Table I) may therefore be attributable to teacher strength rather than BWRF. The authors should re-run QKD and QFD with the same KD-trained ResNet-18 teacher (or, alternatively, report BWRF with a standard pretrained teacher) to support the SOTA claim.
  2. [Section IV-B and Tables I-II] The 'state-of-the-art' claim is not supported at 2-bit precision. In Table I, LCQ (non-uniform) achieves 68.9 top-1 at ResNet-18 2-bit, while BWRF achieves 67.7; in Table II, LCQ also beats BWRF at ResNet-34 2-bit (72.7 vs. 71.7) and ResNet-50 2-bit (75.1 vs. 74.2). The paper acknowledges 'almost all cases' rather than all, but the abstract and contributions state state-of-the-art results without this qualification. The central claim should be revised to reflect the actual comparisons, or additional experiments should be provided to justify the SOTA label for 2-bit cases.
minor comments (6)
  1. [Contributions] The contribution list says '4-, 3-, and 4-bit widths'; this should be '4-, 3-, and 2-bit widths.'
  2. [Section IV-A] The description of the KD-trained ResNet-18 teacher lacks detail: no teacher network, KD temperature, or loss weight is specified, which makes the setup hard to reproduce.
  3. [Section IV-A and Table I] The LSQ† baseline is referred to as a 'preliminary implementation,' but no explanation is given for why its 4-bit accuracy (70.4) is lower than the published LSQ number (71.1) despite sharing the same experimental setting; this discrepancy should be clarified.
  4. [Tables IV and V] The table captions contain the typo 'CIAFR-10'; this should be 'CIFAR-10.'
  5. [Section II and Table II] There are minor typographical errors: 'Reset series' in Section IV-A should be 'ResNet series,' and 'DeReFa-Nets' in Table II should be 'DoReFa-Net.'
  6. [Abstract and Section V] The paper claims the framework is 'a compatible extension for most QAT methods,' but experiments only evaluate LSQ and one non-uniform quantizer (APoT in Table VI). The claim should be tempered or supported by additional base-method experiments.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BWRF's central result is an empirical accuracy comparison against external ImageNet and CIFAR-10 benchmarks, not a quantity defined in terms of itself.

full rationale

The paper's derivation chain is not circular. The proposed training objective L = L_target + L_distill (Eqs. 8-11) combines task cross-entropies with KL distillation losses against the frozen full-precision output yF and mixed-precision outputs yMk; the final reported quantity, low-precision validation accuracy, is measured on external labeled validation sets and compared with published QAT numbers. No parameter is fitted to the reported result: the alpha_k loss weights are hand-chosen hyperparameters, and the full-precision weights are fixed pretrained checkpoints. The ablation tables (Tables IV-V) vary the loss terms and MP branches, giving an internal consistency check but not an identity. There are no load-bearing self-citations, no imported uniqueness theorem, and no ansatz smuggled in by citation; the only self-referential element is the self-distillation average yavg, which is an auxiliary training signal, not the evaluation target. The reviewer's concern that the ResNet-18 FP teacher is KD-trained and QKD/QFD baselines may use weaker teachers is a fairness/confounding issue about whether the SOTA comparison is matched; it is not a circularity in the derivation, because the reported accuracies are still externally measured and the method would need re-benchmarking rather than logical unpacking. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the quality of the pretrained FP teacher, the STE approximation, the hand-chosen alpha_k weights, and the block-partition granularity. The last two are not reported, and the strength of the teacher is not controlled in the headline comparisons. No new physical entities are introduced.

free parameters (2)
  • alpha_k loss weights = not reported
    Introduced in Eqs. 8 and 9 to balance task and distillation losses for each mixed-precision branch. The paper does not state their values or tuning procedure, so the central results depend on hand-chosen weights the reader cannot inspect.
  • block partition granularity = res-block level, n not stated
    Section III-A partitions the network into blocks by code-level res-blocks and image resolution. The number n and exact split are not reported and are not ablated, yet they define the auxiliary branches and thus the training signal.
assumptions (4)
  • domain assumption The pretrained full-precision counterpart is a reliable teacher for every quantized block.
    All auxiliary supervision in Eqs. 8-10 comes from frozen FP blocks. Section III-B relies on F being trustworthy.
  • domain assumption Straight-through estimator gradients (Eqs. 2-3) are an adequate surrogate for the nondifferentiable quantizer.
    The framework inherits STE from LSQ and APoT in Section III-A. BWRF does not correct the gradient mismatch itself.
  • domain assumption Mixed-precision outputs yM^k and their averages yavg_M^k provide useful supervision rather than noise.
    The self-distillation objective Eq. 9 treats branch predictions as soft labels. The paper verifies this only empirically for 4-bit CIFAR-10 ablations.
  • domain assumption Freezing FP weights while training LP weights is beneficial.
    Section III-B states FP weights remain constant during training. No ablation compares this against fine-tuning FP blocks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpart." pith.science (2026). https://pith.science/paper/VS65V4OW

@misc{pith2026241215846,
  author       = {Pith},
  title        = {Pith review of: Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpart},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VS65V4OW}},
  note         = {Machine review of arXiv:2412.15846}
}
read the original abstract

Quantization-aware training (QAT) is a common paradigm for network quantization, in which the training phase incorporates the simulation of the low-precision computation to optimize the quantization parameters in alignment with the task goals. However, direct training of low-precision networks generally faces two obstacles: 1. The low-precision model exhibits limited representation capabilities and cannot directly replicate full-precision calculations, which constitutes a deficiency compared to full-precision alternatives; 2. Non-ideal deviations during gradient propagation are a common consequence of employing pseudo-gradients as approximations in derived quantized functions. In this paper, we propose a general QAT framework for alleviating the aforementioned concerns by permitting the forward and backward processes of the low-precision network to be guided by the full-precision partner during training. In conjunction with the direct training of the quantization model, intermediate mixed-precision models are generated through the block-by-block replacement on the full-precision model and working simultaneously with the low-precision backbone, which enables the integration of quantized low-precision blocks into full-precision networks throughout the training phase. Consequently, each quantized block is capable of: 1. simulating full-precision representation during forward passes; 2. obtaining gradients with improved estimation during backward passes. We demonstrate that the proposed method achieves state-of-the-art results for 4-, 3-, and 2-bit quantization on ImageNet and CIFAR-10. The proposed framework provides a compatible extension for most QAT methods and only requires a concise wrapper for existing codes.

Figures

Figures reproduced from arXiv: 2412.15846 by the authors.

Figure 1
Figure 1. Framework Overview. (a) The fundamental implementation of Quantization-aware Training (QAT) in which weight initialization is performed using full [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Implementation of BWRF Training. Mixed-precision models are [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Validation accuracy of mixed-precision models and low-precision backbone during training. The results are obtained by ResNet-18 on ImageNet. that the replacement between intermediate blocks provides more reliable support. D. Analysis and Discussion Extension to non-uniform quantization. The adaptability of BWRF to the non-uniform design was assessed, as illustrated in Tab. VI. It is not unexpected that performance c… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Measures of feature similarity. The results of cosine distances are obtained by ResNet-18 on CIFAR-10 under 4-bit quantization [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualization results of class activation mapping (CAM). Two visualized targets are established on the third and fourth blocks’ output layers. (a-b) The results of the full-precision model F and low-precision model Q trained with BWRF. (c) The results of the vanilla mo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 35 canonical work pages

  1. [1]

    Post training 4-bit quantization of convolutional networks for rapid-deployment

    Ron Banner, Yury Nahshan, and Daniel Soudry. Post training 4-bit quantization of convolutional networks for rapid-deployment. Advances in Neural Information Processing Systems, 32, 2019

    work page 2019

  2. [2]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. 9

    arXiv 2013

  3. [3]

    Lsq+: Improving low- bit quantization through learnable offsets and better ini- tialization

    Yash Bhalgat, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, and Nojun Kwak. Lsq+: Improving low- bit quantization through learnable offsets and better ini- tialization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 696–697, 2020

    work page 2020

  4. [4]

    Stochastic precision ensemble: self-knowledge distillation for quantized deep neural networks

    Yoonho Boo, Sungho Shin, Jungwook Choi, and Wony- ong Sung. Stochastic precision ensemble: self-knowledge distillation for quantized deep neural networks. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 35, pages 6794–6802, 2021

    work page 2021

  5. [5]

    Pact: Parameterized clipping activation for quantized neural networks

    Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.06085, 2018

    arXiv 2018

  6. [6]

    Low-bit quantization of neural networks for efficient inference

    Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neural networks for efficient inference. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) , pages 3009–3018. IEEE, 2019

    work page 2019

  7. [7]

    Fast and practical neural architecture search

    Jiequan Cui, Pengguang Chen, Ruiyu Li, Shu Liu, Xi- aoyong Shen, and Jiaya Jia. Fast and practical neural architecture search. In Proceedings of the IEEE/CVF in- ternational conference on computer vision , pages 6509– 6518, 2019

    work page 2019

  8. [8]

    Learned step size quantization

    Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. Learned step size quantization. In International Confer- ence on Learning Representations , 2020

    work page 2020

Show all 61 references
  1. [9]

    Fighting quantization bias with bias

    Alexander Finkelstein, Uri Almog, and Mark Grobman. Fighting quantization bias with bias. arXiv preprint arXiv:1906.03193, 2019

    1906 arXiv

  2. [10]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 , 2018

    2018 arXiv

  3. [11]

    Rich feature hierarchies for accurate object detection and semantic segmentation

    Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014

    2014

  4. [12]

    Differentiable soft quantization: Bridging full-precision and low-bit neural networks

    Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proceedings of the IEEE/CVF international conference on computer vision , pages ...

    2019

  5. [13]

    Network sketching: Exploiting binary structure in deep cnns

    Yiwen Guo, Anbang Yao, Hao Zhao, and Yurong Chen. Network sketching: Exploiting binary structure in deep cnns. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 5955–5963, 2017

    2017

  6. [14]

    Improving low-precision network quantization via bin regularization

    Tiantian Han, Dong Li, Ji Liu, Lu Tian, and Yi Shan. Improving low-precision network quantization via bin regularization. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision , pages 5261– 5270, 2021

    2021

  7. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770–778, 2016

    2016

  8. [17]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE in- ternational conference on computer vision , pages 2961– 2969, 2017

    2017

  9. [18]

    Neural networks for machine learning lecture 6a overview of mini-batch gradient descent

    Geoffrey Hinton, Nitish Srivastava, and Kevin Swer- sky. Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited on , 14 (8):2, 2012

    2012

  10. [19]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  11. [20]

    Searching for mobilenetv3

    Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mobilenetv3. In Proceedings of the IEEE/CVF in- ternational conference on computer vision , pages 1314– 1324, 2019

    2019

  12. [21]

    Improving post training neural quantization: Layer-wise calibration and integer pro- gramming

    Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Improving post training neural quantization: Layer-wise calibration and integer pro- gramming. arXiv preprint arXiv:2006.10518 , 2020

    2006 arXiv

  13. [22]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision an...

    2018

  14. [23]

    Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks

    Sambhav Jain, Albert Gural, Michael Wu, and Chris Dick. Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks. Proceedings of Machine Learning and Systems , 2:112– 128, 2020

    2020

  15. [24]

    Layercam: Exploring hierarchical class activation maps for localization

    Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming- Ming Cheng, and Yunchao Wei. Layercam: Exploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing, 30:5875–5888, 2021

    2021

  16. [25]

    Learning to quantize deep networks by optimizing quantization intervals with task loss

    Sangil Jung, Changyong Son, Seohyung Lee, Jinwoo Son, Jae-Joon Han, Youngjun Kwak, Sung Ju Hwang, and Changkyu Choi. Learning to quantize deep networks by optimizing quantization intervals with task loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2019

  17. [26]

    Qkd: Quantization-aware knowledge distillation

    Jangho Kim, Yash Bhalgat, Jinwon Lee, Chirag Patel, and Nojun Kwak. Qkd: Quantization-aware knowledge distillation. arXiv preprint arXiv:1911.12491 , 2019

    1911 arXiv

  18. [27]

    Quantizing deep convo- lutional networks for efficient inference: A whitepaper

    Raghuraman Krishnamoorthi. Quantizing deep convo- lutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342 , 2018

    2018 arXiv

  19. [28]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 10

    2009

  20. [29]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012

    2012

  21. [30]

    Deeply-supervised nets

    Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. Deeply-supervised nets. In Artificial intelligence and statistics, pages 562–570. Pmlr, 2015

    2015

  22. [31]

    Net- work quantization with element-wise gradient scaling

    Junghyup Lee, Dohyung Kim, and Bumsub Ham. Net- work quantization with element-wise gradient scaling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 6448–6457, 2021

    2021

  23. [32]

    Pruning filters for efficient convnets

    Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 , 2016

    2016 arXiv

  24. [33]

    Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks

    Yuhang Li, Xin Dong, and Wei Wang. Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks. arXiv preprint arXiv:1909.13144, 2019

    1909 arXiv

  25. [34]

    Nonuniform-to-uniform quan- tization: Towards accurate quantization via generalized straight-through estimation

    Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric P Xing, and Zhiqiang Shen. Nonuniform-to-uniform quan- tization: Towards accurate quantization via generalized straight-through estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, page...

    2022

  26. [35]

    Rethinking the value of network pruning

    Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. Rethinking the value of network pruning. arXiv preprint arXiv:1810.05270 , 2018

    2018 arXiv

  27. [36]

    Relaxed quanti- zation for discretized neural networks

    Christos Louizos, Matthias Reisser, Tijmen Blankevoort, Efstratios Gavves, and Max Welling. Relaxed quanti- zation for discretized neural networks. arXiv preprint arXiv:1810.01875, 2018

    2018 arXiv

  28. [37]

    Convolutional neural networks using logarithmic data representation

    Daisuke Miyashita, Edward H Lee, and Boris Murmann. Convolutional neural networks using logarithmic data representation. arXiv preprint arXiv:1603.01025 , 2016

    2016 arXiv

  29. [38]

    Data-free quantization through weight equalization and bias correction

    Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling. Data-free quantization through weight equalization and bias correction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1325–1334, 2019

    2019

  30. [39]

    Loss aware post-training quantization

    Yury Nahshan, Brian Chmiel, Chaim Baskin, Evgenii Zheltonozhskii, Ron Banner, Alex M Bronstein, and Avi Mendelson. Loss aware post-training quantization. Machine Learning, 110(11-12):3245–3262, 2021

    2021

  31. [40]

    Fast neural architecture search of compact semantic segmentation models via auxiliary cells

    Vladimir Nekrasov, Hao Chen, Chunhua Shen, and Ian Reid. Fast neural architecture search of compact semantic segmentation models via auxiliary cells. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9126–9135, 2019

    2019

  32. [41]

    Profit: A novel training method for sub-4-bit mobilenet models

    Eunhyeok Park and Sungjoo Yoo. Profit: A novel training method for sub-4-bit mobilenet models. In Computer Vision–ECCV 2020: 16th European Conference, Glas- gow, UK, August 23–28, 2020, Proceedings, Part VI 16 , pages 430–446. Springer, 2020

    2020

  33. [42]

    Model compression via distillation and quantization

    Antonio Polino, Razvan Pascanu, and Dan Alistarh. Model compression via distillation and quantization. arXiv preprint arXiv:1802.05668 , 2018

    2018 arXiv

  34. [43]

    You only look once: Unified, real-time object detection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016

    2016

  35. [44]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems , 28, 2015

    2015

  36. [45]

    Fitnets: Hints for thin deep nets

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Ka- hou, Antoine Chassang, Carlo Gatta, and Yoshua Ben- gio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014

    2014 arXiv

  37. [46]

    Im- agenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Im- agenet large scale visual recognition challenge. Interna- tional journal of computer vision , 115:211–252, 2015

    2015

  38. [47]

    Mobilenetv2: In- verted residuals and linear bottlenecks

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: In- verted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018

    2018

  39. [48]

    Going deeper with convolutions

    Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Ser- manet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1–9, 2015

    2015

  40. [49]

    Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks

    Lin Wang and Kuk-Jin Yoon. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE transactions on pattern analysis and machine intelligence , 44(6):3048–3068, 2021

    2021

  41. [50]

    Resnet strikes back: An improved training procedure in timm

    R Wightman, H Touvron, and H J ´egou. Resnet strikes back: An improved training procedure in timm. arxiv

  42. [51]

    Learnable companding quantization for accurate low-bit neural networks

    Kohei Yamamoto. Learnable companding quantization for accurate low-bit neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5029–5038, 2021

    2021

  43. [52]

    Quantization networks

    Jiwei Yang, Xu Shen, Jun Xing, Xinmei Tian, Houqiang Li, Bing Deng, Jianqiang Huang, and Xian-sheng Hua. Quantization networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7308–7316, 2019

    2019

  44. [53]

    Lq-nets: Learned quantization for highly ac- curate and compact deep neural networks

    Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua. Lq-nets: Learned quantization for highly ac- curate and compact deep neural networks. In Proceedings of the European conference on computer vision (ECCV) , pages 365–382, 2018

    2018

  45. [54]

    Be your own teacher: Improve the performance of convolutional neural networks via self distillation

    Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 3713–3722, 2019

    2019

  46. [55]

    Shufflenet: An extremely efficient convolutional neural network for mobile devices

    Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of 11 the IEEE conference on computer vision and pattern recognition, pages 6848–6856, 2018

    2018

  47. [56]

    Pyramid scene parsing network

    Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2881–2890, 2017

    2017

  48. [57]

    Improving neural network quantization without retraining using outlier channel splitting

    Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Chris De Sa, and Zhiru Zhang. Improving neural network quantization without retraining using outlier channel splitting. In International conference on machine learning , pages 7543–7552. PMLR, 2019

    2019

  49. [58]

    Incremental network quantization: To- wards lossless cnns with low-precision weights

    Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, and Yurong Chen. Incremental network quantization: To- wards lossless cnns with low-precision weights. arXiv preprint arXiv:1702.03044, 2017

    2017 arXiv

  50. [59]

    Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients

    Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160 , 2016

    2016 arXiv

  51. [60]

    Quantized feature distillation for network quantization

    Ke Zhu, Yin-Yin He, and Jianxin Wu. Quantized feature distillation for network quantization. 2023

    2023

  52. [61]

    Training quantized neural networks with a full-precision auxiliary module

    Bohan Zhuang, Lingqiao Liu, Mingkui Tan, Chunhua Shen, and Ian Reid. Training quantized neural networks with a full-precision auxiliary module. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1488–1497, 2020

    2020

  53. [2021]

    arXiv preprint arXiv:2110.00476

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.