Progressive Element-wise Gradient Estimation for Neural Network Quantization

Kaiqi Zhao

REVIEW 4 major objections 6 minor 15 references

Progressive Element-wise Gradient Estimation for Neural Network Quantization

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A simple change to the gradient rule used in quantization-aware training lets low-bit networks match or beat full-precision accuracy.

desk verdict A plausible drop-in gradient estimator for QAT with small consistent gains, but the evidence is a single unseeded run with test-set-tuned schedules. read the letter →

arxiv 2509.00097 v1 pith:6ZMKO7LW submitted 2025-08-27 cs.LG cs.CV

Kaiqi Zhao This is my paper · ORCID

classification cs.LGcs.CV

keywords quantization-awaretrainingstraight-throughestimatorgradientestimationlow-bitquantizationdiscretizationerrorcurriculumlearningmodelcompressionimageclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes PEGE, a direct replacement for the Straight-Through Estimator in quantization-aware training. Instead of passing the gradient through quantization as if the round function were the identity, PEGE progressively mixes full-precision and quantized values during training and adds a scaled discretization-error term to the STE gradient. The central claim is that this simple backward rule consistently improves low-bit accuracy across CIFAR-10 and ImageNet, letting a 2-bit ResNet-20/VGG-16 match its full-precision model and a 4-bit ResNet-18 surpass its full-precision counterpart on ImageNet. If correct, QAT can gain accuracy without changing the forward quantizer or adding meaningful overhead, since PEGE only modifies backpropagation.

What carries the argument

The load-bearing object is the modified gradient estimator in Eq. (5): the STE gradient plus an element-wise correction μ·(x_c − x_q). The correction is scaled by μ, which grows exponentially with training steps, and is paired with a Bernoulli replacement mechanism whose probability follows a logarithmic curriculum, so fully quantized parameters gradually take over from full-precision ones. The mechanism lets the optimizer see both the task gradient and a direct penalty for deviating from quantized values, effectively co-optimizing prediction loss and discretization error.

What would settle it

Train a 2-bit ResNet-20 on CIFAR-10 with PEGE but set μ_max = 0, so the only change from STE is the progressive replacement schedule. If the reported 91.62% accuracy persists, the paper's core gradient-correction mechanism is not doing the work; if the reported accuracy requires specific μ_max and k values that the paper does not disclose, the central claim is not reproducible from the text.

Watch

Extended reading notes

Core claim

The paper's central claim is that the accuracy loss in low-bit quantization-aware training comes mainly from the backward pass, not the forward quantizer, and can be corrected by a gradient rule of the form ∂L/∂x_c = ∂L/∂x_q + μ·(x_c − x_q). Here x_c is the clipped full-precision value, x_q its quantized counterpart, and μ a growing scale. The first term is the standard STE gradient; the second term explicitly feeds the discretization error back into the optimization. PEGE combines this with a stochastic replacement schedule: at each step a Bernoulli variable decides whether the network trains on quantized parameters, with the replacement rate following a logarithmic curriculum, and μ growin

Load-bearing premise

Equation (5) assumes that adding a scaled discretization-error term to the STE gradient steers the full-precision weights toward a better optimum of the quantized loss; the paper gives no derivation or convergence argument for this, and the values of μ_max and the schedule constants are not reported.

Editorial extensions

If this is right

PEGE is backward-only: any existing forward quantizer, including EWGS and PACT, can keep its forward pass and gain accuracy by swapping the backward rule.
Low-bit models become practical: 2-bit CIFAR-10 models come within roughly 0.1–0.3 points of full precision, and 4-bit ResNet-18 exceeds full precision on ImageNet, at a 16x/8x compression ratio.
Training dynamics improve: PEGE converges faster than STE and EWGS in the reported learning curves, and it continues to benefit from longer training, unlike EWGS.
The gains are reported consistently across ResNet-20, VGG-16, and ResNet-18 on both CIFAR-10 and ImageNet, suggesting the effect is not tied to one architecture or dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the μ·(x_c − x_q) term is what matters, PEGE is effectively a penalty or regularization view of QAT; an ablation with μ_max=0 would isolate whether the replacement schedule alone, rather than the correction term, drives the reported gains.
The same correction idea may transfer to other nondifferentiable discrete operations, such as weight pruning masks or binary/ternary networks, where an STE-like identity gradient also ignores discretization error.
Because the paper does not report the values of μ_max, k, and B used in Eqs. (4)–(5), the practical claim is sensitive to schedule hyperparameters; a sensitivity study would be needed before the method can be used as a drop-in.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my read on arXiv:2509.00097. PEGE is a simple gradient-estimation heuristic for quantization-aware training: use STE's gradient plus a scaled discretization-error term, and progressively switch from full-precision to quantized weights/activations with a log-curriculum schedule. Nothing here is individually new—EWGS already scales gradients by discretization error and progressive quantization is an old trick—but the specific combination, framed as a co-optimization of task loss and discretization error, is a reasonable drop-in that claims consistent small gains over STE and EWGS across CIFAR-10 and ImageNet. If the gains hold, it's useful for edge deployment. The writing is clear and the overhead is negligible.

The soft spots are real and load-bearing for the headline claims. The entire evaluation reports single runs: no seeds, no error bars, and the ImageNet 'surpasses full-precision' claim rests on a 0.21 pp advantage over the paper's own FP baseline—well inside run-to-run noise for ResNet-18. The ablations in Section 4.2 pick the logarithmic replacement scheduler and the exponential µ scheduler by test accuracy, with no validation split. So part of the reported gain is a fitted choice, not an independent test. On top of that, the constants in Eqs. (4) and (5)—B, b, k, µ_max—are never reported, so nobody can reproduce the method as described. Equation (5) itself is asserted without derivation; it could be defended as a penalty-style gradient, but the paper doesn't make that case.

I don't think these problems amount to a fatal flaw. The heuristic is plausible, the results are consistently in one direction, and the paper is honest about its experiments. But the central claims—especially 'enables low-precision models to match or outperform full precision'—are not supported at the current level of evidence. The reader's conditional verdict is about right. One small thing: the citation to the author's own pruning paper [14] is not an issue; it's a normal reference.

Who should read it: people working on QAT who want a simple alternative to STE and are willing to test it themselves. The paper is not ready for acceptance, but it deserves a serious referee. I'd send it to review with instructions to require seeds, error bars, validation-based schedule selection, hyperparameter values, and ideally code. If the gains survive that scrutiny, it's a modest but legitimate contribution. Without that, it's a promising tech report.

Referee Report

4 major / 6 minor

Summary. The paper proposes PEGE, a gradient-estimation method for quantization-aware training (QAT). PEGE replaces STE's gradient with dL/dx_c = dL/dx_q + mu*(x_c - x_q), and progressively mixes full-precision and quantized parameters through a Bernoulli mask whose probability follows a logarithmic schedule. The authors claim this improves low-bit accuracy across CIFAR-10 (ResNet-20, VGG-16, W2A2) and ImageNet (ResNet-18, W4A4), and that PEGE enables low-precision models to match or surpass their full-precision counterparts. The method is intended as a drop-in backward-pass replacement compatible with different forward quantizers such as EWGS and PACT.

Significance. If the central claims held, PEGE would be a practically valuable contribution: it is simple, adds negligible overhead, and is forward-method-agnostic, addressing a known weakness of STE at low bit-widths. The paper also usefully frames QAT as a constrained optimization problem and provides ablations of the replacement scheduler and mu scheduler. However, the current evidence is not sufficient to establish the claims: the key gradient formula is asserted heuristically, the schedule hyperparameters are not reported, the ablation selects schedulers on test accuracy, and all experiments appear to be single unseeded runs. These issues directly affect the reproducibility and statistical reliability of the headline 'surpasses full-precision' result.

major comments (4)

[Sec. 3.2, Eq. (5)] The central gradient estimator dL/dx_c = dL/dx_q + mu*(x_c - x_q) is introduced without derivation or theoretical justification. It is not shown to be a descent direction for the quantized loss, nor is it derived from the optimization objective in Eq. (6). The text calls the formulation 'theoretically grounded,' but no argument is provided. At minimum, the authors should derive the estimator from a surrogate loss (e.g., a penalty or proximal term) and discuss when it reduces to STE. The values of mu_max and k for the exponential mu schedule are also never reported, making the method impossible to reproduce.
[Sec. 3.2, Eq. (4) and Sec. 4.2, Tables 3-4, Fig. 3] The logarithmic replacement scheduler p_T = min(log(kT+b)/B, 1.0) is under-specified: the base B, coefficient k, offset b, and the exact base of the logarithm are not given. More importantly, the choice of scheduler and the choice of mu scheduler are both made by comparing test accuracy on CIFAR-10 (Tables 3-4 and Fig. 3) without a held-out validation split. Thus the reported gains partly reflect test-set selection of hyperparameters. The authors should specify all constants and select schedules on a validation split, then report the corresponding test results.
[Sec. 4.1, Tables 1-2 and Fig. 2] All experimental results are reported as single runs with no seeds, error bars, or statistical significance. The main ImageNet claim—PEGE at 70.01% vs. full-precision 69.80%, a 0.21 pp advantage—is within typical run-to-run variance for ResNet-18 on ImageNet, and the CIFAR-10 gains over STE are at most 0.45 pp. The paper needs multiple seeds with mean/std or a significance test to support the 'surpasses full-precision' claim. Without this, the headline conclusion is not robust.
[Sec. 4.1, Table 2] The comparison with 'STE (Orig. with FP: 71)' is not apples-to-apples: the cited PACT baseline uses a full-precision model at 71.0% accuracy, while the paper's own full-precision baseline is 69.80%. The reported -1.80 pp drop is therefore conflated with the baseline difference. The fair comparison is with the paper's own STE run (69.56%) and EWGS run (69.53%), which show PEGE ahead by 0.45-0.48 pp; this is still a small margin from a single run. The authors should either re-run PACT's original protocol under the same baseline or remove the 'Orig.' row.

minor comments (6)

[Title and Sec. 5] Typos: 'N etwork' in the title, and 'backprogation' in Section 5.
[Sec. 4.1, Fig. 2 caption] Figure 2b's caption says 'ResNet-20' but the accompanying text and Table 2 concern 'ResNet-18' on ImageNet.
[Sec. 3.3] The description of Eq. (6) as a 'theoretically grounded' formulation is overstated; it is a standard constrained optimization statement. Consider tempering the language or providing a formal analysis.
[Sec. 3.2, Eq. (3)] The notation is imprecise: 'Consider backpropagation step T, we generate an independent Bernoulli random variables, denoted as r_T'—the variable is a single Bernoulli, not plural; also clarify whether T indexes steps or epochs.
[Implementation Details, Sec. 4] The implementation details omit the initial learning rate, weight decay, data augmentation protocol, and the exact number of training epochs for ImageNet. These are needed for reproduction.
[Fig. 3b] The vertical axis is labeled 'u' instead of the Greek mu used in the text.

Circularity Check

1 steps flagged · score 4.0 of 10

Test-set selection of schedulers makes part of the reported gain a fitted choice; the core Eq. (5) estimator is not circular.

fitted input called prediction [Section 4.2, Tables 3-4 and Figure 3]
"Table 4 shows the top-1 test accuracy of 2-bit ResNet-20 trained using constant, linear, exponential, cosine, and logarithmic replacement schedulers ... The proposed log-curriculum based replacement scheduler achieves the highest accuracy ... Figure 3a shows the top-1 test accuracy evolution ... with different µ scheduling strategies ... Exponential µ Scheduler achieves the best accuracy."

The scheduler choices are made by comparing their top-1 test accuracy, not validation accuracy. The final PEGE configuration (logarithmic replacement, exponential µ) is thus the best-scoring configuration on the test set, so the reported CIFAR-10 improvements over STE/EWGS are partly forced by construction: among several candidate schedulers, the paper reports the one with the highest test accuracy. The ImageNet result is less directly affected because the schedulers were selected on CIFAR-10, but the claim that PEGE 'consistently outperforms' is not fully independent of the test-set selection criterion.

full rationale

The paper contains no self-definitional circularity: Eq. (5) is presented as a heuristic gradient estimator, not derived from Eq. (6), so it is not equivalent to its input by construction. There is no load-bearing self-citation chain: the only self-citation is reference [14], used for background on edge-AI resource constraints. The main circularity-like issue is in Section 4.2: the logarithmic replacement scheduler and exponential µ scheduler are selected by comparing top-1 test accuracy in Tables 4 and Figure 3, and those same schedulers are then used for the headline results in Tables 1–2. This makes part of the reported improvement a fitted choice rather than a prediction. However, the core method (STE plus a discretization-error correction) is an empirical proposal with independent content, and the ImageNet result transfers schedulers chosen on CIFAR-10 rather than being tuned on ImageNet itself, so the circularity is partial and the score is moderate. Concerns about unseeded runs and unreported hyperparameter constants are reproducibility/robustness issues, not circularity per se.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the asserted gradient estimator and the empirically chosen curriculum schedules. The paper provides no derivation of Eq. (5) and does not report the values of its own free parameters, so the reader cannot tell whether the reported gains are robust or the result of tuned choices.

free parameters (6)

mu_max = not reported
Maximum scaling factor for the discretization-error term in Eq. (5); its value is not given and controls gradient correction strength.
k_mu = not reported
Growth rate in the exponential mu scheduler, mu_T = mu_max * (1 - exp(-k T)); not reported.
B = not reported
Base value in the logarithmic replacement scheduler, Eq. (4); not reported.
b = not reported
Offset term in Eq. (4) that sets the basic replacing rate; not reported.
k_p = not reported
Coefficient in Eq. (4) controlling growth rate; not reported.
Scheduler choice = logarithmic for replacement rate, exponential for mu
Selected by highest test accuracy in the ablation in Section 4.2, i.e. fitting model choices to the test set.

assumptions (3)

domain assumption Quantizers can be decomposed as a clipping function followed by a round-based function (Eqs. 1-2), covering PACT, EWGS, LSQ and similar methods.
Section 3.1 asserts this unification; it relies on the prior literature's quantizer designs.
ad hoc to paper The gradient of the quantized loss can be approximated by the STE gradient plus a scaled discretization error (Eq. 5), and this is a valid descent direction.
Eq. (5) is introduced without derivation. The correctness of the whole method depends on this approximation being helpful across architectures, bit widths, and training stages.
ad hoc to paper Mixing full-precision and quantized parameters with a Bernoulli mask whose probability follows a logarithmic schedule improves optimization (Eqs. 3-4).
Section 3.2 states this mechanism without theoretical justification; the schedule itself was chosen empirically in Section 4.2.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Progressive Element-wise Gradient Estimation for Neural Network Quantization." pith.science (2026). https://pith.science/paper/6ZMKO7LW

@misc{pith2026250900097,
  author       = {Pith},
  title        = {Pith review of: Progressive Element-wise Gradient Estimation for Neural Network Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ZMKO7LW}},
  note         = {Machine review of arXiv:2509.00097}
}

read the original abstract

Neural network quantization aims to reduce the bit-widths of weights and activations, making it a critical technique for deploying deep neural networks on resource-constrained hardware. Most Quantization-Aware Training (QAT) methods rely on the Straight-Through Estimator (STE) to address the non-differentiability of discretization functions by replacing their derivatives with that of the identity function. While effective, STE overlooks discretization errors between continuous and quantized values, which can lead to accuracy degradation -- especially at extremely low bit-widths. In this paper, we propose Progressive Element-wise Gradient Estimation (PEGE), a simple yet effective alternative to STE, which can be seamlessly integrated with any forward propagation methods and improves the quantized model accuracy. PEGE progressively replaces full-precision weights and activations with their quantized counterparts via a novel logarithmic curriculum-driven mixed-precision replacement strategy. Then it formulates QAT as a co-optimization problem that simultaneously minimizes the task loss for prediction and the discretization error for quantization, providing a unified and generalizable framework. Extensive experiments on CIFAR-10 and ImageNet across various architectures (e.g., ResNet, VGG) demonstrate that PEGE consistently outperforms existing backpropagation methods and enables low-precision models to match or even outperform the accuracy of their full-precision counterparts.

Figures

Figures reproduced from arXiv: 2509.00097 by the authors.

**Figure 3.** Illustration of a) top-1 test accuracy evolution o [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 2.** Top-1 test accuracy evolution of a) ResNet-20 (W2A [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 7 canonical work pages

[14]

Automatic at- tention pruning: Improving and automating model pruning using attentions

Kaiqi Zhao, Animesh Jain, and Ming Zhao. Automatic at- tention pruning: Improving and automating model pruning using attentions. In International Conference on Artiﬁcial In- telligence and Statistics , pages 10470–10486. PMLR, 2023. 1

work page 2023
[1]

Estimating or propagating gradients through stochastic neurons for conditional computation

Y oshua Bengio, Nicholas L´ eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. 1, 2, 3

arXiv 2013
[2]

Pact: Parameterized clipping activa- tion for quantized neural networks

Jungwook Choi, Zhuo Wang, Swagath V enkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailas h Gopalakrishnan. Pact: Parameterized clipping activa- tion for quantized neural networks. arXiv preprint arXiv:1805.06085, 2018. 2, 4

arXiv 2018
[3]

Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1

Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Y oshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830 , 2016. 2

arXiv 2016
[4]

Learned step size quantization

Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. Learned step size quantization. arXiv preprint arXiv:1902.08153, 2019. 2

arXiv 1902
[5]

Differ- entiable soft quantization: Bridging full-precision and l ow- bit neural networks

Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li , Peng Hu, Jiazhen Lin, Fengwei Y u, and Junjie Yan. Differ- entiable soft quantization: Bridging full-precision and l ow- bit neural networks. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4852–4861, 2019. 2

work page 2019
[6]

Learning to quantize deep networks by op- timizing quantization intervals with task loss

Sangil Jung, Changyong Son, Seohyung Lee, Jinwoo Son, Jae-Joon Han, Y oungjun Kwak, Sung Ju Hwang, and Changkyu Choi. Learning to quantize deep networks by op- timizing quantization intervals with task loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 4350–4359, 2019. 2

work page 2019
[7]

Quantizing deep convolu- tional networks for efﬁcient inference: A whitepaper

Raghuraman Krishnamoorthi. Quantizing deep convolu- tional networks for efﬁcient inference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018. 1, 2

arXiv 2018

Show all 15 references

[8]

Network quantization with element-wise gradient scaling

Junghyup Lee, Dohyung Kim, and Bumsub Ham. Network quantization with element-wise gradient scaling. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6448–6457, 2021. 1, 2, 3

2021
[9]

Additive powers-of- two quantization: An efﬁcient non-uniform discretization for neural networks

Y uhang Li, Xin Dong, and Wei Wang. Additive powers-of- two quantization: An efﬁcient non-uniform discretization for neural networks. arXiv preprint arXiv:1909.13144, 2019. 2

1909 arXiv
[10]

Brecq: Pushing the limit of post-training quantization by block reconstru c- tion

Y uhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Y u, Wei Wang, and Shi Gu. Brecq: Pushing the limit of post-training quantization by block reconstru c- tion. arXiv preprint arXiv:2102.05426, 2021. 2

2021 arXiv
[11]

Mqbench: Towards reproducible and deployable model quantization benchmark

Y uhang Li, Mingzhu Shen, Jian Ma, Yan Ren, Mingxin Zhao, Qi Zhang, Ruihao Gong, Fengwei Y u, and Junjie Yan. Mqbench: Towards reproducible and deployable model quantization benchmark. arXiv preprint arXiv:2111.03759 , 2021. 1, 2

2021 arXiv
[12]

Xnor-net: Imagenet classiﬁcation using bi - nary convolutional neural networks

Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classiﬁcation using bi - nary convolutional neural networks. In European conference on computer vision , pages 525–542. Springer, 2016. 2

2016
[13]

Towards resource-efﬁcient edge ai: From federated learning to semi - supervised model personalization

Zhaofeng Zhang, Sheng Y ue, and Junshan Zhang. Towards resource-efﬁcient edge ai: From federated learning to semi - supervised model personalization. IEEE Transactions on Mobile Computing, 2023. 1

2023
[15]

Dorefa-net: Training low bitwidth convo- lutional neural networks with low bitwidth gradients

Shuchang Zhou, Y uxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Y uheng Zou. Dorefa-net: Training low bitwidth convo- lutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160, 2016. 2

2016 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [14]

Automatic at- tention pruning: Improving and automating model pruning using attentions

Kaiqi Zhao, Animesh Jain, and Ming Zhao. Automatic at- tention pruning: Improving and automating model pruning using attentions. In International Conference on Artiﬁcial In- telligence and Statistics , pages 10470–10486. PMLR, 2023. 1

work page 2023

[2] [1]

Estimating or propagating gradients through stochastic neurons for conditional computation

Y oshua Bengio, Nicholas L´ eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. 1, 2, 3

arXiv 2013

[3] [2]

Pact: Parameterized clipping activa- tion for quantized neural networks

Jungwook Choi, Zhuo Wang, Swagath V enkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailas h Gopalakrishnan. Pact: Parameterized clipping activa- tion for quantized neural networks. arXiv preprint arXiv:1805.06085, 2018. 2, 4

arXiv 2018

[4] [3]

Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1

Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Y oshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830 , 2016. 2

arXiv 2016

[5] [4]

Learned step size quantization

Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. Learned step size quantization. arXiv preprint arXiv:1902.08153, 2019. 2

arXiv 1902

[6] [5]

Differ- entiable soft quantization: Bridging full-precision and l ow- bit neural networks

Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li , Peng Hu, Jiazhen Lin, Fengwei Y u, and Junjie Yan. Differ- entiable soft quantization: Bridging full-precision and l ow- bit neural networks. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4852–4861, 2019. 2

work page 2019

[7] [6]

Learning to quantize deep networks by op- timizing quantization intervals with task loss

Sangil Jung, Changyong Son, Seohyung Lee, Jinwoo Son, Jae-Joon Han, Y oungjun Kwak, Sung Ju Hwang, and Changkyu Choi. Learning to quantize deep networks by op- timizing quantization intervals with task loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 4350–4359, 2019. 2

work page 2019

[8] [7]

Quantizing deep convolu- tional networks for efﬁcient inference: A whitepaper

Raghuraman Krishnamoorthi. Quantizing deep convolu- tional networks for efﬁcient inference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018. 1, 2

arXiv 2018