Pith. sign in

REVIEW 3 major objections 5 minor 38 references

Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training deep networks entirely in 8-bit integers can match floating-point accuracy within a few points.

desk verdict Useful 8-bit training results on ImageNet, but the 'full 8-bit / all bit-wise' framing overstates what is actually quantized and specified. read the letter →

arxiv 1909.02384 v2 pith:TZVEL3KJ submitted 2019-09-05 cs.LG stat.ML

Yukuan Yang , Shuang Wu , Lei Deng , Tianyi Yan , Yuan Xie , Guoqi Li This is my paper · ORCID
classification cs.LGstat.ML
keywords 8-bittrainingfullintegerquantizationbatchnormalizationmomentumoptimizerstraight-throughestimatorquantizederrorpropagationImageNetResbit-wiseoperations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a deep neural network can be trained end-to-end with every data path held in 8-bit integers, including weights, activations, gradients, errors, batch-normalization statistics, parameter updates, and the momentum accumulator. On ImageNet with ResNet18/34/50, the fully 8-bit networks reach 64.79/67.63/67.95 top-1 accuracy, within a few points of the 68.70/71.99/74.66 floating-point baselines. If true, the practical point is that training and inference no longer need floating-point arithmetic at all, so the heavy multiply-accumulate work can be done on simple integer hardware. The paper's contribution is the specific set of quantization functions that keep each data type's precision needs satisfied while keeping the bit width at 8.

What carries the argument

The load-bearing mechanism is a family of three quantization functions matched to the precision needs of different data types: direct quantization for weights, activations, and batch-normalization values; constant quantization for gradients, which keeps the update bit width fixed by normalizing and clipping while preserving gradient orientation; and shift quantization for errors, which preserves magnitude order through a layer-wise scaling factor. A flag-bit variant of the error quantizer expands the coverage of small-magnitude error values so that 8-bit error quantization does not force those values to zero. These functions are fused with quantized batch normalization, $\hat{x} = Q_{BN}\left(\frac{x - \mu_q}{\sigma_q + \epsilon_q}\right)$, and a quantized Momentum optimizer, with bit widths tied by consistency equations so that stored weights, updates, and accumulators have fixed integer widths.

What would settle it

Inspect or reproduce the implementation of Equation (12) and check whether the batch-normalization division is computed with pure 8-bit integer arithmetic or with floating-point or higher-precision intermediate values. If the reported ImageNet accuracies require any floating-point or wider-than-8-bit arithmetic in that step, the central claim that all training operations are bit-wise fails.

Watch

Extended reading notes

Core claim

WAGEUBN is presented as the first complete quantization framework that constrains all training data paths, including W (weights), A (activations), G (gradients), E (errors), U (updates), BN (batch normalization), and the Momentum optimizer, to low-bit integers. The paper reports that on ImageNet, ResNet18/34/50 trained with full 8-bit integers reach top-1 accuracies of 64.79%, 67.63%, and 67.95%, compared with 68.70%, 71.99%, and 74.66% for the FP32 baselines; a variant using 16-bit error quantization recovers part of the gap, reaching 67.40%, 68.50%, and 69.07%. The claim is that every forward and backward operation, including the batch-normalization division and the optimizer accumulation, can be performed as bit-wise operations, shrinking memory by about 4 times and greatly reducing the time, power, and circuit area of the dominant multiply-accumulate operations.

Load-bearing premise

The framework's "all bit-wise" claim rests on the batch-normalization division $\frac{(x - \mu_q)}{(\sigma_q + \epsilon_q)}$ being implementable in integer arithmetic at 8 bits without losing enough precision, and the paper does not specify how that division is done.

Editorial extensions

If this is right

  • Training and inference can run with only integer arithmetic, making online learning on low-power or edge devices a plausible target.
  • The complete 8-bit pipeline yields about 4 times memory savings over FP32 and substantially faster, lower-power multiply-accumulate operations.
  • The reported ImageNet results make full 8-bit integer training competitive for large-scale networks for the first time, not only for small datasets.
  • A 16-bit error-quantization variant recovers most of the accuracy gap, suggesting that error precision is the most sensitive component and can be traded against hardware cost.
  • The published experiments leave the first and last layers in floating point and show visible accuracy loss at batch size 16, so the full 8-bit claim applies to the bulk of the network under standard batch sizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The flag-bit format suggests that the main obstacle to 8-bit training is coverage of small error values, so other training quantizers could adopt similar scaling-and-flag storage to avoid zeroing useful gradient information.
  • The same constant-versus-shift quantizer split could be extended to adaptive optimizers such as Adam, though the paper only quantizes Momentum.
  • The accuracy drop at small batch sizes implies that full-integer training may need deliberate batch-size choices or modified batch-normalization statistics for small-batch regimes.
  • The hardware gains claimed here depend on the as-yet-unspecified integer implementation of the batch-normalization division; realizing the full bit-wise promise requires that step to be solved in pure integer arithmetic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes WAGEUBN, a quantization framework that aims to constrain all training and inference data paths of deep neural networks—weights (W), activations (A), gradients (G), errors (E), updates (U), batch-normalization (BN), and the Momentum optimizer—to low-bit integers. It introduces three quantization functions, a flag-bit data format, and quantized versions of BN and the Momentum optimizer. Experiments on ResNet18/34/50 with ImageNet report Top-1 accuracies of 64.79/67.63/67.95 for the full 8-bit version versus 68.70/71.99/74.66 for the FP32 baselines, with an additional 16-bit E2 version that achieves 67.40/68.50/69.07. The central claim is that this is the first complete quantization framework that advances large-scale DNN training to the full 8-bit integer level, with all operations implemented as bit-wise operations.

Significance. If the central claims are substantiated, this would be an important contribution to low-precision DNN training: a unified framework covering BN and the optimizer, with explicit quantization functions designed for different data distributions, quantitative hardware cost estimates, and ImageNet-scale results. The paper is also transparent about many design choices, such as the distinction between error quantization at different points and the use of flag-bit representation. However, the gap between the claims in the title/abstract and the actual implementation details is substantial: the reported 'full 8-bit' configuration uses 16-bit BN statistics, 15-bit gradient/accumulator paths, 24-bit updates, and unquantized first/last layers, and the bit-wise nature of the BN normalization division is not specified. These issues are load-bearing for the paper's central contribution and must be resolved before the claims can be accepted.

major comments (3)
  1. [III-D(2), Eq. (12), Algorithm 1] The integer implementation of the BN normalization division is unspecified. The text states that 'all operands are quantized and all operations are bit-wise' after Eq. (12), but Eq. (12) contains the division (x−μ_q)/(σ_q+ε_q) and the quantization function QBN is only defined as Q(x,kBN) in Eq. (13), where Eq. (6) applies to continuous values. No fixed-point reciprocal, integer division algorithm, or scaling scheme is provided. The same issue appears in the backward pass: Eq. (3) requires ∂x_2/∂x_1 = 1/(σ_q+ε_q) for Norm&QE2, which again involves a reciprocal. Because this operation occurs in every BN layer, the central claim that all operations are bit-wise integer operations is not established without a concrete and precise integer division specification.
  2. [IV-A and Table I] The 'full 8-bit' claim is contradicted by the actual hyperparameters. Section IV-A states that kBN, kμ, and kσ are 16, and kGγ=kGβ=kGC=15, kAcc=13, klr=10, kWU=24; the same section also states that the first and last layers are not quantized. Thus the experiments labeled 'full 8-bit' in Table I do not constrain all data paths to 8-bit integers. The abstract and title claim that training is advanced to the 'full 8-bit INT level' and that 'all the operations in the training and inference can be bit-wise operations', which is stronger than what the configuration supports. The authors should either revise the claims to specify precisely which tensors are 8-bit and which are wider, or provide a configuration in which truly all parameters and statistics are 8-bit.
  3. [IV-B and Table I] The accuracy comparison is confounded by different training hyperparameters. The paper states that the FP32 baseline uses initial learning rate 0.05 and momentum 0.9, whereas WAGEUBN uses initial learning rate 0.05078125 and momentum 0.75. The reported accuracy gaps (e.g., 3.91, 4.36, and 6.71 Top-1 points for ResNet18/34/50) could result partly from these hyperparameter differences rather than from quantization. The authors should include a controlled comparison—for example, an FP32 baseline trained with WAGEUBN's hyperparameters and learning-rate schedule, or WAGEUBN trained with the official TensorFlow hyperparameters—and ideally report standard deviations over multiple seeds.
minor comments (5)
  1. [III-D(4), Figure 4] The flag-bit format stores 9-bit integers (a flag bit plus 8 data bits), yet the paper calls this part of the 'full 8-bit' framework. The statement that the effective value for computation is INT8 is helpful, but the storage cost should be reported as 9 bits to avoid misleading readers.
  2. [III-C, Eq. (7)] The constant-quantization function uses stochastic rounding, but the paper does not state how the stochastic rounding is seeded or whether the reported results are averaged over multiple runs. This matters for reproducibility of the accuracy numbers.
  3. [Algorithm 2] The notation in Algorithm 2 differs slightly from the equations in Section III-B: for example, the algorithm writes Acc_i and g_iq while the text uses Acc_i^l and g_iq^l; making the notation uniform would improve readability.
  4. [Overall manuscript] No code is provided, and the experimental setup does not mention data augmentation details, weight decay, or the exact learning-rate schedule in epochs. Releasing code or a complete training recipe would substantially strengthen the reproducibility of the reported results.
  5. [Abstract and Section I] There are several typos and grammatical issues, such as 'Our throughout quantization framework' in the abstract and 'lay-wise' instead of 'layer-wise' in Section III-C. A careful proofread is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; WAGEUBN's ImageNet results are external measurements, and the WAGE self-citation is not load-bearing.

full rationale

WAGEUBN is an empirical quantization scheme rather than a derivation that assumes its own conclusion. The quantization functions in Section III-C are standard fixed-point and stochastic-rounding operators, and the bit-width choices are specified a priori in the setup, not fitted to the reported accuracies. The Top-1/Top-5 numbers in Table I are fresh measurements against the external ImageNet benchmark, so they are not fitted inputs renamed as predictions. The only self-referential element is the citation to the authors' earlier WAGE work for the premise that simple scaling layers cannot replace BN; that is a contextual literature citation, not a formal step in the derivation, and the paper's central result—quantized BN plus quantized Momentum achieving 64.79-67.95% Top-1 on ImageNet—is independently validated by the experiments. The unstated implementation of the BN division in Equation (12) is a correctness and completeness concern for the 'all bit-wise operations' claim, but it is not a circularity because the accuracy claims do not assume that division is an INT8 operation. No equation is shown to be equivalent to its own input by construction, no fitted parameter is presented as a prediction, and no load-bearing uniqueness theorem is imported from the authors' prior work.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The framework's design choices are extensive, and most are hand-tuned rather than derived. The bit widths for each data path were selected based on sensitivity experiments; the dynamic range schedule dr was chosen with specific epoch boundaries; and the learning rate and momentum coefficient were changed relative to the FP32 baseline. The central claims rest on these choices. No code is provided to reproduce the experiments.

free parameters (4)
  • Bit widths (kW, kA, kGW, kE1, kE2, kγ, kβ, kBN, kµ, kσ, kGγ, kGβ, kGC, kMom, kAcc, klr, kWU) = 8 for W, A, G, E1, E2, γ, β; 16 for BN, µ, σ; 15 for Gγ, Gβ, GC; 3 for Mom; 13 for Acc; 10 for lr; 24 for WU (Table I…
    Chosen to balance accuracy and overhead. The paper states the update bit width has the greatest impact on accuracy (Section III-D(7)); these values are tuned on the same datasets used for evaluation.
  • dr schedule for constant-quantization function = dr = 128 for epochs 0-30, dr = 64 for epochs 30-60
    Hand-chosen schedule that shrinks the quantization range during training; required for convergence of the gradient quantization (Section IV-A, Equation 7).
  • Learning rate = 0.05078125 (26 × 2^-9, 10-bit integer)
    Fixed-point learning rate chosen for the quantized framework, different from the FP32 baseline's 0.05 (Section IV-B).
  • Momentum coefficient = 0.75 (3 × 2^-2, 3-bit integer)
    Quantized momentum coefficient chosen for the optimizer, different from the FP32 baseline's 0.9 (Section IV-B).
assumptions (4)
  • domain assumption Straight-through estimator (STE) provides a valid gradient for non-differentiable quantization functions (Equation 1).
    The true gradient of a quantized function is zero almost everywhere; STE is standard in quantization literature but is an approximation whose accuracy is not guaranteed.
  • ad hoc to paper The orientation, not the magnitude, of gradients is sufficient for convergence.
    This heuristic justifies the aggressive constant-quantization of gradients (Section IV-A). It is stated as an observation without proof or controlled experiment.
  • ad hoc to paper The first and last layers can be left at full precision without violating the 'complete quantization' claim.
    Section IV-A states that first and last layers are not quantized, following prior work. This weakens the 'full 8-bit' and 'all data paths' claims.
  • ad hoc to paper The BN normalization division (x - µ_q)/(σ_q + ε_q) can be implemented as a bit-wise integer operation without significant accuracy loss.
    Equation (12) divides by quantized σ_q, but the paper does not specify how this division is replaced by bit-wise operations in INT8, nor how precision is maintained.
invented entities (1)
  • Flag bit data format (9-bit integers with a flag bit) independent evidence
    purpose: Expands the data range of the 8-bit QE2 quantization to cover small-magnitude errors (Section III-D(4), Figure 4).
    A new data representation introduced to solve non-convergence of 8-bit error quantization; validated on ImageNet, though only within the same paper's experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers." pith.science (2026). https://pith.science/paper/TZVEL3KJ

@misc{pith2026190902384,
  author       = {Pith},
  title        = {Pith review of: Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TZVEL3KJ}},
  note         = {Machine review of arXiv:1909.02384}
}
read the original abstract

Deep neural network (DNN) quantization converting floating-point (FP) data in the network to integers (INT) is an effective way to shrink the model size for memory saving and simplify the operations for compute acceleration. Recently, researches on DNN quantization develop from inference to training, laying a foundation for the online training on accelerators. However, existing schemes leaving batch normalization (BN) untouched during training are mostly incomplete quantization that still adopts high precision FP in some parts of the data paths. Currently, there is no solution that can use only low bit-width INT data during the whole training process of large-scale DNNs with acceptable accuracy. In this work, through decomposing all the computation steps in DNNs and fusing three special quantization functions to satisfy the different precision requirements, we propose a unified complete quantization framework termed as ``WAGEUBN'' to quantize DNNs involving all data paths including W (Weights), A (Activation), G (Gradient), E (Error), U (Update), and BN. Moreover, the Momentum optimizer is also quantized to realize a completely quantized framework. Experiments on ResNet18/34/50 models demonstrate that WAGEUBN can achieve competitive accuracy on the ImageNet dataset. For the first time, the study of quantization in large-scale DNNs is advanced to the full 8-bit INT level. In this way, all the operations in the training and inference can be bit-wise operations, pushing towards faster processing speed, decreased memory cost, and higher energy efficiency. Our throughout quantization framework has great potential for future efficient portable devices with online learning ability.

Figures

Figures reproduced from arXiv: 1909.02384 by the authors.

Figure 2
Figure 2. According to the derivative chain rules, we have [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 1
Figure 1. Forward quantization of the l-th layer in DNNs. Specifically, we have Input : x l 0 = x l−1 4 Conv : x l 1 = Wl qx l 0 Normalization&QBN : x l 2 = QBN ( x l 1 − µ l q σ l q ) Scale&Offset : x l 3 = γ l qx l 2 + β l q Activation&QA : x l 4 = QA [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Illustration of the constant-quantization function during training. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Data format of 9-bit integers. By introducing a layer-wise scaling factor and a flag bit, we can expand the data coverage greatly. Details can be found in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Overview of the WAGEUBN quantization framework. “BW” denotes bit-wise operations. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Training curves under the WAGEUBN framework: (a) [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Data distribution comparison between W, BN, A, G, and E before and after quantization. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Accuracy sensibility analysis of batch size under [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Data distributions of e 1 3 : (a) 8-bit QE2 , (b) 8-bit Flag QE2 , (c) full precision; (d) Data distribution comparison. respectively. Because the distribution of e l 3 is not uniform, the range covered by different quantization methods varies a lot. The data ratios (P…
Figure 10
Figure 10. Figure 10: Although the larger values take greater impacts on [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Comparison of time, power, and area of single [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 21 canonical work pages

  1. [1]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436, 2015

    2015

  2. [2]

    Image denoising and inpainting with deep neural networks

    Junyuan Xie, Linli Xu, and Enhong Chen. Image denoising and inpainting with deep neural networks. In International Conference on Neural Information Process- ing Systems , 2012

    work page 2012

  3. [3]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems , pages 91–99, 2015

    work page 2015

  4. [4]

    A unified archi- tecture for natural language processing: Deep neural networks with multitask learning

    Ronan Collobert and Jason Weston. A unified archi- tecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning , pages 160–167. ACM, 2008

    work page 2008

  5. [5]

    Creating more intelligent robots through brain-inspired computing

    Bo Zhang, Luping Shi, and Sen Song. Creating more intelligent robots through brain-inspired computing. spe- cial supplement: Brain-inspired intelligent robotics: The intersection of robotics and neuroscience sciences , pages 4–9, Science V ol. 354, Issue 6318, pp.1445 (2016)

    work page 2016

  6. [6]

    Resiliency of deep neural networks under quantization

    Wonyong Sung, Sungho Shin, and Kyuyeon Hwang. Resiliency of deep neural networks under quantization. arXiv preprint arXiv:1511.06488 , 2015

    arXiv 2015

  7. [7]

    Binaryconnect: Training deep neural networks with binary weights during propagations

    Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems , pages 3123– 3131, 2015

    work page 2015

  8. [8]

    Xnor-net: Imagenet classification using binary convolutional neural networks

    Mohammad Rastegari, Vicente Ordonez, Joseph Red- mon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. In Euro- pean Conference on Computer Vision , pages 525–542. Springer, 2016

    work page 2016

Show all 38 references
  1. [9]

    Extremely low bit neural network: Squeeze the last bit out with admm

    Cong Leng, Zesheng Dou, Hao Li, Shenghuo Zhu, and Rong Jin. Extremely low bit neural network: Squeeze the last bit out with admm. 2018

    2018

  2. [10]

    Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients

    Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160 , 2016

    2016 arXiv

  3. [11]

    Gxnor-net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework

    Lei Deng, Peng Jiao, Jing Pei, Zhenzhi Wu, and Guoqi Li. Gxnor-net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework. Neural Networks, 100:49–58, 2018

    2018

  4. [12]

    Training deep neural networks with 8-bit floating point numbers

    Naigang Wang, Jungwook Choi, Daniel Brand, Chia-Yu Chen, and Kailash Gopalakrishnan. Training deep neural networks with 8-bit floating point numbers. In Advances in neural information processing systems , pages 7675– 7684, 2018

    2018

  5. [13]

    Scalable methods for 8-bit training of neural networks

    Ron Banner, Itay Hubara, Elad Hoffer, and Daniel Soudry. Scalable methods for 8-bit training of neural networks. In Advances in Neural Information Processing Systems, pages 5145–5153, 2018

    2018

  6. [14]

    Mixed precision training

    Paulius Micikevicius, Sharan Narang, Jonah Alben, Gre- gory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXiv:1710.03740, 2017

    arXiv

  7. [15]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 , 2015

    2015 arXiv

  8. [16]

    Mixed precision training of convolutional neural networks using integer operations

    Dipankar Das, Naveen Mellempudi, Dheevatsa Mudi- gere, Dhiraj Kalamkar, Sasikanth Avancha, Kunal Baner- jee, Srinivas Sridharan, Karthik Vaidyanathan, Bharat Kaul, Evangelos Georganas, et al. Mixed precision training of convolutional neural networks using integer operations. a...

    2018 arXiv

  9. [17]

    Per-tensor fixed- point quantization of the back-propagation algorithm

    Charbel Sakr and Naresh Shanbhag. Per-tensor fixed- point quantization of the back-propagation algorithm. arXiv preprint arXiv:1812.11732 , 2018

    2018 arXiv

  10. [18]

    Training and inference with integers in deep neural networks

    Shuang Wu, Guoqi Li, Feng Chen, and Luping Shi. Training and inference with integers in deep neural networks. arXiv preprint arXiv:1802.04680 , 2018

    2018 arXiv

  11. [19]

    Gradient-based learning applied to document recognition

    Yann LeCun, L ´eon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278–2324, 1998

    1998

  12. [20]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014

    2014 arXiv

  13. [21]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012

    2012

  14. [22]

    On the momentum term in gradient descent learning algorithms

    Ning Qian. On the momentum term in gradient descent learning algorithms. Neural networks , 12(1):145–151, 1999

    1999

  15. [23]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  16. [24]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248–255. Ieee, 2009

    2009

  17. [25]

    Binarized neural net- works: Training deep neural networks with weights and activations constrained to+ 1 or-1

    Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural net- works: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830, 2016

    2016 arXiv

  18. [26]

    Neural networks for machine learning

    Geoffrey Hinton, Nitsh Srivastava, and Kevin Swersky. Neural networks for machine learning. Coursera, video lectures, 264, 2012

    2012

  19. [27]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013

    2013 arXiv

  20. [28]

    Quantized neural net- works: Training neural networks with low precision weights and activations

    Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Quantized neural net- works: Training neural networks with low precision weights and activations. The Journal of Machine Learn- ing Research, 18(1):6869–6898, 2017

    2017

  21. [29]

    Balanced quantization: An effec- tive and efficient approach to quantized neural net- works

    Shu-Chang Zhou, Yu-Zhi Wang, He Wen, Qin-Yao He, and Yu-Heng Zou. Balanced quantization: An effec- tive and efficient approach to quantized neural net- works. Journal of Computer Science and Technology , 32(4):667–682, 2017

    2017

  22. [30]

    Effective quan- tization methods for recurrent neural networks

    Qinyao He, He Wen, Shuchang Zhou, Yuxin Wu, Cong Yao, Xinyu Zhou, and Yuheng Zou. Effective quan- tization methods for recurrent neural networks. arXiv preprint arXiv:1611.10176, 2016

    2016 arXiv

  23. [31]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision , pages 1026–1034, 2015

    2015

  24. [32]

    Tbn: Convolutional neural network with ternary inputs and binary weights

    Diwen Wan, Fumin Shen, Li Liu, Fan Zhu, Jie Qin, Ling Shao, and Heng Tao Shen. Tbn: Convolutional neural network with ternary inputs and binary weights. In Proceedings of the European Conference on Computer Vision (ECCV), pages 315–332, 2018

    2018

  25. [33]

    Learning low precision deep neural networks through regularization

    Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee. Learning low precision deep neural networks through regularization. arXiv preprint arXiv:1809.00095 , 2018

    2018 arXiv

  26. [34]

    Ten- sorflow: Large-scale machine learning on heterogeneous distributed systems

    Mart ´ın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. Ten- sorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 , 2016

    2016 arXiv

  27. [35]

    Using learning rate schedules for deep learning models in python with keras, 2016

    J Brownlee. Using learning rate schedules for deep learning models in python with keras, 2016

    2016

  28. [36]

    Learning rate schedules and adaptive learning rate methods for deep learning

    Suki Lau. Learning rate schedules and adaptive learning rate methods for deep learning. Towards Data Science , 2017

    2017

  29. [37]

    Finite-time bound- edness of large-scale systems with actuator faults and gain fluctuations

    V Tharanidharan, R Sakthivel, B Kaviarasan, Faris Alzahrani, and S Marshal Anthoni. Finite-time bound- edness of large-scale systems with actuator faults and gain fluctuations. International Journal of Robust and Nonlinear Control, 2019

    2019

  30. [38]

    Single precision in weather forecasting models: An evaluation with the ifs

    Filip V ´aˇna, Peter D ¨uben, Simon Lang, Tim Palmer, Martin Leutbecher, Deborah Salmond, and Glenn Carver. Single precision in weather forecasting models: An evaluation with the ifs. Monthly Weather Review , 145(2):495–502, 2017

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.