pith. sign in

arxiv: 1502.02551 · v1 · pith:CHDVJQH3new · submitted 2015-02-09 · 💻 cs.LG · cs.NE· stat.ML

Deep Learning with Limited Numerical Precision

Suyog Gupta , Ankur Agrawal , Kailash Gopalakrishnan , Pritish Narayanan This is my paper
classification 💻 cs.LG cs.NEstat.ML
keywords deepfixed-pointroundingtraininglimitedlow-precisionnetworknetworks
0
0 comments X
read the original abstract

Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a crucial role in determining the network's behavior during training. Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy. We also demonstrate an energy-efficient hardware accelerator that implements low-precision fixed-point arithmetic with stochastic rounding.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Template-Based Posit Multiplication for Training and Inferring in Neural Networks

    cs.CV 2019-07 unverdicted novelty 7.0

    Posit multiplier template in FloPoCo enables neural network training and inference with reduced-precision posits, matching FP accuracy on MNIST but not CIFAR-10.

  2. Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

    cs.CL 2016-09 accept novelty 6.0

    GNMT deploys 8-layer LSTMs with attention, wordpieces, low-precision inference, and coverage-penalized beam search to match state-of-the-art on WMT'14 En-Fr and En-De while cutting translation errors by 60% in human e...

  3. The Thermodynamic Costs of Simple Linear Regression

    cond-mat.stat-mech 2026-05 unverdicted novelty 5.0

    Thermodynamic lower bounds are approximated for exact and SGD linear regression, producing energy-aware scaling laws for optimal training dataset size given a target generalization error.

  4. Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics

    cs.LG 2022-12 unverdicted novelty 2.0

    A comprehensive review of deep learning techniques for computational mechanics, including LSTM for constitutive modeling, PINNs for PDE solving, optimizers, and kernel methods.