Pith. sign in

REVIEW 5 major objections 5 minor 47 references

Cheetah: Mixed Low-Precision Hardware & Software Co-Design Framework for DNNs on the Edge

T0 review · 5 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The Cheetah framework shows 16-bit posits train small DNNs better than 16-bit floats, and 5–8-bit posits win the accuracy-versus-energy tradeoff for edge inference.

desk verdict Useful engineering framework and the first same-bit-width posit-vs-float training comparison, but the headline training claim rests on a vanilla FP16 baseline and an internal inconsistency that need fixing before I would trust it. read the letter →

arxiv 1908.02386 v1 pith:2DX52CGV submitted 2019-08-06 cs.LG cs.NEstat.ML

Hamed F. Langroudi , Zachariah Carmichael , David Pastuch , Dhireesha Kudithipudi This is my paper · ORCID
classification cs.LGcs.NEstat.ML
keywords positarithmeticlow-precisiontrainingedgeinferenceenergy-delayproductquantizationexactmultiply-accumulatemixedprecisiondeepneuralnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Cheetah is a hardware/software co-design framework for low-precision deep networks aimed at edge devices. It emulates training and inference in three numerical formats—posit, floating point, and fixed-point—under two quantization schemes, and pairs them with an exact multiply-accumulate (EMAC) soft core to measure energy-delay product. The paper's central claim is that the posit format is not only competitive but superior at low precision: 16-bit posits train small feedforward networks more accurately than 16-bit floats, and 5–8-bit posits give better inference accuracy per energy-delay cost than floating point or fixed-point. A sympathetic reader would care because, if true, edge hardware could run smaller and cheaper arithmetic without sacrificing accuracy.

What carries the argument

The central object is the posit number format, a Type III unum whose tapered precision allocates more fraction bits to values near 1, with a regime field providing wide dynamic range. The argument is carried by pairing this format with the EMAC—an exact multiply-accumulate soft core that multiplies posits without intermediate rounding and accumulates into a wide fixed-point register (the quire) before a single convergent rounding step. Cheetah combines these with rounding or linear quantization and searches over bit-width and exponent/regime settings to find the point where accuracy and energy-delay product meet a design constraint.

What would settle it

Retrain the same 4-layer feedforward networks on MNIST and Fashion-MNIST with a standard mixed-precision FP16 schedule (32-bit weight updates, loss scaling, or stochastic rounding) and compare test accuracy with the 16-bit posit runs; if FP16 then reaches within about 0.1–0.5 percentage points of 32-bit float while 16-bit posit stays about 6 points lower, the training-advantage claim would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that posits outperform IEEE-style floating point and fixed-point at matched low bit widths for both training and inference on small image-classification tasks. In the paper's experiments, 16-bit posit training reaches 96.535% and 87.400% on MNIST and Fashion-MNIST, compared with 90.646% and 81.725% for 16-bit float; at 5 bits, posit inference beats float by 28.5 percentage points and fixed-point by 31.62 on CIFAR-10. The authors attribute this to two mechanisms: posit's tapered precision concentrates representation accuracy near 1, matching DNN weight and activation distributions, and the EMAC core defers rounding until after all products are accumulated.

Load-bearing premise

The load-bearing premise is that the FP16 baseline in Table V is a fair comparator for 16-bit training; it uses no mixed-precision accumulation, loss scaling, or stochastic rounding, so if a stronger FP16 recipe were used the claimed posit advantage could shrink.

Editorial extensions

If this is right

  • Small edge DNNs can be trained end-to-end at 16-bit posit precision with simple round-to-nearest arithmetic, without loss scaling or stochastic rounding, and still land within about 1% of 32-bit float accuracy.
  • Inference at 5–8 bits with posits, especially with linear quantization and exponent width es=0 or 1, keeps accuracy close to full precision while reducing EMAC energy-delay product.
  • Linear quantization with shift lets designers replace general multipliers with constant-power-of-two scalings, lowering hardware cost and enabling power-of-two scaling.
  • Cheetah-style search over bit-width and posit exponent settings can pick a configuration from a target accuracy and energy-delay constraint rather than relying on ad hoc choices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 16-bit posit training advantage may depend on the comparison margin: a stronger FP16 baseline using 32-bit accumulation and loss scaling could close the gap, and testing that baseline on the same networks would separate format benefit from baseline weakness.
  • The energy-delay results are measured on a Virtex-7 FPGA soft core; an ASIC implementation could shift the comparison, especially because posit encode/decode is resource-heavy while avoiding subnormals saves latency.
  • Because posit's tapered precision is matched to weight distributions in small networks, deeper networks with wider dynamic ranges (such as ImageNet-scale CNNs) may need more exponent bits, and the framework's search over es is a natural way to test this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents Cheetah, a hardware/software co-design framework for low-precision DNN training and inference that supports posit, floating-point, and fixed-point formats under rounding and linear quantization. The framework couples software emulation in Keras/TensorFlow with an FPGA soft-core implementation of exact multiply-and-accumulate (EMAC) operations. The authors evaluate inference on MNIST, Fashion-MNIST, and CIFAR-10 at 5–8 bit widths and training on MNIST and Fashion-MNIST at 16 and 32 bits. The two headline claims are that 16-bit posits outperform 16-bit floating point in DNN training, and that 5–8-bit posits improve the accuracy-versus-energy-delay-product trade-off over float and fixed-point for inference. The paper also reports on the EMAC FPGA implementation and compares resource usage, latency, and energy-delay product across formats.

Significance. If substantiated, the training result would be a notable contribution: it would show a simple round-to-nearest posit pipeline matching or exceeding FP16 training without loss scaling or master-weight copies, which would be attractive for edge training. The inference study is also useful in that it compares formats at equal bit-widths, uses a consistent EMAC hardware baseline, and reports a concrete FPGA implementation. The distortion-rate analysis (Fig. 4) is a reasonable way to connect format properties to DNN parameter statistics. The framework itself, with support for multiple formats and quantization schemes, is a useful artifact for the community. However, the central training claim rests on a comparison against a vanilla FP16 baseline that lacks established mixed-precision techniques, and the paper contains a direct numerical inconsistency in the magnitude of the claimed accuracy degradation. As a result, the headline result is not yet established by the evidence presented.

major comments (5)
  1. [Section V-C, Table V] The claim that '16-bit posits outperform 16-bit floating point in DNN training' is not supported as stated because the FP16 baseline is a bare pipeline with no loss scaling, no FP32 master-weight copies, no stochastic rounding, and no mixed-precision accumulation. The cited prior work on FP16 training (e.g., [14], [34]) uses these techniques specifically to prevent the accuracy collapse that Table V shows for Float-16 (90.646% on MNIST and 81.725% on Fashion-MNIST, roughly 7.4 points below Float-32). Without a fair FP16 baseline, the observed gap may reflect FP16's limited 5-bit exponent range rather than any advantage of the posit format. Please rerun the comparison with a state-of-the-art mixed-precision FP16 baseline, or explicitly restrict and justify the claim to a 'vanilla FP16 without mixed-precision techniques' comparison.
  2. [Section V-C, final paragraph and Table V] The text states that 16-bit posits show 'less than 1% accuracy degradation' without exhaustively analyzing DNN training parameters. Table V, however, shows Posit-16 at 96.535% versus Posit-32 at 98.131% on MNIST (a 1.60 percentage-point drop) and at 87.400% versus 89.263% on Fashion-MNIST (a 1.86 percentage-point drop). This is an internal inconsistency between the text and the reported numbers, and it must be corrected. In addition, although Table V reports averages over 10 independent runs, no variance, standard deviation, or significance test is given, so the claimed 'slightly better accuracy' of Posit-16 over Float-16 cannot be assessed statistically.
  3. [Section V-C, Table V and Section II-B] The table reports accuracy for 'Posit-16' without specifying the posit exponent size es. Since a posit's value, dynamic range, and precision depend critically on es (Eq. 3), and since the paper elsewhere shows that es ∈ {0,1,2} materially changes accuracy (Table IV), the reported Posit-16 numbers are not reproducible or comparable to Float-16 without knowing which es was used. Please report the exact posit configuration and the analogous float exponent-bit width for every training run.
  4. [Section V-A, Table III] The inference comparison in Table III reports 'respective best results' for each format, with posit es ∈ {0,1,2} and floating-point exponent width we ∈ {3,4}. This is a form of selecting the best exponent configuration per format, but the table does not indicate which es or we was chosen for each bit-width and dataset. If the best configuration is selected per cell on the test set, the comparison is optimistic for both formats and should be justified or replaced with a fixed configuration policy; at minimum, the selected configurations should be listed so that the reader can assess whether the comparison is balanced.
  5. [Section V-C and Section IV-A] The description of the training pipeline is incomplete in ways that are load-bearing for the training claim. The paper says networks are trained 'using only the specified numerical format,' but it does not state how gradients, weight updates, and biases are quantized, whether the EMAC/quire accumulator is used during training as well as inference, how the learning rate is represented, or what optimizer and hyperparameters are used. Without this information, the FP16 and Posit-16 results in Table V cannot be reproduced, and it is unclear whether the comparison isolates the numerical format or conflates it with pipeline choices.
minor comments (5)
  1. [Section I, Introduction] The sentence 'However, the scope of these studies is limited, as highlighted below' is followed by item 1: 'None of the previous works explore the propriety of the posit numerical format'; 'propriety' should be 'appropriateness' or 'suitability'.
  2. [Figures 8 and 9] The legends in Figures 8 and 9 contain garbled tokens such as '/uni00000029/uni0000004c/...' instead of readable format labels. These figures are otherwise central to the EDP comparison and need to be regenerated with correct text.
  3. [Table VI and References] Table VI lists 'Courbariaux et al. [45]' as a separate row, but reference [45] appears to be the same arXiv paper as reference [32] (both cite arXiv:1412.7024) with different titles. This duplication should be cleaned up, and the table's 'DNN library' entry for Wang et al. ('Home Suite') should be clarified.
  4. [Section IV-A and Table I] The text says that β is selected from {1, 2, 4, 8} to provide 'a wide coverage (~82%) of the dynamic range of each numerical format, as shown in Table I,' but Table I reports coverage percentages for the numerical formats themselves, not for the β values. The connection between the β grid and the coverage percentages is not explained and should be made explicit.
  5. [Abstract and Section IV] The abstract states that Cheetah 'supports mixed-precision floating point and fixed-point numerical formats,' but the training experiments reported here use only a single 16- or 32-bit format per network with no mixed-precision configuration. Please either add such experiments or soften the abstract to describe the framework's capability rather than claiming demonstrated mixed-precision training.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims are empirical benchmarks, not derivations from fitted inputs or self-citation chains.

full rationale

The paper's main claims are comparative accuracy and energy-delay measurements reported in Tables III, IV, and V, produced by software emulation and FPGA synthesis. No fitted constant is fed back as a prediction, and no equation is shown to reduce to its own inputs by construction. The EMAC design is taken from the authors' prior work [20], [21], but that is design reuse rather than a load-bearing circular argument: the posit-versus-float comparisons are measured results, not consequences of the citation. Concerns about the fairness of the FP16 training baseline and the apparent inconsistency between the '<1% accuracy degradation' text and the Table V numbers are correctness risks, not circularity. They do not amount to a claim that a prediction is equivalent to its input by definition.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims are empirical, so the ledger holds the choices that shape the measurements: the beta search range, data-dependent scale factors, best exponent-configuration selection, the assumption that DNN weights resemble posit's distribution, the EMAC arithmetic core, and the per-MAC EDP extrapolation.

free parameters (3)
  • Linear quantization range beta = beta in {1, 2, 4, 8}
    Section V picks beta from {1,2,4,8} to limit search; beta sets alpha1 and alpha2 in Equation (5) and directly changes accuracy results.
  • Scale factors alpha1 and alpha2 = alpha1 = beta / Max(A_i); alpha2 = 2 * beta / (Max(W_i) - Min(W_i))
    Computed from the min and max of activations and weights in Section IV-A. This is standard data-dependent quantization scaling, but it is a fitted per-layer value.
  • Best exponent configuration per format = posit es in {0,1,2}; float we in {3,4}, best per bit-width
    Table III reports 'respective best results' for each format. This post hoc selection is a free choice that affects the claimed accuracy advantage.
assumptions (4)
  • standard math Posit arithmetic and EMAC quire properties from [20], [25] are correct and applicable to DNN numeric ranges.
    The framework's behavior depends on the posit encoding in Equation (3) and the quire width in Equation (6); these are taken as given from prior literature.
  • domain assumption DNN training and inference parameters are concentrated around one, matching posit's tapered precision distribution.
    Section V-A uses this to explain posit's accuracy advantage; it is an empirical claim tested only on three small datasets and is not independently established in this paper.
  • domain assumption The exact MAC (EMAC) with a wide quire register preserves precision and is a fair arithmetic core for comparing formats.
    Algorithm 1 and Equation (6) define the EMAC; the accuracy and EDP comparisons inherit this design from [20], so results for other MAC designs could differ.
  • domain assumption Per-MAC energy-delay-product on one FPGA is representative of full DNN edge hardware cost.
    Figures 5-9 measure only the EMAC soft core, not a complete network accelerator; the conclusion that 6-bit posit is the best tradeoff extrapolates from this single unit.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cheetah: Mixed Low-Precision Hardware & Software Co-Design Framework for DNNs on the Edge." pith.science (2026). https://pith.science/paper/2DX52CGV

@misc{pith2026190802386,
  author       = {Pith},
  title        = {Pith review of: Cheetah: Mixed Low-Precision Hardware & Software Co-Design Framework for DNNs on the Edge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2DX52CGV}},
  note         = {Machine review of arXiv:1908.02386}
}
read the original abstract

Low-precision DNNs have been extensively explored in order to reduce the size of DNN models for edge devices. Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision in [5..8]-bits. However, previous studies were limited to studying posit for DNN inference only. In this paper, we propose the Cheetah framework, which supports both DNN training and inference using posits, as well as other commonly used formats. Additionally, the framework is amenable for different quantization approaches and supports mixed-precision floating point and fixed-point numerical formats. Cheetah is evaluated on three datasets: MNIST, Fashion MNIST, and CIFAR-10. Results indicate that 16-bit posits outperform 16-bit floating point in DNN training. Furthermore, performing inference with [5..8]-bit posits improves the trade-off between performance and energy-delay-product over both [5..8]-bit float and fixed-point.

Figures

Figures reproduced from arXiv: 1908.02386 by the authors.

Figure 1
Figure 1. The Cheetah High-level Hardware & Software Co-design framework for DNNs on the edge. EDP: Energy-Delay Product. 32-bit high-precision floating point with soft-core MACs to a low-precision DNN model with either posit, floating point, or fixed-point arithmetic soft-core exact-MACs (EMACs). This optimization is performed iteratively, reducing the bit￾precision by one at each step; the performance degradation and hardwa… view at source ↗
Figure 2
Figure 2. The Cheetah software framework for feedforward neural networks with one hidden layer. The framework scales to any DNN architecture. paper is calculated using the EMAC algorithm [20]. In the EMAC, to preserve precision in computing the products, the posit weights and activations are multiplied in a posit format without truncation or rounding at the end of multiplications. To avoid rounding during accumulation, the pr… view at source ↗
Figure 3
Figure 3. A parameterized (n total bits, es exponent bits) FPGA soft core design of the posit exact multiply-and-accumulate (EMAC) operation [20]. mentioned beforehand, the fixed-point values of Nop products are accumulated in a wide register sized as given by (6). The posit EMAC, illustrated by [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Layer-wise delta distortion rate ∆(d(R)) heatmaps compare the precision (rates) of [5..8]-bit numerical formats for representing 32-bit floating point DNN parameters. The average ∆(d(R)) among all weights in a DNN are shown in the final column of each heatmap. (a) d(R)…
Figure 5
Figure 5. Figure 5: The average accuracy degradation from 32-bit floating point across the two classification tasks vs. the energy-delay [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: The average accuracy degradation from 32-bit floating point across the two classification tasks vs. the latency of the [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: The average accuracy degradation from 32-bit floating point across the two classification tasks vs. the cost of the [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: The average accuracy degradation from 32-bit floating point across the two classification tasks vs. the energy-delay [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: The average accuracy degradation from 32-bit floating point across the two classification tasks vs. the latency of [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 30 canonical work pages

  1. [14]

    Training deep neural networks with 8-bit floating point numbers,

    N. Wang, J. Choi, D. Brand, C.-Y . Chen, and K. Gopalakrishnan, “Training deep neural networks with 8-bit floating point numbers,” in Advances in neural information processing systems , 2018, pp. 7686– 7695

    work page 2018

  2. [34]

    Mixed precision training,

    P. Micikevicius, S. Narang, J. Alben, G. F. Diamos, E. Elsen et al. , “Mixed precision training,” in Conference Track Proceedings of the 6th International Conference on Learning Representations, ICLR . Vancouver, BC, Canada: OpenReview.net, 2018. [Online]. Available: https://openreview.net/forum?id=r1gs9JgRZ

    work page 2018

  3. [1]

    Edge intelligence: On-demand deep learning model co-inference with device-edge synergy,

    E. Li, Z. Zhou, and X. Chen, “Edge intelligence: On-demand deep learning model co-inference with device-edge synergy,” in Proceedings of the 2018 Workshop on Mobile Edge Communications . ACM, 2018, pp. 31–36

    work page 2018

  4. [2]

    Edge computing: Vision and challenges,

    W. Shi, J. Cao, Q. Zhang, Y . Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE Internet of Things Journal , vol. 3, no. 5, pp. 637–646, 2016

    2016

  5. [3]

    The emergence of edge computing,

    M. Satyanarayanan, “The emergence of edge computing,” Computer, vol. 50, no. 1, pp. 30–39, 2017

    2017

  6. [4]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems, NeurIPS , P. L. Bartlett, F. C. N. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds., Lake Tahoe, Nevada, US...

    work page 2012

  7. [5]

    1.1 computing’s energy problem (and what we can do about it),

    M. Horowitz, “1.1 computing’s energy problem (and what we can do about it),” in 2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC) . IEEE, 2014, pp. 10–14

    2014

  8. [6]

    Scaling for edge inference of deep neural networks,

    X. Xu, Y . Ding, S. X. Hu, M. Niemier, J. Cong et al., “Scaling for edge inference of deep neural networks,” Nature Electronics, vol. 1, no. 4, p. 216, 2018

    work page 2018

Show all 47 references
  1. [7]

    Ma- chine learning at facebook: Understanding inference at the edge,

    C.-J. Wu, D. Brooks, K. Chen, D. Chen, S. Choudhury et al. , “Ma- chine learning at facebook: Understanding inference at the edge,” in 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 2019, pp. 331–344

    2019

  2. [8]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang et al. , “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861 , 2017

    2017 arXiv

  3. [9]

    Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution,

    Y . Chen, H. Fang, B. Xu, Z. Yan, Y . Kalantidis et al. , “Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution,” arXiv preprint arXiv:1904.05049 , 2019

    1904 arXiv

  4. [10]

    MEC: Memory-efficient convolution for deep neural network,

    M. Cho and D. Brand, “MEC: Memory-efficient convolution for deep neural network,” in Proceedings of the 34th International Conference on Machine Learning, ICML , ser. Proceedings of Machine Learning Research, D. Precup and Y . W. Teh, Eds., vol. 70. Sydney, NSW, Australia: PMLR...

    2017

  5. [11]

    Sbnet: Sparse blocks network for fast inference,

    M. Ren, A. Pokrovsky, B. Yang, and R. Urtasun, “Sbnet: Sparse blocks network for fast inference,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 8711–8720

    2018

  6. [12]

    Revisiting the importance of individual units in cnns via ablation,

    B. Zhou, Y . Sun, D. Bau, and A. Torralba, “Revisiting the importance of individual units in cnns via ablation,” arXiv preprint arXiv:1806.02891, 2018

    2018 arXiv

  7. [13]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference,

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang et al. , “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    2018

  8. [15]

    Understanding the impact of precision quantization on the accuracy and energy of neural networks,

    S. Hashemi, N. Anthony, H. Tann, R. I. Bahar, and S. Reda, “Understanding the impact of precision quantization on the accuracy and energy of neural networks,” in Design, Automation & Test in Europe Conference & Exhibition, DATE , D. Atienza and G. D. Natale, Eds. Lausanne, Swi...

    2017

  9. [16]

    Ristretto: A frame- work for empirical study of resource-efficient inference in convolutional neural networks,

    P. Gysel, J. Pimentel, M. Motamedi, and S. Ghiasi, “Ristretto: A frame- work for empirical study of resource-efficient inference in convolutional neural networks,” IEEE Transactions on Neural Networks and Learning Systems, 2018

    2018

  10. [17]

    A survey on methods and theories of quantized neural net- works,

    Y . Guo, “A survey on methods and theories of quantized neural net- works,” arXiv preprint arXiv:1808.04752 , 2018

    2018 arXiv

  11. [18]

    Quantizing deep convolutional networks for effi- cient inference: A whitepaper,

    R. Krishnamoorthi, “Quantizing deep convolutional networks for effi- cient inference: A whitepaper,” arXiv preprint arXiv:1806.08342, 2018

    2018 arXiv

  12. [19]

    Deep learning infer- ence on embedded devices: Fixed-point vs posit,

    S. H. F. Langroudi, T. Pandit, and D. Kudithipudi, “Deep learning infer- ence on embedded devices: Fixed-point vs posit,” in 2018 1st Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2) , March 2018, pp. 19–23

    2018

  13. [20]

    Deep positron: A deep neural network using the posit number system,

    Z. Carmichael, H. F. Langroudi, C. Khazanov, J. Lillie, J. L. Gustafson, and D. Kudithipudi, “Deep positron: A deep neural network using the posit number system,” in Design, Automation & Test in Europe Conference & Exhibition, DATE . Florence, Italy: IEEE, Mar. 2019, pp. 1421–...

    2019

  14. [21]

    Performance-efficiency trade-off of low-precision numerical formats in deep neural networks,

    Z. Carmichael, H. F. Langroudi, C. Khazanov, J. Lillie, J. L. Gustafson, and D. Kudithipudi, “Performance-efficiency trade-off of low-precision numerical formats in deep neural networks,” in Proceedings of the Conference for Next Generation Arithmetic, ser. CoNGA’19. Singapore,...

    2019

  15. [22]

    Rethinking floating point for deep learning,

    J. Johnson, “Rethinking floating point for deep learning,” arXiv preprint arXiv:1811.01721, 2018

    2018 arXiv

  16. [23]

    Gradient-based learning applied to document recognition,

    Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998

    1998

  17. [24]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, and A. Courville, Deep Learning. MIT Press, 2016, http://www.deeplearningbook.org

    2016

  18. [25]

    Beating floating point at its own game: Posit arithmetic,

    J. L. Gustafson and I. T. Yonemoto, “Beating floating point at its own game: Posit arithmetic,”Supercomputing Frontiers and Innovations, vol. 4, no. 2, pp. 71–86, 2017

    2017

  19. [26]

    Unums 2.0: An interview with John L. Gustafson,

    W. Tichy, “Unums 2.0: An interview with John L. Gustafson,” Ubiquity, vol. 2016, no. September, p. 1, 2016

    2016

  20. [27]

    VLSI implementation of a neural network model,

    H. P. Graf, L. D. Jackel, and W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer , vol. 21, no. 3, pp. 41–49,

  21. [28]

    An arti- ficial neural network accelerator using general purpose 24 bits floating point digital signal processors,

    A. Iwata, Y . Yoshida, S. Matsuda, Y . Sato, and N. Suzumura, “An arti- ficial neural network accelerator using general purpose 24 bits floating point digital signal processors,” in International Joint Conference on Neural Networks, IJCNN , vol. 2, 1989, pp. 171–175

    1989

  22. [29]

    A VLSI architecture for high-performance, low- cost, on-chip learning,

    D. W. Hammerstrom, “A VLSI architecture for high-performance, low- cost, on-chip learning,” in IJCNN 1990, International Joint Conference on Neural Networks. San Diego, CA, USA: IEEE, Jun. 1990, pp. 537–

    1990

  23. [30]

    Experimental determination of precision requirements for back-propagation training of artificial neural networks,

    K. Asanovic and N. Morgan, “Experimental determination of precision requirements for back-propagation training of artificial neural networks,” in In Proceedings of the 2nd International Conference on Microelectron- ics for Neural Networks , 1991, pp. 9–15. 13

    1991

  24. [31]

    Training with noise is equivalent to tikhonov regular- ization,

    C. M. Bishop, “Training with noise is equivalent to tikhonov regular- ization,” Neural computation, vol. 7, no. 1, pp. 108–116, 1995

    1995

  25. [33]

    Deep learning with limited numerical precision,

    S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, “Deep learning with limited numerical precision,” in Proceedings of the 32nd International Conference on Machine Learning, ICML , ser. JMLR Workshop and Conference Proceedings, F. R. Bach and D. M. Blei, Eds., vol. 37....

    2015

  26. [35]

    Flexpoint: An adaptive numerical format for efficient training of deep neural networks,

    U. K ¨oster, T. Webb, X. Wang, M. Nassar, A. K. Bansalet al., “Flexpoint: An adaptive numerical format for efficient training of deep neural networks,” inAdvances in Neural Information Processing Systems, 2017, pp. 1742–1752

    2017

  27. [36]

    Mixed precision training with 8-bit floating point,

    N. Mellempudi, S. Srinivasan, D. Das, and B. Kaul, “Mixed precision training with 8-bit floating point,” arXiv preprint arXiv:1905.12334 , 2019

    1905 arXiv

  28. [37]

    A study of BFLOAT16 for deep learning training,

    D. Kalamkar, D. Mudigere, N. Mellempudi, D. Das, K. Banerjee et al., “A study of BFLOAT16 for deep learning training,” arXiv preprint arXiv:1905.12322, 2019

    1905 arXiv

  29. [38]

    Serving DNNs in real time at datacenter scale with Project Brainwave,

    E. Chung, J. Fowers, K. Ovtcharov, M. Papamichael, A. Caulfield et al., “Serving DNNs in real time at datacenter scale with Project Brainwave,” IEEE Micro, vol. 38, no. 2, pp. 8–20, 2018

    2018

  30. [39]

    Rounding errors in algebraic processes,

    J. H. Wilkinson, “Rounding errors in algebraic processes,” in IFIP Congress, 1959, pp. 44–53

    1959

  31. [40]

    Posits: the good, the bad and the ugly,

    F. de Dinechin, L. Forget, J.-M. Muller, and Y . Uguen, “Posits: the good, the bad and the ugly,” Dec. 2018, working paper or preprint. [Online]. Available: https://hal.inria.fr/hal-01959581

    2018

  32. [41]

    The use of multiple measurements in taxonomic prob- lems,

    R. A. Fisher, “The use of multiple measurements in taxonomic prob- lems,” Annals of eugenics , vol. 7, no. 2, pp. 179–188, 1936

    1936

  33. [42]

    Kulisch, Computer arithmetic and validity: theory, implementation, and applications , 1st ed., ser

    U. Kulisch, Computer arithmetic and validity: theory, implementation, and applications , 1st ed., ser. de Gruyter Studies in Mathematics. Berlin, New York, USA: Walter de Gruyter, 2008, vol. 33

    2008

  34. [43]

    Chollet et al., “Keras,” https://github.com/keras-team/keras, 2015

    F. Chollet et al., “Keras,” https://github.com/keras-team/keras, 2015

    2015

  35. [44]

    TensorFlow: Large-scale machine learning on heterogeneous systems,

    M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen et al. , “TensorFlow: Large-scale machine learning on heterogeneous systems,”

  36. [45]

    Training deep neu- ral networks with low precision multiplications,

    M. Courbariaux, Y . Bengio, and J.-P. David, “Training deep neu- ral networks with low precision multiplications,” arXiv preprint arXiv:1412.7024, 2014

    2014 arXiv

  37. [544]

    Available: https://doi.org/10.1109/IJCNN.1990.137621

    [Online]. Available: https://doi.org/10.1109/IJCNN.1990.137621

    1990

  38. [1988]

    Available: https://doi.org/10.1109/2.30

    [Online]. Available: https://doi.org/10.1109/2.30

    doi

  39. [2015]

    Available: https://www.tensorflow.org/

    [Online]. Available: https://www.tensorflow.org/

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.