Pith. sign in

REVIEW 9 cited by

Training and Inference with Integers in Deep Neural Networks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1802.04680 v1 pith:GQYEQHBY submitted 2018-02-13 cs.LG

Shuang Wu , Guoqi Li , Feng Chen , Luping Shi This is my paper · ORCID
classification cs.LG
keywords inferencetrainingdeepintegersbeendiscretelearninglow-bitwidth
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Researches on deep neural networks with discrete parameters and their deployment in embedded systems have been active and promising topics. Although previous works have successfully reduced precision in inference, transferring both training and inference processes to low-bitwidth integers has not been demonstrated simultaneously. In this work, we develop a new method termed as "WAGE" to discretize both training and inference, where weights (W), activations (A), gradients (G) and errors (E) among layers are shifted and linearly constrained to low-bitwidth integers. To perform pure discrete dataflow for fixed-point devices, we further replace batch normalization by a constant scaling layer and simplify other components that are arduous for integer implementation. Improved accuracies can be obtained on multiple datasets, which indicates that WAGE somehow acts as a type of regularization. Empirically, we demonstrate the potential to deploy training in hardware systems such as integer-based deep learning accelerators and neuromorphic chips with comparable accuracy and higher energy efficiency, which is crucial to future AI applications in variable scenarios with transfer and continual learning demands.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Refining Datapath for Microscaling ViTs

    cs.AR 2025-05 conditional novelty 6.0 of 10

    MXInt-based datapath designs put all ViT nonlinear operators on an FPGA at 2-5 bit mantissas, with under 1% ImageNet accuracy loss on DeiT models.

  2. Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers

    cs.LG 2019-09 conditional novelty 6.0 of 10

    WAGEUBN trains ResNet models on ImageNet using 8-bit integers for weights, activations, gradients, errors, batch normalization, and the Momentum optimizer, with moderate accuracy loss.

  3. QUOTIENT: Two-Party Secure Neural Network Training and Prediction

    cs.CR 2019-07 unverdicted novelty 6.0 of 10

    QUOTIENT achieves 50X faster WAN training time and 6% higher absolute accuracy for secure two-party DNN training by jointly optimizing a discretized training algorithm with a tailored secure protocol.

  4. APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning

    cs.CV 2026-08 conditional novelty 5.0 of 10

    A profiling-guided, LLM-driven framework combines structured pruning and mixed-precision quantization-aware training, reporting 13-18x bit-operation reductions with modest accuracy loss on ImageNet and CIFAR-10.

  5. Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A one-shot training framework uses dynamic BN zeroization/recovery and an ML-based latency predictor to compress or scale DNNs to meet hard latency constraints on edge devices.

  6. A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models

    cs.CL 2025-09 conditional novelty 4.0 of 10

    A hybrid-granularity FP8 training recipe achieves near-lossless accuracy versus BF16 on reasoning benchmarks with up to 22% faster training and 14% lower memory.

  7. Exploring Vision Neural Network Pruning via Screening Methodology

    cs.LG 2025-02 unverdicted novelty 4.0 of 10

    A unified F-statistic screening and weighted evaluation method prunes both unstructured and structured parameters in FNNs and CNNs, claiming order-of-magnitude size reduction with competitive accuracy on vision datasets.

  8. Characterization and Mitigation of ADC Noise by Reference Tuning in RRAM-Based Compute-In-Memory

    cs.ET 2025-02 reject novelty 4.0 of 10

    A measured effective-bit noise model is used to simulate accuracy loss on four workloads; per-module or per-ADC reference tuning helps some tasks, but the drone result contradicts the claimed general effectiveness.

  9. Group Pruning using a Bounded-Lp norm for Group Gating and Regularization

    stat.ML 2019-08 conditional novelty 4.0 of 10

    A bounded-L1 regularizer combined with exponential gating layers prunes neural network channels to exactly zero during training, compressing standard models by 30 to 75 percent with little accuracy loss.

Pith tools