StoMPP progressively binarizes BNN layers layerwise from input to output via stochastic masks, delivering depth-scalable accuracy gains in a fully STE-free regime by controlling activation-induced gradient blockades.
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
3 Pith papers cite this work, alongside 45 external citations. Polarity classification is still indexing.
3
Pith papers citing it
45
external citations · Pith
abstract
Although weight and activation quantization is an effective approach for Deep Neural Network (DNN) compression and has a lot of potentials to increase inference speed leveraging bit-operations, there is still a noticeable gap in terms of prediction accuracy between the quantized model and the full-precision model. To address this gap, we propose to jointly train a quantized, bit-operation-compatible DNN and its associated quantizers, as opposed to using fixed, handcrafted quantization schemes such as uniform or logarithmic quantization. Our method for learning the quantizers applies to both network weights and activations with arbitrary-bit precision, and our quantizers are easy to train. The comprehensive experiments on CIFAR-10 and ImageNet datasets show that our method works consistently well for various network structures such as AlexNet, VGG-Net, GoogLeNet, ResNet, and DenseNet, surpassing previous quantization methods in terms of accuracy by an appreciable margin. Code available at https://github.com/Microsoft/LQ-Nets
verdicts
UNVERDICTED 3representative citing papers
The authors propose BIHIWE, a microarchitecture that accelerates DNN dot-products by bit-partitioning them into spatially parallel low-bitwidth MAC units operating in the charge domain and sharing A/D converters.
Evolutionary fine-tuning of select weights in pre-quantized convolutional networks improves accuracy over standard rounding for VGG, ResNet, and autoencoder models.
citing papers explorer
-
Layerwise Progressive Freezing: A Training Scaffold for Depth-Scalable Binary Networks
StoMPP progressively binarizes BNN layers layerwise from input to output via stochastic masks, delivering depth-scalable accuracy gains in a fully STE-free regime by controlling activation-induced gradient blockades.
-
Mixed-Signal Charge-Domain Acceleration of Deep Neural networks through Interleaved Bit-Partitioned Arithmetic
The authors propose BIHIWE, a microarchitecture that accelerates DNN dot-products by bit-partitioning them into spatially parallel low-bitwidth MAC units operating in the charge domain and sharing A/D converters.
-
Evolutionary fine tuning of quantized convolution-based deep learning models
Evolutionary fine-tuning of select weights in pre-quantized convolutional networks improves accuracy over standard rounding for VGG, ResNet, and autoencoder models.