pith. sign in

arxiv: 1702.04008 · v2 · pith:3756NG4Pnew · submitted 2017-02-13 · 📊 stat.ML · cs.LG

Soft Weight-Sharing for Neural Network Compression

Karen Ullrich , Edward Meeds , Max Welling This is my paper
classification 📊 stat.ML cs.LG
keywords compressionnetworkneuralpruningquantizationratessoftweight-sharing
0
0 comments X p. Extension
pith:3756NG4P Add to your LaTeX paper What is a Pith Number? 
\usepackage{pith}
\pithnumber{3756NG4P}

Prints a linked pith:3756NG4P badge after your title and writes the identifier into PDF metadata. Compiles on arXiv with no extra files. Learn more

read the original abstract

The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices. This however, conflicts with their computationally, memory and energy intense nature, leading to a growing interest in compression. Recent work by Han et al. (2015a) propose a pipeline that involves retraining, pruning and quantization of neural network weights, obtaining state-of-the-art compression rates. In this paper, we show that competitive compression rates can be achieved by using a version of soft weight-sharing (Nowlan & Hinton, 1992). Our method achieves both quantization and pruning in one simple (re-)training procedure. This point of view also exposes the relation between compression and the minimum description length (MDL) principle.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Efficient compression of neural networks and datasets

    cs.LG 2025-05 unverdicted novelty 5.0

    Refined probabilistic and smooth l0 pruning techniques approximate minimum description length for neural networks, achieving high compression with minimal accuracy loss and empirically verifying better sample efficien...