Pith. sign in

REVIEW 6 cited by

Soft Weight-Sharing for Neural Network Compression

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1702.04008 v2 pith:3756NG4P submitted 2017-02-13 stat.ML cs.LG

Karen Ullrich , Edward Meeds , Max Welling This is my paper · ORCID
classification stat.MLcs.LG
keywords compressionnetworkneuralpruningquantizationratessoftweight-sharing
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices. This however, conflicts with their computationally, memory and energy intense nature, leading to a growing interest in compression. Recent work by Han et al. (2015a) propose a pipeline that involves retraining, pruning and quantization of neural network weights, obtaining state-of-the-art compression rates. In this paper, we show that competitive compression rates can be achieved by using a version of soft weight-sharing (Nowlan & Hinton, 1992). Our method achieves both quantization and pruning in one simple (re-)training procedure. This point of view also exposes the relation between compression and the minimum description length (MDL) principle.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Information-Bottleneck Driven Binary Neural Network for Change Detection

    cs.CV 2025-07 conditional novelty 6.0 of 10

    BiCD is a 1-bit change detection network whose auxiliary IB-style losses improve F1 by about 1 to 3 points over other binary networks, with no extra inference cost.

  2. Hyperpruning: Efficient Search through Pruned Variants of Recurrent Neural Networks Leveraging Lyapunov Spectrum

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A Lyapunov-spectrum-based distance to the dense network lets hyperparameter search for pruned RNNs stop early and select models that beat both loss-based baselines and the dense originals.

  3. Stochastic Weight Sharing for Bayesian Neural Networks

    cs.LG 2025-05 conditional novelty 6.0 of 10

    2DGBNN compresses Bayesian neural networks by clustering weight means and variances into shared 2D Gaussians, reducing parameter counts by up to 99% on ImageNet-scale models with small accuracy losses.

  4. Efficient compression of neural networks and datasets

    cs.LG 2025-05 unverdicted novelty 5.0 of 10

    Refined probabilistic and smooth l0 pruning techniques approximate minimum description length for neural networks, achieving high compression with minimal accuracy loss and empirically verifying better sample efficien...

  5. Learning Multimodal Fixed-Point Weights using Gradient Descent

    cs.LG 2019-07 unverdicted novelty 5.0 of 10

    Gradient-based optimization learns symmetric Gaussian mixture modes for 2-bit fixed-point weight quantization, claiming state-of-the-art performance and self-adaptive weights.

  6. Neuron ranking -- an informed way to condense convolutional neural networks architecture

    cs.LG 2019-07 unverdicted novelty 5.0 of 10

    Shapley value and variational importance switch methods produce consistent rankings of filter importance in CNNs, enabling compression and interpretability.

Pith tools