arxiv: 1306.0543 · v2 · pith:37WR2NVEnew · submitted 2013-06-03 · 💻 cs.LG · cs.NE· stat.ML

Predicting Parameters in Deep Learning

Misha Denil , Babak Shakibi , Laurent Dinh , Marc'Aurelio Ranzato , Nando de Freitas This is my paper

classification 💻 cs.LG cs.NEstat.ML

keywords learningonlyvaluesdeeppredictpredictingseveralweights

0 comments p. Extension

Add this Pith Number to your LaTeX paper

\usepackage{pith}
\pithnumber{37WR2NVE}

Prints a linked pith:37WR2NVE badge after your title and writes the identifier into PDF metadata. Compiles on arXiv with no extra files. Learn more

read the original abstract

We demonstrate that there is significant redundancy in the parameterization of several deep learning models. Given only a few weight values for each feature it is possible to accurately predict the remaining values. Moreover, we show that not only can the parameter values be predicted, but many of them need not be learned at all. We train several different architectures by learning only a small number of weights and predicting the rest. In the best case we are able to predict more than 95% of the weights of a network without any drop in accuracy.

This paper has not been read by Pith yet.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Vanishing Contributions: A Unified Framework for Smooth and Iterative Model Compression
cs.LG 2025-10 unverdicted novelty 5.0

VCON is a unified framework for smooth iterative DNN compression that uses parallel execution and an affine combination to progressively replace the original model with its compressed form during fine-tuning.