REVIEW 3 cited by
Training Sparse Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Deep neural networks with lots of parameters are typically used for large-scale computer vision tasks such as image classification. This is a result of using dense matrix multiplications and convolutions. However, sparse computations are known to be much more efficient. In this work, we train and build neural networks which implicitly use sparse computations. We introduce additional gate variables to perform parameter selection and show that this is equivalent to using a spike-and-slab prior. We experimentally validate our method on both small and large networks and achieve state-of-the-art compression results for sparse neural network models.
Forward citations
Cited by 3 Pith papers
-
Architecture-aware Network Pruning for Vision Quality Applications
An architecture-aware pruning method cuts SID and EDSR network MACs by 58% and 37% with no measured PSNR or SSIM drop.
-
Principled Approximation Methods for Efficient and Scalable Deep Learning
A thesis that synthesizes the author's published work on continuous approximations to discrete deep learning problems, with experiments showing efficiency gains, but offering little new beyond the author's prior papers.
-
Smaller Models, Better Generalization
A regularizer claimed to minimize a VC dimension bound for neural networks is proposed, but the bound derivation drops a required term and the empirical gains over L2 regularization are inconsistent.
Discussion (0). Continue with ORCID to comment.