REVIEW 2 cited by
Single Shot Structured Pruning Before Training
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning before training which prune individual weights, our work develops a methodology to remove entire channels and hidden units with the explicit aim of speeding up training and inference. We introduce a compute-aware scoring mechanism which enables pruning in units of sensitivity per FLOP removed, allowing even greater speed ups. Our method is fast, easy to implement, and needs just one forward/backward pass on a single batch of data to complete pruning before training begins.
Forward citations
Cited by 2 Pith papers
-
LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservation
A post-pruning method uses pivoted QR decomposition to delete linearly dependent CNN filters and least-squares kernel adjustment to preserve output, gaining small extra compression without fine-tuning.
-
Pruning-based Data Selection and Network Fusion for Efficient Deep Learning
PruneFuse combines pruning at initialization with weight fusion and knowledge distillation to make active learning data selection cheaper and to initialize the final model.
Discussion (0). Continue with ORCID to comment.