REVIEW 1 cited by
Faster Neural Network Training with Approximate Tensor Operations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We propose a novel technique for faster deep neural network training which systematically applies sample-based approximation to the constituent tensor operations, i.e., matrix multiplications and convolutions. We introduce new sampling techniques, study their theoretical properties, and prove that they provide the same convergence guarantees when applied to SGD training. We apply approximate tensor operations to single and multi-node training of MLP and CNN networks on MNIST, CIFAR-10 and ImageNet datasets. We demonstrate up to 66% reduction in the amount of computations and communication, and up to 1.37x faster training time while maintaining negligible or no impact on the final test accuracy.
Forward citations
Cited by 1 Pith paper
-
Accelerated CNN Training Through Gradient Approximation
Approximating the weight gradient for a subset of layers and batches yields 3.5% to 7% wall-clock training speedup on CIFAR-10 deep CNNs with minimal validation accuracy loss.
Discussion (0). Continue with ORCID to comment.