Speeding up Resnet Architecture with Layers Targeted Low Rank Decomposition

Austin Wen; Habib Hajimolahoseini; Walid Ahmed; Yang Liu

Speeding up Resnet Architecture with Layers Targeted Low Rank Decomposition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.12412 v1 pith:7VFU2BLN submitted 2023-09-21 cs.CV cs.LG

Walid Ahmed , Habib Hajimolahoseini , Austin Wen , Yang Liu This is my paper · ORCID

classification cs.CVcs.LG

keywords compressionhardwarelayersnetworktrainingascend910decompositioninference

verification ladder T0 review T1 audit T2 compute T3 formal

0 comments

read the original abstract

Compression of a neural network can help in speeding up both the training and the inference of the network. In this research, we study applying compression using low rank decomposition on network layers. Our research demonstrates that to acquire a speed up, the compression methodology should be aware of the underlying hardware as analysis should be done to choose which layers to compress. The advantage of our approach is demonstrated via a case study of compressing ResNet50 and training on full ImageNet-ILSVRC2012. We tested on two different hardware systems Nvidia V100 and Huawei Ascend910. With hardware targeted compression, results on Ascend910 showed 5.36% training speedup and 15.79% inference speed on Ascend310 with only 1% drop in accuracy compared to the original uncompressed model

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training
cs.LG 2025-05 reject novelty 4.0 of 10

Shared KV caching across the last 25% to 50% of layers, adapted incrementally from pretrained LLaMA weights, is claimed to improve training speed and inference throughput with little accuracy loss.

Pith tools