A spatio-channel clustering framework for CNN compression reduces FLOPs by 81% and raises brain tumor MRI classification accuracy from 87.76% to 89.80% compared with global SVD and Tucker baselines.
Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
abstract
Although the latest high-end smartphone has powerful CPU and GPU, running deeper convolutional neural networks (CNNs) for complex tasks such as ImageNet classification on mobile devices is challenging. To deploy deep CNNs on mobile devices, we present a simple and effective scheme to compress the entire CNN, which we call one-shot whole network compression. The proposed scheme consists of three steps: (1) rank selection with variational Bayesian matrix factorization, (2) Tucker decomposition on kernel tensor, and (3) fine-tuning to recover accumulated loss of accuracy, and each step can be easily implemented using publicly available tools. We demonstrate the effectiveness of the proposed scheme by testing the performance of various compressed CNNs (AlexNet, VGGS, GoogLeNet, and VGG-16) on the smartphone. Significant reductions in model size, runtime, and energy consumption are obtained, at the cost of small loss in accuracy. In addition, we address the important implementation level issue on 1?1 convolution, which is a key operation of inception module of GoogLeNet as well as CNNs compressed by our proposed scheme.
representative citing papers
Introduces partially trainable tensor decompositions that split RGB convolutional filters into spatial and spectral parts, training only the latter for hyperspectral transfer learning.
ASVD compresses LLMs by 10-30% and KV caches by 50% via activation-aware SVD that absorbs outliers into transformed weights and calibrates per-layer sensitivity.
FPGA accelerator for Tucker decomposition reports 2.16-30.2x speedup versus CPU/GPU toolboxes on cardiac MRI data via fixed-point design and warm-start SVD.
WNQ uses weight normalization to reshape weight distributions and reduce quantization error, outperforming baselines on CIFAR-100 and ImageNet.
citing papers explorer
-
Hierarchical Spatio-Channel Clustering for Efficient Model Compression in Medical Image Analysis
A spatio-channel clustering framework for CNN compression reduces FLOPs by 81% and raises brain tumor MRI classification accuracy from 87.76% to 89.80% compared with global SVD and Tucker baselines.
-
Transfer learning RGB models to hyperspectral images with trainable tensor decompositions
Introduces partially trainable tensor decompositions that split RGB convolutional filters into spatial and spectral parts, training only the latter for hyperspectral transfer learning.
-
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
ASVD compresses LLMs by 10-30% and KV caches by 50% via activation-aware SVD that absorbs outliers into transformed weights and calibrates per-layer sensitivity.
-
Tucker Tensor Decomposition on FPGA
FPGA accelerator for Tucker decomposition reports 2.16-30.2x speedup versus CPU/GPU toolboxes on cardiac MRI data via fixed-point design and warm-start SVD.
-
Weight Normalization based Quantization for Deep Neural Network Compression
WNQ uses weight normalization to reshape weight distributions and reduce quantization error, outperforming baselines on CIFAR-100 and ImageNet.
- Certainty Is Redundant: Token Sparsification for Efficient Camouflaged Object Detection with Vision Foundation Models