pith. machine review for the scientific record. sign in

arxiv: 1810.02340 · v2 · submitted 2018-10-04 · 💻 cs.CV · cs.LG

Recognition: unknown

SNIP: Single-shot Network Pruning based on Connection Sensitivity

Namhoon Lee , Thalaiyasingam Ajanthan , Philip H. S. Torr
Authors on Pith no claims yet
classification 💻 cs.CV cs.LG
keywords pruningnetworkgivennetworksapproachconnectionconnectionsexisting
0
0 comments X
read the original abstract

Pruning large neural networks while maintaining their performance is often desirable due to the reduced space and time complexity. In existing methods, pruning is done within an iterative optimization procedure with either heuristically designed pruning schedules or additional hyperparameters, undermining their utility. In this work, we present a new approach that prunes a given network once at initialization prior to training. To achieve this, we introduce a saliency criterion based on connection sensitivity that identifies structurally important connections in the network for the given task. This eliminates the need for both pretraining and the complex pruning schedule while making it robust to architecture variations. After pruning, the sparse network is trained in the standard way. Our method obtains extremely sparse networks with virtually the same accuracy as the reference network on the MNIST, CIFAR-10, and Tiny-ImageNet classification tasks and is broadly applicable to various architectures including convolutional, residual and recurrent networks. Unlike existing methods, our approach enables us to demonstrate that the retained connections are indeed relevant to the given task.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. XTinyU-Net: Training-Free U-Net Scaling via Initialization-Time Sensitivity

    eess.IV 2026-05 unverdicted novelty 7.0

    XTinyU-Net identifies the smallest stable U-Net configuration using a Jacobian sensitivity metric on unlabeled images at initialization, achieving comparable accuracy to full models with 400-1600 times fewer parameters.

  2. Not How Many, But Which: Parameter Placement in Low-Rank Adaptation

    cs.LG 2026-05 unverdicted novelty 6.0

    Gradient-informed placement of LoRA parameters recovers full performance under GRPO while random placement does not, due to differences in gradient rank and stability across training regimes.

  3. XTinyU-Net: Training-Free U-Net Scaling via Initialization-Time Sensitivity

    eess.IV 2026-05 unverdicted novelty 6.0

    A Jacobian sensitivity curve computed at initialization identifies the narrowest U-Net configuration that avoids performance collapse, matching nnU-Net accuracy with 400-1600x fewer parameters on six medical datasets.

  4. Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compression

    cs.LG 2026-05 unverdicted novelty 6.0

    PARSE trains a prompt-aware linear router on dense-model outputs to select dynamic SVD ranks, improving accuracy up to 10% at 0.6 compression ratio on LLaMA-7B while delivering 2.5x prefill and 2.4x decode speedups.

  5. Representation-Guided Parameter-Efficient LLM Unlearning

    cs.CL 2026-04 unverdicted novelty 6.0

    REGLU guides LoRA-based unlearning via representation subspaces and orthogonal regularization to outperform prior methods on forget-retain trade-off in LLM benchmarks.

  6. Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression

    cs.LG 2026-04 unverdicted novelty 4.0

    The prune-quantize-distill ordering produces a better accuracy-size-latency frontier on CIFAR-10/100 than any single technique or other orderings, with INT8 QAT providing the main runtime gain.