Pith. sign in

REVIEW 3 major objections 4 minor 45 references

LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservation

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LinDeps finds linearly dependent filters, removes them, and rewrites the next layer's kernels so accuracy survives with no fine-tuning.

desk verdict A useful fine-tuning-free post-pruning add-on with sound per-layer math, but the 'guaranteed preservation' claim is not established for ResNets because residual connections are never addressed. read the letter →

arxiv 2507.21573 v1 pith:3F76BY3Y submitted 2025-07-29 cs.CV

Maxim Henry , Adrien Deliège , Anthony Cioppa , Marc Van Droogenbroeck This is my paper · ORCID
classification cs.CV
keywords CNNsnetworkpruningpost-pruninglineardependencyanalysispivotedQRdecompositionsignalrecoveryfilterredundancymodelcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LinDeps is a post-pruning add-on that can be placed on top of any existing pruning technique, and the paper's central claim is that it removes the redundancy that other methods leave behind. That redundancy is expressed as linear dependence between feature maps: for a layer with C channels, batch images are flattened into a matrix A, a pivoted QR decomposition exposes how many channels are truly independent, and channels that are (near-)linear combinations of the rest are pruned. A recovery matrix L, obtained by least squares so that L A' ≈ A, is then applied to the next layer's kernels, which keeps the layer's output intact on the calibration batch and, in the experiments, leaves reported top-1 accuracy unchanged without any fine-tuning. Across CIFAR-10 and ImageNet with VGG-16, ResNet-56/110, and ResNet-50, LinDeps increased FLOPs reduction by about 0.16 to 5.7 percentage points on top of APIB and NORTON, and in a no-retraining low-resource setting it added up to 30 percentage points of pruning with under 0.3% accuracy loss. The paper therefore positions LinDeps as a universal complement that makes any pruning method more aggressive at almost no cost.

What carries the argument

The load-bearing object is the pair formed by pivoted QR decomposition and the recovery matrix L. Pivoted QR writes A^T P = QR, with the diagonal of R11 sorted by decreasing magnitude; values below τ(R11)11 mark channels to prune, and the permutation P maps back to original channel indices. The recovery matrix L ∈ $R^{{C×C'}}$ is the least-squares solution to L A' ≈ A, encoding each removed channel as a linear combination of survivors. Multiplying a next-layer kernel, flattened to $p^{2}$ × C, by L and reshaping to C'×p×p makes the next layer accept the pruned feature maps and emit approximately the same outputs, which is what lets many layers be pruned sequentially without fine-tuning.

What would settle it

Use a trained VGG-16, calibrate LinDeps on one 256-image batch with τ=0, and feed a held-out batch through the original and pruned networks; if the relative output error of the first pruned layer, ||L A'_test − A_test||_2 / ||A_test||_2, is much larger on the held-out batch than on the calibration batch, the recovery is batch-specific. Repeating the whole procedure with several different calibration batches and checking whether the pruned-channel sets or test accuracies vary would directly test the stability assumption on which the claimed performance preservation rests.

Watch

Extended reading notes

Core claim

LinDeps's discovery is that a layer's redundant filters can be identified exactly as linear dependencies among its feature maps and removed without retraining, provided the next layer is algebraically compensated. Concretely, the feature maps of a layer over a batch of B images are flattened and stacked into a matrix A ∈ $R^{{C×(B·H·W)}}$. A pivoted QR decomposition of A^T orders the channels by importance and reveals the effective rank on the diagonal of the triangular factor R11; a threshold τ zeroes small diagonal entries, leaving C' channels. LinDeps then solves L = argmin_{L} ||L A' − A||_2, where A' holds the surviving channels, and multiplies each flattened next-layer kernel (a $p^{2}$ × C matrix) by L to obtain a C'×p×p kernel. At τ = 0 this makes the next layer's output identical on the calibration batch; at τ > 0 the reconstruction is approximate. Empirically, the paper reports that this post-pruning step raises FLOPs and parameter reduction on top of APIB and NORTON while the reported top-1 accuracy stays the same across VGG-16, ResNet-56, ResNet-110, and ResNet-50.

Load-bearing premise

The load-bearing premise is that linear dependencies measured on one 256-image batch persist on unseen test data, so the least-squares recovery matrix computed on that batch keeps later layers' outputs nearly unchanged after many pruning steps, and that residual and skip connections in ResNet are reconciled even though the paper's equations do not state how.

Editorial extensions

If this is right

  • LinDeps can be stacked on top of any current or future pruning method, so its compression gain is additive rather than an alternative to importance-based pruning.
  • With τ = 0 (implemented as τ = 10^{-6}) the layer output is preserved exactly on the calibration batch, making the method usable where retraining is impossible, such as test-time adaptation or on-device updates.
  • On VGG-16/CIFAR-10, NORTON+LinDeps reaches 96.12% FLOPs reduction and 98.66% parameter reduction while keeping 92.54% top-1 accuracy, exceeding NORTON alone at 95.58% FLOPs.
  • The gains transfer to ResNet-56, ResNet-110, and ResNet-50, but shrink on ImageNet, which the authors attribute to noisy feature maps in a lower-accuracy model.
  • On low-resource devices, LinDeps itself runs quickly (13 s on an RTX 3070, 247 s on a Raspberry Pi) and shortens inference by 21-33% after NORTON pruning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test the paper does not run is seed-dependence: applying LinDeps with different 256-image calibration batches and comparing pruned-channel sets and test accuracy would reveal how much the method relies on the batch being representative.
  • Because the recovery step is a pure least-squares linear map, the same compensation could in principle be applied to other linear layers such as attention projections in transformers, though the paper only mentions transformer pruning as future work.
  • The held-out reconstruction residual ||L A'_test − A_test||_2 relative to the calibration residual could serve as a per-layer confidence score, letting practitioners skip layers where the linear model is unstable instead of using one global threshold τ.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces LinDeps, a post-pruning method that operates on top of an existing structured pruning technique. For each layer, it flattens a batch of feature maps into a matrix A, applies pivoted QR decomposition to A^T, identifies (near-)linearly dependent channels via a threshold tau, prunes those channels, and computes a least-squares recovery matrix L such that L A' approximately reconstructs A. The next layer's kernels are then reshaped through L so that the layer can consume the pruned representation without fine-tuning. Experiments combine LinDeps with APIB and NORTON on VGG-16, ResNet-56, ResNet-110, and ResNet-50 on CIFAR-10 and ImageNet, reporting additional FLOPs/parameter reductions at essentially unchanged accuracy, plus a low-resource setting without retraining where larger pruning gains are obtained with small accuracy losses. The central claim is that LinDeps provides a fine-tuning-free, performance-preserving way to remove layer-wise linear dependencies.

Significance. If the central claim is fully supported, LinDeps would be a useful and simple add-on: the per-layer derivation is clear, the method is generic across pruning techniques, and the authors make an honest effort to report multi-seed averages for the APIB combination and to show standalone limitations. Credit is due for presenting a concrete, falsifiable experimental protocol and for separating the performance-preserving regime from the lossy low-resource regime. However, the incremental gains in the main performance-preserving tables are small (mostly 0.16--2.82% additional FLOPs reduction), and the strongest headline gain (+30% pruning in the low-resource setup) comes with accuracy loss rather than with the claimed guarantee. The principal gap is that the theoretical derivation in Section 3 covers only feed-forward layer chains, while the experiments include ResNet architectures whose residual additions are not handled in the equations. Given that gap, the 'guaranteed performance preservation' claim is not yet substantiated for a substantial part of the experimental evaluation.

major comments (3)
  1. [Sec. 3, Eq. (3)--(4); Tables 2--4] The recovery derivation assumes that the pruned layer's output is consumed only by a parameterized next convolution, but this is not true for the residual blocks used in ResNet-56/110/50. If a filter of the second convolution of a block is pruned, the main branch produces C' channels while the shortcut still produces C channels, making the elementwise addition dimensionally invalid. If LinDeps is restricted to pruning only the first convolution of each block, or to layers whose outputs are not added to a shortcut, that restriction is never stated in Section 3 or in the implementation details. The equations therefore do not substantiate the 'guaranteed performance preservation' claim for the ResNet results reported in Tables 2--4.
  2. [Sec. 3, Eq. (4); Implementation details] The title and abstract promise a guarantee, but the recovery matrix L is obtained by a least-squares solve on a single batch of 256 images and the practical pruning threshold is tau = 1e-6, so the identity L A' ≈ A is approximate even on the calibration batch and unverified on test data. The paper's 'lossless pruning' terminology should be weakened to something like 'approximately output-preserving on the calibration batch,' and the performance-preservation claim should be presented as an empirical property unless a stability argument is added.
  3. [Sec. 3, Step 2; Eq. (1)--(2)] Equation (1) writes an exact zero block in R, but with numerical feature maps the factorization has small nonzero entries everywhere; the thresholding in Eq. (2) then changes the effective rank, and the subsequent description of selecting C' retained rows is not fully precise. Please state explicitly that Eq. (1) is the exact-dependency idealization, and define C' as the number of diagonal entries of R11 that survive the threshold, along with the exact rule for mapping the permutation P back to channel indices to form A'.
minor comments (4)
  1. [Sec. 4.2, 'LinDeps as a standalone pruning technique'] The text says 'for tau = 0' but the implementation section defines lossless pruning as tau = 1e-6; please reconcile these two statements so the reader knows which threshold produced the reported 24.9% pruning result.
  2. [Sec. 4.3, Fig. 4 and Table 5] The low-resource gains are reported as single numbers without variability or confidence intervals; adding multiple runs or error bars would strengthen the claim, especially because the gains reach 30% and the accuracy losses are up to 0.3%.
  3. [Reproducibility] The manuscript cites the public code of APIB and NORTON but does not provide a link to the authors' own implementation or checkpoints of LinDeps; please add a code release or a detailed pseudocode listing.
  4. [Fig. 3] The legend in Fig. 3 is dense and hard to read because many methods overlap; consider splitting the benchmark into separate panels or using a table-style legend.

Circularity Check

1 steps flagged · score 2.0 of 10

Recovery matrix is least-squares fit on the same batch that defines the pruning, making the batch-level preservation identity definitional; test-set accuracy claims remain independent, so circularity is minor.

  1. fitted input called prediction [Section 3, Step 3 'Signal recovery by weight adjustment' (Eqs. 3-4); Section 3 'Implementation details' ('lossless pruning')]
    "we compute a recovery matrix L ∈ R^{C×C′} such that L A′ ≈ A ... L = argmin_{eL} ∥eL A′ − A∥2 ... Thus, the new kernel can ingest C′ channels instead of C, becoming compatible with the pruned layer i, and produces equivalent outputs under the no-approximation (τ = 0) scenario."

    A′ is formed by selecting rows of the same batch-aggregated feature matrix A that is the target of the least-squares fit, so L A′ ≈ A holds on the calibration batch by construction; the claimed 'no performance cost' guarantee at τ=0 is therefore the definition of the fit rather than an independent prediction. The central reported results are held-out test accuracies that are not forced by this identity, so this is a minor, localized circularity rather than one that makes the whole evaluation vacuous.

full rationale

The core pruning decision (PQR-based detection of linearly dependent feature maps) is adopted from LDFM [34] and is explicitly acknowledged; the new contribution is the recovery mechanism. The recovery matrix L is fit to the same batch used to define A and A′, so the statement that the next layer's output is preserved on that batch is true by construction. However, the paper's headline performance claims are evaluated on held-out CIFAR-10 and ImageNet test sets, which are not fitted by the least-squares problem, so the main empirical findings are not circular. The self-citations present in the paper, such as [36] for the choice of top-1 accuracy as a ranking score and [6] for test-time adaptation, are auxiliary and not load-bearing for the pruning derivation. No uniqueness theorem or ansatz is smuggled in via self-citation, and the method is not a renaming of a known result. The only circular element is the batch-level preservation identity, which is inherent to the least-squares formulation. Given that the actual benchmarks are external and independent, the appropriate circularity score is low.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on one fitted threshold, one batch size, and per-layer recovery matrices, plus assumptions about distribution shift, linearity, batch norm, and residual connections. No new physical or architectural entities are introduced.

free parameters (3)
  • pruning threshold tau = 1e-6 for lossless, 0.1 for low-resource
    Controls how aggressively near-dependent channels are pruned; chosen per experiment, not derived.
  • batch size B = 256
    Used for feature map aggregation; must satisfy B*H*W > C, a condition that shapes the approximation.
  • recovery matrix L (per layer) = least-squares solution of L A' = A
    A fitted linear map that determines how well pruned channels are reconstructed; the output-preservation guarantee depends on this fit.
assumptions (4)
  • domain assumption Linear dependencies in feature maps computed on a batch of 256 training images generalize to the test distribution.
    The pruning decision and recovery matrix are computed from a finite batch; the guarantee relies on these dependencies holding for unseen data. See Section 3, Step 2.
  • standard math The network's subsequent layer is treated as linear in its input, so multiplying its kernels by a recovery matrix preserves its output when the input is reconstructed.
    Convolution is linear, and the activation function between the pruned layer and the next layer's kernel application is not present because the recovery targets the pre-activation input. See Section 3, Step 3.
  • domain assumption Batch normalization parameters can be removed with the pruned channels without recomputing running statistics.
    The paper states 'we prune the corresponding BN parameters together with the channel' but does not describe recalibration of running mean or variance.
  • domain assumption Residual connections in ResNet can be reconciled with pruned channel counts using an undocumented procedure.
    The methodology does not describe how skip connections are handled; the ResNet-56/110 results imply a necessary adjustment that is not present in the equations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservation." pith.science (2026). https://pith.science/paper/3F76BY3Y

@misc{pith2026250721573,
  author       = {Pith},
  title        = {Pith review of: LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3F76BY3Y}},
  note         = {Machine review of arXiv:2507.21573}
}
read the original abstract

Convolutional Neural Networks (CNN) are widely used in many computer vision tasks. Yet, their increasing size and complexity pose significant challenges for efficient deployment on resource-constrained platforms. Hence, network pruning has emerged as an effective way of reducing the size and computational requirements of neural networks by removing redundant or unimportant parameters. However, a fundamental challenge with pruning consists in optimally removing redundancies without degrading performance. Most existing pruning techniques overlook structural dependencies across feature maps within a layer, resulting in suboptimal pruning decisions. In this work, we introduce LinDeps, a novel post-pruning method, i.e., a pruning method that can be applied on top of any pruning technique, which systematically identifies and removes redundant filters via linear dependency analysis. Particularly, LinDeps applies pivoted QR decomposition to feature maps to detect and prune linearly dependent filters. Then, a novel signal recovery mechanism adjusts the next layer's kernels to preserve compatibility and performance without requiring any fine-tuning. Our experiments on CIFAR-10 and ImageNet with VGG and ResNet backbones demonstrate that LinDeps improves compression rates of existing pruning techniques while preserving performances, leading to a new state of the art in CNN pruning. We also benchmark LinDeps in low-resource setups where no retraining can be performed, which shows significant pruning improvements and inference speedups over a state-of-the-art method. LinDeps therefore constitutes an essential add-on for any current or future pruning technique.

Figures

Figures reproduced from arXiv: 2507.21573 by the authors.

Figure 1
Figure 1. Overview of our LinDeps post-pruning method. We propose LinDeps, a post-pruning method applicable on top of any pruning method. LinDeps identifies and removes linear depen￾dencies between input neurons xi and output neurons y of a layer using a pivoted QR decomposition, and provides a signal recovery mechanism to preserve the network’s performance without fine￾tuning leading to compression improvements with guarante… view at source ↗
Figure 2
Figure 2. Pipeline of LinDeps applied to a single layer. Our method proceeds in three steps. [Step 1] Aggregation: (a) A batch of images of size B is passed through (b) the network up to the layer to be pruned. (c) This produces C feature maps per image, which are (d) flattened and concatenated into a matrix A ∈ R C×(B·H·W) , where each row corresponds to one channel, aggregated over all spatial locations and all images in th… view at source ↗
Figure 3
Figure 3. Benchmark of pruning techniques with VGG-16 trained on CIFAR-10. LinDeps, applied on top of NORTON, achieves a new state of the art, surpassing all previously published pruning techniques by a significant margin in terms of compres￾sion level while maintaining the top-1 accuracy. mance, outperforming previously published techniques. ResNet-56/110 on CIFAR-10. Tables 2 and 3 respec￾tively, show the results of LinDeps… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: LinDeps pruning improvement without retraining. In low-resource environments where post-pruning retraining can￾not be afforded, LinDeps gives a significant pruning boost for a comparatively negligible decrease in performance. Device NORTON time (s) LinDeps time (s) Inf…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 41 canonical work pages

  1. [1]

    DECORE: Deep compression with reinforcement learning

    Manoj Alwani, Yang Wang, and Vashisht Madhavan. DECORE: Deep compression with reinforcement learning. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 12339–12349, New Orleans, LA, USA, 2022. 7

    work page 2022

  2. [2]

    Automatic Neural Network Pruning that Efficiently Preserves the Model Accuracy

    Thibault Castells and Seul-Ki Yeom. Automatic neural net- work pruning that efficiently preserves the model accuracy. arXiv, abs/2111.09635, 2021. 7

    work page Pith review arXiv 2021

  3. [3]

    RGP: Neural network pruning through regular graph with edges swapping

    Zhuangzhi Chen, Jingyang Xiang, Yao Lu, Qi Xuan, Zhen Wang, Guanrong Chen, and Xiaoniu Yang. RGP: Neural network pruning through regular graph with edges swapping. IEEE Trans. Neural Networks Learn. Syst. , 35(10):14671– 14683, 2024. 7

    work page 2024

  4. [4]

    A survey on deep neural network pruning: Taxonomy, compar- ison, analysis, and recommendations

    Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. A survey on deep neural network pruning: Taxonomy, compar- ison, analysis, and recommendations. IEEE Trans. Pattern Anal. Mach. Intell., 46(12):10558–10578, 2024. 1, 2

    work page 2024

  5. [5]

    Krishna Teja Chitty-Venkata, Sparsh Mittal, Murali Emani, Venkatram Vishwanath, and Arun K. Somani. A survey of techniques for optimizing transformer inference. J. Syst. Arch., 144, 2023. 3

    work page 2023

  6. [6]

    ARTHuS: Adaptive real-time human segmentation in sports through online distillation

    Anthony Cioppa, Adrien Deliege, Maxime Istasse, Christophe De Vleeschouwer, and Marc Van Droogen- broeck. ARTHuS: Adaptive real-time human segmentation in sports through online distillation. In IEEE Int. Conf. Comput. Vis. Pattern Recognit. Work. (CVPRW), CVsports , pages 2505–2514, Long Beach, CA, USA, 2019. 8

    work page 2019

  7. [7]

    ImageNet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 248–255, Miami, FL, USA, 2009. 5

    work page 2009

  8. [8]

    Network pruning via feature shift mini- mization

    Yuanzhi Duan, Yue Zhou, Peng He, Qiang Liu, Shukai Duan, and Xiaofang Hu. Network pruning via feature shift mini- mization. In Asian Conf. Comput. Vis. (ACCV), pages 618–

    work page

Show all 45 references
  1. [9]

    A differ- entiable framework for end-to-end learning of hybrid struc- tured compression

    Moonjung Eo, Suhyun Kang, and Wonjong Rhee. A differ- entiable framework for end-to-end learning of hybrid struc- tured compression. arXiv, abs/2309.13077, 2023. 7

    2023 arXiv

  2. [10]

    The lottery ticket hy- pothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hy- pothesis: Finding sparse, trainable neural networks. In Int. Conf. Learn. Represent. (ICLR) , pages 1–42, New Orleans, LA, USA, 2019. 2

    2019

  3. [11]

    Jointly training and pruning CNNs via learnable agent guid- ance and alignment

    Alireza Ganjdanesh, Shangqian Gao, and Heng Huang. Jointly training and pruning CNNs via learnable agent guid- ance and alignment. In IEEE/CVF Conf. Comput. Vis. Pat- tern Recognit. (CVPR) , pages 16058–16069, Seattle, W A, USA, 2024. 2

    2024

  4. [12]

    Gene H. Golub. Numerical methods for solving linear least squares problems. Numer. Math., 7(3):206–216, 1965. 3

    1965

  5. [13]

    Automatic network pruning via Hilbert-Schmidt indepen- dence criterion lasso under information bottleneck principle

    Song Guo, Lei Zhang, Xiawu Zheng, Yan Wang, Yuchao Li, Fei Chao, Chenglin Wu, Shengchuan Zhang, and Rongrong Ji. Automatic network pruning via Hilbert-Schmidt indepen- dence criterion lasso under information bottleneck principle. In IEEE/CVF Int. Conf. Comput. Vis. (ICCV), pag...

    2023

  6. [14]

    Dynamic net- work surgery for efficient DNNs

    Yiwen Guo, Anbang Yao, and Yurong Chen. Dynamic net- work surgery for efficient DNNs. InAdv. Neural Inf. Process. Syst. (NeurIPS), pages 1387–1395, Barcelona, Spain, 2016. 2

    2016

  7. [15]

    Song Han, Jeff Pool, John Tran, and William J. Dally. Learn- ing both weights and connections for efficient neural net- works. In Adv. Neural Inf. Process. Syst. (NeurIPS) , pages 1135–1143, Montr´eal, Can., 2015. 2

    2015

  8. [16]

    Stephen Jos ´e Hanson and Lorien Y . Pratt. Comparing biases for minimal network construction with back-propagation. In Adv. Neural Inf. Process. Syst. (NeurIPS) , pages 177–185, Denver, CO, USA, 1988. 2

    1988

  9. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Int. Conf. Comput. Vis. Pattern Recognit. (CVPR) , pages 770– 778, Las Vegas, NV , USA, 2016. 5

    2016

  10. [18]

    Structured pruning for deep con- volutional neural networks: A survey

    Yang He and Lingao Xiao. Structured pruning for deep con- volutional neural networks: A survey. IEEE Trans. Pattern Anal. Mach. Intell., 46(5):2900–2919, 2024. 2

    2024

  11. [19]

    Soft filter pruning for accelerating deep convolutional neural networks

    Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang. Soft filter pruning for accelerating deep convolutional neural networks. In Int. Jt. Conf. Artif. Intell. (IJCAI), pages 2234–2240, Stockholm, Sweden, 2018. 2

    2018

  12. [20]

    Filter pruning via geometric median for deep convolutional neural networks acceleration

    Yang He, Ping Liu, Ziwei Wang, Zhilan Hu, and Yi Yang. Filter pruning via geometric median for deep convolutional neural networks acceleration. In IEEE/CVF Conf. Com- put. Vis. Pattern Recognit. (CVPR), pages 4335–4344, Long Beach, CA, USA, 2019. 2

    2019

  13. [21]

    Network trimming: A data-driven neuron pruning approach towards efficient deep architectures

    Hengyuan Hu, Rui Peng, Yu-Wing Tai, and Chi-Keung Tang. Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv, abs/1607.03250,

    arXiv

  14. [22]

    Accelerating transformer pre-training with 2:4 sparsity

    Yuezhou Hu, Kang Zhao, Weiyu Huang, Jianfei Chen, and Jun Zhu. Accelerating transformer pre-training with 2:4 sparsity. In Int. Conf. Mach. Learn. (ICML) , pages 19531– 19543, 2024. 2

    2024

  15. [23]

    A survey of FPGA and ASIC designs for transformer inference acceleration and optimization

    Beom Jin Kang, Hae In Lee, Seok Kyu Yoon, Young Chan Kim, Sang Beom Jeong, Seong Jun O, and Hyun Kim. A survey of FPGA and ASIC designs for transformer inference acceleration and optimization. J. Syst. Arch., 155, 2024. 3

    2024

  16. [24]

    Learning multiple layers of features from tiny images, 2009

    Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009. Technical report, University of Toronto. 5

    2009

  17. [25]

    Denker, and Sara A

    Yann LeCun, John S. Denker, and Sara A. Solla. Optimal brain damage. In Adv. Neural Inf. Process. Syst. (NeurIPS), pages 598–605, Denver, CO, USA, 1989. 2

    1989

  18. [26]

    Pruning filters for efficient ConvNets

    Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient ConvNets. In Int. Conf. Learn. Represent. (ICLR), pages 1–13, 2017. 2

    2017

  19. [27]

    Pruning filters for efficient ConvNets

    Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient ConvNets. In 9 Int. Conf. Learn. Represent. (ICLR) , pages 1–13, Toulon, France, 2017. 2

    2017

  20. [28]

    Pruning and quantization for deep neural network acceleration: A survey

    Tailin Liang, John Glossner, Lei Wang, Shaobo Shi, and Xi- aotong Zhang. Pruning and quantization for deep neural network acceleration: A survey. Neurocomputing, 461:370– 403, 2021. 2

    2021

  21. [29]

    HRank: Filter pruning using high-rank feature map

    Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, and Ling Shao. HRank: Filter pruning using high-rank feature map. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 1526– 1535, Seattle, W A, USA, 2020. 2, 7

    2020

  22. [30]

    Network pruning us- ing adaptive exemplar filters

    Mingbao Lin, Rongrong Ji, Shaojie Li, Yan Wang, Yongjian Wu, Feiyue Huang, and Qixiang Ye. Network pruning us- ing adaptive exemplar filters. IEEE Trans. Neural Networks Learn. Syst., 33(12):7357–7366, 2022. 7

    2022

  23. [31]

    EZCrop: Energy-zoned channels for robust out- put pruning

    Rui Lin, Jie Ran, Dongpeng Wang, King Hung Chiu, and Ngai Wong. EZCrop: Energy-zoned channels for robust out- put pruning. In IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), pages 3595–3604, Waikoloa, HI, USA, 2022. 7

    2022

  24. [32]

    Learning efficient convolutional networks through network slimming

    Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. Learning efficient convolutional networks through network slimming. In IEEE Int. Conf. Comput. Vis. (ICCV) , pages 2755–2763, Venice, Italy, 2017. 2

    2017

  25. [33]

    SGDR: Stochastic gradi- ent descent with warm restarts

    Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradi- ent descent with warm restarts. In Int. Conf. Learn. Repre- sent. (ICLR), pages 1–16, Toulon, France, 2017. 6

    2017

  26. [34]

    Network pruning using linear dependency analysis on feature maps

    Hao Pan, Zhongdi Chao, Jiang Qian, Bojin Zhuang, Shao- jun Wang, and Jing Xiao. Network pruning using linear dependency analysis on feature maps. In IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), pages 1720–1724, Toronto, Ontario, Can., 2021. 2, 3

    2021

  27. [35]

    Enhanced network compression through tensor decomposi- tions and pruning

    Van Tien Pham, Yassine Zniyed, and Thanh Phuong Nguyen. Enhanced network compression through tensor decomposi- tions and pruning. IEEE Trans. Neural Networks Learn. Syst., 36(3):4358–4370, 2025. 2, 3, 5, 6, 7

    2025

  28. [36]

    Foundations of the theory of performance-based ranking

    S ´ebastien Pi ´erard, Ana ¨ıs Halin, Anthony Cioppa, Adrien Deli`ege, and Marc Van Droogenbroeck. Foundations of the theory of performance-based ranking. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 14293– 14302, Nashville, TN, USA, 2025. 5

    2025

  29. [37]

    Very deep convo- lutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv, abs/1409.1556, 2014. 5

    2014 arXiv

  30. [38]

    ptflops: a flops counting tool for neu- ral networks in pytorch framework

    Vladislav Sovrasov. ptflops: a flops counting tool for neu- ral networks in pytorch framework. https://github. com/sovrasov/flops-counter.pytorch, 2024. 5

    2024

  31. [39]

    CHIP: channel independence-based pruning for compact neural networks

    Yang Sui, Miao Yin, Yi Xie, Huy Phan, Saman Zonouz, and Bo Yuan. CHIP: channel independence-based pruning for compact neural networks. In Adv. Neural Inf. Process. Syst. (NeurIPS), pages 24604–24616, 2021. 7

    2021

  32. [40]

    A survey on transformer compression

    Yehui Tang, Yunhe Wang, Jianyuan Guo, Zhijun Tu, Kai Han, Hailin Hu, and Dacheng Tao. A survey on transformer compression. arXiv, abs/2402.05964, 2024. 3

    2024 arXiv

  33. [41]

    Deep learning and the information bottleneck principle

    Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In IEEE Inf. Theory Work. (ITW), pages 1–5, Jerusalem, Israel, 2015. 3

    2015

  34. [42]

    Single shot structured pruning before training

    Joost van Amersfoort, Milad Alizadeh, Sebastian Farquhar, Nicholas Lane, and Yarin Gal. Single shot structured pruning before training. arXiv, abs/2007.00389, 2020. 2

    2007 arXiv

  35. [43]

    HALOC: Hardware-aware automatic low-rank compression for com- pact neural networks

    Jinqi Xiao, Chengming Zhang, Yu Gong, Miao Yin, Yang Sui, Lizhi Xiang, Dingwen Tao, and Bo Yuan. HALOC: Hardware-aware automatic low-rank compression for com- pact neural networks. In AAAI Conf. Artif. Intell. , pages 10464–10472, Washington DC, USA, 2023. 7

    2023

  36. [44]

    Toward compact deep neural networks via energy-aware pruning

    Seul-Ki Yeom, Kyung-Hwan Shim, and Jee-Hyun Hwang. Toward compact deep neural networks via energy-aware pruning. arXiv, abs/2103.10858, 2021. 7

    2021 arXiv

  37. [45]

    Yuyao Zhang and Nikolaos M. Freris. Adaptive filter prun- ing via sensitivity feedback. IEEE Trans. Neural Networks Learn. Syst., 35(8):10996–11008, 2024. 7 10

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.