Pith. sign in

REVIEW 4 major objections 6 minor 69 references

Layer Pruning with Consensus: A Triple-Win Solution

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Combining several similarity metrics into one pruning score identifies layers whose removal least hurts the model, keeping accuracy high and improving adversarial robustness.

desk verdict A plausible new criterion for layer pruning, but the triple-win claim rests on uncontrolled baselines and single runs. read the letter →

arxiv 2411.14345 v1 pith:6D2MWAWN submitted 2024-11-21 cs.LG cs.CV

Leandro Giusti Mugnaini , Carolina Tavares Duarte , Anna H. Reali Costa , Artur Jordao This is my paper · ORCID
classification cs.LGcs.CV
keywords layerpruningsimilaritymetricsConsensuscriterionadversarialrobustnessFLOPsreductionrepresentationout-of-distributiongeneralizationstructured
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Layer pruning removes whole layers from a trained network to cut compute, but existing criteria rank layers with a single similarity metric and can pick layers that hurt accuracy or generalization. This paper argues that combining several similarity metrics into one ranking, called the Consensus criterion, finds layers whose removal least disturbs the network's final feature representation. The claim is a triple win: pruned models stay accurate, run with up to 78.8% fewer FLOPs, and hold up better under adversarial attacks and out-of-distribution shifts than models pruned with a single metric such as CKA (centered kernel alignment). If true, this makes pruning a more trustworthy compression tool and a cheap robustness defense, since no adversarial training is required.

What carries the argument

The Consensus criterion is the central object: for each layer, compute several similarity scores between the pruned and unpruned networks' final representations, convert each metric's scores into a rank ordering of layers, sum the ranks across metrics, and remove the layer ranked most similar overall. The similarity metrics come from a metric space for stochastic neural network representations, including Procrustes and Bures distances and interpolated variants, so the comparison is invariant to rotations and accounts for noise structure in neural responses. The load-bearing operation is the rank-sum: it turns incomparable score magnitudes from different metrics into a single ordering, and it operationalizes the paper's thesis that no single metric should decide layer importance.

What would settle it

Compare the consensus ranking against exhaustive leave-one-layer-out accuracy after fine-tuning across several random seeds: if the layer with the smallest summed rank is not among the layers whose removal costs the least accuracy, then the representation-distance proxy is not tracking true layer importance.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a pruning criterion formed by summing rank positions across multiple representation-similarity metrics identifies low-importance layers more reliably than any single metric. Concretely, for each candidate layer the method removes it, extracts the feature map at the layer before classification, measures how similar that representation is to the unpruned network's under several metrics, ranks layers per metric, and prunes the layer with the smallest summed rank. Across ResNet models on CIFAR-10 and ImageNet, Consensus matches or beats state-of-the-art pruning methods in accuracy drop at equal FLOP reduction, reaches 78.80% FLOP reduction with minimal accuracy loss, and improves accuracy under FGSM, CIFAR-C, CIFAR-10.2, and ImageNet-C by up to roughly 4 percentage points over the CKA criterion. The authors interpret the gains as avoiding shortcut learning: a single metric can overfit to clean-image statistics, while a consensus of metrics selects layers that preserve features useful under distribution shift.

Load-bearing premise

The ranking is only meaningful if how much removing a layer changes the network's final feature representation faithfully measures how important that layer is, and if summing rank positions across metrics preserves that signal.

Editorial extensions

If this is right

  • Pruned models can reach 78.80% FLOP reduction on ResNet56/CIFAR-10 while keeping accuracy nearly unchanged, with gains that are often larger than single-criterion baselines at the same compression level.
  • Adversarial robustness improves without adversarial training: consensus-pruned models beat CKA-pruned ones by up to roughly 4 percentage points on FGSM and 1.2 percentage points on CIFAR-10.2 at matched compression.
  • The same criterion transfers to shallow ResNets (ResNet32/44) and to a Transformer on tabular human-activity data, cutting FLOPs up to about 80% with small accuracy change.
  • Reported energy and carbon savings follow mechanically from the FLOP reduction: the paper estimates 66.99% lower energy and 68.75% lower carbon emissions on ResNet56.
  • Because the method needs no architectural search or specialized sparse hardware, it offers a drop-in layer-pruning criterion for standard residual networks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the rank-sum interpretation is the active ingredient, the same consensus procedure could be applied to filter pruning or to LLM layer dropping, where single-criterion rankings are known to be unstable; the paper notes the LLM extension but does not test it.
  • A testable extension is to use the dispersion of rank positions across metrics as a confidence measure: layers on which metrics agree may be safer to prune than layers on which they disagree, something the paper does not examine.
  • The robustness gains suggest that representation-preservation pruning could serve as a zero-cost complement to adversarial training; combining Consensus with adversarial training is a natural next experiment the paper does not run.
  • The method's dependence on a single reference representation from the unpruned network means its ranking may drift as training data changes; re-running selection on different data subsets would test how stable the pruned layer choice is.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Consensus, a layer-pruning criterion that aggregates multiple representation-similarity metrics by summing per-metric layer rankings, and claims a 'triple win': low accuracy drop, high FLOP/latency reduction, and improved robustness to adversarial attacks and OOD shifts. The method is evaluated on CIFAR-10 and ImageNet with ResNet architectures, on robustness benchmarks (FGSM, CIFAR-10.2, CIFAR-C, ImageNet-C), on shallow ResNets, on Transformers for wearable-sensor HAR, and with GreenAI cost estimates. The authors report state-of-the-art or on-par accuracy at high FLOP reductions and robustness gains over the CKA baseline, and they attribute the gains to avoiding shortcut learning.

Significance. If the central claim holds, the paper makes a useful practical contribution: a simple, code-available pruning criterion that improves robustness and compression simultaneously, without adversarial training. The evaluation breadth is commendable, and the method's rank-aggregation formulation is easy to reproduce. However, the evidence for the aggregation benefit and for the robustness advantage over single-metric baselines is not yet statistically established, because the controlled comparisons lack multiple seeds and the SOTA comparisons use numbers from heterogeneous original papers.

major comments (4)
  1. [§4, 'Comparison with the State of the Art' (Tables 1-4)] The paper explicitly states 'For a fair comparison, we report the results of each method according to the original paper.' This means the baseline numbers in Tables 1-4 come from different training and fine-tuning pipelines (different epochs, learning rates, augmentation, and hardware), so the comparisons are not controlled. The only apparently shared-protocol comparison is Figure 2, which shows single trajectories without seeds or confidence intervals. The headline robustness margins of 4.14 and 4.29 pp on FGSM at 41.28% and 56.29% FLOP reduction are within the typical run-to-run variance of fine-tuned CIFAR ResNets, so they do not yet demonstrate that Consensus is more robust than CKA. The authors should run a same-protocol, multi-seed comparison against CKA (and at least one other SOTA baseline) and report means and standard deviations or confidence intervals.
  2. [§4, 'Effectiveness of the Proposed Consensus Criterion'] The single-pruning-iteration experiment on ResNet32 reports mean accuracy across benchmarks and a maximum margin of 0.60 pp over the best individual metric, with no variance or per-benchmark breakdown. This is too weak to support the central claim that combining multiple similarity metrics is reliably better than its components. The authors should provide multi-iteration pruning trajectories for each constituent metric and for Consensus, with multiple seeds, and report the distribution of accuracy deltas on each robustness benchmark.
  3. [§4-§5, shortcut learning claims] The paper repeatedly attributes Consensus's robustness gains to avoiding 'shortcut learning' (e.g., Section 4: 'our results suggest that our criterion yields pruned models that avoid shortcut opportunities'; Section 5: 'it avoids shortcut learning'), but no direct evidence for shortcut avoidance is provided. The manuscript does not test for reliance on spurious correlations, background or texture shortcuts, or any shortcut-detection metric. Since this is a load-bearing part of the 'trustworthy pruned models' claim, the authors should either add a concrete shortcut-detection experiment (e.g., evaluating on a biased variant of the dataset or using feature attribution) or soften the claim to a hypothesis.
  4. [§3, Algorithm 1, and §4, ablation of metric set] The Consensus score is a simple sum of rankings over the chosen metric set S, and S is a free design choice (the paper does not study how varying the composition of S affects pruning outcomes). The claim that consensus is beneficial depends on which metrics are included and on their interpolation parameters. The paper should include an ablation that varies the metric set (e.g., each individual metric, pairs, and the full set) and shows the effect on the pruning trajectory, to establish that the aggregation itself, rather than any single component metric, drives the reported gains.
minor comments (6)
  1. [Figure 1 caption] The left panel's label 'FSGM' should be 'FGSM'.
  2. [Figure 2 caption] 'botton-right' should be 'bottom-right'.
  3. [Section 1 and Section 3] Typos: 'traning' should be 'training'; 'mantaining' should be 'maintaining'.
  4. [Table 1] The row labeled 'Consensus20 (Ours)' should probably be 'Consensus (Ours)' to match the other rows; please check this row and the corresponding run.
  5. [§4, 'GreenAI and Computational Costs'] The carbon emission and cost reduction figures (68.75% and 66.99%) are reported with a footnote referring to an impact calculator, but the hardware, runtime, power draw, and training time assumptions are not given. Please state these assumptions for reproducibility.
  6. [§4, 'Effectiveness in Transformer Architectures'] The Transformer experiments show accuracy improvements across datasets, but there is no comparison to any prior layer-pruning method on these HAR datasets. The claim that the results 'confirm its effectiveness' would be stronger with a CKA baseline or a standard filter-pruning baseline.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Consensus criterion is an aggregation heuristic evaluated on external benchmarks, with no fitted parameter that reproduces the reported gains.

full rationale

The paper's derivation chain does not reduce to its inputs. The Consensus criterion is defined as a rank-sum over a fixed set of similarity metrics taken from Duong et al. [10], where each metric compares the final-layer representation of a layer-removed network with that of the unpruned network (Algorithm 1, Section 3). Nothing in this definition encodes the target outcomes of low accuracy drop, FLOPs reduction, or adversarial robustness; the reported triple-win results are empirical measurements obtained after pruning and fine-tuning, not consequences enforced by construction. No parameter is fitted to the evaluation benchmarks and then renamed as a prediction. The only notable self-citations are to the authors' own prior CKA layer-pruning work [47] and to their earlier pruning-robustness study [29], but these are used as baselines and motivation rather than as load-bearing justification for the central claim. No uniqueness theorem is imported from the authors, and no ansatz is smuggled in via citation: the shape metrics are externally defined by Duong et al. [10]. Concerns that baselines are reported from their original papers and that Figure 2 shows single unseeded trajectories are legitimate experimental-rigor issues, but they are not circularity. The central claim is therefore self-contained against the external benchmarks, and the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that representation similarity is a valid proxy for layer importance, on the residual-network redundancy principle, and on the validity of the adopted shape metrics. No numerical constants are fitted to the benchmark results.

free parameters (3)
  • Similarity metric set S = not reported
    The choice and number of metrics composing the Consensus criterion are selected by the authors and not justified or ablated; the method's performance depends on this choice.
  • Shape metric interpolation parameter = not reported
    Duong et al. metrics include interpolated versions balancing mean and covariance differences; the paper does not report the interpolation weights used.
  • Representation sample X = not reported
    M(F_l^s, X) uses a set of training samples; the size and composition are not specified, which affects the similarity estimates.
assumptions (4)
  • domain assumption Representational similarity of the penultimate layer is a valid proxy for layer importance.
    Used to score every candidate layer in Section 3; inherited from Pons et al. [47] without independent validation.
  • domain assumption Residual networks contain redundant layers, so removing one layer and fine-tuning can preserve accuracy.
    Stated in Section 3 as principles (I) and (II); load-bearing for the pruning paradigm.
  • standard math The shape metrics of Duong et al. are valid similarity measures for stochastic neural representations.
    Adopted from [10]; the paper does not re-derive them.
  • domain assumption Greedy iterative pruning with fine-tuning after each removal yields near-optimal layer subsets.
    Algorithm 1 removes one layer at a time; no comparison to joint or global layer selection is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Layer Pruning with Consensus: A Triple-Win Solution." pith.science (2026). https://pith.science/paper/6D2MWAWN

@misc{pith2026241114345,
  author       = {Pith},
  title        = {Pith review of: Layer Pruning with Consensus: A Triple-Win Solution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6D2MWAWN}},
  note         = {Machine review of arXiv:2411.14345}
}
read the original abstract

Layer pruning offers a promising alternative to standard structured pruning, effectively reducing computational costs, latency, and memory footprint. While notable layer-pruning approaches aim to detect unimportant layers for removal, they often rely on single criteria that may not fully capture the complex, underlying properties of layers. We propose a novel approach that combines multiple similarity metrics into a single expressive measure of low-importance layers, called the Consensus criterion. Our technique delivers a triple-win solution: low accuracy drop, high-performance improvement, and increased robustness to adversarial attacks. With up to 78.80% FLOPs reduction and performance on par with state-of-the-art methods across different benchmarks, our approach reduces energy consumption and carbon emissions by up to 66.99% and 68.75%, respectively. Additionally, it avoids shortcut learning and improves robustness by up to 4 percentage points under various adversarial attacks. Overall, the Consensus criterion demonstrates its effectiveness in creating robust, efficient, and environmentally friendly pruned models.

Figures

Figures reproduced from arXiv: 2411.14345 by the authors.

Figure 1
Figure 1. Left: Comparison with state-of-the-art on the popular ResNet56 + CIFAR-10 setting. Our Consensus technique [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Trade-offs between predictive accuracy and com [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Performance of the Consensus criterion on Trans [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 69 canonical work pages

  1. [1]

    DECORE: deep compression with reinforce- ment learning

    Manoj Alwani, Yang Wang, and Vashisht Madha- van. DECORE: deep compression with reinforce- ment learning. In CVPR, 2022. 6

    work page 2022

  2. [2]

    Anna Bair, Hongxu Yin, Maying Shen, Pavlo Molchanov, and Jose M. Alvarez. Adaptive sharpness-aware pruning for robust sparse net- works. In ICLR, 2024. 1, 4

    work page 2024

  3. [3]

    Bartoldson, Ari S

    Brian R. Bartoldson, Ari S. Morcos, Adrian Barbu, and Gordon Erlebacher. The generalization-stability tradeoff in neural network pruning. In NeurIPS, 2020. 1, 4

    work page 2020

  4. [4]

    Shallowing deep networks: Layer-wise pruning based on feature representa- tions

    Shi Chen and Qi Zhao. Shallowing deep networks: Layer-wise pruning based on feature representa- tions. (IEEE) TPAMI, 2019. 1, 3, 7

    work page 2019

  5. [5]

    Dynamical channel pruning by conditional accuracy change for deep neural networks

    Zhiqiang Chen, Ting-Bing Xu, Changde Du, Cheng-Lin Liu, and Huiguang He. Dynamical channel pruning by conditional accuracy change for deep neural networks. (IEEE) TNNLS , 2021. 8

    work page 2021

  6. [6]

    A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommen- dations

    Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommen- dations. (IEEE) TPAMI, 2024. 1, 3, 4

    work page 2024

  7. [7]

    The efficiency mis- nomer

    Mostafa Dehghani, Yi Tay, Anurag Arnab, Lucas Beyer, and Ashish Vaswani. The efficiency mis- nomer. In ICLR, 2022. 1

    work page 2022

  8. [8]

    Attention is not all you need: pure attention loses rank doubly exponentially with depth

    Yihe Dong, Jean-Baptiste Cordonnier, and An- dreas Loukas. Attention is not all you need: pure attention loses rank doubly exponentially with depth. In ICML, 2021. 4

    work page 2021

Show all 69 references
  1. [9]

    Layer folding: Neu- ral network depth reduction using activation lin- earization

    Amir Ben Dror and et al. Layer folding: Neu- ral network depth reduction using activation lin- earization. In BMVC, 2022. 3

    2022

  2. [10]

    Duong and et al

    Lyndon R. Duong and et al. Representational dis- similarity metric spaces for stochastic neural net- works. In ICLR, 2023. 2, 4

    2023

  3. [11]

    Utku Evci, Vincent Dumoulin, Hugo Larochelle, and Michael C. Mozer. Head2toe: Utilizing inter- mediate representations for better transfer learn- ing. In ICML, 2022. 4

    2022

  4. [12]

    Llmcarbon: Modeling the end-to-end carbon foot- print of large language models

    Ahmad Faiz, Sotaro Kaneda, Ruhan Wang, Rita Osi, Parteek Sharma, Fan Chen, and Lei Jiang. Llmcarbon: Modeling the end-to-end carbon foot- print of large language models. ICLR, 2024. 8

    2024

  5. [13]

    Sparsegpt: Mas- sive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. Sparsegpt: Mas- sive language models can be accurately pruned in one-shot. In ICML, 2023. 3

    2023

  6. [14]

    Depthshrinker: A new com- pression paradigm towards boosting real-hardware efficiency of compact neural networks

    Yonggan Fu and et al. Depthshrinker: A new com- pression paradigm towards boosting real-hardware efficiency of compact neural networks. In ICML,

  7. [15]

    Jointly training and pruning cnns via learnable agent guidance and alignment

    Alireza Ganjdanesh, Shangqian Gao, and Heng Huang. Jointly training and pruning cnns via learnable agent guidance and alignment. CVPR,

  8. [16]

    Bilevelpruning: Unified dynamic and static channel pruning for convolutional neu- ral networks

    Shangqian Gao, Yanfu Zhang, Feihu Huang, and Heng Huang. Bilevelpruning: Unified dynamic and static channel pruning for convolutional neu- ral networks. CVPR, 2024. 6

    2024

  9. [17]

    Shortcut learning in deep neural networks

    Robert Geirhos and et al. Shortcut learning in deep neural networks. Nature Machine Intelli- gence, 2020. 2, 7

    2020

  10. [18]

    DAIS: automatic chan- nel pruning via differentiable annealing indicator search

    Yushuo Guan and et al. DAIS: automatic chan- nel pruning via differentiable annealing indicator search. (IEEE) TPAMI, 2023. 6, 8

    2023

  11. [19]

    Blending pruning criteria for convolutional neural networks

    Wei He and et al. Blending pruning criteria for convolutional neural networks. In ICANN, 2021. 4

    2021

  12. [20]

    Structured pruning for deep convolutional neural networks: A survey

    Yang He and Lingao Xiao. Structured pruning for deep convolutional neural networks: A survey. (IEEE) TPAMI, 2023. 1, 3, 5, 6

    2023

  13. [21]

    Dietterich

    Dan Hendrycks and Thomas G. Dietterich. Bench- marking neural network robustness to common corruptions and perturbations. In ICLR, 2019. 1, 3, 5

    2019

  14. [22]

    Pixmix: Dreamlike pic- tures comprehensively improve safety measures

    Dan Hendrycks and et al. Pixmix: Dreamlike pic- tures comprehensively improve safety measures. In CVPR, 2022. 4

    2022

  15. [23]

    Hermann and et al

    Katherine L. Hermann and et al. On the founda- tions of shortcut learning. ICLR, 2024. 2, 7

    2024

  16. [24]

    Deep networks with stochas- tic depth

    Gao Huang and et al. Deep networks with stochas- tic depth. In ECCV, 2016. 4

    2016

  17. [25]

    Rethinking the prun- ing criteria for convolutional neural network

    Zhongzhan Huang, Wenqi Shao, Xinjiang Wang, Liang Lin, and Ping Luo. Rethinking the prun- ing criteria for convolutional neural network. In NeurIPS, 2021. 2, 3, 4

    2021

  18. [26]

    On the channel pruning using graph convolution network for convolutional neural network acceleration

    Di Jiang, Yuan Cao, and Qiang Yang. On the channel pruning using graph convolution network for convolutional neural network acceleration. In IJCAI, 2022. 6

    2022

  19. [27]

    Discriminative layer prun- ing for convolutional neural networks

    Artur Jordao and et al. Discriminative layer prun- ing for convolutional neural networks. J-STSP,

  20. [28]

    When layers play the lottery, all tickets win at initialization

    Artur Jordao and et al. When layers play the lottery, all tickets win at initialization. In ICCV,

  21. [29]

    On the effect of pruning on adversarial robustness

    Artur Jord˜ ao and H´ elio Pedrini. On the effect of pruning on adversarial robustness. In ICCV, 2021. 1, 4, 5, 6, 8

    2021

  22. [30]

    Shortened llama: A simple depth pruning for large language models

    Bo-Kyeong Kim, Geonmin Kim, Tae-Ho Kim, Thibault Castells, Shinkook Choi, Junho Shin, and Hyoung-Kyu Song. Shortened llama: A simple depth pruning for large language models. ICLR - Workshop , 2024. 1, 3

    2024

  23. [31]

    Last layer re-training is suf- ficient for robustness to spurious correlations

    Polina Kirichenko, Pavel Izmailov, and An- drew Gordon Wilson. Last layer re-training is suf- ficient for robustness to spurious correlations. In ICLR, 2023. 4

    2023

  24. [32]

    Ma- honey, Joseph Hassoun, Kurt Keutzer, and Amir Gholami

    Woosuk Kwon, Sehoon Kim, Michael W. Ma- honey, Joseph Hassoun, Kurt Keutzer, and Amir Gholami. A fast post-training pruning framework for transformers. In NeurIPS, 2022. 3

    2022

  25. [33]

    Quantifying the carbon emissions of machine learning

    Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. Quantifying the carbon emissions of machine learning. InNeurIPS,

  26. [34]

    Can pruning improve certi- fied robustness of neural networks? TMLR, 2023

    Zhangheng Li, Tianlong Chen, Linyi Li, Bo Li, and Zhangyang Wang. Can pruning improve certi- fied robustness of neural networks? TMLR, 2023. 4

    2023

  27. [35]

    Pruning networks with cross-layer ranking & k-reciprocal nearest filters

    Mingbao Lin, Liujuan Cao, Yuxin Zhang, Ling Shao, Chia-Wen Lin, and Rongrong Ji. Pruning networks with cross-layer ranking & k-reciprocal nearest filters. (IEEE) TNNLS , 2023. 6

    2023

  28. [36]

    Hrank: Filter pruning using high-rank feature map

    Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, and Ling Shao. Hrank: Filter pruning using high-rank feature map. In CVPR, 2020. 6

    2020

  29. [37]

    SOKS: automatic searching of the optimal kernel shapes for stripe-wise network pruning

    Guangzhe Liu, Ke Zhang, and Meibo Lv. SOKS: automatic searching of the optimal kernel shapes for stripe-wise network pruning. (IEEE) TPAMI,

  30. [38]

    UPDP: A unified progressive depth pruner for CNN and vision transformer

    Ji Liu and et al. UPDP: A unified progressive depth pruner for CNN and vision transformer. In AAAI, 2024. 3

    2024

  31. [39]

    Harder or different? a closer look at distribution shift in dataset repro- duction

    Shangyun Lu and et al. Harder or different? a closer look at distribution shift in dataset repro- duction. In ICML, 2020. 3, 5

    2020

  32. [40]

    LLM-pruner: On the structural pruning of large language models

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. LLM-pruner: On the structural pruning of large language models. In NeurIPS, 2023. 1

    2023

  33. [41]

    The tunnel effect: Building data representations in deep neural net- works

    Wojciech Masarczyk and et al. The tunnel effect: Building data representations in deep neural net- works. In NeurIPS, 2023. 4

    2023

  34. [42]

    What makes a good prune? maximal unstructured pruning for maximal cosine similarity

    Gabryel Mason-Williams and Fredrik Dahlqvist. What makes a good prune? maximal unstructured pruning for maximal cosine similarity. In ICLR,

  35. [43]

    Investigating calibration and corruption robust- ness of post-hoc pruned perception cnns: An im- age classification benchmark study

    Pallavi Mitra, Gesina Schwalbe, and Nadja Klein. Investigating calibration and corruption robust- ness of post-hoc pruned perception cnns: An im- age classification benchmark study. In CVPR,

  36. [44]

    SOSP: efficiently captur- ing global correlations by second-order structured pruning

    Manuel Nonnenmacher, Thomas Pfeil, Ingo Stein- wart, and David Reeb. SOSP: efficiently captur- ing global correlations by second-order structured pruning. In ICLR, 2022. 6, 8

    2022

  37. [45]

    An introduction to adversarially robust deep learning

    Jonathan Peck, Bart Goossens, and Yvan Saeys. An introduction to adversarially robust deep learning. (IEEE) TPAMI, 2024. 1

    2024

  38. [46]

    Huy Phan, Miao Yin, Yang Sui, Bo Yuan, and Saman A. Zonouz. CSTAR: towards compact and structured deep neural networks with adversarial robustness. In AAAI, 2023. 1, 4

    2023

  39. [47]

    Re- ali Costa, and Artur Jordao

    Ian Pons, Bruno Yamamoto, Anna H. Re- ali Costa, and Artur Jordao. Effective layer pruning through similarity metric per- spective. In ICPR (To appear). ArXiv: https://arxiv.org/html/2405.17081v1, 2024. 1, 2, 3, 4, 5, 6, 7, 8, 9

    2024 arXiv

  40. [48]

    Smith, and Oren Etzioni

    Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. Green AI. ACM, 2020. 8

    2020

  41. [49]

    Human activity recognition based on smartphone and wearable sensors using multiscale dcnn ensemble

    Jessica Sena, Jesimon Barreto, Carlos Cae- tano, Guilherme Cramer, and William Robson Schwartz. Human activity recognition based on smartphone and wearable sensors using multiscale dcnn ensemble. Neurocomputing, 2021. 9

    2021

  42. [50]

    ´Alvarez

    Maying Shen, Hongxu Yin, Pavlo Molchanov, Lei Mao, Jianna Liu, and Jos´ e M. ´Alvarez. Struc- tural pruning via latency-saliency knapsack. In NeurIPS, 2022. 1, 6

    2022

  43. [51]

    Energy and policy considerations for deep learning in NLP

    Emma Strubell, Ananya Ganesh, and Andrew Mc- Callum. Energy and policy considerations for deep learning in NLP. In ACL, 2019. 8, 9

    2019

  44. [52]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models. In ICLR, 2024. 1, 3

    2024

  45. [53]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izac- ard, Xavier Martinet, Marie-Anne Lachaux, Tim- oth´ ee Lacroix, Baptiste Rozi` ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aur´ elien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundatio...

    2023

  46. [54]

    Mobileone: An improved one millisecond mobile backbone

    Pavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan. Mobileone: An improved one millisecond mobile backbone. In CVPR, 2023. 1

    2023

  47. [55]

    Wilber, and Serge J

    Andreas Veit, Michael J. Wilber, and Serge J. Be- longie. Residual networks behave like ensembles of relatively shallow networks. In NeurIPS, 2016. 4

    2016

  48. [56]

    Recent advances on neural network prun- ing at initialization

    Huan Wang, Can Qin, Yue Bai, Yulun Zhang, and Yun Fu. Recent advances on neural network prun- ing at initialization. In IJCAI, 2022. 3

    2022

  49. [57]

    Channel pruning via lookahead search guided reinforcement learn- ing

    Zi Wang and Chengcheng Li. Channel pruning via lookahead search guided reinforcement learn- ing. In WACV, 2022. 6

    2022

  50. [58]

    Generalized shape metrics on neural representations

    Alex H Williams, Erin Kunz, Simon Kornblith, and Scott Linderman. Generalized shape metrics on neural representations. In NeurIPS, 2021. 2

    2021

  51. [59]

    Auto-train-once: Con- troller network guided automatic network pruning from scratch

    Xidong Wu, Shangqian Gao, Zeyu Zhang, Zhen- zhen Li, Runxue Bao, Yanfu Zhang, Xiaoqian Wang, and Heng Huang. Auto-train-once: Con- troller network guided automatic network pruning from scratch. CVPR, 2024. 6

    2024

  52. [60]

    Sheared llama: Accelerating lan- guage model pre-training via structured pruning

    Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. Sheared llama: Accelerating lan- guage model pre-training via structured pruning. In ICLR, 2024. 1, 3

    2024

  53. [61]

    Imagenet-OOD: Deciphering modern out-of-distribution detection algorithms

    William Yang, Byron Zhang, and Olga Rus- sakovsky. Imagenet-OOD: Deciphering modern out-of-distribution detection algorithms. In ICLR,

  54. [62]

    Auto graph encoder-decoder for neural network pruning

    Sixing Yu, Arya Mazaheri, and Ali Jannesari. Auto graph encoder-decoder for neural network pruning. In ICCV, 2021. 8

    2021

  55. [63]

    Topology-aware network pruning using multi- stage graph embedding and reinforcement learn- ing

    Sixing Yu, Arya Mazaheri, and Ali Jannesari. Topology-aware network pruning using multi- stage graph embedding and reinforcement learn- ing. In ICML, 2022. 6

    2022

  56. [64]

    Are all layers created equal? JMLR, 2022

    Chiyuan Zhang, Samy Bengio, and Yoram Singer. Are all layers created equal? JMLR, 2022. 1, 3, 4

    2022

  57. [65]

    Layer pruning for obtaining shallower resnets

    Ke Zhang and Guangzhe Liu. Layer pruning for obtaining shallower resnets. (IEEE) SPL, 2022. 1, 3, 4, 6, 7

    2022

  58. [66]

    Carrying out CNN channel prun- ing in a white box

    Yuxin Zhang, Mingbao Lin, Chia-Wen Lin, Jie Chen, and et al. Carrying out CNN channel prun- ing in a white box. (IEEE) TNNLS , 2023. 6

    2023

  59. [67]

    Revisit kernel pruning with lottery regulated grouped convolutions

    Shaochen Zhong, Guanqun Zhang, Ningjia Huang, and Shuai Xu. Revisit kernel pruning with lottery regulated grouped convolutions. In ICLR, 2022. 6, 8

    2022

  60. [68]

    Learning N: M fine-grained structured sparse neural networks from scratch

    Aojun Zhou, Yukun Ma, Junnan Zhu, Jianbo Liu, Zhijie Zhang, Kun Yuan, Wenxiu Sun, and Hong- sheng Li. Learning N: M fine-grained structured sparse neural networks from scratch. In ICLR,

  61. [69]

    Yen, and Zhang Yi

    Yao Zhou, Gary G. Yen, and Zhang Yi. Evolu- tionary shallowing deep neural networks at block levels. (IEEE) TNNLS , 2022. 1, 3, 4, 5, 7 12

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.