Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers

Enzo Tartaglione; Nour Hezbri; Van-Tam Nguyen; Victor Qu\'etu; Zhu Liao

REVIEW 4 major objections 6 minor 39 references

Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Batch-norm statistics can delete most layers from a deep network while keeping accuracy nearly unchanged.

desk verdict A simple, empirical depth-pruning method worth engaging, but its theoretical story is thin and one ImageNet result contradicts the abstract. read the letter →

arxiv 2412.15077 v1 pith:LKF7WYUS submitted 2024-12-19 cs.LG cs.CLcs.CV

Zhu Liao , Nour Hezbri , Victor Quétu , Van-Tam Nguyen , Enzo Tartaglione This is my paper · ORCID

classification cs.LGcs.CLcs.CV MSC 68T07

keywords modelcompressionlayerpruningdepthreductionbatchnormalizationneuralnetworkefficiencyrectifiedactivationinferencelatency

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a deep network can be made much shallower by treating each batch-normalization layer as a set of per-neuron switches. For every neuron, the batch-norm shift $\beta_{l,i}$ tells which side of its rectifier the pre-activation typically falls on: positive $\beta$ means the neuron is almost always in its linear region, so the activation can be replaced by the identity and folded into the next layer, while non-positive $\beta$ means the neuron is almost always silent, so it can be deleted. The paper then removes whole layers, ranked by the accuracy drop each removal causes, and retrains between removals. Across ResNet-18, MobileNet-V2, Swin-T, and VGG-16 on CIFAR-10 it reports removing 12/17, 17/35, 6/12, and 7/15 layers while keeping top-1 accuracy within about one point of the dense model, with comparable results on larger image sets and on the language models BERT and RoBERTa. The reason this matters is that inference latency is set by the serial chain of layers, so deleting layers attacks exactly the cost that parameter-count pruning misses.

What carries the argument

The object that carries the argument is the per-neuron error likelihood $E_{l,i} = \Phi(-|\beta_{l,i}|/\gamma_{l,i})$ (Eq. 3), evaluated separately for each neuron in a batch-normalization layer. It converts 'is this layer removable?' into a yes/no decision per neuron: positive $\beta_{l,i}$ means the neuron is almost always ON (rectifier in its linear region), so the activation function is replaced by the identity and the neuron's weights are folded into the following layer; non-positive $\beta_{l,i}$ means the neuron is almost always OFF, so the neuron is dropped. Removing a layer is therefore a surgical operation on the BatchNorm parameters rather than a retraining-dependent search. A layer's rank is fixed by measuring validation accuracy after its surgical removal (Eq. 4), and the outer algorithm removes the lowest-ranked layers while the validation accuracy stays above $\theta \cdot A_\mathrm{init}$, retraining after each accepted removal. For transformer models without BatchNorm, the paper attaches the same statistics by computing the mean and standard deviation of the fully-connected layer's output.

What would settle it

Compute the actual change in the next layer's activations when a layer is removed via TLC, and compare it with the predicted per-neuron error sum $\sum_i \Phi(-|\beta_{l,i}|/\gamma_{l,i})$. If the actual perturbation is large when the predicted error is small, or if the predicted ranking of layers disagrees with the ranking by true accuracy loss, the central mechanism is falsified. A second, cheaper check: take a trained ResNet-18 on CIFAR-10, measure the empirical distribution of each pre-activation $z_{l,i}$ over the validation set, and test whether $z_{l,i}$ is close enough to Gaussian that $\Phi(-|\beta|/\gamma)$ matches the observed fraction of wrong-side crossings for the chosen rectifier.

Watch

Extended reading notes

Core claim

The central discovery is that the boundary between removable and unremovable computation is visible in the batch-norm parameters themselves. Under the paper's Gaussian model, each pre-activation is distributed as $z_{l,i} \sim \mathcal{N}(\beta_{l,i}, \gamma_{l,i}^2)$, so the probability that the rectifier's choice is wrong if we force the neuron to a single state is $\Phi(-|\beta_{l,i}|/\gamma_{l,i})$. The paper's move is to take the binary sign of $\beta_{l,i}$ as the state: neurons with $\beta_{l,i}>0$ are linearized and merged into the next layer via a linear combination, neurons with $\beta_{l,i}\le0$ are simply removed, and the whole layer is then gone. TLC evaluates how much accuracy the model loses when each layer is removed in this way, removes the least costly layers one by one while validation accuracy stays above a relative threshold $\theta$, and retrains after each accepted removal. The paper reports that this procedure removes a large fraction of layers on convolutional, transformer, and language models while keeping validation accuracy near the dense model's, and that it avoids the layer-collapse failure mode that afflicts entropy-based baselines on networks without skip connections.

Load-bearing premise

The load-bearing premise is that each pre-activation $z_{l,i}$ is Gaussian with mean $\beta_{l,i}$ and scale $\gamma_{l,i}$, so the formula $\Phi(-|\beta_{l,i}|/\gamma_{l,i})$ really measures the perturbation caused by linearizing or deleting that neuron, and that the hard sign threshold on $\beta$ is therefore the right ON/OFF rule.

Editorial extensions

If this is right

Inference latency should drop roughly in proportion to the fraction of removed layers, because the critical serial path through the network, not total parameter count, is what limits GPU latency.
The same BatchNorm-derived ON/OFF slicing can be applied to already-trained models, making depth compression a post-training operation that needs only a validation set and a fine-tuning pass.
Skip-connection architectures tolerate much deeper removal than plain feedforward ones, so TLC's practical ceiling depends on architecture family, not just on dataset.
For transformer models, replacing LayerNorm-based statistics with output mean and standard deviation of the linear layer lets the method transfer to BERT and RoBERTa with comparable layer removals.
The short-finetune variant (TLC-finetuning) suggests the method can scale to larger models, though it removes fewer layers than full retraining between rounds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The Gaussian assumption in Eq. (3) is untested; a validation-set histogram of pre-activations would show how far real distributions deviate, and replacing $\Phi$ with empirical crossing rates would make TLC's error estimate trustworthy.
Nothing in the method prevents using the same ON/OFF score to prune individual channels inside surviving layers, so TLC's neuron-level decisions could be extended into a joint depth-and-width compression algorithm.
The hard $\beta>0$ threshold suggests a training-time regularizer that pushes pre-activation means away from zero, which would increase the margin $|\beta|/\gamma$ and make even more layers removable; the paper does not try this.
The comparison is made in removed-layer counts and FLOPs; actual latency gains will depend on hardware and tensor shapes, and reporting end-to-end inference time on the target device would sharpen the practical claim.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

TLC is a heuristic depth-pruning method that works well in many settings, but the paper oversells the theory and glosses over a real accuracy drop on ImageNet MobileNet-V2.

The genuinely new part is the per-neuron ON/OFF split using the sign of the BN shift beta: ON neurons are linearized and merged into the next layer, OFF neurons are removed outright. That's simple and clean, and it's distinct from entropy-based depth reduction (EGP, EASIER) and from BN filter pruning (Liu et al., Oh et al.). The experimental coverage is broad: five image datasets, three NLP tasks, and architectures including ResNet, Swin, MobileNet, BERT, and RoBERTa. In many cases TLC removes a large fraction of layers with small accuracy loss (12/17 on ResNet-18 CIFAR-10, 6/12 on RoBERTa SST-2, 8/12 on RoBERTa QNLI). It also has a real training-time advantage over EASIER.

The soft spots are real but mostly fixable. Eq. (3) is a per-neuron probability of being on the wrong side of the rectifier, not an output-perturbation bound; it ignores error magnitude, downstream weight scales, and propagation. The Gaussian assumption is asserted, not verified. The stress-test is right that the validation-driven selection loop does much of the work, but it overstates the consequence: the per-neuron rule still makes the removal safe enough to be empirically effective. The theoretical narrative is weaker than the paper suggests, but the method stands on its experiments.

Mechanical issues: Algorithm 1 uses AMtest before it is assigned (line 11); theta is never specified; residual architectures are asserted to work without explanation of how linearization interacts with skip connections.

The strongest factual concern is ImageNet MobileNet-V2: 59.43 vs 71.87 dense with only 2/35 layers removed, a 12.4-point drop that contradicts the abstract's 'without significantly compromising model performance.' Baselines are single runs without error bars, and the code link has no commit hash.

Still, I agree with the reader's CONDITIONAL verdict. The method is practical, simple, and broadly validated. I'd send it to peer review; it deserves a serious look. The authors should fix the pseudocode, report theta and seeds, validate or de-emphasize the Gaussian assumption, and acknowledge the ImageNet case. I'd cite it as a baseline if I were working on depth pruning. I'd also bring it to a reading group for a discussion of what counts as a justification in pruning papers.

Referee Report

4 major / 6 minor

Summary. The paper introduces TLC (Till the Layers Collapse), a depth-compression method for deep neural networks that uses batch normalization parameters to decide, per neuron, whether to remove the neuron (OFF, β≤0) or linearize its activation and merge it into the next layer (ON, β>0). Whole layers are then removed by this surgical operation. Layer importance is ranked by the validation accuracy after individually removing each layer, and Algorithm 1 iteratively removes the least important layers, retrains, and continues until the validation accuracy falls below a relative threshold θ. The method is evaluated on ResNet-18, Swin-T, MobileNet-V2, VGG-16bn across CIFAR-10, Tiny-ImageNet, ImageNet, PACS, and VLCS, and on BERT/RoBERTa across SST-2, QNLI, and RTE, showing that many layers can be removed with small accuracy degradation and that TLC generally outperforms EGP, LF, EASIER, and magnitude/gradient-based layer removal baselines.

Significance. If the method holds, it provides a practical and computationally efficient way to reduce network depth, which is a well-motivated target because depth largely determines latency on modern hardware. The empirical scope is broad—five image datasets, three NLP tasks, and several architecture families—and the paper reports code release, training details, and wall-clock times, which are strengths. The method's core contribution is a cheap, post-hoc importance signal (batch norm statistics) that avoids expensive per-layer retraining during ranking. However, the analytical justification for the per-neuron ON/OFF rule is heuristic: Eq. (3) measures per-neuron sign-error probability, not actual output perturbation, and the paper does not prove or empirically verify that this metric ranks operations by true perturbation. The method's final layer-selection loop is validation-guided, so the empirical success could be driven by that selection rather than by the batch-norm-based importance signal. These gaps do not invalidate the empirical findings, but they do limit the strength of the theoretical claims.

major comments (4)

[§3.3, Eq. (4)] The importance relation is stated as I(l) < I(l′) ⇔ A(M_rem{l}) < A(M_rem{l′}). This ordering says a layer is less important exactly when removing it produces a smaller validation accuracy (i.e., a larger drop), which is the reverse of what is needed for the 'ascending ranking' described in §3.4 that starts from the 'least important layer.' If taken literally, Algorithm 1 would remove the most damaging layers first, contradicting the reported success. This is likely a typo (the intended relation is probably >), but it must be corrected and the direction of the ranking should be clarified with a concrete example.
[§3.2, Eq. (3), Fig. 2] The per-neuron error likelihood E_l,i is the probability that a Gaussian pre-activation lies on the wrong side of the rectifier when the neuron is linearized (ON) or zeroed (OFF). This is not a measure of the actual output perturbation induced by TLC. It ignores error magnitude (a rare large deviation can dominate), the scaling of the downstream weights, and the propagation of errors through depth. The area-under-the-curve argument in Fig. 2 only compares unweighted per-neuron misclassification probabilities; it does not bound the change in the next layer's activations or the network output. Please either provide an actual perturbation bound that tracks the change in the subsequent layer's pre-activations (accounting for weight norms and merging), or explicitly reframe Eq. (3) as a heuristic motivation and add an empirical check that the ON/OFF decision correlates with the true activation perturbation for the models used.
[§3.2, §3.4, Fig. 1] The description of the layer removal and linearization/merge operation assumes a strictly sequential chain of layers (conv/FC → BN → activation → next layer). For residual architectures such as ResNet-18, Swin-T, and MobileNet-V2, the paper does not specify how the removal is performed when the target layer is part of a residual block with a skip connection. It is unclear whether the skip path is folded into the merge, whether the batch norm is absorbed, or how the 'subsequent layer' is defined in the presence of an additive shortcut. Since the reported results on these architectures are central to the paper's claims, the surgical procedure for residual blocks must be detailed (or the release code must be referenced with explicit pointers).
[§4.1, Table 1] The baselines 'Smallest weights' and 'Smallest gradients' are not precisely defined. It is not stated whether they remove the layer with the smallest sum of absolute weight values (or gradient norms) and whether the same retraining procedure as TLC is applied after each removal. Without this detail, the comparison can be unfair or unreproducible. Please specify the exact criterion and the fine-tuning protocol used for these baselines.

minor comments (6)

[§3.1] The phrase 'bach norm' should be 'batch norm'.
[Table 1] The entry 'Tiny-Inet' should be 'Tiny-ImageNet', and '82,90' uses a comma as a decimal separator; please unify the decimal notation.
[Table 2] The header 'BERT RoBERTatop-1' is missing a space; it should be 'BERT RoBERTa top-1'.
[§4.2] The sentence 'removing the layer with the lowest sum of weights/gradients results performs close to TLC' is grammatically awkward and should be reworded.
[§3.4] The accuracy retention threshold θ is a free parameter but no default value is reported in the experiments. Please state the θ used for Tables 1 and 2.
[Fig. 2] The green curve labeled 'TLC' is not defined in the caption; please add the formula or description of how it is computed.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: Eq. (3) is a stated Gaussian modeling assumption and the validation-based stopping rule is standard model selection; the one self-citation (Layerfold) is not load-bearing.

full rationale

The paper's central derivation is not circular in the sense of fitting a parameter and then reporting that parameter as a prediction. The per-neuron error criterion Eq. (3), E_l,i = Phi(-|beta_l,i|/gamma_l,i), is introduced under the explicit modeling assumption that the batch-normalized pre-activation satisfies z_l,i ~ N(beta, gamma^2). The ON/OFF rule (linearize if beta > 0, remove if beta <= 0) is then chosen to minimize this defined error likelihood; the minimization is true by construction of E_l,i, but the paper's empirical claim of maintaining accuracy is supported by experiments, not derived from the formula. The layer importance ranking in Eq. (4) is defined by actually measuring validation accuracy after removing each layer, and Algorithm 1 stops removing layers when validation accuracy falls below a relative threshold theta. This makes the reported number of removable layers a validation-guided model-selection outcome rather than a prediction from the BN statistics, but that is a standard use of a validation set, not a reduction of the result to a fitted constant. The only self-citation used in the method, Pilo et al. 2024 (Layerfold), supports the linear-fusion implementation ("linearized and merged within the subsequent layer via linear combination similarly to (Pilo et al. 2024)"); this is a routine linear-algebra operation and is not the load-bearing justification for the approach. The main weaknesses--lack of verification of the Gaussian assumption, the gap between per-neuron state error and true output perturbation, and the paper's own admission that for transformers it uses the empirical mean of the fully connected layer output rather than batch-norm parameters--are correctness and validity risks, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the Gaussianity of pre-activations, the independence of per-neuron errors, the ability to merge linearized neurons in an exact way (which is questionable for residual blocks), and the stability of the importance ranking during iterative removal. These are all plausible heuristics but none are formally validated.

free parameters (2)

theta (accuracy retention threshold)
Algorithm 1 stops when validation accuracy drops below theta times initial accuracy; theta is never specified in the paper, yet it controls the number of removed layers in all reported results.
batch norm shift sign threshold (0) = 0
The ON/OFF criterion beta_l,i > 0 versus beta_l,i <= 0 is a hand-chosen boundary from the Gaussian model; no sensitivity analysis is reported.

assumptions (4)

domain assumption Pre-activations z_l,i are Gaussian with mean beta_l,i and variance gamma^2_l,i
Invoked in Sec. 3.2 to derive Eq. (3). No normality test or robustness analysis is provided.
domain assumption Per-neuron errors are independent and the expected error of the whole layer is the sum of per-neuron misclassification probabilities
Used implicitly to justify the green curve in Fig. 2; the downstream effect on the next layer's output is not bounded.
ad hoc to paper Layers can be treated as a sequence of affine-BN-rectifier blocks so that linearized neurons can be exactly merged into the next layer
Residual connections in ResNet/Swin and LayerNorm in transformers do not fit this template; the paper does not explain how merging is performed in residual blocks.
domain assumption The importance ranking computed on the complete model remains valid after removing earlier layers
Algorithm 1 computes Lranked once per outer loop but then removes layers cumulatively; the ranking can be stale after structural changes.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers." pith.science (2026). https://pith.science/paper/LKF7WYUS

@misc{pith2026241215077,
  author       = {Pith},
  title        = {Pith review of: Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LKF7WYUS}},
  note         = {Machine review of arXiv:2412.15077}
}

read the original abstract

Today, deep neural networks are widely used since they can handle a variety of complex tasks. Their generality makes them very powerful tools in modern technology. However, deep neural networks are often overparameterized. The usage of these large models consumes a lot of computation resources. In this paper, we introduce a method called \textbf{T}ill the \textbf{L}ayers \textbf{C}ollapse (TLC), which compresses deep neural networks through the lenses of batch normalization layers. By reducing the depth of these networks, our method decreases deep neural networks' computational requirements and overall latency. We validate our method on popular models such as Swin-T, MobileNet-V2, and RoBERTa, across both image classification and natural language processing (NLP) tasks.

Figures

Figures reproduced from arXiv: 2412.15077 by the authors.

**Figure 1.** Overview of the key steps for TLC: identification of layer to remove, removal of irrelevant channels, and linearization [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** Error plot for the i-th neuron of the l-th layer as a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Validation loss for the complete Resnet-18 model [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Test performance (top-1) for models trained on [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 6.** Figure 6: Validation accuracy for the complete ResNet-18 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 25 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

H.; and Disselhoff, J

Ali Mehmeti-G\" o pel, C. H.; and Disselhoff, J. 2023. Nonlinear Advantage: Trained Networks Might Not Be As Complex as You Think. In ICML

work page 2023
[4]

A.; Tartaglione, E.; Berzovini, C.; Calandri, M.; and Grangetto, M

Barbano, C. A.; Tartaglione, E.; Berzovini, C.; Calandri, M.; and Grangetto, M. 2022. A Two-Step Radiologist-Like Approach for Covid-19 Computer-Aided Diagnosis from Chest X-Ray Images. In ICIAP

work page 2022
[5]

Bentivogli, L.; Clark, P.; Dagan, I.; and Giampiccolo, D. 2009. The Fifth PASCAL Recognizing Textual Entailment Challenge. TAC

work page 2009
[6]

D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. NeurIPS

work page 2020
[7]

Carion, N.; Massa, F.; Synnaeve, G.; Usunier, N.; Kirillov, A.; and Zagoruyko, S. 2020. End-to-end object detection with transformers. In ECCV

2020
[8]

Chaudhry, H. A. H.; Renzulli, R.; Perlo, D.; Santinelli, F.; Tibaldi, S.; Cristiano, C.; Grosso, M.; Fiandrotti, A.; Lucenteforte, M.; and Cavagnino, D. 2022. Lung Nodules Segmentation with DeepHealth Toolkit. In ICIAP

work page 2022

Show all 39 references

[9]

Chen, S.; and Zhao, Q. 2019. Shallowing Deep Networks: Layer-Wise Pruning Based on Feature Representations. IEEE Transactions on Pattern Analysis and Machine Intelligence

2019
[10]

Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. ImageNet: A large-scale hierarchical image database. In CVPR

2009
[11]

B.; Zehngut, N.; Raviv, A.; Artyomov, E.; Vitek, R.; and Jevnisek, R

Dror, A. B.; Zehngut, N.; Raviv, A.; Artyomov, E.; Vitek, R.; and Jevnisek, R. 2021. Layer folding: Neural network depth reduction using activation linearization

2021
[12]

Gulrajani, I.; and Lopez-Paz, D. 2020. In Search of Lost Domain Generalization. In ICLR

2020
[13]

Han, S.; Mao, H.; and Dally, W. J. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149

2015 arXiv
[14]

Han, S.; Pool, J.; Tran, J.; and Dally, W. 2015. Learning both Weights and Connections for Efficient Neural Network. In NeurIPS

2015
[15]

He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In CVPR

2016
[16]

Hestness, J.; Narang, S.; Ardalani, N.; Diamos, G.; Jun, H.; Kianinejad, H.; Patwary, M. M. A.; Yang, Y.; and Zhou, Y. 2017. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409

2017 arXiv
[17]

Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

2015 arXiv
[18]

G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; and Adam, H

Howard, A. G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; and Adam, H. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

2017 arXiv
[19]

Kenton, J. D. M.-W. C.; and Toutanova, L. K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT

2019
[20]

Krizhevsky, A.; Hinton, G.; et al. 2009. Learning multiple layers of features from tiny images

2009
[21]

Le, Y.; and Yang, X. 2015. Tiny imagenet visual recognition challenge. CS 231N

2015
[22]

Lee, N.; Ajanthan, T.; and Torr, P. 2019. SNIP: Single-shot network pruning based on connection sensitivity. In ICLR

2019
[23]

Liao, Z.; Qu \'e tu, V.; Nguyen, V.-T.; and Tartaglione, E. 2023. Can Unstructured Pruning Reduce the Depth in Deep Neural Networks? In ICCV

2023
[24]

Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692

2019 arXiv
[25]

Liu, Z.; Li, J.; Shen, Z.; Huang, G.; Yan, S.; and Zhang, C. 2017. Learning Efficient Convolutional Networks through Network Slimming. In ICCV

2017
[26]

Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; and Guo, B. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV

2021
[27]

Molchanov, P.; Tyree, S.; Karras, T.; Aila, T.; and Kautz, J. 2016. Pruning Convolutional Neural Networks for Resource Efficient Inference. In ICLR

2016
[28]

Oh, J.; Kim, H.; Baik, S.; Hong, C.; and Lee, K. M. 2022. Batch normalization tells you which filter is important. In WACV

2022
[29]

Peer, D.; Stabinger, S.; Engl, S.; and Rodr \' guez-S \'a nchez, A. 2022. Greedy-layer pruning: Speeding up transformer models for natural language processing. Pattern Recognition Letters

2022
[30]

Pilo, G.; Hezbri, N.; Pereira e Ferreira, A.; Qu \'e tu, V.; and Tartaglione, E. 2024. Layerfold: A Python Library to Reduce the Depth of Neural Networks

2024
[31]

Qu \'e tu, V.; Liao, Z.; and Tartaglione, E. 2024. The simpler the better: An entropy-based importance metric to reduce neural networks’ depth. In ECML PKDD

2024
[32]

Qu \'e tu, V.; and Tartaglione, E. 2024. DSD ^2 : Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? In AAAI

2024
[33]

Simonyan, K.; and Zisserman, A. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv:1409.1556

2015 arXiv
[34]

D.; Ng, A

Socher, R.; Perelygin, A.; Wu, J.; Chuang, J.; Manning, C. D.; Ng, A. Y.; and Potts, C. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP

2013
[35]

Sun, D.; Wang, M.; and Li, A. 2019. A Multimodal Deep Neural Network for Human Breast Cancer Prognosis Prediction by Integrating Multi-Dimensional Data. IEEE/ACM Transactions on Computational Biology and Bioinformatics

2019
[36]

Tartaglione, E.; Bragagnolo, A.; Fiandrotti, A.; and Grangetto, M. 2022. Loss-based sensitivity regularization: towards deep sparse neural networks. Neural Networks

2022
[37]

Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

2023 arXiv
[38]

Williams, A.; Nangia, N.; and Bowman, S. R. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In NAACL HLT

2018
[39]

Xu, Q.; Zhang, R.; Zhang, Y.; Wang, Y.; and Tian, Q. 2021. A Fourier-Based Framework for Domain Generalization. In CVPR

2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

H.; and Disselhoff, J

Ali Mehmeti-G\" o pel, C. H.; and Disselhoff, J. 2023. Nonlinear Advantage: Trained Networks Might Not Be As Complex as You Think. In ICML

work page 2023

[4] [4]

A.; Tartaglione, E.; Berzovini, C.; Calandri, M.; and Grangetto, M

Barbano, C. A.; Tartaglione, E.; Berzovini, C.; Calandri, M.; and Grangetto, M. 2022. A Two-Step Radiologist-Like Approach for Covid-19 Computer-Aided Diagnosis from Chest X-Ray Images. In ICIAP

work page 2022

[5] [5]

Bentivogli, L.; Clark, P.; Dagan, I.; and Giampiccolo, D. 2009. The Fifth PASCAL Recognizing Textual Entailment Challenge. TAC

work page 2009

[6] [6]

D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. NeurIPS

work page 2020

[7] [7]

Carion, N.; Massa, F.; Synnaeve, G.; Usunier, N.; Kirillov, A.; and Zagoruyko, S. 2020. End-to-end object detection with transformers. In ECCV

2020

[8] [8]

Chaudhry, H. A. H.; Renzulli, R.; Perlo, D.; Santinelli, F.; Tibaldi, S.; Cristiano, C.; Grosso, M.; Fiandrotti, A.; Lucenteforte, M.; and Cavagnino, D. 2022. Lung Nodules Segmentation with DeepHealth Toolkit. In ICIAP

work page 2022