Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks

Bingbing Ni; Chenglong Zhao; Haoran Deng; Jian Zhang; Yunxiang Zhang

REVIEW 3 major objections 4 minor 48 references

Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that convolutional channels with similar batch-normalization statistics are redundant, letting a hierarchical-clustering prune remove 45.9% of ResNet-50's FLOPs at a 0.34% Top-5 accuracy cost.

desk verdict A simple, usable BN-statistics channel pruning heuristic with competitive results, but the distance metric's independence assumption is load-bearing and never quantitatively validated on the ImageNet regime. read the letter →

arxiv 1908.02620 v1 pith:CH6QMPS4 submitted 2019-08-06 cs.LG cs.CVstat.ML

Yunxiang Zhang , Chenglong Zhao , Bingbing Ni , Jian Zhang , Haoran Deng This is my paper · ORCID

classification cs.LGcs.CVstat.ML

keywords channelpruningsimilaritybatchnormalizationhierarchicalclusteringnetworkaccelerationrepresentationalredundancyResNet-50ImageNet

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that redundant channels in a deep CNN are best identified by similarity of feature information rather than by magnitude of weights or activations. It derives a cheap similarity metric from batch-normalization parameters, groups channels with hierarchical clustering, and removes all but one representative per cluster. The paper claims this yields a pruned ImageNet ResNet-50 that loses only 0.34% Top-5 accuracy while shedding 45.90% of FLOPs, and that at a 30% FLOPs cut the pruned model actually beats the baseline's Top-1 accuracy. The broader payoff is a pruning recipe that needs no sparsity training or data-driven optimization and works directly on pretrained networks.

What carries the argument

The central object is the batch-normalization-derived channel distance $\mathrm{Dist}(N^{(i)},N^{(j)}) = (\beta^{(i)}-\beta^{(j)})^2 + (\gamma^{(i)})^2 + (\gamma^{(j)})^2$, treating each channel's normalized activations as i.i.d. random variables with mean $\beta$ and variance $\gamma^2$. This distance matrix feeds an agglomerative hierarchical clustering with one global threshold per network; after normalizing each layer's matrix to $[0,1]$, the algorithm keeps one representative channel per cluster and discards the rest. The theoretical backbone is Proposition 2, which upper-bounds the shift induced on the next layer's activations by $\lambda$ times the distance to the nearest remaining channel, where $\lambda$ is a small coefficient depending on kernel size and norm. This replaces an $O(H W B C^2)$ data-dependent comparison with an $O(C^2)$ computation using only BN parameters.

What would settle it

Keep a channel's $\beta$ and $\gamma$ fixed but permute its spatial activation patterns, or exchange it with another channel having the same $(\beta,\gamma)$ but different edge-selectivity; if the next layer's response shifts substantially even though Equation 4 is unchanged, then the BN-only distance misses spatial structure and the redundancy claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that redundant channels in a convolutional network are not the small-magnitude ones but the similar ones: two channels whose batch-normalized activations have nearly the same distribution carry overlapping feature information. For batch-normalized channels, the expected squared distance between channels converges in probability to $(\beta^{(i)}-\beta^{(j)})^2+(\gamma^{(i)})^2+(\gamma^{(j)})^2$, so the learned shift and scale parameters $\beta,\gamma$ alone define a channel-similarity metric. The paper then claims that channels clustered by this metric are functionally redundant, and that deleting all but the cluster member with the largest $|\gamma|$, followed by a short fine-tune, does not meaningfully reduce representational power. It supports this with a bound showing that removing one channel and merging its kernel into a similar channel shifts the next layer's activations at most by a small layer-dependent factor times the original channel distance.

Load-bearing premise

The load-bearing assumption is that two channels whose batch-normalization mean and scale parameters are close are truly interchangeable, so spatial structure and downstream effects can be ignored when deciding what to prune.

Editorial extensions

If this is right

On ImageNet, a pretrained ResNet-50 pruned with threshold $t=0.10$ drops only 0.34% Top-5 accuracy while removing 45.90% of FLOPs.
At a gentler 30% FLOPs reduction, the pruned ResNet-50 gains 0.08% Top-1 accuracy over the uncompressed baseline.
Because the metric uses only BN parameters, the pruning is one-shot on a pretrained model and does not require training with sparsity constraints, unlike SSL or NS.
The probabilistic distance estimate is stable across data batches and batch sizes, whereas direct activation-value distances fluctuate with the input sample.
Normalizing each layer's distance matrix lets a single global threshold automatically set per-layer pruning ratios, avoiding hand-designed target architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves implicit that the same beta-gamma distance could serve as a cheap layer-wise width-allocation signal during architecture search, or as a live duplication detector during training.
Since the proof bounds only a single-layer, one-to-one substitution while the algorithm prunes many channels across all layers and then fine-tunes, the method's practical success likely depends on fine-tuning repairing multi-layer coupling; testing this directly would clarify where the gain comes from.
For BN-free architectures, the metric would need an estimated analogue from activation moments, so portability beyond batch-normalized CNNs is a natural next test.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Punchline first: this is a simple, genuinely usable channel-pruning heuristic, and the empirical results on CIFAR and ImageNet hold up against the contemporary state of the art. The soft spot is exactly where the stress-test puts it: Eq. 4 drops the inter-channel covariance term, and the paper never quantitatively validates that independence assumption where the headline numbers live.

What is new: the specific package — BN (gamma, beta) as a data-free channel distance, hierarchical clustering on normalized distance matrices, one global threshold per experiment — is not in the cited RDF, Srinivas and Babu, diversity networks, or kernel-clustering work. That is a legitimate if incremental contribution. The WLLN derivation is correct as far as it goes, the normalization ablation in Section 4.4 is a real finding, and comparing the BN-based distance matrix against activation-based ones in Section 4.3 is the right check to run. The ImageNet ResNet-50 figures (45.9% FLOPs cut at 0.34% Top-5 drop; beating baseline at 30% cut) are competitive with the baselines they cite.

Where it softens. First, the covariance problem. The paper admits in Section 3.1 that the independence assumption may appear strong, then points to Figure 3, but Figure 3 is a visual comparison for VGG-16 on CIFAR-10. No error norm, no correlation coefficient, nothing on ResNet-50/ImageNet. Two channels with identical (beta, gamma) but anti-correlated behavior get distance 0 under Eq. 4 while their true squared difference can be several times gamma^2; conversely, redundant channels that differ by a scale or offset are pushed apart. This is the load-bearing assumption of the entire clustering step. It does not sink the method — the cross-dataset empirical results say the approximation is serviceable — but it is under-validated in exactly the regime the paper sells.

Second, Proposition 2 bounds only a single-layer, one-to-one substitution with kernel compensation. Section 3.3 says the one-to-one strategy is mainly for theoretical concern, so the authors are honest that the bound does not cover the actual many-channel clustering plus fine-tuning. I read that as a disclosed limitation, but the theory section promises more than it delivers.

Third, the abstract's outperforms-the-baseline claim rests on a +0.08% Top-1 difference, which without error bars or multiple seeds is noise. There is also no code release, which matters for a method this easy to re-implement. Minor: the appendix renumbers Propositions 1 and 2 as 3 and 4, a sign of hasty assembly.

Who this is for: anyone needing a training-free, data-free pruning step for pretrained models. It deserves a serious referee, not a desk reject. I would send it out and ask for quantitative distance-metric validation on ImageNet-scale models, error bars for the baseline claims, and code or step-by-step reproducibility instructions.

Referee Report

3 major / 4 minor

Summary. The paper proposes a channel-pruning method for pre-trained CNNs based on the idea that channels with similar feature information are functionally redundant. It derives a distance between two channels from probabilistic modeling, approximating the expected squared activation difference under i.i.d. assumptions; using batch-normalization parameters, the distance reduces to (β_i−β_j)^2 + γ_i^2 + γ_j^2 (Eq. 4). The algorithm then clusters channels per layer with hierarchical clustering, retains one representative channel per cluster (largest |γ|), and fine-tunes the resulting model. The paper reports strong empirical results, including ResNet-50 on ImageNet with 45.90% FLOPs reduction at only 0.34% Top-5 accuracy drop, and 30% FLOPs reduction with slightly improved Top-1 accuracy. It also provides a theoretical bound (Proposition 2) for a single-layer, one-channel substitution.

Significance. If the proposed metric is valid, the method is practically attractive: it requires no sparsity training, no complex data-driven optimization, can be applied directly to pre-trained models, and uses a single global threshold to control pruning across all layers. The empirical results on CIFAR and ImageNet are competitive with or better than several existing channel-pruning methods. The derivation of the distance is principled under the stated assumptions, and the paper makes a good-faith effort to validate the approximation against activation-based distances (Section 4.3, Figure 3). However, the load-bearing assumption that per-channel batch-normalization parameters fully characterize functional similarity is not quantitatively validated, and the theoretical support is narrower than the actual algorithm. These issues affect the central claim, so the manuscript needs revision before the results can be considered fully supported.

major comments (3)

[Section 3.2, Eq. (4); Proposition 1 (Eq. 2)] The distance formula omits the cross-channel covariance term. Under the stated independence assumption, E[(N_i−N_j)^2] = (β_i−β_j)^2 + γ_i^2 + γ_j^2, but without independence the exact expansion contains −2Cov(N_i,N_j). Batch-normalization parameters β and γ are per-channel and cannot encode this covariance. For example, if β_i=β_j and γ_i=γ_j, Eq. 4 gives 2γ^2, whereas the actual expected squared difference is 2γ^2−2Cov; this can be much larger for negatively correlated channels and near zero for positively correlated near-duplicates. Since the hierarchical clustering and all pruning decisions are driven entirely by this distance, the central claim that most channels in a similarity group can be removed is not secured by the provided derivation. The validation in Figure 3 compares Eq. 4 with Eq. 1 only visually for VGG-16 on CIFAR-10, and does not quantify rank agreement or test the ResNet-50/ImageNet regime where the headline results are reported. Please provide a quantitative comparison (e.g., Spearman rank correlation between Eq. 1 and Eq. 4 distances) on ImageNet layers, and either relax the independence assumption or incorporate a covariance-corrected term.
[Section 3.3, Proposition 2 (Eq. 8)] Proposition 2 bounds only a single-layer, one-to-one substitution of one channel by one similar channel using a specifically updated kernel. The actual algorithm in Section 3.4 removes all but one channel per cluster, in every layer, and relies on fine-tuning to restore accuracy; the proof does not bound the cumulative effect of many removals, cross-layer propagation, or the interaction with the channel-selection layers inserted in residual blocks (Appendix A). As stated, Proposition 2 supports the weaker statement that removing one channel is safe if a sufficiently similar channel exists, not the abstract's claim that 'most channels within each such similarity group can be removed.' Please either extend the analysis to multi-channel removal or explicitly present Proposition 2 as heuristic motivation rather than a proof of the algorithm's safety.
[Section 4.3, Figure 3] The validation of the probabilistic distance is weakened by the instability shown in the left panel of Figure 3: activation-based distances vary substantially across random batches, so comparing a 20-trial average to the BN-based distance does not establish that Eq. 4 is a reliable proxy for the distance on a given input. The right panel shows only visual agreement and no numerical error or rank-correlation statistic. Since the metric is the load-bearing component, please report a quantitative measure (e.g., mean absolute error, Spearman rank correlation) across multiple layers and architectures, including ResNet on ImageNet.

minor comments (4)

[Section 3.2, Eq. (4)] The approximation symbol '⋍' is unusual; please define it explicitly or use a standard notation such as '≈'.
[Appendix B] The numbering of propositions in the appendix is inconsistent with the main text: Proposition 1 appears as Proposition 3 and Proposition 2 appears as Proposition 4. Please align the numbering.
[Table 1(c)] The column header 'Time Pruned' is ambiguous; clarify whether it reports absolute wall-clock time per batch or relative time saving, and specify the unit.
[Figure 3 caption] The caption refers to the 'accuracy of the latter' for the red dotted line, but the left panel title and text describe the 'probabilistic approach'; use consistent terminology in the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the BN-distance metric follows from the stated probabilistic assumptions, and the accuracy/FLOPs results are measured after pruning rather than being defined by the metric.

full rationale

The derivation chain is self-contained. Equation 4 is obtained algebraically from Proposition 1 under the paper's explicitly stated i.i.d. within-channel and mutual-independence assumptions; it is not fitted to pruning accuracy or to the reported ImageNet results. The hierarchical clustering uses this fixed distance metric with a global threshold t, but t controls pruning ratio, and the reported accuracies are measured after fine-tuning, so the headline results are empirical outcomes rather than consequences of the definition. Proposition 2 provides an independent mathematical upper bound on the distance shift from one-to-one channel substitution; it does not presuppose the success of the final pruning experiments. The paper does not rely on a self-citation chain or a uniqueness theorem to force its choice of metric, and its references to prior work such as [22] and [31] are external sources that motivate rather than define the result. The skeptic's concern that omitted inter-channel covariance weakens the metric is a validity or robustness critique of the independence assumption, not evidence that any claimed prediction reduces to its own input by construction. No load-bearing step is self-definitional, fitted-and-renamed, or forced by citation.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method rests on learning-derived BN parameters, one user-set threshold, and a fine-tuning schedule. No new physical or mathematical entities are introduced.

free parameters (2)

Threshold t = 0.25, 0.225, 0.20, 0.19, 0.10, 0.06, 0.08, 0.11, 0.13 depending on dataset and model
Global distance threshold that determines pruning ratio per layer; chosen post hoc to reach target FLOPs reductions, not predicted a priori. Reported accuracy is for these selected thresholds.
Fine-tuning epochs = 60 on CIFAR; ImageNet settings kept the same
Hyperparameter for fine-tuning after pruning; standard but affects final accuracy.

assumptions (3)

domain assumption Activations belonging to the same channel are i.i.d. and activations from different channels are mutually independent (Proposition 1).
Underlies the WLLN derivation of Eq. 2 and the BN-based distance in Eq. 4. The paper validates empirically in Section 4.3 but does not prove it.
domain assumption Batch-normalized activations follow a distribution with mean beta(c) and variance (gamma(c))^2.
Assumes BN outputs are i.i.d. with these moments; ignores residual mean and variance of normalized inputs, though BN training encourages them to be 0 and 1.
ad hoc to paper The single-layer shift bound (Prop. 2) extends to full-network pruning with fine-tuning.
The algorithm prunes all layers at once and fine-tunes, whereas the bound analyzes a single layer with one-to-one kernel compensation. The paper assumes multi-layer greedy pruning plus fine-tuning preserves accuracy despite the bound being local.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks." pith.science (2026). https://pith.science/paper/CH6QMPS4

@misc{pith2026190802620,
  author       = {Pith},
  title        = {Pith review of: Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CH6QMPS4}},
  note         = {Machine review of arXiv:1908.02620}
}

read the original abstract

To address the limitations of existing magnitude-based pruning algorithms in cases where model weights or activations are of large and similar magnitude, we propose a novel perspective to discover parameter redundancy among channels and accelerate deep CNNs via channel pruning. Precisely, we argue that channels revealing similar feature information have functional overlap and that most channels within each such similarity group can be removed without compromising model's representational power. After deriving an effective metric for evaluating channel similarity through probabilistic modeling, we introduce a pruning algorithm via hierarchical clustering of channels. In particular, the proposed algorithm does not rely on sparsity training techniques or complex data-driven optimization and can be directly applied to pre-trained models. Extensive experiments on benchmark datasets strongly demonstrate the superior acceleration performance of our approach over prior arts. On ImageNet, our pruned ResNet-50 with 30% FLOPs reduced outperforms the baseline model.

Figures

Figures reproduced from arXiv: 1908.02620 by the authors.

**Figure 2.** Accuracy-acceleration curve on CIFAR dataset. Best viewed in color. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Left: Performance comparison between activation-value-based approach and its probabilis [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Left: Distribution of values in both the unnormalized distance matrix and the normalized [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Architecture of pruned VGG-16 model with 80% reduction of FLOPs on CIFAR-10 dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Visualization of the channel distance matrix computed using different approaches. The [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 44 canonical work pages

[1]

Imagenet classiﬁcation with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. “Imagenet classiﬁcation with deep convolutional neural networks”. In: Advances in neural information processing systems. 2012, pp. 1097–1105

work page 2012
[2]

Very deep convolutional networks for large-scale image recognition

Karen Simonyan and Andrew Zisserman. “Very deep convolutional networks for large-scale image recognition”. In: International Conference on Learning Representations. 2015, pp. 1–14

work page 2015
[3]

Deep residual learning for image recognition

Kaiming He et al. “Deep residual learning for image recognition”. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 770–778

2016
[4]

Predicting parameters in deep learning

Misha Denil et al. “Predicting parameters in deep learning”. In:Advances in neural information processing systems. 2013, pp. 2148–2156

work page 2013
[5]

Do deep nets really need to be deep?

Jimmy Ba and Rich Caruana. “Do deep nets really need to be deep?” In:Advances in neural information processing systems. 2014, pp. 2654–2662

work page 2014
[6]

Learning Efﬁcient Convolutional Networks through Network Slimming

Zhuang Liu et al. “Learning Efﬁcient Convolutional Networks through Network Slimming”. In: International Conference on Computer Vision. 2017, pp. 2755–2763

work page 2017
[7]

Channel pruning for accelerating very deep neural networks

Yihui He, Xiangyu Zhang, and Jian Sun. “Channel pruning for accelerating very deep neural networks”. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, pp. 1389–1397

work page 2017
[8]

Speeding Up Convolutional Neural Networks with Low Rank Expansions

Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. “Speeding Up Convolutional Neural Networks with Low Rank Expansions”. In: British Machine Vision Conference. 2014

work page 2014

Show all 48 references

[9]

Exploiting linear structure within convolutional networks for efﬁcient evaluation

Emily L Denton et al. “Exploiting linear structure within convolutional networks for efﬁcient evaluation”. In: Advances in neural information processing systems. 2014, pp. 1269–1277

2014
[10]

Binarized neural networks: Training deep neural networks with weights and activations constrained to + 1 or -1

Matthieu Courbariaux et al. “Binarized neural networks: Training deep neural networks with weights and activations constrained to + 1 or -1”. In: arXiv preprint arXiv:1602.02830 (2016)

2016 arXiv
[11]

Xnor-net: Imagenet classiﬁcation using binary convolutional neural networks

Mohammad Rastegari et al. “Xnor-net: Imagenet classiﬁcation using binary convolutional neural networks”. In: European Conference on Computer Vision. Springer. 2016, pp. 525–542

2016
[12]

Distilling the Knowledge in a Neural Network

Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. “Distilling the Knowledge in a Neural Network”. In: NIPS Deep Learning Workshop. 2014

2014
[13]

Fitnets: Hints for thin deep nets

Adriana Romero et al. “Fitnets: Hints for thin deep nets”. In: International Conference on Learning Representations. 2014

2014
[14]

Pruning Filter via Geometric Median for Deep Convolutional Neural Net- works Acceleration

Yang He et al. “Pruning Filter via Geometric Median for Deep Convolutional Neural Net- works Acceleration”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2019

2019
[15]

Optimal brain damage

Yann LeCun, John S Denker, and Sara A Solla. “Optimal brain damage”. In: Advances in neural information processing systems. 1990, pp. 598–605

1990
[16]

Second order derivatives for network pruning: Optimal brain surgeon

Babak Hassibi and David G Stork. “Second order derivatives for network pruning: Optimal brain surgeon”. In: Advances in neural information processing systems. 1993, pp. 164–171

1993
[17]

Learning both weights and connections for efﬁcient neural network

Song Han et al. “Learning both weights and connections for efﬁcient neural network”. In: Advances in neural information processing systems. 2015, pp. 1135–1143

2015
[18]

Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding

Song Han, Huizi Mao, and William J Dally. “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding”. In: 2015

2015
[19]

Dynamic network surgery for efﬁcient dnns

Yiwen Guo, Anbang Yao, and Yurong Chen. “Dynamic network surgery for efﬁcient dnns”. In: Advances In Neural Information Processing Systems. 2016, pp. 1379–1387

2016
[20]

Compressing Neural Networks using the Variational Information Bottleneck

Bin Dai et al. “Compressing Neural Networks using the Variational Information Bottleneck”. In: Proceedings of the 35th International Conference on Machine Learning. 2018, pp. 1143– 1152. 9

2018
[21]

Variational Dropout Sparsiﬁes Deep Neural Networks

Dmitry Molchanov, Arsenii Ashukha, and Dmitry P. Vetrov. “Variational Dropout Sparsiﬁes Deep Neural Networks”. In: Proceedings of the 34th International Conference on Machine Learning. 2017, pp. 2498–2507

2017
[22]

Data-free Parameter Pruning for Deep Neural Net- works

Suraj Srinivas and R. Venkatesh Babu. “Data-free Parameter Pruning for Deep Neural Net- works”. In: Proceedings of the British Machine Vision Conference. 2015, pp. 31.1–31.12

2015
[23]

Diversity networks: neural network compression using determi- nantal point processes

Zelda Mariet and Suvrit Sra. “Diversity networks: neural network compression using determi- nantal point processes”. In: 2015

2015
[24]

Clustering convolutional kernels to compress deep neural networks

Sanghyun Son, Seungjun Nah, and Kyoung Mu Lee. “Clustering convolutional kernels to compress deep neural networks”. In: Proceedings of the European Conference on Computer Vision (ECCV). 2018, pp. 216–232

2018
[25]

Learning structured sparsity in deep neural networks

Wei Wen et al. “Learning structured sparsity in deep neural networks”. In:Advances in Neural Information Processing Systems. 2016, pp. 2074–2082

2016
[26]

Fast convnets using group-wise brain damage

Vadim Lebedev and Victor Lempitsky. “Fast convnets using group-wise brain damage”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2016, pp. 2554–2564

2016
[27]

Data-driven sparse structure selection for deep neural networks

Zehao Huang and Naiyan Wang. “Data-driven sparse structure selection for deep neural networks”. In: Proceedings of the European Conference on Computer Vision (ECCV). 2018, pp. 304–320

2018
[28]

Less is more: Towards compact cnns

Hao Zhou, Jose M Alvarez, and Fatih Porikli. “Less is more: Towards compact cnns”. In: European Conference on Computer Vision. Springer. 2016, pp. 662–677

2016
[29]

Thinet: A ﬁlter level pruning method for deep neu- ral network compression

Jian-Hao Luo, Jianxin Wu, and Weiyao Lin. “Thinet: A ﬁlter level pruning method for deep neu- ral network compression”. In: Proceedings of the IEEE international conference on computer vision. 2017, pp. 5058–5066

2017
[30]

Discrimination-aware channel pruning for deep neural networks

Zhuangwei Zhuang et al. “Discrimination-aware channel pruning for deep neural networks”. In: Advances in Neural Information Processing Systems. 2018, pp. 875–886

2018
[31]

Reducing duplicate ﬁlters in deep neural networks

Aruni RoyChowdhury et al. “Reducing duplicate ﬁlters in deep neural networks”. In: NIPS workshop on Deep Learning: Bridging Theory and Practice. V ol. 1. 2017

2017
[32]

Shufﬂenet: An extremely efﬁcient convolutional neural network for mobile devices

Xiangyu Zhang et al. “Shufﬂenet: An extremely efﬁcient convolutional neural network for mobile devices”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018, pp. 6848–6856

2018
[33]

Mobilenets: Efﬁcient convolutional neural networks for mobile vision applications

Andrew G Howard et al. “Mobilenets: Efﬁcient convolutional neural networks for mobile vision applications”. In: arXiv preprint arXiv:1704.04861 (2017)

2017 arXiv
[34]

Designing Neural Network Architectures using Reinforcement Learning

Bowen Baker et al. “Designing Neural Network Architectures using Reinforcement Learning”. In: 5th International Conference on Learning Representations. 2017

2017
[35]

Neural Architecture Search with Reinforcement Learning

Barret Zoph and Quoc V . Le. “Neural Architecture Search with Reinforcement Learning”. In: 5th International Conference on Learning Representations. 2017

2017
[36]

Large-scale evolution of image classiﬁers

Esteban Real et al. “Large-scale evolution of image classiﬁers”. In:Proceedings of the 34th International Conference on Machine Learning. 2017, pp. 2902–2911

2017
[37]

Hierarchical Representations for Efﬁcient Architecture Search

Hanxiao Liu et al. “Hierarchical Representations for Efﬁcient Architecture Search”. In: 6th International Conference on Learning Representations. 2018

2018
[38]

Neural architecture optimization

Renqian Luo et al. “Neural architecture optimization”. In: Advances in neural information processing systems. 2018, pp. 7816–7827

2018
[39]

Darts: Differentiable architecture search

Hanxiao Liu, Karen Simonyan, and Yiming Yang. “Darts: Differentiable architecture search”. In: arXiv preprint arXiv:1806.09055 (2018)

2018 arXiv
[40]

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Sergey Ioffe and Christian Szegedy. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, pp. 448–456

2015
[41]

Deep sparse rectiﬁer neural networks

Xavier Glorot, Antoine Bordes, and Yoshua Bengio. “Deep sparse rectiﬁer neural networks”. In: Proceedings of the 14th international conference on artiﬁcial intelligence and statistics. 2011, pp. 315–323

2011
[42]

Densely connected convolutional networks

Gao Huang et al. “Densely connected convolutional networks”. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, pp. 4700–4708

2017
[43]

Rethinking the value of network pruning

Zhuang Liu et al. “Rethinking the value of network pruning”. In: International Conference on Learning Representations. 2018. 10

2018
[44]

Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks

Yang He et al. “Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks”. In: Proceedings of the Twenty-Seventh International Joint Conference on Artiﬁcial Intelligence. 2018, pp. 2234–2240

2018
[45]

Learning multiple layers of features from tiny images

Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech. rep. Citeseer, 2009

2009
[46]

Imagenet: A large-scale hierarchical image database

Jia Deng et al. “Imagenet: A large-scale hierarchical image database”. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2009, pp. 248–255

2009
[47]

Automatic differentiation in PyTorch

Adam Paszke et al. “Automatic differentiation in PyTorch”. In: NIPS Workshop. 2017

2017
[48]

Identity mappings in deep residual networks

Kaiming He et al. “Identity mappings in deep residual networks”. In: European conference on computer vision. Springer. 2016, pp. 630–645. A. Implementation Implementation details. On CIFAR dataset, we make use of a variant of VGG-16 [6] and a 3-stage pre-activation ResNet [48]...

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Imagenet classiﬁcation with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. “Imagenet classiﬁcation with deep convolutional neural networks”. In: Advances in neural information processing systems. 2012, pp. 1097–1105

work page 2012

[2] [2]

Very deep convolutional networks for large-scale image recognition

Karen Simonyan and Andrew Zisserman. “Very deep convolutional networks for large-scale image recognition”. In: International Conference on Learning Representations. 2015, pp. 1–14

work page 2015

[3] [3]

Deep residual learning for image recognition

Kaiming He et al. “Deep residual learning for image recognition”. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 770–778

2016

[4] [4]

Predicting parameters in deep learning

Misha Denil et al. “Predicting parameters in deep learning”. In:Advances in neural information processing systems. 2013, pp. 2148–2156

work page 2013

[5] [5]

Do deep nets really need to be deep?

Jimmy Ba and Rich Caruana. “Do deep nets really need to be deep?” In:Advances in neural information processing systems. 2014, pp. 2654–2662

work page 2014

[6] [6]

Learning Efﬁcient Convolutional Networks through Network Slimming

Zhuang Liu et al. “Learning Efﬁcient Convolutional Networks through Network Slimming”. In: International Conference on Computer Vision. 2017, pp. 2755–2763

work page 2017

[7] [7]

Channel pruning for accelerating very deep neural networks

Yihui He, Xiangyu Zhang, and Jian Sun. “Channel pruning for accelerating very deep neural networks”. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, pp. 1389–1397

work page 2017

[8] [8]

Speeding Up Convolutional Neural Networks with Low Rank Expansions

Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. “Speeding Up Convolutional Neural Networks with Low Rank Expansions”. In: British Machine Vision Conference. 2014

work page 2014