Pith. sign in

REVIEW 3 major objections 6 minor 33 references

Rethinking the Number of Channels for the Convolutional Neural Network

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Width-only search lifts CNN accuracy up to 2.33 percent while cutting parameters

desk verdict A cheap channel-width search that finds lower-parameter networks with small CIFAR gains, plus a genuinely interesting qualitative rule (fluctuating widths beat monotone schedules), but the unvalidated 15-epoch fitness proxy and single search runs leave the central claim under-supported. read the letter →

arxiv 1909.01861 v1 pith:XYA2IFMH submitted 2019-09-04 cs.CV

Hui Zhu , Zhulin An , Chuanguang Yang , Xiaolong Hu , Kaiqiang Xu , Yongjun Xu This is my paper · ORCID
classification cs.CV
keywords neuralarchitecturesearchchannelwidthfunction-preservingtransformationevolutionaryalgorithmconvolutionalnetworksCIFAR-10CIFAR-100networkmorphism
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the number of channels in a convolutional network can be searched automatically with very little compute, and that the channel patterns found this way outperform both the sharp channel doublings used in classic networks and the gradual pyramid increases used in others. The search starts from a small uniform-width version of the target network and repeatedly applies function-preserving mutations that widen individual layers along a family of curve shapes, with an evolutionary selector keeping the mutations with the highest validation accuracy. The paper reports improved test accuracy of about 0.5 percent on CIFAR-10 and up to 2.33 percent on CIFAR-100 while using fewer parameters, with total search cost of 0.4 to 1.3 GPU-days. If the claim holds, width is a cheap, largely independent design axis that can be tuned without a large search budget.

What carries the argument

The machinery is a function-preserving layer-widening operation: to add channels to a convolutional layer, existing filters are replicated and the incoming weights of the next layer are divided by the replication count, with a small random noise added, so the network's input-output function is unchanged. Search wraps this operation in a set of width-curve mutation functions, including increasing, constant, and decreasing slopes, step shapes at downsampling points, symmetric counterparts, and a uniform addition, applied cumulatively to an initial uniform-width model. Because each mutation preserves the function, mutated networks can be evaluated after only 15 epochs of training, which is what makes the search cheap.

What would settle it

Take the mutated networks evaluated during one search run, record their validation accuracies at 15 epochs, train each one from scratch to full convergence, and measure the rank correlation between the two orderings. If the correlation is low, or if a control network with the same parameter count and a monotonic width profile converges to equal or better accuracy, the claim that the discovered width rules explain the gains is not supported.

Watch

Extended reading notes

Core claim

The central discovery is that the widths of a convolutional network can be optimized separately from its connectivity, and that the winning width profiles are neither the classic double-at-every-downsampling pattern nor the continuous pyramid pattern; they are irregular, fluctuating allocations that keep total parameters below the baseline while lowering error. The paper establishes this by running its functionally incremental search on several classical network families, selecting the highest-fitness individual, and retraining it from scratch with the same recipe as the original network. In every reported case the modified network has fewer parameters and lower test error, with the largest reported gain being 2.33 percent on CIFAR-100.

Load-bearing premise

The search selects networks by validation accuracy after only 15 epochs of continued training, and the paper assumes that this quick ranking predicts test accuracy after the final training from scratch to convergence; if that correlation fails, the reported improvements could be selection noise rather than consequences of the width rules.

Editorial extensions

If this is right

  • The best width profiles improve CIFAR-10 accuracy by about 0.5 percent and CIFAR-100 accuracy by up to 2.33 percent while using fewer parameters than the original networks.
  • Classic width rules, namely sharp doublings at downsampling and continuous pyramid growth, are outperformed by fluctuating profiles at lower parameter counts in the reported settings.
  • The search is cheap enough to run per network and per dataset: 0.4 to 1.3 GPU-days on a single GPU.
  • The same function-preserving search can be applied to almost any convolutional architecture, since it only needs the layer structure as input.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: if the 15-epoch fitness ranking carries over to converged accuracy on larger datasets, the same set of width-curve mutations could be applied to ImageNet-scale backbones, where full architecture search is usually unaffordable; the paper does not test this.
  • Inference: the fluctuating width profiles suggest a separation between parameter efficiency and FLOPs efficiency, so a width profile that wins on accuracy-per-parameter may not win on accuracy-per-FLOP, which matters for deployment.
  • Inference: because the mutation machinery acts layer by layer, width profiles discovered for one backbone could be transplanted onto a differently connected backbone to test whether the gains transfer with the widths alone.
  • Inference: adding parameter count and FLOPs to the fitness function, as the paper says it plans to do, would let the search target a specific compute budget and would likely change which width curves win.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an efficient evolutionary search method, termed functionally incremental search, for discovering the number of channels at each convolutional layer of a network. The method uses function-preserving mutations (Net2Net-style) to explore a family of hand-designed width-profiles (exponential, linear, step-shaped, and their symmetric variants) while keeping the parameter count close to the original network. On CIFAR-10 and CIFAR-100, the authors report that the modified ResNet-18, ResNet-34, VGG-16, and SE-ResNet-50 achieve lower test error with fewer parameters than the original baselines, at a search cost of 0.4 to 1.3 GPU-days. Supplementary experiments on ResNet-18 and PyramidNet-110 suggest that 'fluctuating' channel widths can outperform both the conventional sharp increase at downsampling and the gradual pyramid-shaped increase. The paper claims this as evidence for rethinking the design rule for the number of channels.

Significance. If the reported results hold, the paper makes a useful empirical contribution: it demonstrates that a function-preserving evolutionary search over channel widths can improve the accuracy of several standard CNNs on CIFAR-10/100 while reducing parameter counts, with a modest search budget. The PyramidNet-110 transfer experiment is genuine independent support for the claim that fluctuating widths can generalize beyond the searched ResNet-18. The authors also provide code for the search and for some discovered architectures, which is a credit to reproducibility. However, the strength of the conclusions is limited by the lack of a check that the short-episode validation fitness used for selection correlates with final converged accuracy, by the absence of error bars for two of the searched networks, and by the omission of FLOPs, which are needed to support the 'efficiency' framing.

major comments (3)
  1. [§4.3–4.4] The search selects the individual with the highest validation accuracy after 15 epochs of training on a mutated network (§4.3), but the final comparison is obtained by retraining the selected architecture from scratch for 511 epochs with Cutout and lmax=0.1 (§4.4). The paper does not verify that the 15-epoch validation ranking correlates with converged test accuracy. Since the search intentionally stops when the parameter count approaches the baseline, the accuracy improvements in Table 1 could partly arise from selecting among many random mutations rather than from the width rule. Please provide a correlation analysis (e.g., a scatter plot of 15-epoch validation accuracy versus 511-epoch test error for a sample of visited architectures) and report error bars across multiple independent search runs.
  2. [Table 1 and §4.3] For VGG-16 and SE-ResNet-50, the text states that the search was performed only once, so the reported improvements of 0.52% and 0.43% are single draws. Without multiple seeds or a variance estimate, these gains cannot be distinguished from selection noise. The three rows for ResNet-18 provide some evidence of reproducibility, but the paper should either add multiple search runs for the other two networks or explicitly temper the claim that the method 'can discover more efficient rules.'
  3. [Table 1 and §4.5] The paper's efficiency claim rests on parameter counts and search time, but inference FLOPs are not reported. The modified ResNet-18 in Table 4 has roughly 200 channels in the first layer, compared with 64 in the original, so the computational cost of early layers could be substantially higher even though the total parameter count is lower. Since the abstract and conclusions describe the discovered widths as 'more efficient' and the authors themselves acknowledge in §4.5 that FLOPs are important, please report FLOPs for all architectures in Table 1 (and Table 3 if possible), or restrict the claim to parameter efficiency explicitly.
minor comments (6)
  1. [§4.3] The stopping criterion 'until the number of parameters of the individual with the highest fitness is similar to the comparison network' is not quantified; please state the tolerance used.
  2. [Table 2] The parameter count for the original ResNet-18 is 11.18M in Table 2 but 11.54M in Table 1; please reconcile this discrepancy or clarify that the two tables use different ResNet-18 variants.
  3. [Table 2 and §4.5] The row labels 'Increasing number of channels (Modified ResNet-18)' and 'Decreasing number of channels (Modified ResNet-18)' are ambiguous; the text says the opposite order was used, so state explicitly which row corresponds to the searched order and which to its reverse.
  4. [§4.1] Please specify the initial channel count used for each network (ResNet-34, VGG-16, SE-ResNet-50), not only for ResNet-18, to clarify the search-space construction.
  5. [Section 3.2] There are several typos: 'slop' should be 'slope', 'convoluaional' in Section 4.1 should be 'convolutional', and 'filed' in Section 2 should be 'field'. Please also check the author list for spacing issues.
  6. [Related Work] The paper does not compare the proposed search to prior automatic width-search methods such as DenseNAS, EfficientNet, or FBNet; a brief comparison or discussion would help position the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the search fitness and reported test accuracy are separate measurements, and the self-citations are not load-bearing.

full rationale

This is an empirical architecture-search paper, not a derivation, and its central claims do not reduce to their inputs by construction. The search selects width configurations using validation accuracy after 15 epochs of training on a mutated, function-preserving network (Sec. 4.3), while the reported Table 1 results are test errors after retraining the selected configurations on the full training set with Cutout and SGDR for 511 epochs (Sec. 4.4). The fitness and the reported quantity are therefore distinct measurements, so the accuracy improvement is not equal to the selection objective by definition. The stopping rule constrains parameter counts, but the accuracy comparison remains an independent empirical measurement. The "rules" of channel widths are post-hoc descriptions of the selected configurations, supported by supplementary comparisons in Tables 2 and 3, including a transfer to PyramidNet-110 that does not depend on the search that produced the ResNet widths. The paper cites the authors' own EENA work (Zhu et al. 2019) in related-work contexts, but the search procedure is specified in the present paper with equations (2)-(13), and the function-preserving machinery is attributed to Net2Net and network morphism, which are external sources; the self-citation is not load-bearing. The concern that the 15-epoch validation ranking may not correlate with converged test error is a correctness and generalization risk, not a circularity, because no equation or fitted parameter forces the converged test result to equal the search fitness.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

Everything the central claim rests on that the reader did not pay for upstream: fixed lambda, initial channel count, population and tournament sizes, the 15-epoch fitness proxy, and the hand-designed increment function family, plus assumptions about Net2Net and validation accuracy. These are mostly standard for NAS papers, but the fitness proxy and function family are not stress-tested.

free parameters (5)
  • rate of increase lambda = 0.2
    Fixed for all searches; controls how much channels can grow per mutation, thus defining the magnitude of all increment functions.
  • initial channel count = 32
    Set to about half the minimum channel count of ResNet-18; defines the starting point of the search and constrains the final widths.
  • population and tournament sizes = P1=12, P2=20, k=3
    Hand-selected evolutionary control parameters; no sensitivity analysis is given.
  • fitness training epochs = 15
    Individuals are trained for only 15 epochs before validation accuracy is used to rank candidates; this proxy is load-bearing and unvalidated.
  • increment function family = 9 functions (fa to fh plus lambda/2)
    The search space is defined by these hand-designed curves; if the best width rule is not expressible as their multiplicative composition, search cannot find it.
assumptions (4)
  • standard math Net2Net function-preserving widening operation (Eqs. 2 to 4) preserves network output up to symmetry-breaking noise.
    Inherited from prior work by Chen et al. 2016; the paper relies on it to evaluate mutated networks without full retraining.
  • domain assumption Validation accuracy on a 10k stratified split of the training set is a reliable reward for architecture search.
    Used throughout the evolution in Section 4.2; standard for NAS but not probed for bias.
  • domain assumption Fifteen epochs of SGDR training is enough to rank candidate width patterns by final converged accuracy.
    The entire selection is based on this proxy in Section 4.3; the paper provides no correlation study.
  • domain assumption Projection shortcuts used when channel width changes at every unit do not distort the search comparison.
    Discussed in the ResNet-18 implementation details; the paper adds one supporting experiment but does not fully analyze shortcut effects.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking the Number of Channels for the Convolutional Neural Network." pith.science (2026). https://pith.science/paper/XYA2IFMH

@misc{pith2026190901861,
  author       = {Pith},
  title        = {Pith review of: Rethinking the Number of Channels for the Convolutional Neural Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYA2IFMH}},
  note         = {Machine review of arXiv:1909.01861}
}
read the original abstract

Latest algorithms for automatic neural architecture search perform remarkable but few of them can effectively design the number of channels for convolutional neural networks and consume less computational efforts. In this paper, we propose a method for efficient automatic architecture search which is special to the widths of networks instead of the connections of neural architecture. Our method, functionally incremental search based on function-preserving, will explore the number of channels rapidly while controlling the number of parameters of the target network. On CIFAR-10 and CIFAR-100 classification, our method using minimal computational resources (0.4~1.3 GPU-days) can discover more efficient rules of the widths of networks to improve the accuracy by about 0.5% on CIFAR-10 and a~2.33% on CIFAR-100 with fewer number of parameters. In particular, our method is suitable for exploring the number of channels of almost any convolutional neural network rapidly.

Figures

Figures reproduced from arXiv: 1909.01861 by the authors.

Figure 1
Figure 1. Visualization of the Functionally incremental se [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Several complex rules of the width for the network [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. The schematic diagram visualized one search proce [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 27 canonical work pages

  1. [1]

    Cai, H.; Chen, T.; Zhang, W.; Yu, Y.; and Wang, J. 2017. Reinforcement learning for architecture search by network transformation. CoRR abs/1707.04873

    arXiv 2017

  2. [2]

    Cai, H.; Chen, T.; Zhang, W.; Yu, Y.; and Wang, J. 2018a. Efficient architecture search by network transformation. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018 , 2787--2794

    work page 2018

  3. [3]

    Cai, H.; Yang, J.; Zhang, W.; Han, S.; and Yu, Y. 2018b. Path-level network transformation for efficient architecture search. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm \" a ssan, Stockholm, Sweden, July 10-15, 2018 , 677--686

    work page 2018

  4. [4]

    Cai, H.; Zhu, L.; and Han, S. 2018. Proxylessnas: Direct neural architecture search on target task and hardware. CoRR abs/1812.00332

    arXiv 2018

  5. [5]

    J.; and Shlens, J

    Chen, T.; Goodfellow, I. J.; and Shlens, J. 2016. Net2net: Accelerating learning via knowledge transfer. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings

    work page 2016

  6. [6]

    Chollet, F. 2017. Xception: Deep learning with depthwise separable convolutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , 1800--1807

    work page 2017

  7. [7]

    Devries, T., and Taylor, G. W. 2017. Improved regularization of convolutional neural networks with cutout. CoRR abs/1708.04552

    arXiv 2017

  8. [8]

    Fang, J.; Sun, Y.; Zhang, Q.; Li, Y.; Liu, W.; and Wang, X. 2019. Densely connected search space for more flexible neural architecture search. CoRR abs/1906.09607

    arXiv 2019

Show all 33 references
  1. [9]

    E., and Deb, K

    Goldberg, D. E., and Deb, K. 1990. A comparative analysis of selection schemes used in genetic algorithms. In Proceedings of the First Workshop on Foundations of Genetic Algorithms. Bloomington Campus, Indiana, USA, July 15-18 1990. , 69--93

    1990

  2. [10]

    Han, D.; Kim, J.; and Kim, J. 2017. Deep pyramidal residual networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , 6307--6315

    2017

  3. [11]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016a. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 , 770--778

    2016

  4. [12]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016b. Identity mappings in deep residual networks. In Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV , 630--645

    2016

  5. [13]

    Hu, J.; Shen, L.; and Sun, G. 2018. Squeeze-and-excitation networks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018 , 7132--7141

    2018

  6. [14]

    Huang, G.; Liu, Z.; van der Maaten, L.; and Weinberger, K. Q. 2017. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , 2261--2269

    2017

  7. [15]

    Kandasamy, K.; Neiswanger, W.; Schneider, J.; P \' o czos, B.; and Xing, E. P. 2018. Neural architecture search with bayesian optimisation and optimal transport. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems...

    2018

  8. [16]

    Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held ...

    2012

  9. [17]

    L.; Huang, J.; and Murphy, K

    Liu, C.; Zoph, B.; Neumann, M.; Shlens, J.; Hua, W.; Li, L.; Fei - Fei, L.; Yuille, A. L.; Huang, J.; and Murphy, K. 2018a. Progressive neural architecture search. In Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Pa...

    2018

  10. [18]

    Liu, H.; Simonyan, K.; Vinyals, O.; Fernando, C.; and Kavukcuoglu, K. 2018b. Hierarchical representations for efficient architecture search. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track ...

    2018

  11. [19]

    Liu, H.; Simonyan, K.; and Yang, Y. 2018. DARTS: differentiable architecture search. CoRR abs/1806.09055

    2018 arXiv

  12. [20]

    Loshchilov, I., and Hutter, F. 2017. SGDR: stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings

    2017

  13. [21]

    Luo, R.; Tian, F.; Qin, T.; Chen, E.; and Liu, T. 2018. Neural architecture optimization. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montr \' e al, Canada. , 7827--7838

    2018

  14. [22]

    Y.; Zoph, B.; Le, Q

    Pham, H.; Guan, M. Y.; Zoph, B.; Le, Q. V.; and Dean, J. 2018. Efficient neural architecture search via parameter sharing. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm \" a ssan, Stockholm, Sweden, July 10-15, 2018 , 4092--4101

    2018

  15. [23]

    Real, E.; Aggarwal, A.; Huang, Y.; and Le, Q. V. 2018. Regularized evolution for image classifier architecture search. CoRR abs/1802.01548

    2018 arXiv

  16. [24]

    Simonyan, K., and Zisserman, A. 2015. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

    2015

  17. [25]

    Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; and Wojna, Z. 2016. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 , 2818--2826

    2016

  18. [26]

    Tan, M., and Le, Q. V. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , 6105--6114

    2019

  19. [27]

    Wei, T.; Wang, C.; Rui, Y.; and Chen, C. W. 2016. Network morphism. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016 , 564--572

    2016

  20. [28]

    Wistuba, M. 2018. Deep learning architecture search by neuro-cell-based evolution with function-preserving mutations. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Dublin, Ireland, September 10-14, 2018, Proceedings, Part II , 243--258

    2018

  21. [29]

    B.; Doll \' a r, P.; Tu, Z.; and He, K

    Xie, S.; Girshick, R. B.; Doll \' a r, P.; Tu, Z.; and He, K. 2017. Aggregated residual transformations for deep neural networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , 5987--5995

    2017

  22. [30]

    Zagoruyko, S., and Komodakis, N. 2016. Wide residual networks. In Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, September 19-22, 2016

    2016

  23. [31]

    Zhu, H.; An, Z.; Yang, C.; Xu, K.; and Xu, Y. 2019. EENA: efficient evolution of neural architecture. CoRR abs/1905.07320

    2019 arXiv

  24. [32]

    Zoph, B., and Le, Q. V. 2017. Neural architecture search with reinforcement learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings

    2017

  25. [33]

    Zoph, B.; Vasudevan, V.; Shlens, J.; and Le, Q. V. 2018. Learning transferable architectures for scalable image recognition. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018 , 8697--8710

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.