Pith. sign in

REVIEW 2 major objections 5 minor 24 references

Collaborative pruning of depth, width and resolution yields more efficient CNNs for embedded devices than single-dimension cuts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 05:20 UTC pith:VT3EQMEH

load-bearing objection Practical three-axis pruner with real board measurements that beats dual-axis SOTA on the accuracy-latency front; latency surrogate is the softest piece but not load-bearing. the 2 major comments →

arxiv 2607.11473 v1 pith:VT3EQMEH submitted 2026-07-13 cs.CV cs.LG

Towards Efficient Convolutional Neural Network for Embedded Hardware via Multi-Dimensional Pruning

Hao Kong , Di Liu , Xiangzhong Luo , Shuo Huai , Ravi Subramaniam , Christian Makaya , Qian Lin , Weichen Liu This is my paper
classification cs.CV cs.LG
keywords multi-dimensional pruningCNN compressionembedded hardwaredepth-width-resolutionlatency-aware pruningheuristic architecture descentImageNet
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern CNNs are too heavy for edge hardware. Most compression methods thin only one axis—layers, channels or input resolution—and therefore leave accuracy on the table. TECO instead prunes all three axes together. It first ranks units inside each axis by a cheap local score, then compares the three cheapest candidates with a single global score that balances accuracy loss against both MAC reduction and measured on-device latency. A short iterative loop repeatedly removes the lowest-scoring unit and fine-tunes. On ImageNet and CIFAR-10 the resulting models deliver higher top-1 accuracy at equal or lower compute and lower measured latency on Jetson Nano, TX2 and Xavier than prior single- or dual-dimension pruners.

Core claim

A two-stage importance framework (inner-dimensional local scores plus an inter-dimensional global score that jointly accounts for accuracy, MACs and device latency) enables a heuristic architecture-descent procedure that collaboratively prunes depth, width and resolution, producing networks that dominate existing single- and dual-dimension SOTA methods on both ImageNet accuracy and real embedded latency.

What carries the argument

The global-importance metric I(u)=A(u)/(αM(u)+(1-α)T(u)), where A is accuracy loss, M is MAC reduction and T is latency reduction estimated by a dimension-wise linear model; this metric, computed only on the three candidates pre-selected by local gate-gradient scores, drives the progressive pruning decisions.

Load-bearing premise

A linear latency-versus-MACs model fitted on ResNet-50 samples from one device family remains accurate enough for other residual networks and power budgets that the global ranking still picks the true best unit to prune.

What would settle it

Measure end-to-end latency of TECO-pruned models versus the same models re-pruned by exhaustive single-dimension search on a non-residual architecture or a different power envelope; if the TECO models are no longer faster at equal accuracy, the linear latency model has failed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Edge CNNs can be made simultaneously smaller and more accurate by jointly shrinking layers, channels and input size rather than any single axis.
  • On-device latency, not just FLOPs, can be folded into the pruning criterion via a cheap linear model, so the search directly optimizes wall-clock time.
  • The same two-stage scoring loop can be reused for any residual backbone once the three latency coefficients are re-fitted.
  • Even already-compact models (ResNet-110 on CIFAR-10) can gain both speed and accuracy once residual redundancy across all three dimensions is removed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same global-importance idea could be extended to include memory-bandwidth or energy if those quantities also admit simple linear predictors.
  • Because resolution pruning is performed uniformly rather than per-image, the method may still leave spatial redundancy that a learned cropping policy could reclaim.
  • If the linear latency model is replaced by a small neural surrogate trained on the target SoC, the framework could transfer to non-residual or transformer-based vision models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces TECO, a multi-dimensional pruning framework that jointly compresses the depth, width, and resolution of CNNs for embedded hardware. It defines a two-stage importance evaluation: INES computes local importance of units (layers/channels) inside each dimension via squared gradients of fully-gated residual blocks (Eqs. 7-8), while ITES ranks the three candidate units by a global importance score I(u) that combines accuracy impact A(u) (cross-entropy increase), MACs reduction M(u), and estimated latency reduction T(u) derived from a dimension-wise linear model (Eqs. 2-6). A heuristic architecture-descent algorithm then iteratively prunes the lowest-I(u) unit, fine-tunes for one epoch, and finally trains the resulting architecture from scratch. Experiments on ImageNet (ResNet-50) and CIFAR-10 (ResNet-110) report higher top-1 accuracy at equal or lower MACs and lower measured latency on Jetson Nano/TX2/Xavier than single- or dual-dimension SOTA methods (HRank, GAL, Taylor, DECORE, etc.), supported by ablation of INES/ITES and CAM visualizations.

Significance. If the empirical trade-offs hold, TECO advances practical edge-CNN compression by demonstrating that coordinated three-dimension pruning, guided by a cheap two-stage importance metric and progressive descent, consistently outperforms prior single- or dual-dimension pruners on both accuracy-MACs and measured on-device latency. Strengths include open-source code and pretrained models, multi-platform latency measurements under realistic power budgets, component ablation (Table III), and interpretability analysis via class-activation maps. The work is of clear interest to the embedded-vision and efficient-DNN communities; the latency-aware global score and fully-gated block are reusable building blocks even if later refined.

major comments (2)
  1. [III-A] Section III-A, Eqs. (4)-(5) and Fig. 3: The dimension-wise affine latency model l_s = a_s · m + c_s is fitted exclusively on ResNet-50 samples (one device family). While the reported MSEs are low, the paper asserts without further evidence that the same slopes remain accurate for other residual CNNs and power modes. Because T(u) = a_s M(u) directly enters the global importance I(u) that selects which unit to prune each iteration, imperfect a_s can systematically bias the ranking. A short sensitivity study (re-fitting a_s on a second architecture or power budget and re-running a few pruning trajectories) would confirm that the final measured superiority is robust to this modeling choice.
  2. [IV] Tables I-II and Figs. 1/5: All accuracy and latency numbers are single-run point estimates with no error bars, multiple random seeds, or statistical tests. Several claimed gains are small (e.g., TECO-L 76.34 % vs. DECORE-8 76.31 %; TECO-S vs. GAL-1). Without variance estimates it is impossible to judge whether the ranking against SOTA is stable. Reporting mean ± std over at least three independent pruning+retraining runs for the main TECO variants would make the superiority claims load-bearing rather than anecdotal.
minor comments (5)
  1. [Abstract] Abstract and Sec. I claim “optimal trade-off”; the algorithm is a greedy heuristic (Alg. 1). Soften the wording to “improved” or “near-optimal under the chosen importance metric.”
  2. [III-B] Fig. 2 caption and Sec. III-B: the decision to skip INES for resolution is justified only by computational cost; a one-sentence note that uniform down-sampling is used (rather than per-image selection) would clarify the design choice.
  3. [III-A] Hyper-parameter α is fixed at 0.5 with no sensitivity plot. A brief ablation of α ∈ {0, 0.5, 1} on one ImageNet setting would show how the MACs-versus-latency trade-off behaves.
  4. [IV-D] Table III “Cost (hour)” column: clarify whether the reported times include the final from-scratch training or only the iterative pruning+fine-tune loop.
  5. Minor typos: “Sampleed data” in Fig. 3; “Xaiver” in Sec. IV-B; “coin this algorithm” → “coin” should be “call” (Sec. III-C).

Circularity Check

0 steps flagged

No significant circularity; INES/ITES importance scores and the fitted latency surrogate guide pruning decisions only, while reported accuracy/latency gains are independently measured after from-scratch training.

full rationale

The paper's derivation chain is self-contained and empirical. Local importance (Eqs. 7-8) is the squared gradient of introduced gates; global importance (Eq. 6) simply normalizes accuracy impact A(u) by a linear combination of MACs reduction M(u) and estimated latency reduction T(u). The dimension-wise latency model (Eq. 4, fitted by least-squares on ResNet-50 samples, Fig. 3) is used solely inside ITES to rank the three candidate units each iteration of the heuristic descent (Alg. 1); it is never presented as a prediction of the final reported numbers. Final top-1 accuracy, MACs and on-device latency (Tables I-II, Figs. 1/5, ablation Table III) are obtained by training the pruned architectures from scratch and measuring them directly on the target platforms. Minor self-citations of the authors' prior edge-AI surveys or tools appear only in the introduction/related-work and do not underwrite any uniqueness claim or load-bearing equation. No equation reduces a claimed accuracy or latency gain to a quantity that was fitted into the importance score itself. Hence the strongest claim (superior measured trade-offs versus HRank/GAL/etc.) stands independently of the internal ranking surrogate.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 3 invented entities

The method rests on a handful of free hyper-parameters (α, number of gradient images, fine-tune epochs) and on the modeling assumptions that gate gradients rank units and that latency is linear in MACs per dimension. The invented scoring constructs (INES, ITES, global importance) are defined inside the paper and evaluated empirically; they do not introduce new physical entities.

free parameters (4)
  • α (trade-off between MACs and latency in global importance) = 0.5
    Empirically fixed at 0.5; controls the ranking that decides which dimension is pruned.
  • latency-model slopes a_s and intercepts c_s = a_d=2.31, a_w=1.78, a_r=2.49 (and corresponding c_s)
    Least-squares fit on sampled ResNet-50 MACs-vs-latency points for each dimension; used inside T(u).
  • number of images for INES gradient estimation = 5000
    Chosen as 5 000 after informal accuracy-vs-cost check; directly affects local-importance reliability.
  • pruning iterations n and fine-tune schedule = n≈10, lr=1e-4
    n set by target budget (≈10); 1-epoch fine-tune at 1e-4 after each prune; both affect final accuracy.
axioms (3)
  • domain assumption Squared gradient of a channel/layer gate approximates the unit's contribution to loss (local importance).
    Imported from Taylor pruning literature and extended to layers; used as the sole ranking signal inside each dimension (Eqs. 7-8).
  • ad hoc to paper On-device latency is an affine function of MACs within each pruning dimension.
    Justified by low MSE on ResNet-50 samples (Fig. 3); enables cheap T(u) without continuous hardware measurement.
  • domain assumption Uniform resolution shrinking is a valid proxy for resolution pruning (no per-image selection).
    Stated as necessary for tractability on ImageNet-scale data; skips INES for the resolution axis.
invented entities (3)
  • global importance I(u) = A(u) / (α M(u) + (1-α) T(u)) no independent evidence
    purpose: Single scalar that lets units of different granularities (layer, channel, resolution) be compared for pruning.
    Defined in Eq. 6; no independent existence outside the ranking procedure.
  • fully gated residual bottleneck (channel gates + layer gate) no independent evidence
    purpose: Enables simultaneous gradient collection for all width and depth units in one back-prop.
    Architectural device introduced in Fig. 4 solely to support INES.
  • heuristic architecture descent no independent evidence
    purpose: Progressive search that repeatedly applies INES then ITES instead of combinatorial NAS.
    Algorithm 1; claimed to find good trade-offs efficiently but not proven optimal.

pith-pipeline@v1.1.0-grok45 · 15903 in / 2745 out tokens · 29813 ms · 2026-07-14T05:20:47.829557+00:00 · methodology

0 comments
read the original abstract

In this paper, we propose TECO, a multi-dimensional pruning framework to collaboratively prune the three dimensions (depth, width, and resolution) of convolutional neural networks (CNNs) for better execution efficiency on embedded hardware. In TECO, we first introduce a two-stage importance evaluation framework, which efficiently and comprehensively evaluates each pruning unit according to both the local importance inside each dimension and the global importance across different dimensions. Based on the evaluation framework, we present a heuristic pruning algorithm to progressively prune the three dimensions of CNNs towards the optimal trade-off between accuracy and efficiency. Experiments on multiple benchmarks validate the advantages of TECO over existing state-of-the-art (SOTA) approaches. The code and pre-trained models are available at https://github.com/ntuliuteam/Teco.

Figures

Figures reproduced from arXiv: 2607.11473 by Christian Makaya, Di Liu, Hao Kong, Qian Lin, Ravi Subramaniam, Shuo Huai, Weichen Liu, Xiangzhong Luo.

Figure 1
Figure 1. Figure 1: Comparison of different methods in terms of model MACs, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overview of TECO, where INES evaluates the local importance of units inside each dimension and ITES evaluates the global [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Latency distributions obtained by separately pruning the three [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of inference latency on Jetson Nano and Jetson [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The class activation map (CAM) for different pruning methods. [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 1 linked inside Pith

  1. [1]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778

    2016

  2. [2]

    Bringing ai to edge: From deep learning’s perspective,

    D. Liu, H. Kong, X. Luo, W. Liu, and R. Subramaniam, “Bringing ai to edge: From deep learning’s perspective,”Neurocomputing, vol. 485, pp. 297–320, 2022

    2022

  3. [3]

    Edge computing: Vision and challenges,

    W. Shi, J. Cao, Q. Zhang, Y . Li, and L. Xu, “Edge computing: Vision and challenges,”IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637–646, 2016

    2016

  4. [4]

    Edlab: A benchmark for edge deep learning accelerators,

    H. Kong, S. Huai, D. Liu, L. Zhang, H. Chen, S. Zhu, S. Li, W. Liu, M. Rastogi, R. Subramaniamet al., “Edlab: A benchmark for edge deep learning accelerators,”IEEE Design and Test, 2021

    2021

  5. [5]

    Data-driven sparse structure selection for deep neural networks,

    Z. Huang and N. Wang, “Data-driven sparse structure selection for deep neural networks,” inProceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 304–320

    2018

  6. [6]

    Learning versatile filters for efficient convolutional neural networks,

    Y . Wang, C. Xu, C. Xu, C. Xu, and D. Tao, “Learning versatile filters for efficient convolutional neural networks,”Advances in Neural Information Processing Systems (NeurIPS), vol. 31, 2018

    2018

  7. [7]

    Importance estimation for neural network pruning,

    P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz, “Importance estimation for neural network pruning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 11 264–11 272

    2019

  8. [8]

    Hrank: Filter pruning using high-rank feature map,

    M. Lin, R. Ji, Y . Wang, Y . Zhang, B. Zhang, Y . Tian, and L. Shao, “Hrank: Filter pruning using high-rank feature map,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 1529–1538

    2020

  9. [9]

    Towards optimal structured cnn pruning via generative adversarial learning,

    S. Lin, R. Ji, C. Yan, B. Zhang, L. Cao, Q. Ye, F. Huang, and D. Doermann, “Towards optimal structured cnn pruning via generative adversarial learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2790–2799

    2019

  10. [10]

    Decore: Deep compression with reinforcement learning,

    M. Alwani, Y . Wang, and V . Madhavan, “Decore: Deep compression with reinforcement learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 12 349– 12 359

    2022

  11. [11]

    Hessian-aware pruning and optimal neural implant,

    S. Yu, Z. Yao, A. Gholami, Z. Dong, S. Kim, M. W. Mahoney, and K. Keutzer, “Hessian-aware pruning and optimal neural implant,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 3880–3891

    2022

  12. [12]

    Dnr: A tunable robust pruning framework through dynamic network rewiring of dnns,

    S. Kundu, M. Nazemi, P. A. Beerel, and M. Pedram, “Dnr: A tunable robust pruning framework through dynamic network rewiring of dnns,” inProceedings of the 26th Asia and South Pacific Design Automation Conference (ASP-DAC), 2021, pp. 344–350

    2021

  13. [13]

    Acceleration-aware fine-grained channel pruning for deep neural networks via residual gating,

    K. Huang, S. Chen, B. Li, L. Claesen, H. Yao, J. Chen, X. Jiang, Z. Liu, and D. Xiong, “Acceleration-aware fine-grained channel pruning for deep neural networks via residual gating,”IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems (TCAD), vol. 41, no. 6, pp. 1902–1915, 2021

    1902

  14. [14]

    Dbp: Discrimination based block-level pruning for deep model acceleration,

    W. Wang, S. Zhao, M. Chen, J. Hu, D. Cai, and H. Liu, “Dbp: Discrimination based block-level pruning for deep model acceleration,” arXiv preprint arXiv:1912.10178, 2019

    Pith/arXiv arXiv 1912

  15. [15]

    Learning structured sparsity in deep neural networks,

    W. Wen, C. Wu, Y . Wang, Y . Chen, and H. Li, “Learning structured sparsity in deep neural networks,”Advances in Neural Information Processing Systems (NeurIPS), vol. 29, 2016

    2016

  16. [16]

    Dy- namic resolution network,

    M. Zhu, K. Han, E. Wu, Q. Zhang, Y . Nie, Z. Lan, and Y . Wang, “Dy- namic resolution network,”Advances in Neural Information Processing Systems (NeurIPS), vol. 34, pp. 27 319–27 330, 2021

    2021

  17. [17]

    Resolu- tion adaptive networks for efficient inference,

    L. Yang, Y . Han, X. Chen, S. Song, J. Dai, and G. Huang, “Resolu- tion adaptive networks for efficient inference,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 2369–2378

    2020

  18. [18]

    Smart scissor: Coupling spatial redundancy reduction and cnn compression for embedded hardware,

    H. Kong, D. Liu, S. Huai, X. Luo, W. Liu, R. Subramaniam, C. Makaya, and Q. Lin, “Smart scissor: Coupling spatial redundancy reduction and cnn compression for embedded hardware,” inProceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2022

    2022

  19. [19]

    Provable filter pruning for efficient neural networks,

    L. Liebenwein, C. Baykal, H. Lang, D. Feldman, and D. Rus, “Provable filter pruning for efficient neural networks,” inInternational Conference on Learning Representations (ICLR), 2020

    2020

  20. [20]

    Mobilenetv2: Inverted residuals and linear bottlenecks,

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4510–4520

    2018

  21. [21]

    Zerobn: Learning compact neural networks for latency-critical edge systems,

    S. Huai, L. Zhang, D. Liu, W. Liu, and R. Subramaniam, “Zerobn: Learning compact neural networks for latency-critical edge systems,” in 2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, 2021, pp. 151–156

    2021

  22. [22]

    Centripetal sgd for pruning very deep convolutional networks with complicated structure,

    X. Ding, G. Ding, Y . Guo, and J. Han, “Centripetal sgd for pruning very deep convolutional networks with complicated structure,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 4943–4953

    2019

  23. [23]

    SGDR: Stochastic gradient descent with warm restarts,

    I. Loshchilov and F. Hutter, “SGDR: Stochastic gradient descent with warm restarts,” inInternational Conference on Learning Representations (ICLR), 2017

    2017

  24. [24]

    Grad-cam: Visual explanations from deep networks via gradient- based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Ba- tra, “Grad-cam: Visual explanations from deep networks via gradient- based localization,” inProceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), 2017, pp. 618–626

    2017