Pith. sign in

REVIEW 4 major objections 5 minor 44 references

Model compression using knowledge distillation with integrated gradients

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that overlaying a teacher's integrated-gradient maps onto a fraction of training images improves knowledge distillation for compressed models, lifting CIFAR-10 accuracy at 4.1x compression from 91.5% to 92.6% with p<0.001.

desk verdict A careful, reproducible KD augmentation result whose main mechanistic claim is not yet supported by its ablations—worth refereeing, but the authors need a noise/edge-map control and less selective reporting. read the letter →

arxiv 2506.14440 v1 pith:UN6PLW6N submitted 2025-06-17 cs.CV cs.AIcs.LG

David E. Hernandez , Jose Chang , Torbjörn E. M. Nordling This is my paper
classification cs.CVcs.AIcs.LG
keywords modelcompressionknowledgedistillationintegratedgradientsdeeplearningexplainableAIattentiontransferdataaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that knowledge distillation can be improved by overlaying the teacher's integrated-gradient maps onto a fraction of the student's training images, so the small model is repeatedly shown which pixels the big model actually uses to decide. At 4.1x compression on CIFAR-10, adding this overlay to KD raises test accuracy from 91.5% to 92.6% (p<0.001), a gain that also appears in Monte Carlo simulations over random 80% data splits and on a curated ImageNet subset. The authors argue that precomputing the attribution maps makes this a one-time cost, and that the benefit is concentrated in the 4-30x compression range most relevant for edge deployment. If correct, the paper establishes that attribution maps, tools usually built for explanation, can double as a cheap training signal that improves compressed-model accuracy while retaining interpretability.

What carries the argument

The load-bearing mechanism is the IG overlay operator defined by Equations 3-6: integrated gradients of the teacher with respect to the true-class logit, summed across channels, scaled by a log-uniform factor s in [1,2], min-max normalized to [0,1], and blended with the original image as x_augmented = 0.5x + 0.5*IG_hat with probability p = 0.1. This turns an explanation tool into a data augmentation: the student never has to imitate attributions in a loss, it only has to classify images in which the teacher-highlighted pixels are brightened. The scheme is coupled to the standard KD objective (temperature T = 2.5, weight alpha = 0.01) and optionally to an attention-transfer loss. The key design choice is sparsity: applying the overlay to only 10% of training images balances feature emphasis against preserving the natural image distribution.

What would settle it

Train the same 4.1x student with the same KD objective but replace the IG overlay with a control overlay, such as random Gaussian noise or an edge map, normalized and blended identically at p = 0.1; if the control reaches 92.6% on CIFAR-10 or matches KD & IG within noise, the specific contribution of integrated gradients is falsified, and the same test should be run on the filtered ImageNet subset to check whether the claimed generalization is overlay-specific.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that using the teacher's integrated gradients as a data-augmentation overlay, rather than as an extra loss term, is the most effective way found to compress a MobileNetV2 teacher into a 4.1x smaller student: KD & IG reaches 92.6% test accuracy on CIFAR-10, the highest of all ablations, with a statistically significant 1.1 percentage-point improvement over the non-distilled student (91.5%) and a relative improvement of 44.8% in the teacher-student accuracy gap. The overlay is applied sparsely, to 10% of training images, blended as 0.5 times the original image plus 0.5 times the min-max normalized integrated-gradient map, with a log-uniform scale factor in [1,2], and the IG maps are precomputed before training. The same configuration also leads on a filtered ImageNet subset aligned with CIFAR-10 classes (85.7% vs 83.8% for the student baseline), and the advantage persists across the moderate compression range (2.2x-12x) but vanishes at extreme compression, where all methods converge. The authors further report that adding attention transfer on top of KD & IG slightly reduces accuracy (92.4%), suggesting some competition between the two guidance signals.

Load-bearing premise

The paper attributes the accuracy gain specifically to the information in the teacher's integrated-gradient maps, but its ablations never overlay a control signal such as random noise, edge maps, or another attribution method at the same probability; if any attention-like overlay produces the same gain, the causal role of integrated gradients is unsupported and the method reduces to generic input augmentation.

Editorial extensions

If this is right

  • At 4.1x compression, KD & IG achieves 92.6% test accuracy on CIFAR-10, a statistically significant 1.1-point gain over the student baseline, and preserves 98.6% of teacher accuracy.
  • Inference time drops from 140 ms to about 13 ms per batch at 4.12x compression, a speedup that exceeds the compression ratio.
  • The advantage transfers to a filtered ImageNet subset (85.7% vs 83.8%), indicating the learned focus is not dataset-specific.
  • The benefit is concentrated in the moderate compression range (2.2x-12x); beyond about 28x all methods converge as capacity limits dominate.
  • Precomputing IG maps before training, around 2 hours for CIFAR-10, replaces what would otherwise be per-epoch attribution computation, making the overhead a one-time cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A control experiment is the immediate next test: overlaying random noise, edge maps, or a different attribution method (e.g., Grad-CAM) at p = 0.1 would reveal whether the gain comes from IG's specific information or from any saliency-like input perturbation; the paper does not run this control.
  • If the gain survives control overlays, the method's contribution is better described as input-space regularization rather than distillation of attributions, and the claimed link to teacher decision-making would need to be weakened.
  • The same precomputed-overlay trick could transfer to other compression settings, such as quantization-aware training, pruning fine-tuning, or large-model distillation, where one shared teacher annotates a dataset once and many small models reuse the maps.
  • The reported benefit is small in absolute terms, 1.1 points; whether that margin justifies the two-hour precomputation and the extra hyperparameter p depends on the deployment budget, where even a one-point gain on a four-times-smaller model can matter on edge hardware.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes augmenting knowledge distillation with teacher integrated-gradient (IG) maps overlaid on a fraction of training images, so that the student receives 'deeper insights' into the teacher's decision process. The method is tested on CIFAR-10 with a MobileNetV2 teacher and layer-pruned MobileNetV2 students, reporting 92.6% test accuracy at 4.1x compression versus 91.5% for the non-distilled student, with paired t-tests and 60-run Monte Carlo simulations. Additional experiments include attention transfer combinations, a compression-factor sweep up to 1122x, and a cross-dataset evaluation on an ImageNet subset.

Significance. If the reported effect is causal, the contribution is modest but useful: a simple, precomputable input-space augmentation that adds roughly 0.8-1.1 percentage points to KD-compressed students in the moderate compression range, with careful attention to statistical design. The paper's strengths are its reproducible code repository, explicit hyperparameter search, detailed accounting of precomputation costs, and the 60-run Monte Carlo analysis. However, the central attribution of the gain to the information content of IG is not yet established, and the ImageNet generalisation evidence is weakened by restricting evaluation to teacher-correct samples. With a control-overlay experiment and corrected evaluation reporting, the paper would be suitable for publication.

major comments (4)
  1. [§2.3, Eq. (6); Table 4] The causal claim that teacher IG maps drive the +1.1pp gain is untested. Eq. (6) is a fixed-weight convex blend, and Table 4 only compares IG-present vs IG-absent configurations; no condition replaces the IG map with a content-free control (e.g., scaled Gaussian noise), a structural non-attribution signal (edge maps), or another attribution method (e.g., Grad-CAM) at the same p=0.1 and the same 0.5/0.5 blend weights. Without such a control, the improvement over the Student baseline in Table 4 and the Monte Carlo means in Table 5 (91.29% vs 90.05%) remain compatible with a null IG-specific effect, i.e., the method could reduce to generic input mixing. Please add the control conditions and report mean ± std for each.
  2. [§3.1.2; Table 6] The ImageNet evaluation set is the 3,537 samples that the teacher classifies correctly out of 5,250 (Table S14); the teacher 'baseline' of 100.00 in Table 6 is therefore true by construction, and the student scores in §4.5 are conditional on teacher correctness. This biases the cross-dataset comparison in favour of teacher-aligned inputs and makes the 'strong performance relative to the 100% baseline' statement in §4.5 non-informative. Please report accuracy on the full 5,250-image subset, with per-class results for the eight classes that have samples, and state the filtering effect explicitly in the main text.
  3. [Abstract; Table 4] The headline 92.6% and 91.5% are maxima over 10 runs, while Table 4 lists means of 92.02% and 91.18%; the paired t-test p<0.001 applies to the runs, but the '1.1 percentage point improvement' in the Abstract is a comparison of maxima, not of the mean difference (0.84pp on CIFAR-10 with 100% training data). Please report the mean difference and confidence interval in the Abstract and main text, or explicitly frame the headline as best-of-10. The Monte Carlo mean difference of 1.24pp uses 80% training data and should not be mixed with the 100%-data CIFAR-10 comparison.
  4. [Abstract; Table S15; §4.3] The claim that IG-based KD 'consistently outperforms conventional approaches across varied architectures and compression ratios' is contradicted by the paper's own Table S15: at 54.59x, IG (85.20) exceeds KD & IG (84.84); at 139.43x, both KD (79.13) and IG (78.47) exceed KD & IG (78.24); and at 1121.71x, KD & IG (55.03) is the worst configuration, below Student (55.48). Since §4.3 states each compression point is a single run, the 'consistent performance advantage' in Figure 5 is not statistically supported. Please restrict conclusions to the 4-30x range or provide multi-run statistics.
minor comments (5)
  1. [Tables S13-S14; §3.1.2] The ImageNet subset contains no samples for Deer and Horse (both mapped to 'None'), so it covers eight rather than ten CIFAR-10 classes; the text 'aligned with CIFAR-10 classes' should be qualified accordingly.
  2. [Figure S4; Table 6] The CIFAR-10 and ImageNet accuracies for Student (91.43 vs 91.50) and KD & IG (92.45 vs 92.58) differ between the figure and the table; if these come from different runs, this should be stated explicitly.
  3. [§4.2] The phrase 'highest relative improvement of 44.8%' is undefined; specify the formula, for example (92.58-91.50)/(93.91-91.50), so the reader can reproduce the number.
  4. [§4.1.2; Table S11] The text states that p=0.09 provides 'insufficient guidance', but Table S11 shows p=0.09 at 92.28%, only 0.30pp below p=0.1; with 10 runs per configuration, this difference requires a standard error or significance test before drawing that conclusion.
  5. [Abstract; Table S6] The abstract's 'varied architectures' overstates the experimental scope: all student models are MobileNetV2 with different numbers of removed blocks, not architecturally distinct model families.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are empirical evaluations on held-out test data, with no fitted parameter renamed as a prediction and no load-bearing self-citation chain.

full rationale

The paper's methodology and results are empirical rather than derivational. Section 2.3 defines IG augmentation through Eqs. (3)-(6), but the reported accuracies in Table 4 and the Monte Carlo results of Section 4.4 are measured outcomes of training procedures, not quantities obtained by algebraic substitution. Hyperparameters are stated to be selected on a validation split before final retraining and test evaluation (Section 3.1.1), so the main +1.1 pp result is not defined in terms of the method's own fitted parameters. The only self-referential element is that IG maps are computed from the same teacher that supplies the distillation soft labels, which is standard in knowledge distillation and does not make the student's test accuracy a restatement of the teacher's output. No external or self-cited uniqueness theorem is invoked to force the method choice, and no known empirical pattern is renamed as a new result. The absence of a control overlay (e.g., noise or Grad-CAM) in Section 4.2 weakens causal attribution of the gain to IG-specific information, but this is a validity concern, not circularity: the paper does not reduce the alleged IG effect to the overlay operation by construction. The code repository and fixed benchmarks make the reported numbers externally checkable. Overall, no circular step is exhibited.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on five fitted hyperparameters (alpha, T, p, s range, gamma), none of which are derived from theory, plus standard assumptions about IG attribution quality, validation-split representativeness, attention-map comparability, and teacher-filtered ImageNet validity. No new entities are introduced. The free parameters are all tuned on a validation split, so the reported test accuracy inherits the usual selection risk.

free parameters (6)
  • alpha (KD loss weight) = 0.01
    Chosen by grid search on a 20% validation split (Table S10); weights the KL distillation term in Eq. 1.
  • T (distillation temperature) = 2.5
    Chosen by grid search (Table S10); softens teacher logits in Eq. 1.
  • p (IG overlay probability) = 0.1
    Chosen by grid search (Table S11); fraction of training images blended with IG maps in Eq. 6.
  • s (IG scale factor range) = [1,2] log-uniform
    Selected empirically in Section 2.3; scales IG maps in Eq. 4 before normalization.
  • gamma (AT loss weight) = 0.8
    Chosen by grid search (Table S12); weight of attention-transfer loss in Eq. 2.
  • Blending weights (0.5, 0.5) = 0.5/0.5
    Fixed in Eq. 6; not tuned, but determines the visible strength of the IG overlay.
assumptions (5)
  • domain assumption Integrated gradients with a zero baseline and true-class label provide meaningful pixel attributions for the teacher model.
    Invoked in Section 2.3 and ESM Section 1.3.3; if attributions are noisy or misaligned, the overlay guidance would not help or could hurt.
  • domain assumption The 20% validation split of CIFAR-10 is representative for selecting KD/IG/AT hyperparameters.
    Section 3.1.1; hyperparameters chosen on this split are applied to full-data retraining and test evaluation.
  • domain assumption The attention maps from the selected middle layers (e.g., block 9) are comparable across teacher and student architectures after L2 normalization.
    Section 2.2 and ESM Table S6; attention transfer loss (Eq. 2) requires dimensionally compatible maps, and the authors move the attention source earlier for highly compressed students.
  • ad hoc to paper The teacher-filtered ImageNet subset is a valid evaluation set for measuring compression effects.
    Section 3.1.2; only teacher-correct images are retained (3537 of 5250), and classes Deer and Horse are absent, so the subset is not representative of the original classes.
  • domain assumption Standard deep learning training (Adam, lr=0.001, 100 epochs) is sufficient for convergence of all student configurations.
    Section 3.1 and ESM; no learning-rate schedule or early stopping is described, and convergence is assumed from the epoch count.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Model compression using knowledge distillation with integrated gradients." pith.science (2026). https://pith.science/paper/UN6PLW6N

@misc{pith2026250614440,
  author       = {Pith},
  title        = {Pith review of: Model compression using knowledge distillation with integrated gradients},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UN6PLW6N}},
  note         = {Machine review of arXiv:2506.14440}
}
abstract

Model compression is critical for deploying deep learning models on resource-constrained devices. We introduce a novel method enhancing knowledge distillation with integrated gradients (IG) as a data augmentation strategy. Our approach overlays IG maps onto input images during training, providing student models with deeper insights into teacher models' decision-making processes. Extensive evaluation on CIFAR-10 demonstrates that our IG-augmented knowledge distillation achieves 92.6% testing accuracy with a 4.1x compression factor-a significant 1.1 percentage point improvement ($p<0.001$) over non-distilled models (91.5%). This compression reduces inference time from 140 ms to 13 ms. Our method precomputes IG maps before training, transforming substantial runtime costs into a one-time preprocessing step. Our comprehensive experiments include: (1) comparisons with attention transfer, revealing complementary benefits when combined with our approach; (2) Monte Carlo simulations confirming statistical robustness; (3) systematic evaluation of compression factor versus accuracy trade-offs across a wide range (2.2x-1122x); and (4) validation on an ImageNet subset aligned with CIFAR-10 classes, demonstrating generalisability beyond the initial dataset. These extensive ablation studies confirm that IG-based knowledge distillation consistently outperforms conventional approaches across varied architectures and compression ratios. Our results establish this framework as a viable compression technique for real-world deployment on edge devices while maintaining competitive accuracy.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 36 canonical work pages

  1. [1]

    arXiv preprint arXiv:170906030

    Ashok A, Rhinehart N, Beainy F, et al (2017) N2n learning: Network to network compression via policy gradient reinforcement learning. arXiv preprint arXiv:170906030

    work page 2017

  2. [2]

    ACM Transactions on Embedded Computing Systems 18:1--22

    Bhardwaj K, Lin CY, Sartor A, et al (2019) Memory- and communication-aware model compression for distributed deep learning inference on iot. ACM Transactions on Embedded Computing Systems 18:1--22. doi:10.1145/3358205

    work page doi:10.1145/3358205 2019

  3. [3]

    Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression

    Blakeney C, Li X, Yan Y, et al (2020) Parallel blockwise knowledge distillation for deep neural network compression. arXiv preprint arXiv201203096 doi:10.48550/arXiv.2012.03096

    work page Pith review arXiv doi:10.48550/arxiv.2012.03096 2020

  4. [4]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11933--11942

    Chen D, Mei JP, Zhang H, et al (2022) Knowledge distillation with the reused teacher classifier. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11933--11942

    work page 2022

  5. [5]

    In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3514--3522

    Chen H, Wang Y, Xu C, et al (2019 a ) Data-free learning of student networks. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3514--3522

    work page 2019

  6. [6]

    In: Computer Vision--ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2--6, 2018, Revised Selected Papers, Part III 14, Springer, pp 200--215

    Chen WC, Chang CC, Lee CR (2019 b ) Knowledge distillation with feature maps for image classification. In: Computer Vision--ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2--6, 2018, Revised Selected Papers, Part III 14, Springer, pp 200--215

    work page 2019

  7. [7]

    In: Proceedings of the AAAI conference on artificial intelligence

    Chen Y, Wang N, Zhang Z (2018) Darkrank: Accelerating deep metric learning via cross sample similarities transfer. In: Proceedings of the AAAI conference on artificial intelligence

    work page 2018

  8. [8]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 710--711

    Choi Y, Choi J, El-Khamy M, et al (2020) Data-free network quantization with adversarial knowledge distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 710--711

    work page 2020

Show all 44 references
  1. [9]

    IEEE Transactions on Industrial Informatics 19(5):7099--7109

    Gou J, Sun L, Yu B, et al (2022) Multilevel attention-based sample correlations for knowledge distillation. IEEE Transactions on Industrial Informatics 19(5):7099--7109

    2022

  2. [10]

    ACM Transactions on Multimedia Computing, Communications and Applications 20(2):1--20

    Gou J, Sun L, Yu B, et al (2023) Hierarchical multi-attention transfer for knowledge distillation. ACM Transactions on Multimedia Computing, Communications and Applications 20(2):1--20

    2023

  3. [11]

    Applied Soft Computing 159:111579

    Hossain MI, Akhter S, Hong CS, et al (2024) Purf: Improving teacher representations by imposing smoothness constraints for knowledge distillation. Applied Soft Computing 159:111579

    2024

  4. [12]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510--4520

    Sandler M, Howard A, Zhu M, et al (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510--4520

    2018

  5. [13]

    IEEE Transactions on Neural Networks and Learning Systems 34(12):10051--10064

    Su T, Zhang J, Yu Z, et al (2022) Stkd: Distilling knowledge from synchronous teaching for efficient model compression. IEEE Transactions on Neural Networks and Learning Systems 34(12):10051--10064

    2022

  6. [14]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 1190--1197

    Wang J, Bao W, Sun L, et al (2019) Private model compression via knowledge distillation. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 1190--1197

    2019

  7. [15]

    Journal of Ambient Intelligence and Humanized Computing 12(2):2149--2161

    Xie H, Jiang W, Luo H, et al (2021) Model compression via pruning and knowledge distillation for person re-identification. Journal of Ambient Intelligence and Humanized Computing 12(2):2149--2161

    2021

  8. [16]

    IEEE Transactions on Cybernetics 52(4):2070--2081

    Zhao H, Sun X, Dong J, et al (2020) Highlight every step: Knowledge distillation via collaborative teaching. IEEE Transactions on Cybernetics 52(4):2070--2081

    2020

  9. [17]

    In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3514--3522

    Chen H, Wang Y, Xu C, et al (2019) Data-free learning of student networks. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3514--3522

    2019

  10. [18]

    arXiv preprint arXiv:240906857 ://doi.org/10.48550/arXiv.2409.06857

    Chen L, Varoquaux G (2024) What is the role of small models in the llm era: A survey. arXiv preprint arXiv:240906857 ://doi.org/10.48550/arXiv.2409.06857

    2024 doi arXiv

  11. [19]

    arXiv preprint arXiv:171009282

    Cheng Y, Wang D, Zhou P, et al (2017) A survey of model compression and acceleration for deep neural networks. arXiv preprint arXiv:171009282

    2017

  12. [20]

    Proceedings of the IEEE 108(4):485--532

    Deng L, Li G, Han S, et al (2020) Model compression and hardware acceleration for neural networks: A comprehensive survey. Proceedings of the IEEE 108(4):485--532

    2020

  13. [21]

    International Journal of Computer Vision 129(6):1789--1819

    Gou J, Yu B, Maybank SJ, et al (2021) Knowledge distillation: A survey. International Journal of Computer Vision 129(6):1789--1819

    2021

  14. [22]

    arXiv preprint arXiv:151000149

    Han S, Mao H, Dally WJ (2015) Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:151000149

    2015

  15. [23]

    doi:10.48550/ARXIV.1503.02531, ://arxiv.org/abs/1503.02531

    Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. doi:10.48550/ARXIV.1503.02531, ://arxiv.org/abs/1503.02531

    2015 doi arXiv

  16. [24]

    arXiv preprint arXiv:170404861

    Howard AG, Zhu M, Chen B, et al (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:170404861

    2017

  17. [25]

    Advances in neural information processing systems 29

    Hubara I, Courbariaux M, Soudry D, et al (2016) Binarized neural networks. Advances in neural information processing systems 29

    2016

  18. [26]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8049--8059

    Idelbayev Y, Carreira-Perpin \'a n MA (2020) Low-rank compression of neural nets: Learning the rank of each layer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8049--8059

    2020

  19. [27]

    arXiv preprint arXiv:14053866

    Jaderberg M, Vedaldi A, Zisserman A (2014) Speeding up convolutional neural networks with low rank expansions. arXiv preprint arXiv:14053866

    2014

  20. [28]

    arXiv preprint arXiv:180608342

    Krishnamoorthi R (2018) Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:180608342

    2018

  21. [29]

    Neurocomputing 485:297--320

    Liu D, Kong H, Luo X, et al (2022) Bringing ai to edge: From deep learning’s perspective. Neurocomputing 485:297--320

    2022

  22. [30]

    Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS) Https://arxiv.org/abs/1705.07874

    Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS) Https://arxiv.org/abs/1705.07874

    2017 arXiv

  23. [31]

    In: 2018 IEEE international parallel and distributed processing symposium workshops (IPDPSW), IEEE, pp 522--531

    Markidis S, Der Chien SW, Laure E, et al (2018) Nvidia tensor core programmability, performance & precision. In: 2018 IEEE international parallel and distributed processing symposium workshops (IPDPSW), IEEE, pp 522--531

    2018

  24. [32]

    arXiv preprint arXiv:161106440

    Molchanov P, Tyree S, Karras T, et al (2016) Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:161106440

    2016

  25. [33]

    IEEE Transactions on Magnetics 54(11):1--5

    Pan Y, Ouyang P, Zhao Y, et al (2018) A multilevel cell stt-mram-based computing in-memory accelerator for binary convolutional neural network. IEEE Transactions on Magnetics 54(11):1--5

    2018

  26. [34]

    why should i trust you?

    Ribeiro MT, Singh S, Guestrin C (2016) "why should i trust you?": Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining Https://arxiv.org/abs/1602.04938

    2016 arXiv

  27. [35]

    In: Proceedings of the 34th International Conference on Machine Learning, PMLR, pp 3319--3328

    Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: Proceedings of the 34th International Conference on Machine Learning, PMLR, pp 3319--3328

    2017

  28. [36]

    Proceedings of the IEEE 105(12):2295--2329

    Sze V, Chen YH, Yang TJ, et al (2017) Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE 105(12):2295--2329

    2017

  29. [37]

    In: Thirty-First AAAI Conference on Artificial Intelligence

    Szegedy C, Ioffe S, Vanhoucke V, et al (2017) Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning . In: Thirty-First AAAI Conference on Artificial Intelligence

    2017

  30. [38]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2078--2087

    Wang T, Wang K, Cai H, et al (2020 a ) Apq: Joint search for network architecture, pruning and quantization policy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2078--2087

    2020

  31. [39]

    arXiv preprint arXiv:240209748

    Wang W, Chen W, Luo Y, et al (2024) Model compression and efficient inference for large language models: A survey. arXiv preprint arXiv:240209748

    2024

  32. [40]

    In: Proceedings of the AAAI conference on artificial intelligence, pp 12273--12280

    Wang Y, Zhang X, Xie L, et al (2020 b ) Pruning from scratch. In: Proceedings of the AAAI conference on artificial intelligence, pp 12273--12280

    2020

  33. [41]

    arXiv preprint arXiv:230510010

    Wu S, Chen H, Quan X, et al (2023) Ad-kd: Attribution-driven knowledge distillation for language model compression. arXiv preprint arXiv:230510010

    2023

  34. [42]

    arXiv preprint arXiv:220400408

    Xia M, Zhong Z, Chen D (2022) Structured pruning learns compact and accurate models. arXiv preprint arXiv:220400408

    2022

  35. [43]

    sn-basic.bst

    FUNCTION identify.basic.version "sn-basic.bst" " [2024/07/19 v1.1 bibliography style]" * top ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key keywords month note number organization pages publisher school series ...

    2024

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap dupl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.