REVIEW 4 major objections 5 minor 25 references
Compressing Deep Neural Networks Using Explainable AI
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims that LRP relevance scores can act as a single importance criterion for both pruning and mixed-precision quantization of deep neural networks.
desk verdict The 42% accuracy claim is unsupported by the paper's own tables; the LRP-guided pruning + mixed-precision quantization idea is sensible but incremental, and the experiments are too weak to support publication. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the per-weight LRP relevance score, obtained by the layer-wise redistribution rule $R_{i\leftarrow j} = (a_i w_{ij}/z_j) R_j$ with the conservation guarantee $\sum_i R_{i\leftarrow j}=R_j$. The paper aggregates these weight scores into filter scores, prunes filters whose scores are non-positive, and uses the per-layer median of the surviving filter scores as the threshold that assigns each filter to a low- or high-precision quantizer. This mechanism carries the argument because the same score is used for both removal and bit-width allocation.
What would settle it
Take the same trained three-layer network and prune the same number of filters chosen by the smallest weight-norm, or by random selection, instead of by LRP score, keeping the same 1.2 MB mixed-precision budget. If the accuracy of these control schemes matches or exceeds the 90.5% reported for the LRP-based scheme, then the relevance ordering, not the pruning and quantization procedure itself, is not the cause of the improvement.
Extended reading notes
Core claim
The paper's central claim is that LRP relevance scores, which redistribute the network output back through the layers according to a conservation rule, carry the information needed to decide both which parameters to remove and how many bits each remaining parameter deserves. Negative or zero-scored filters are dropped outright; the remaining filters in each layer are split at the per-layer median of their scores, with below-median filters stored at lower bit-widths and above-median filters at higher bit-widths. Reported on a three-layer ReLU network with 1000 hidden units per layer and the 'multi' toy dataset (4 classes), the scheme reduces storage from 8 MB to 1.2 MB while holding accuracy near 90.5%, compared with 79.25% for the LRP-only filter-pruning baseline and 71.49% for a weight-magnitude-based method at the same model size. The authors interpret these numbers as showing that explainability-driven importance scoring outperforms magnitude-based criteria for mixed-precision allocation.
Load-bearing premise
The method assumes that per-weight LRP relevance scores, aggregated into filter scores, rank each filter's global importance correctly, so that every negative-or-zero-score filter can be removed without meaningful accuracy loss.
Editorial extensions
If this is right
- Memory footprint of the tested dense network drops to 15% of its original size (8 MB to 1.2 MB) with accuracy staying above 90%.
- LRP relevance can replace magnitude as the default importance metric for pruning and quantization pipelines, offering better accuracy at equal size (90.5% versus 71.49% at 1.2 MB).
- A single XAI pass supplies both the pruning mask and the precision map, making compression a one-shot post-training procedure on this architecture.
- The method needs no reference activation and no Hessian inverse, avoiding two common costs of prior compression criteria.
- Because scores are computed per layer, the bit-width map can be tuned per layer, giving a low-cost knob for accuracy-versus-size trade-offs.
Reading between the lines
- The filter-score aggregation rule is never specified, so the method's portability to convolutional or transformer architectures depends on how per-weight relevance is summed over a filter; a reader should test whether max-pooling versus summing changes results.
- The per-layer median split is one of many possible thresholds; a testable extension is to search bit-width assignment by validation loss instead of median.
- The 42% accuracy-improvement figure in the abstract does not obviously match Table III (90.5% versus 79.25% is about 14% relative), so a reader comparing claims to tables should identify the exact baseline before relying on that number.
- If LRP scores are re-computed after pruning, a second compression pass may yield further size reduction; the paper does not test iterative application.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a DNN compression method that combines pruning and mixed-precision quantization using Layer-wise Relevance Propagation (LRP) importance scores. The method computes relevance scores for weights, prunes filters with negative or zero scores, and assigns bit-widths based on whether a layer's filter score is above or below the median score of that layer. The experiments are conducted on a small three-layer dense network trained on a toy four-class dataset. The abstract and conclusion claim a 64% model-size reduction and a 42% accuracy improvement over a state-of-the-art XAI-based compression method, and the paper includes comparisons with a non-XAI Taylor-expansion method and two named baselines in Table III.
Significance. If the central claims were sound, the paper would demonstrate a useful synergy between LRP-based explainability and joint pruning/quantization, potentially offering a parameter-importance criterion that goes beyond weight magnitude. The authors are to be credited for comparing against a non-XAI baseline in Table II and for examining two mixed-precision configurations in Table I. However, the significance is severely limited by the absence of a precise algorithmic specification, the lack of any comparison with the cited state-of-the-art ECQx method, and a quantitative discrepancy between the headline '42% improvement' and the results actually reported in Table III. The paper also does not supply code or machine-checked proofs, and the core per-weight relevance derivation in Eq. (4) is not rigorously established. As it stands, the work is not reproducible from the manuscript, and its main quantitative claim is unsupported.
major comments (4)
- [Abstract and Section III.D (Table III)] The central claim that the proposed method improves accuracy by 42% over a state-of-the-art XAI-based compression method is not supported by the reported data. In Table III, the Pruning+MPQ row shows the proposed method at 90.5% versus 79.25% for the filter-pruning baseline [2], a relative improvement of about 14%, not 42%. The only occurrence of '42%' in the experimental section is in the caption of Fig. 1, where the comparison is with the authors' own Pruning+SPQ variant, whose accuracy is 90.125% or 90.8%, under one percentage point different from the proposed method. Moreover, the named state-of-the-art XAI-based compression method ECQx [1] is never evaluated. The abstract's headline figure must be corrected or removed, and the claim in the Conclusion that accuracy 'improved to 42% compared to previous work' is likewise inaccurate.
- [Section II.A, Eq. (4) and Algorithm 1 (lines 1-5)] The paper never defines how individual neuron relevance scores are aggregated into filter scores, yet Algorithm 1 line 5 prunes filters with 'negative or zero' scores. Eq. (4) states the standard LRP message formula in an unusual form, replacing a_i * ∂z_j/∂a_i with w_ij * ∂z_j/∂w_ij, but it does not derive a per-weight relevance; it still describes neuron-to-neuron messages. No equation or textual description specifies whether a filter's score is the sum, mean, maximum, or some other statistic of the relevance scores of its constituent weights or neurons. Without this specification, the pruning and quantization rules in Algorithm 1 are not reproducible, and the reader cannot verify whether the method actually uses LRP-derived weight importance as claimed.
- [Section II.B and Section III.B] The method's hyperparameters, specifically the per-layer median threshold and the mixed-precision bit-width configurations (e.g., Layer0: 8,16 bits vs. Layer2: 4,8 bits in Table I), were selected after observing accuracy on the same toy test set. The text states that 'other metrics like weighted mean where tested but the best accuracy was related to the experiment which classification was done using median,' which is an explicit selection on the evaluation set. This creates a circularity that inflates the reported accuracy numbers. A validation split or a sensitivity analysis over thresholds and bit-widths is necessary to support the claim that the proposed importance criterion, rather than the chosen hyperparameters, is responsible for the observed performance.
- [Section III.A and Table III] The experimental evaluation is restricted to a single toy model (three dense layers, 1000 hidden units) and a single four-class synthetic dataset. The paper does not report training details such as the number of training samples, optimizer, learning rate, or epochs, and no results are given on standard benchmarks (e.g., MNIST, CIFAR-10, or any real-world dataset). Given that the abstract generalizes to 'deep neural networks' and claims deployability on edge devices, the lack of any additional experiment is a load-bearing gap in the empirical support for the method's general validity.
minor comments (5)
- [Section II.B] Algorithm 1 is referenced in the text but the algorithm listing is not included in the manuscript; please provide the pseudocode or a complete textual specification of the algorithm's steps.
- [Section III.A] The description of the model and dataset is incomplete: 'a k-class toy dataset "multi" (k = 4)' should specify the input dimensionality, the number of samples, the train/test split, and the data generator used. The sentence 'The proposed method is used to compress a DNN model which is used by the previous works' is unclear and should name the earlier work that introduced this model.
- [Eq. (4)] The notation in Eq. (4) is confusing: the relation a_i w_ij = a_i ∂z_j/∂a_i = w_ij ∂z_j/∂w_ij holds for a linear layer, but the text should explicitly state this and define all symbols (a_i, w_ij, z_j, R_j) before presenting the formula.
- [Table I and Table II] The bit-width allocations in Table I are not fully specified: it is not stated which weights (above or below the median) receive the higher bit-width in each layer, and the two Pruning+MPQ rows should be clearly distinguished in the text. In Table II, the 'Non-XAI' and 'XAI' columns should indicate which method's accuracy is reported for each row.
- [Throughout the manuscript] There are several typographical and grammatical errors, including 'stae-of-the art' (Section III.D), 'cereticia' (Introduction), 'Pruinng' (Section III.D), and 'multi-precision quantization' used inconsistently with 'mixed-precision quantization'. The Conclusion's phrase 'accuracy improved to 42%' should be corrected to 'improved by 42%' if retained, but see Major Comment 1.
Circularity Check
Mild selection circularity in threshold and bit-width choice; the central LRP-based compression idea is otherwise not derived from its own outputs.
-
fitted input called prediction
[Section II-B (Compression Algorithm) and Section III-B (Table I)]
"In this regard, other metrics like weighted mean where tested but the best accuracy was related to the experiment which classification was done using median. ... Different mixed-precision bit-widths are tried in order to find the best performance."
The median threshold and the per-layer bit-width assignments are selected by comparing accuracy on the same toy test set and keeping the configuration with the highest accuracy. The reported 90.5% for Pruning + MP Q is therefore the output of a selection procedure whose objective is that same accuracy value, not an independent estimate of the method's performance. The 'best' configuration is defined by the reported metric, so part of the result is forced by the fitting process rather than by LRP scores alone. The core LRP computation is not itself derived from accuracy, so this is a mild selection circularity rather than a full reduction.
full rationale
Most of the derivation is not circular: LRP scores are computed from the network's weights and activations by Eqs. (1)-(4) and used as pruning and quantization criteria; no parameter is defined in terms of the final accuracy, and no load-bearing uniqueness claim is imported from the authors' own prior work. The one circular element is hyperparameter selection: the paper explicitly chooses the median threshold and per-layer bit-widths by trying alternatives and keeping the best accuracy on the same test set, making the reported 90.5% partly a selected maximum rather than an independent validation. Separately, the abstract's '42% improvement over the state-of-the-art XAI-based compression method' is not supported by Table III (the relative gain over filter pruning [2] is about 14.2%) and appears to refer to a comparison with the authors' own Pruning + SP Q variant; this is a reporting and consistency problem, not a derivation-level circularity. Score 3 reflects the mild selection circularity while acknowledging that the central LRP-based idea still has independent content.
Assumptions & free parameters
free parameters (3)
- per-layer median threshold =
median of LRP scores in each layer
- per-layer mixed-precision bit-widths =
Layer0: 8,16; Layer1: 8,16; Layer2: 8,16 (best reported configuration)
- pruning sign rule =
prune all filters with negative or zero LRP score
assumptions (3)
- domain assumption LRP relevance scores represent each unit's true contribution to the network output (LRP conservation property)
- ad hoc to paper Weights with negative or zero relevance are unimportant and can be pruned without significant accuracy loss
- ad hoc to paper The per-layer median of relevance scores is a good split point for assigning high and low bit-widths
Cite this review
Pith. "Pith review of Compressing Deep Neural Networks Using Explainable AI." pith.science (2026). https://pith.science/paper/7KPPBLZ5
@misc{pith2026250705286,
author = {Pith},
title = {Pith review of: Compressing Deep Neural Networks Using Explainable AI},
year = {2026},
howpublished = {\url{https://pith.science/paper/7KPPBLZ5}},
note = {Machine review of arXiv:2507.05286}
}
read the original abstract
Deep neural networks (DNNs) have demonstrated remarkable performance in many tasks but it often comes at a high computational cost and memory usage. Compression techniques, such as pruning and quantization, are applied to reduce the memory footprint of DNNs and make it possible to accommodate them on resource-constrained edge devices. Recently, explainable artificial intelligence (XAI) methods have been introduced with the purpose of understanding and explaining AI methods. XAI can be utilized to get to know the inner functioning of DNNs, such as the importance of different neurons and features in the overall performance of DNNs. In this paper, a novel DNN compression approach using XAI is proposed to efficiently reduce the DNN model size with negligible accuracy loss. In the proposed approach, the importance score of DNN parameters (i.e. weights) are computed using a gradient-based XAI technique called Layer-wise Relevance Propagation (LRP). Then, the scores are used to compress the DNN as follows: 1) the parameters with the negative or zero importance scores are pruned and removed from the model, 2) mixed-precision quantization is applied to quantize the weights with higher/lower score with higher/lower number of bits. The experimental results show that, the proposed compression approach reduces the model size by 64% while the accuracy is improved by 42% compared to the state-of-the-art XAI-based compression method.
Reference graph
Works this paper leans on
-
[2]
Pruning by explaining: A nove l criterion for deep neural network pruning
Yeom, Seul-Ki, et al. "Pruning by explaining: A nove l criterion for deep neural network pruning." Pattern Recognition 115 (2021): 107899
work page 2021
-
[1]
ECQ: Explainability -Driven Quantization for Low-Bit and Sparse DNNs
Becking, Daniel, et al. "ECQ: Explainability -Driven Quantization for Low-Bit and Sparse DNNs." International Workshop on Extending Explainable AI Beyond Deep Models and Classifiers. Springer, Cham, 2022
work page 2022
-
[21]
Optimal brain compression: A framework for accurate post -training quantization and pruning
Frantar, Elias, and Dan Alistarh. "Optimal brain compression: A framework for accurate post -training quantization and pruning." Advances in Neural Information Processing Systems 35 (2022): 4475- 4488
work page 2022
-
[3]
Utilizing Explainable AI for Quantization and Pruning of Deep Neural Networks
Sabih, Muhammad, Frank Hannig, and Juergen Teich. "Utilizing explainable AI for quantization and pruning of deep neural networks." arXiv preprint arXiv:2008.09072 (2020)
work page Pith review arXiv 2020
-
[4]
Explaining deep neural networks and beyond: A review of methods and applications
Samek, Wojciech, et al. "Explaining deep neural networks and beyond: A review of methods and applications." Proceedings of the IEEE 109.3 (2021): 247-278
work page 2021
-
[5]
Layer -wise relevance propagation: an overview
Montavon, Grégoire, et al. "Layer -wise relevance propagation: an overview." Explainable AI: interpre ting, explaining and visualizing deep learning (2019): 193-209
work page 2019
-
[6]
Learning important features through propagating activation differences
Shrikumar, Avanti, Peyton Greenside, and Anshul Kundaje. "Learning important features through propagating activation differences." International conference on machine learning. PMLR, 2017
work page 2017
-
[7]
Model compression and hardware acceleration for neural networks: A comprehensive survey
Deng, Lei, et al. "Model compression and hardware acceleration for neural networks: A comprehensive survey." Proceedings of the IEEE 108.4 (2020): 485-532
work page 2020
Show all 25 references
-
[8]
Pruning convolutional neural networks for resource efficient inference
Molchanov, P., et al. "Pruning convolutional neural networks for resource efficient inference." 5th Int ernational Conference on Learning Representations, ICLR 2017-Conference Track Proceedings. 2019
2017
-
[9]
Channel pruning based on mean gradient for accelerating convolutional neural networks
Liu, Congcong, and Huaming Wu. "Channel pruning based on mean gradient for accelerating convolutional neural networks." Signal Processing 156 (2019): 84-91
2019
-
[10]
Nisp: Pruning networks using neuron importance score propagation
Yu, Ruichi, et al. "Nisp: Pruning networks using neuron importance score propagation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018
2018
-
[11]
Pruning and quantization for deep neural network acceleration: A survey
Liang, Tailin, et al. "Pruning and quantization for deep neural network acceleration: A survey." Neurocomputing 461 (2021): 370-403
2021
-
[12]
Convolutional neural network pruning: A survey
Xu, Sheng, et al. "Convolutional neural network pruning: A survey." 2020 39th Chinese Control Conference (CCC). IEEE, 2020
2020
-
[13]
Quantizing deep convolutional networks for efficient inference: A whitepaper
Krishnamoorthi, Raghuraman. "Quantizing deep convolutional networks for efficient inference: A whitepaper." arXiv preprint arXiv:1806.08342 (2018)
2018 arXiv
-
[14]
Automatic pruning for quantized neural networks
Guerra, Luis, and Tom Drummond. "Automatic pruning for quantized neural networks." 2021 Digital Image Comput ing: Techniques and Applications (DICTA). IEEE, 2021
2021
-
[15]
Model compression and hardware acceleration for neural networks: A comprehensive survey
Deng, Lei, Guoqi Li, Song Han, Luping Shi, and Yuan Xie. "Model compression and hardware acceleration for neural networks: A comprehensive survey." Proceedings of the IEEE 108, no. 4 (2020): 485-532
2020
-
[16]
Autocompress: An automatic dnn structured pruning framework for ultra -high compression rates
Liu, Ning, Xiaolong Ma, Zhiyuan Xu, Yanzhi Wang, Jian Tang, and Jieping Ye. "Autocompress: An automatic dnn structured pruning framework for ultra -high compression rates." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, pp. 4876-
-
[17]
Clip -q: Deep network compression learning by in -parallel pruning -quantization
Tung, Frederick, and Greg Mori. "Clip -q: Deep network compression learning by in -parallel pruning -quantization." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7873-
-
[18]
Admm-nn: An algorithm-hardware co-design framework of dnns using alternating direction methods of multipliers
Ren, Ao, Tianyun Zhang, Shaokai Ye, Jiayu Li, Wenyao Xu, Xuehai Qian, Xue Lin, and Yanzhi Wang. "Admm-nn: An algorithm-hardware co-design framework of dnns using alternating direction methods of multipliers." In Proceedings of the Twenty -Fourth International Conference on Arc...
2019
-
[19]
Automatic neural network compression by sparsity-quantization joint learning: A constrained optimization -based ap proach
Yang, Haichuan, Shupeng Gui, Yuhao Zhu, and Ji Liu. "Automatic neural network compression by sparsity-quantization joint learning: A constrained optimization -based ap proach." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2178-2188. 2020
2020
-
[20]
Accelerator -aware pruning for convolutional neural networks
Kang, Hyeong -Ju. "Accelerator -aware pruning for convolutional neural networks." IEEE Transactions on Circuits and Systems for Video Technology 30.7 (2019): 2093-2103
2019
-
[22]
Nisp: Pru ning networks using neuron importance score propagation
Yu, Ruichi, et al. "Nisp: Pru ning networks using neuron importance score propagation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018
2018
-
[23]
Thinet: pruning cnn filters for a thinner net
Luo, Jian -Hao, et al. "Thinet: pruning cnn filters for a thinner net." IEEE transactions on pattern analysis and machine intelligence 41.10 (2018): 2525-2538
2018
-
[24]
Compressing the CNN architecture for in -air handwritten Chinese character recognition
Gan, Ji, Weiqiang Wang, and Ke Lu. "Compressing the CNN architecture for in -air handwritten Chinese character recognition." Pattern Recognition Letters 129 (2020): 190-197
2020
-
[25]
NeST: A neural network synthesis tool based on a grow -and-prune paradigm
Dai, Xiaoliang, Hongxu Yin, and Niraj K. Jha. "NeST: A neural network synthesis tool based on a grow -and-prune paradigm." IEEE Transactions on Computers 68.10 (2019): 1487-1497
2019
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.