RECORD
Neurons Activation Visualization and Information Theoretic Analysis
Computer Vision and Pattern Recognition · Submitted 14 May 2019
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim It is found that for the well trained deep neural networks model, the entropy of the neuron activation pattern is monotonically reduced with the depth of the layers.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Understanding the inner working mechanism of deep neural networks (DNNs) is essential and important for researchers to design and improve the performance of DNNs. In this work, the entropy analysis is leveraged to study the neurons activation behavior of the fully connected layers of DNNs. The entropy of the activation patterns of each layer can provide a performance metric for the evaluation of the network model accuracy. The study is conducted based on a well trained network model. The activation patterns of shallow and deep layers of the fully connected layers are analyzed by inputting the images of a single class. It is found that for the well trained deep neural networks model, the entropy of the neuron activation pattern is monotonically reduced with the depth of the layers. That is, the neuron activation patterns become more and more stable with the depth of the fully connected layers. The entropy pattern of the fully connected layers can also provide guidelines as to how many fully connected layers are needed to guarantee the accuracy of the model. The study in this work provides a new perspective on the analysis of DNN, which shows some interesting results.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Understanding the inner working mechanism of deep neural networks (DNNs) is essential and important for researchers to design and improve the performance of DNNs. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | In this work, the entropy analysis is leveraged to study the neurons activation behavior of the fully connected layers of DNNs. | method 0.60 | 0.91 | 0.04 | asserted 0.98 |
| s2 | The entropy of the activation patterns of each layer can provide a performance metric for the evaluation of the network model accuracy. | contribution 0.30 | 0.37 | 0.10 | asserted 0.87 |
| s3 | The study is conducted based on a well trained network model. | method 0.97 | 0.52 | 0.02 | asserted 0.98 |
| s4 | The activation patterns of shallow and deep layers of the fully connected layers are analyzed by inputting the images of a single class. | method 0.99 | 0.93 | 0.03 | asserted 0.95 |
| s5 | It is found that for the well trained deep neural networks model, the entropy of the neuron activation pattern is monotonically reduced with the depth of the layers. | result 1.00 | 0.94 | 0.77 | asserted 1.00 |
| s6 | That is, the neuron activation patterns become more and more stable with the depth of the fully connected layers. | interpretation 0.93 | 0.84 | 0.31 | asserted 1.00 |
| s7 | The entropy pattern of the fully connected layers can also provide guidelines as to how many fully connected layers are needed to guarantee the accuracy of the model. | interpretation 0.93 | 0.78 | 0.19 | hedged possibility 0.66 |
| s8 | The study in this work provides a new perspective on the analysis of DNN, which shows some interesting results. | contribution 0.54 | 0.54 | 0.06 | asserted 0.78 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Understanding the inner working mechanism of deep neural networks (DNNs) is essential and important for researchers to design and improve the performance of DNNs. In this work, the entropy analysis is leveraged to study the neurons activation behavior of the fully connected layers of DNNs. The entropy of the activation patterns of each layer can provide a performance metric for the evaluation of the network model accuracy. The study is conducted based on a well trained network model. The activation patterns of shallow and deep layers of the fully connected layers are analyzed by inputting the images of a single class. It is found that for the well trained deep neural networks model, the entropy of the neuron activation pattern is monotonically reduced with the depth of the layers. That is, the neuron activation patterns become more and more stable with the depth of the fully connected layers. The entropy pattern of the fully connected layers can also provide guidelines as to how many fully connected layers are needed to guarantee the accuracy of the model. The study in this work provides a new perspective on the analysis of DNN, which shows some interesting results.
Human review
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.