REVIEW 1 cited by
Flow of Information in Feed-Forward Deep Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Flow of Information in Feed-Forward Deep Neural Networks
read the original abstract
Feed-forward deep neural networks have been used extensively in various machine learning applications. Developing a precise understanding of the underling behavior of neural networks is crucial for their efficient deployment. In this paper, we use an information theoretic approach to study the flow of information in a neural network and to determine how entropy of information changes between consecutive layers. Moreover, using the Information Bottleneck principle, we develop a constrained optimization problem that can be used in the training process of a deep neural network. Furthermore, we determine a lower bound for the level of data representation that can be achieved in a deep neural network with an acceptable level of distortion.
Forward citations
Cited by 1 Pith paper
-
Post-Training Neural Network Pruning using Graph Curvature
An activation-aware graph-curvature score ranks neural connections for post-training pruning, preserving accuracy as well as or better than magnitude, SNIP, or SynFlow on MNIST/CIFAR models in one-shot removal tests.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.