Information Theoretic Interpretation of Deep learning

Tianchen Zhao

RECORD

Information Theoretic Interpretation of Deep learning

Tianchen Zhao

Machine Learning · Submitted 21 Mar 2018

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim Inspired by these results, we established a conjecture of the dynamics of the machinary of deep neural network.

how it is stated · asserted 0.78 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 2,867 input tokens.

What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.08 · human participants 0.03 · controlled comparison 0.03 · randomized assignment 0.02 · states quantitative result 0.03 · compares with prior results 0.83 · mentions code or data availability 0.02.

The abstract, sentence by sentence

arXiv listing abstract v2. Results and the main claim are underlined; show every role.

We interpret part of the experimental results of Shwartz-Ziv and Tishby [2017]. Inspired by these results, we established a conjecture of the dynamics of the machinary of deep neural network. This conjecture can be used to explain the counterpart result by Saxe et al. [2018].

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	We interpret part of the experimental results of Shwartz-Ziv and Tishby [2017].	contribution 0.46	0.54	0.04	asserted 0.97
s1	Inspired by these results, we established a conjecture of the dynamics of the machinary of deep neural network.	contribution 1.00	0.91	0.31	asserted 0.78
s2	This conjecture can be used to explain the counterpart result by Saxe et al. [2018].	interpretation 0.53	0.39	0.24	hedged possibility 0.54

What kind of paper the classifier reports this is: position or commentary 0.77 · evidence from analysis of existing data 0.60.

What was examined, and what was not

sources read

arXiv listing abstract v2

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

We interpret part of the experimental results of Shwartz-Ziv and Tishby [2017]. Inspired by these results, we established a conjecture of the dynamics of the machinary of deep neural network. This conjecture can be used to explain the counterpart result by Saxe et al. [2018].

Human review

How Pith checks papers → · Full audit of this paper →

How the literature uses it

1 arXiv paper cite this in 2 sentences

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

Papers citing it

Cites it as background 1 paper

Current approaches of neural network interpretation include Bayesian probabilistic interpretations [ref] and information theoretic interpretations [ref] .

Seeing Convolution Through the Eyes of Finite Transformation Semigroup Theory: An Abstract Algebraic Interpretation of Convolutional Neural Networks · Introduction · 0.99

What it cites · 13 sentences without a confident label

Uses data or values from 2 papers

Here we use full reference to experimental results from [ref] .

Identity Mappings in Deep Residual Networks · Deep Residual Network · 0.86
Here we use full reference to experimental results from [ref] .

Empirical Analysis of the Hessian of Over-Parametrized Neural Networks · Flat Basins · 0.88

Cites as support 1 paper

[ref] and [ref] showed that the searching path of SGD needs time of exponential order to jump from one local min to another local min, which is intuitively why the compression phase is slow.

Local Optimality and Generalization Guarantees for the Langevin Algorithm via Empirical Metastability · Machinary of SGD · 0.97

Cites as background 3 papers

[ref] designed a variational approximation to the IB framework.

Deep Variational Information Bottleneck · Related work · 0.96
Residual Network by [ref] , the winner of ILSVRC2015, is one of the best existing deep network structures.

Deep Residual Learning for Image Recognition · Interpretation of Res-Net · 0.95
[ref] used entropy to detect wide valleys for SGD algorithm.

Entropy-SGD: Biasing Gradient Descent Into Wide Valleys · Related work · 0.93

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.