REVIEW 2 major objections 4 minor 7 references
The many faces of deep learning
T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Deep learning is best understood by combining seven disciplinary perspectives, not by a single theory.
desk verdict A readable, honest survey of seven perspectives on deep learning, but the promised synthesis of those perspectives is asserted rather than actually carried out. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The organizing device is the "faces" metaphor applied to a single common object: the composition of nonlinear transformations in a feedforward neural network trained by supervised learning. Each face selects one aspect of that object—the geometry of the data cloud, the metric induced by embeddings, the information flow through layers, the energy landscape of the weights, the computational graph, and the biological analogy—and the argument is carried by the interrelations among these selections rather than by a new theorem, identity, or mechanism. The geometric folding picture, with its image of training as folding an elastic cloud along learned directions, serves as the book's most concrete recurring intuition, but it is one face among several.
What would settle it
A concrete way to test the synthesis would be to find a widely used deep learning system that cannot be described by any of the seven faces, such as a purely attention-based, self-supervised model whose success depends on a mechanism none of the perspectives covers. A narrower test: measure mutual information between hidden layers and the input during training; if a well-generalizing network increases this quantity while the information face predicts compression, that face is falsified.
Extended reading notes
Core claim
The central claim is that no single community's intuition explains deep learning, and that the field is better served by collecting, juxtaposing, and interconnecting the different intuitions developed in mathematics, physics, computation, and neuroscience. The paper grounds this claim in the concrete setting of a feedforward network trained by supervised learning, and then walks through each perspective: training as high-dimensional origami that folds a point cloud into linearly separable regions; embeddings as creating a meaningful metric on symbolic inputs; layers as a Markov chain that necessarily discards information, ideally toward a minimal sufficient statistic of the input for the target; supervised learning as operating purely at the level of association, leaving causal reasoning and counterfactual transfer to approaches involving intervention; networks as statistical-mechanical systems with energy landscapes whose local minima are often nearly as good as global ones; architectures as computational graphs whose training cost and resource scaling must be measured; and neuroscience as a two-way source of representational and algorithmic ideas. The paper's own stated conclusion is that "the deepness in this case should come from putting all these faces of deep learning together in the reader's mind and entertain their interrelations."
Load-bearing premise
The synthesis assumes that insights drawn from feedforward networks trained by supervised learning—geometric folding, information compression, spin-glass energy landscapes—transfer to the full diversity of deep learning architectures and learning paradigms, since the paper explicitly restricts its scope to that setting.
Editorial extensions
If this is right
- Each perspective suggests its own practical levers: geometry points to disentangling and folding directions, information theory points to compression regularizers, and physics points to noise, annealing, and flat-minima methods such as dropout.
- Adversarial examples and mode collapse appear as natural byproducts of the same continuous folding and non-invertible compression that make learning work, so defenses should target those mechanisms rather than individual attacks.
- Because supervised feedforward models only capture associations, gains in transfer learning and explainability are more likely to require adding interventions, causal structure, or reinforcement-style credit assignment than simply adding data or capacity.
- The information face predicts that representations compressing the input while retaining target-relevant information should generalize better, making layer-wise mutual information a usable diagnostic during training.
- Fair comparison of learning systems requires measuring how performance scales with computational resources and problem complexity, not only reporting final accuracy.
Reading between the lines
- If the seven faces are truly projections of one object, an integrated theory could quantify trade-offs between geometric disentanglement, information compression, and flatness of minima; a regularizer optimizing all three together should outperform any one alone.
- The continuous-folding account of adversarial examples implies that adversarial perturbation directions should align with the dominant expansion directions of the network's Jacobian—a spectral prediction the paper does not make but that could be tested.
- The paper's feedforward scope leaves attention and transformers implicit; since attention layers are learned pairwise similarity functions, the metric face seems the natural starting point for extending the synthesis to modern architectures.
- If the information-loss view is right, invertible architectures such as normalizing flows, which deliberately avoid discarding information, should generalize by a different mechanism than the one the paper describes, so the framework would need a separate account of their success.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This perspective paper collects seven disciplinary viewpoints on why deep learning works: topological, metric, information-theoretic, causal, statistical-physics, computational, and neuroscience. Each section presents a self-contained vignette of relevant intuitions, and the Discussion states that the 'deepness' of the paper should come from the reader combining these faces and exploring their interrelations. The paper explicitly disclaims formal results and restricts attention to feedforward networks trained by supervised learning, with a brief note on how other settings reduce to this one.
Significance. The individual vignettes are generally accurate, well-written, and supported by an appropriate set of citations; the paper is honest about the shallow coverage of each perspective. As a result, the manuscript could serve as an accessible interdisciplinary overview for newcomers. However, the paper's stated central contribution—a synthesis in which the perspectives are interrelated—is not realized, because the sections remain parallel and the Discussion provides no worked example of interrelations. The value of the paper therefore depends on the reader doing the synthesis work on their own, which weakens the claimed contribution for a journal publication.
major comments (2)
- [Section 3 (Discussion)] The central claim that deepness comes from 'putting all these faces ... together in the reader's mind and entertain their interrelations' is asserted but not delivered: the preceding sections are largely self-contained vignettes, and cross-perspective references (e.g., 'folding' in Sections 2.1 and 2.3, energy landscapes in Sections 2.5 and 2.6) are not explained or compared. The Discussion would need at least one worked example of how two perspectives mutually illuminate a problem to scaffold the promised synthesis, but none is provided.
- [Section 2.4 (Causal Perspective)] The introduction (Section 1) restricts the scope to feedforward supervised networks, yet Section 2.4 develops the causal perspective almost entirely through reinforcement learning, without explaining how the RL discussion transfers back to the feedforward setting or how the earlier 'conversion' argument justifies this. This mismatch leaves the causal perspective disconnected from the rest of the survey.
minor comments (4)
- [Section 2.6 (Computational Perspective)] The statement that finding the absolute minimum is an 'NP-problem' should be sharpened to 'NP-hard' (or 'NP-complete', with the precise reference to Blum and Rivest (1989)); as written it is imprecise.
- [Section 2.1 (Topological Perspective)] The sentence attributing the almost-linearity explanation of adversarial examples to Athalye et al. (2018) appears to misattribute this idea; Goodfellow et al. (2014b), cited later in the same paragraph, is the standard reference for that explanation.
- [Section 2.3 (Information Perspective)] The information-theoretic section presents the information bottleneck idea without acknowledging known criticisms of the claim that compression directly causes generalization; adding a caveat would make the survey more balanced.
- [Section 2.2 (Metric Perspective)] The term 'one-shot transfer learning' is used without an illustrative example or definition; the term is more specific than the surrounding text suggests.
Circularity Check
No circularity: the paper is a self-contained perspective essay with no fitted parameters, derived predictions, or load-bearing self-citation chain.
full rationale
This manuscript is an opinion/perspective piece that surveys existing intuitions about deep learning from topology, metrics, information theory, causality, physics, computation, and neuroscience. It introduces no equations that predict a quantity from fitted parameters, and its central claim is explicitly pedagogical: 'the deepness in this case should come from putting all these faces of deep learning together in the reader's mind and entertain their interrelations.' That claim is a stated purpose rather than a derived result, so there is no derivation chain whose output could be equivalent to its input by construction. All technical statements are attributed to external prior work (e.g., Montufar et al. 2014 for folding, Tishby and Zaslavsky 2015 for information bottleneck, Choromanska et al. 2015 for spin-glass mappings), and the paper does not import any uniqueness theorem or ansatz from the author's own prior publications. The acknowledged limitation that the discussion of each perspective is 'necessarily shallow' is a completeness concern, not a circularity concern. The skeptical observation that the Discussion does not work through cross-perspective interrelations in detail is a merit critique, not evidence that any claim is defined in terms of itself. Accordingly, the circularity score is 0.
Assumptions & free parameters
Cite this review
Pith. "Pith review of The many faces of deep learning." pith.science (2026). https://pith.science/paper/YTWHRT3Q
@misc{pith2026190810206,
author = {Pith},
title = {Pith review of: The many faces of deep learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/YTWHRT3Q}},
note = {Machine review of arXiv:1908.10206}
}
read the original abstract
Deep learning has sparked a network of mutual interactions between different disciplines and AI. Naturally, each discipline focuses and interprets the workings of deep learning in different ways. This diversity of perspectives on deep learning, from neuroscience to statistical physics, is a rich source of inspiration that fuels novel developments in the theory and applications of machine learning. In this perspective, we collect and synthesize different intuitions scattered across several communities as for how deep learning works. In particular, we will briefly discuss the different perspectives that disciplines across mathematics, physics, computation, and neuroscience take on how deep learning does its tricks. Our discussion on each perspective is necessarily shallow due to the multiple views that had to be covered. The deepness in this case should come from putting all these faces of deep learning together in the reader's mind, so that one can look at the same problem from different angles.
Reference graph
Works this paper leans on
-
[1]
Mathematicians will rightly contend that we do not really know
Introduction Why deep learning works so well? The typical answer to this question depends on to which community is addressed. Mathematicians will rightly contend that we do not really know. Apart from theorems concerning the expressibility of neural networks (e.g. they are universal approximators of continuous functions given some mild conditions), there ...
work page 2015
-
[2]
Perspectives Different disciplines have naturally developed distinct interests and points of view in regard to the theory and application of deep learning. Here, we wish to expose the varied views on deep learning phenomena by discussing a few illustrative examples from each of the considered perspectives. 2.1. Topological perspective While random numbers...
work page 2016
-
[3]
Discussion Looking at a problem through multiple angles can stimulate novel ideas. Our discussion on each perspective and examples of intuitions have been necessarily shallow due to the multiple views that had to be covered. The deepness in this case should come from putting all these faces of deep learning together in the reader's mind and entertain thei...
-
[4]
Conflict of Interest The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest
-
[5]
Funding RV thanks the financial support by the Estonian Research Council (project number PUT 1476), and the Estonian Centre of Excellence in IT (EXCITE) (project number TK148)
-
[6]
Acknowledgments RV thanks all members of the group of computational neuroscience (Jaan Aru, Tambet Matiisen, Ardi Tampuu, Ilya Kuzovkin, Daniel Majoral, Aqeel Labash, Oriol Andreu, Roman Ring, Kristjan Korjus, Abdullah Makkeh) for enlightening discussions and patient explanations over the years on many of the concepts covered in this article
-
[7]
References Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks, 61, 85-117. Goodfellow, I., Bengio, Y ., & Courville, A. (2016). Deep learning. MIT press. Montufar, G. F., Pascanu, R., Cho, K., & Bengio, Y . (2014). On the number of linear regions of deep neural networks. In Advances in neural information processing syst...
arXiv 2015
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.