REVIEW 4 major objections 5 minor 18 references
What are Neural Networks made of?
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper hypothesizes that neural network training is a form of genetic programming, with gradient descent handling evaluation and selection and Transformer attention carrying out random crossover.
desk verdict GP-as-NN framing is novel but under-supported; the attention-crossover claim confuses initialization with learning and doesn't survive a second look. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the correspondence between GP operations and NN components, anchored by an information-theoretic decomposition of the maximum-likelihood objective. The loss splits into a Kullback-Leibler term plus the target's conditional mutual information with the features, so adding a feature can improve the objective only up to its mutual information with the target; gradient descent is then read as a feature-selection mechanism. The concrete crossover mechanism is Transformer attention, where a value tensor is combined with the value at a softmax-selected position $j^*$ via key-query inner products, and the paper claims this selection is random because those tensors inherit random initialization.
What would settle it
Compare the attention distribution over positions $j$ on a fixed input set before and after training. If the chosen positions remain statistically indistinguishable from a random baseline, the randomness assumption holds; if they concentrate on task-relevant tokens, attention is selecting by learning and the claimed GP crossover is not random. A second test: freeze or randomize the key and query projections after initialization and see whether training still works; if it collapses, the learned component of pairing is doing essential work.
Extended reading notes
Core claim
On the author's own terms, the discovery is a structural correspondence between the three defining operations of GP and the mechanics of modern deep learning. A neuron or tensor is treated as a program; the computation graph is its program tree; gradient steps reinforce features that correlate with the target, which the author identifies with GP's evaluation-and-selection step; and the attention mechanism in the Transformer, which sums the value tensor at a selected position into the current position, is identified with GP's random crossover. The author is explicit that this is a hypothesis and that the randomness of attention selection is an assumption, not a demonstrated fact.
Load-bearing premise
The claim rests on attention pairings being random rather than learned: if training actually shapes which positions get paired, the random-crossover step of the GP analogy fails, and the argument loses its main recombination mechanism.
Editorial extensions
If this is right
- Better architectures are better GP implementations: skip connections and attention make evaluation paths short and recombinations easy, which is why the Transformer outperforms earlier architectures.
- Overparameterization matters because it enlarges the candidate population: BERT's roughly 8 million neurons per layer provide enough random pairings for the search to work.
- The hierarchical abstraction seen in CNNs, from low-level features to high-level concepts, is a natural result of lower-layer features being recombined into higher-layer programs.
- SGD noise and random initialization are not merely annoyances; they are the source of the random variation that GP requires.
- Multiple random restarts and redundant feature representation early in training help because they increase the pool of random combinations.
Reading between the lines
- A quantitative test: track whether attention's chosen positions shift with training; if they do, the randomness premise must be relaxed, but gradient-based selection of features could still be the main GP-like operation.
- Design implication: deliberately injecting recombination noise, such as stochastic attention or random feature dropout, might improve training by supplying the variation that GP needs.
- The paper's population-size arithmetic suggests an efficiency prediction: models with too few parameters or too narrow layers should converge less reliably, which is testable in controlled scaling experiments.
- The GP framing invites looking for punctuated generation effects in loss curves, where progress stalls and then jumps when a new feature combination crosses the noise threshold.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper advances the hypothesis that deep neural network training should be understood as a form of genetic programming (GP). Section 2 derives an information-theoretic decomposition of the conditional maximum-likelihood loss, following Brown et al., to argue that gradient descent sequentially selects features. Sections 3 and 4 draw a GP analogy in which gradient steps perform fitness evaluation and selection, random initialization and sgd noise supply randomness, and Transformer attention implements random pairwise crossover. Section 5 identifies attention in BERT as the GP crossover mechanism, Section 6 estimates candidate population sizes, and Section 7 proposes architectural guidance based on GP. The manuscript is explicit that this is a hypothesis rather than a demonstrated equivalence, and it contains no empirical tests.
Significance. If the GP hypothesis were established, it would offer a novel conceptual bridge between evolutionary computation and deep learning and could motivate new architectures. The paper's useful contributions are the correct application of the Brown et al. decomposition to the sequential-feature-selection view and its candor in framing the claim as a hypothesis. However, the central mechanistic argument is not demonstrated, and the attention-crossover claim contains an internal inconsistency about trainability; as it stands, the paper does not provide a testable or internally consistent basis for the GP equivalence.
major comments (4)
- [Section 4 and Eq. (1)] The claim that gradient descent performs GP evaluation and selection is asserted rather than derived. Eq. (1) supports only the weaker statement that adding a feature can improve the objective by up to its mutual information with the target; it says nothing about the trajectory of gradient dynamics or about selection as GP understands it. Since the rest of the paper builds on this step, the argument is load-bearing and unsupported.
- [Sections 5 and 6, Eqs. (2)-(4)] The claim that Transformer attention provides random pairwise crossover contradicts the paper's own treatment of parameters. The linear projections producing q and k in Eqs. (2)-(3) are learned parameters updated by backpropagation; after any gradient step, the selected position j* is a deterministic, input-dependent function of trained weights, not a random GP mating operation. The statement that 'it would be much harder to argue that it is mediated by the gradient' does not establish randomness. This invalidates the only concrete architectural mechanism proposed for crossover.
- [Section 6] The sgd diffusion process described as tau' = tau + alpha(grad l + epsilon) perturbs parameters, but the forward attention operation remains a deterministic function of the input for each fixed parameter vector. Parameter-level noise does not make the softmax selection in Eq. (3) a stochastic crossover operator in the GP sense; hence this source of randomness cannot rescue the crossover analogy.
- [Section 7] The paper offers no falsifiable predictions or experimental discriminations. For a hypothesis whose value depends on distinguishing GP-like random combination from learned, input-dependent combination, the absence of any stated empirical criterion makes the central claim untestable as presented.
minor comments (5)
- [Section 1] The word 'somwhat' in the Discussion preview should be 'somewhat'.
- [Section 5] The notation 'question tensor' is nonstandard; 'query tensor' would align with the terminology of Eqs. (2)-(4) and with Vaswani et al.
- [References] The Vaswani et al. entry contains a formatting artifact '/suppress Lukasz Kaiser'; the citation should be cleaned.
- [Section 6] The population estimate of 'over 8 million' per layer counts neurons as potential programs without defining what constitutes an independent GP candidate, so the relevance of the number is unclear.
- [Section 3] The mapping between GP concepts and NN mechanisms would benefit from a systematic table; currently the mapping shifts across sections.
Circularity Check
Minor definitional circularity: the GP identity of tensors is stipulated in Section 1.1, but the main training-as-GP hypothesis is offered as a hypothesis and is not fitted to its own conclusion.
-
self definitional
[Section 1.1 (Preliminaries)]
"We interchangeably make use of the terms features, programs, and tensors, to highlight specific aspects, or simplify the exposition."
The later statement that 'NN neurons or tensors are comparable to GP programs' rests on this interchangeability. If 'program' is defined as the DAG of operations from which a tensor is computed, and 'tensor' is then used interchangeably with 'program', the equivalence of tensors and GP programs is partly a definitional stipulation rather than an independently derived result. The paper does not then fit parameters or use self-citation to force the conclusion, and the overall claim is explicitly a hypothesis, so this is a minor definitional issue rather than a full circular derivation.
full rationale
The paper makes no fitted-input-calls-prediction moves: there are no fitted parameters, no benchmark predictions, and no empirical quantities that are defined in terms of the conclusion. It also does not rely on self-citation: the only references are standard external works on information bottleneck, GP, and Transformers, not prior papers by the same author. The central claim ('neural network training is a form of Genetic Programming') is presented as a hypothesis and supported by analogical mappings: gradient-based feature weighting as GP evaluation/selection, random initialization plus sgd noise as a randomness reservoir, and Transformer attention as pairwise combination. These mappings are genuinely substantive analogies; their weakness (e.g., attention selection being learned rather than random after the first gradient step) is a correctness/falsifiability problem, not a circularity problem. The only notable circular element is the Section 1.1 stipulation that the terms features, programs, and tensors be used interchangeably, which makes the statement that NN tensors are 'comparable to GP programs' true partly by definition. Because this definitional framing is explicit and the main hypothesis still has independent content, the circularity score is low.
Assumptions & free parameters
assumptions (6)
- ad hoc to paper Features, programs, and tensors are interchangeable by definition.
- domain assumption Universal approximation implies no trade-off between feature quality and distribution-fitting quality.
- ad hoc to paper Gradient steps in a neural network are equivalent to fitness evaluation and selection in GP.
- ad hoc to paper Attention pairing in the Transformer is random, not gradient-mediated.
- domain assumption Random initialization and sgd noise provide a reservoir of random combinations across training.
- standard math The chain rule for mutual information used in Eq. (1).
Cite this review
Pith. "Pith review of What are Neural Networks made of?." pith.science (2026). https://pith.science/paper/TGRAVYL5
@misc{pith2026190909588,
author = {Pith},
title = {Pith review of: What are Neural Networks made of?},
year = {2026},
howpublished = {\url{https://pith.science/paper/TGRAVYL5}},
note = {Machine review of arXiv:1909.09588}
}
read the original abstract
The success of Deep Learning methods is not well understood, though various attempts at explaining it have been made, typically centered on properties of stochastic gradient descent. Even less clear is why certain neural network architectures perform better than others. We provide a potential opening with the hypothesis that neural network training is a form of Genetic Programming.
Reference graph
Works this paper leans on
-
[1]
Emergence of invariance and disentanglement in deep representations
Alessandro Achille and Stefano Soatto. Emergence of invariance and disentanglement in deep representations. The Journal of Machine Learning Research, 19 0 (1): 0 1947--1980, 2018
work page 1947
-
[2]
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016
arXiv 2016
-
[3]
Generalized denoising auto-encoders as generative models
Yoshua Bengio, Li Yao, Guillaume Alain, and Pascal Vincent. Generalized denoising auto-encoders as generative models. In Advances in neural information processing systems, pages 899--907, 2013
work page 2013
-
[4]
Gavin Brown, Adam Pocock, Ming-Jie Zhao, and Mikel Luj \'a n. Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. Journal of machine learning research, 13 0 (Jan): 0 27--66, 2012
work page 2012
-
[5]
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018
arXiv 2018
-
[6]
Computational rationality: A converging paradigm for intelligence in brains, minds, and machines
Samuel J Gershman, Eric J Horvitz, and Joshua B Tenenbaum. Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science, 349 0 (6245): 0 273--278, 2015
work page 2015
-
[7]
Estimating information flow in deep neural networks
Ziv Goldfeld, Ewout Van Den Berg, Kristjan Greenewald, Igor Melnyk, Nam Nguyen, Brian Kingsbury, and Yury Polyanskiy. Estimating information flow in deep neural networks. In International Conference on Machine Learning, pages 2299--2308, 2019
work page 2019
-
[8]
Recent advances in convolutional neural networks
Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, Gang Wang, Jianfei Cai, et al. Recent advances in convolutional neural networks. Pattern Recognition, 77: 0 354--377, 2018
work page 2018
Show all 18 references
-
[9]
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015
2015 arXiv
-
[10]
Genetic programming: on the programming of computers by means of natural selection, volume 1
John R Koza. Genetic programming: on the programming of computers by means of natural selection, volume 1. MIT press, 1992
1992
-
[11]
Genetic programming IV: Routine human-competitive machine intelligence, volume 5
John R Koza, Martin A Keane, Matthew J Streeter, William Mydlowec, Jessen Yu, and Guido Lanza. Genetic programming IV: Routine human-competitive machine intelligence, volume 5. Springer Science & Business Media, 2006
2006
-
[12]
The expressive power of neural networks: A view from the width
Zhou Lu, Hongming Pu, Feicheng Wang, Zhiqiang Hu, and Liwei Wang. The expressive power of neural networks: A view from the width. In Advances in neural information processing systems, pages 6231--6239, 2017
2017
-
[13]
Compositional inductive biases in function learning
Eric Schulz, Joshua B Tenenbaum, David Duvenaud, Maarten Speekenbrink, and Samuel J Gershman. Compositional inductive biases in function learning. Cognitive psychology, 99: 0 44--79, 2017
2017
-
[14]
Learning and generalization with the information bottleneck
Ohad Shamir, Sivan Sabato, and Naftali Tishby. Learning and generalization with the information bottleneck. Theoretical Computer Science, 411 0 (29-30): 0 2696--2711, 2010
2010
-
[15]
Opening the black box of deep neural networks via information
Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810, 2017
2017 arXiv
-
[16]
The information bottleneck method
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000
2000 arXiv
-
[17]
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998--6008, 2017
2017
-
[18]
A connection between score matching and denoising autoencoders
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23 0 (7): 0 1661--1674, 2011
2011
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.