REVIEW 4 major objections 4 minor 54 references
A change of variables reorganizes the training Lagrangian into a data-independent bulk and a data-dependent boundary.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 00:15 UTC pith:XVK64RE4
load-bearing objection A checkable algebraic decomposition and an honest sketch of a field theory, but the promised energy continuity equation is missing from the paper. the 4 major comments →
Bulk-boundary decomposition of neural networks
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is an algebraic identity: in the basis where the weights W and the neuron pre-activations z are the independent degrees of freedom, the training Lagrangian of Eq. (5) reorganizes into L = L_bulk + L_boundary. L_bulk couples only adjacent layers and contains no reference to the training data, while L_boundary contains all dependence on the input X and target Y through the loss. This makes manifest that information flow along depth is local, that repeated layer structures give a discrete translational symmetry, and that data enter only at the two ends of the network.
What carries the argument
The key mechanism is a change of variables: the recursive relation z_i^{(m+1)} = sum_j W_ij^{(m)} sigma(z_j^{(m)}) + b_i^{(m)} is solved for the bias, b_i^{(m)} = z_i^{(m+1)} - sum_j W_ij^{(m)} sigma(z_j^{(m)}), and the biases are eliminated as independent parameters. This promotes the pre-activations z to degrees of freedom, rewriting the kinetic and potential terms so that adjacent layers couple only locally, and the loss depends only on the top layer z^{(M)}. The resulting bulk-boundary decomposition (Eq. 7) is then the object from which locality, symmetry, and a field-theoretic continuum limit are extracted.
Load-bearing premise
The load-bearing premise is that the discrete stochastic SGD updates of Eq. (2) can be replaced by the deterministic continuous-time gradient flow of Eq. (3) on a fixed training sample; if that replacement misrepresents the stochastic sampling, all subsequent Lagrangian and field-theoretic claims inherit the error.
What would settle it
Simulate the discrete SGD updates of Eq. (2) on a small network with a fixed sample and compare the resulting trajectories to the continuous-time flow of Eq. (3) at decreasing learning rates; if the flow does not converge to the discrete dynamics, the continuous-time Lagrangian premise fails. (Alternatively, the paper's own text can be checked: the abstract announces an energy continuity equation that the main text never actually writes down.)
If this is right
- One can study architecture-dependent dynamics and data-dependent dynamics separately, isolating the role of the loss from the role of network structure.
- The explicit depth-direction locality permits a continuum limit in which the network becomes a field theory with an emergent coordinate along depth.
- Repeated layer structures give a discrete translational symmetry in depth, opening the door to momentum/energy conservation arguments and renormalization-group-style analyses.
- Boundary stochasticity is naturally a candidate for a statistical-mechanics treatment, potentially linking generalization to effective thermal ensembles.
- The decomposition suggests long-range order may emerge from local training interactions, possibly characterizing successfully trained networks.
Where Pith is reading between the lines
- If the bulk term is truly data-independent, then two networks with identical architecture and different training tasks share the same interior dynamics; a concrete test is to measure bulk-term statistics across tasks.
- The boundary/bulk split suggests that architectural design could be viewed as engineering the bulk Lagrangian, while data curation tunes the boundary, potentially giving a new design principle.
- The missing energy continuity equation, promised in the abstract but absent from the text, is the most direct next step: deriving it from Noether's theorem would give a testable conservation law for training.
- One could test the field-theoretic limit by training narrow-but-deep local architectures (like the illustrated chain) and checking whether coarse-grained observables obey the continuum equations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a "bulk–boundary decomposition" of the training Lagrangian of a deep neural network. Starting from a stochastic-gradient-descent update (Eq. 2) and a continuous-time deterministic gradient-flow approximation (Eq. 3), the authors change variables from weights and biases to weights and pre-activations (Eq. 6), obtaining a Lagrangian (Eq. 7) split into a data-independent bulk term and a data-dependent boundary term. They then sketch continuum field-theory Lagrangians for the boundary and bulk sectors (Eqs. 9–10) and claim, in the abstract, that locality and homogeneity imply an energy continuity equation. The paper is short, programmatic, and explicitly leaves the complete field-theoretic construction to future work.
Significance. The algebraic identity in Eq. (7) is checkable and appears valid; promoting pre-activations to dynamical variables to expose adjacent-layer locality is a reasonable idea. If the advertised energy continuity equation were actually derived and the continuum limit made rigorous, the framework could offer a useful physical perspective on deep-learning dynamics. However, as submitted, the central advertised consequence is absent from the text, the field-theory section is explicitly illustrative rather than derived, and the connection to stochastic gradient descent is not justified. The paper does not currently establish the new framework it claims.
major comments (4)
- [Abstract; Sections 'Bulk–Boundary Decomposition' and 'Field Description'] The abstract states: 'As a physical consequence of locality and homogeneity, we derive the energy continuity equation within a deep neural network.' The main text contains no such equation, no Noether current, and no conservation law. The only symmetry discussed is the discrete translational symmetry m→m+1 of L_bulk, but no continuity equation is constructed from it. This is the advertised physical consequence, so the paper's central claim is unsupported.
- [Field Description, Eqs. (9)–(10), footnote 5] The continuum Lagrangians in Eqs. (9)–(10) are not derived. Footnote 5 states that the complete Lagrangian depends on a specific coordinate choice 'which we do not detail here' and that only representative terms are shown. No lattice-spacing expansion, no coordinate assignment for X, Y, z, w, and no limit is specified. The reader cannot reproduce or verify Eqs. (9)–(10); they are at most a sketch, not a field-theoretic formulation.
- [Fundamentals, Eq. (3)] The continuous-time limit Eq. (3) is not the SGD dynamics defined in Eq. (2). In SGD the pair (X, Y) is drawn randomly at each step, so the loss ℓ is a stochastic function of time; replacing it by a fixed-sample deterministic gradient flow changes the process. Consequently, the Lagrangian L(t) in Eq. (5) and any conservation law derived from it do not describe stochastic gradient descent. This is load-bearing because the paper motivates the entire framework from the SGD formulation.
- [Bulk–Boundary Decomposition, Eq. (7)] The claimed bulk–boundary separation is a definitional regrouping: terms explicitly containing X, Y, or ℓ are assigned to L_boundary, and the remaining terms to L_bulk. The identity is valid, but the paper does not demonstrate that this regrouping yields new analytic power. No observable, conserved quantity, or training-related quantity is computed from the decomposition to show that it 'reveals' or 'governs' the claimed structure beyond the algebra.
minor comments (4)
- [Eq. (7)] The first term in L_boundary appears to be missing the factor 1/2 that follows from expanding (1/2)(b_i^(0))^2 in Eq. (5) for m=0; please check the coefficient.
- [Footnote 5 and Eqs. (9)–(10)] Footnote 5 contradicts the presentation of Eqs. (9)–(10) as derived results; the text should either label these as a proposed expansion or provide the missing coordinate assignment and expansion.
- [Eq. (8)] The integral kernel W(x,x') is not related explicitly to the discrete weights W_ij^(m); the continuum limit that gives Eq. (8) should be stated.
- [Field Description, Eq. (10)] The term 'homogeneous' is used loosely: L_bulk in Eq. (10) contains terms of different orders in a_x and a_y, so translational symmetry holds only in the continuum limit and for a repeated layer structure, which should be clarified.
Circularity Check
No significant circularity: Eq. (7) is an exact algebraic reorganization after a change of variables; the advertised energy continuity equation is absent, but that is a missing result, not a circular reduction.
full rationale
The paper's derivation is self-contained algebra. Starting from Eq. (1), it substitutes b_i^(m) = z_i^(m+1) - sum_j W_ij^(m) sigma(z_j^(m)) (Eq. 6) into the Lagrangian (5), and Eq. (7) is the resulting identity reorganized into terms with no explicit X,Y (L_bulk) and terms with explicit X,Y (L_boundary). The split is therefore grouping by construction, but this is not a prediction, a fitted parameter, or a fitted-quantity-renamed-as-prediction: no parameter is fit to data. The self-citations (Refs. [6] and [43]) are contextual in footnote/citation lists and do not carry the load-bearing substitution or any uniqueness claim. I flag, as required, that the Abstract states 'As a physical consequence of locality and homogeneity, we derive the energy continuity equation within a deep neural network,' but the body never derives such an equation: no continuity equation, Noether current, or conservation law appears in 'Field Description' or 'Discussion and Outlook.' That absence is a substantial completeness/correctness gap in the advertised physical consequence, but it is not a circular step and does not make Eq. (7) equivalent to its input. The bulk-boundary identity is independently checkable, so the circularity score is 0.
Axiom & Free-Parameter Ledger
free parameters (2)
- a_x (lattice spacing along depth)
- a_y (lattice spacing along width)
axioms (6)
- standard math The e^{gamma*t}-weighted Lagrangian (Eq. 5) yields the damped equations of motion (Eq. 4) via the Euler-Lagrange equations.
- domain assumption SGD can be approximated by continuous-time deterministic gradient flow Wdot=-eta*d_W*loss, bdot=-eta*d_b*loss for a sample (X,Y).
- standard math The substitution b_i^(m) = z_i^(m+1) - sum_j W_ij^(m) sigma(z_j^(m)) (Eq. 6) is a valid change of variables that preserves the Lagrangian dynamics.
- domain assumption Locality in the width direction requires restricting to a local nearest-neighbor architecture with periodic boundary conditions.
- domain assumption Discrete translation symmetry of the bulk requires repeating layer structure.
- ad hoc to paper A well-defined coordinate assignment of z,w,X,Y exists for the boundary field theory; the paper does not specify it.
invented entities (2)
-
Bulk-boundary decomposition (bulk vs. boundary degrees of freedom)
no independent evidence
-
Energy continuity equation
no independent evidence
read the original abstract
We present the bulk--boundary decomposition as a new framework for understanding the training dynamics of deep neural networks. Starting from the stochastic gradient descent formulation, we show that the Lagrangian can be reorganized into a data-independent bulk term and a data-dependent boundary term. The bulk captures the intrinsic dynamics set by network architecture and activation functions, while the boundary reflects stochastic interactions from training samples at the input and output layers. This decomposition exposes the local and homogeneous structure underlying deep networks. As a physical consequence of locality and homogeneity, we derive the energy continuity equation within a deep neural network.
Figures
Reference graph
Works this paper leans on
-
[1]
F.-L. Fan, J. Xiong, M. Li, and G. Wang, On inter- pretability of artificial neural networks: A survey, IEEE Trans. Radiat. Plasma Med. Sci.5, 741 (2021)
2021
-
[2]
X. Li, H. Xiong, X. Li, X. Wu, X. Zhang, J. Liu, J. Bian, and D. Dou, Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond, Knowl. Inf. Syst.64, 3197 (2022)
2022
-
[3]
Bahri, J
Y. Bahri, J. Kadmon, J. Pennington, S. S. Schoenholz, J. Sohl-Dickstein, and S. Ganguli, Statistical mechanics of deep learning, Annu. Rev. of Condens. Matter Phys. 11, 501 (2020)
2020
-
[4]
S. S. Schoenholz, J. Gilmer, S. Ganguli, and J. Sohl-Dickstein, Deep information propagation, arXiv:1611.01232
-
[5]
G. A. D’Inverno, Z. Hu, L. Davy, M. Unser, G. Rozza, and J. Dong, Revisiting deep information propagation: Fractal frontier and finite-size effects, arXiv:2508.03222
-
[6]
D. Lee, H.-S. Lee, and J. Yi, Dynamic neuron approach to deep neural networks: Decoupling neurons for renormal- ization group analysis, Phys. Rev. Res.7, 023276 (2025), arXiv:2410.00396 [cond-mat.stat-mech]
Pith/arXiv arXiv 2025
-
[7]
H. You, Y. Yu, M. D’Elia, T. Gao, and S. Silling, Non- local kernel network (nkn): A stable and resolution- independent deep neural network, Journal of Computa- tional Physics469, 111536 (2022)
2022
-
[8]
S. Lanthaler, Z. Li, and A. M. Stuart, Nonlocality and nonlinearity implies universality in operator learning, arXiv:2304.13221
-
[9]
Carnevali and S
P. Carnevali and S. Patarnello, Exhaustive thermody- namical analysis of boolean learning networks, Europhys. Lett.4, 1199 (1987)
1987
-
[10]
Tishby, E
N. Tishby, E. Levin, and S. A. Solla, Consistent inference of probabilities in layered networks: Predictions and gen- eralizations, inProceedings IEEE International Confer- ence on Neural Networks(IEEE, Piscataway, NJ, 1989) pp. 403–409
1989
-
[11]
Sompolinsky, N
H. Sompolinsky, N. Tishby, and H. S. Seung, Learning from examples in large neural networks, Phys. Rev. Lett. 65, 1683 (1990)
1990
-
[12]
Levin, N
E. Levin, N. Tishby, and S. A. Solla, A statistical ap- proach to learning and generalization in layered neural networks, Proc. IEEE78, 1568 (1990)
1990
-
[13]
H. S. Seung, H. Sompolinsky, and N. Tishby, Statistical mechanics of learning from examples, Phys. Rev. A45, 6056 (1992)
1992
-
[14]
Engel and C
A. Engel and C. Van den Broeck,Statistical mechanics of learning(Cambridge University Press, 2001)
2001
-
[15]
M´ ezard and A
M. M´ ezard and A. Montanari,Information, physics, and computation(Oxford University Press, 2009)
2009
-
[16]
Gy¨ orgyi, First-order transition to perfect generaliza- tion in a neural network with binary synapses, Phys
G. Gy¨ orgyi, First-order transition to perfect generaliza- tion in a neural network with binary synapses, Phys. Rev. A41, 7097 (1990)
1990
-
[17]
Koebarle and W
R. Koebarle and W. K. Theumann,Neural Networks and Spin Glasses—Proceedings of the Statphys 17 Workshop (World Scientific, Singapore, 1990)
1990
-
[18]
Advani, S
M. Advani, S. Lahiri, and S. Ganguli, Statistical mechan- ics of complex neural systems and high dimensional data, J. Stat. Mech. , P03014 (2013)
2013
-
[19]
P. Mehta and D. J. Schwab, Exact mapping between the variational renormalization group and deep learning, arXiv:1410.3831
-
[20]
L. Zdeborov´ a and F. Krzakala, Statistical physics of inference: thresholds and algo- rithms, Advances in Physics65, 453 (2016), https://doi.org/10.1080/00018732.2016.1211393
arXiv 2016
-
[21]
D. Ghio, Y. Dandi, F. Krzakala, and L. Zdeborov´ a, Sam- pling with flows, diffusion, and autoregressive neural net- works from a spin-glass perspective, Proceedings of the National Academy of Sciences121, e2311810121 (2024), https://www.pnas.org/doi/pdf/10.1073/pnas.2311810121
-
[22]
J. Lee, Y. Bahri, R. Novak, S. S. Schoenholz, J. Pen- nington, and J. Sohl-Dickstein, Deep neural networks as gaussian processes, arXiv:1711.00165 [stat.ML]
-
[23]
S. S. Schoenholz, J. Pennington, and J. Sohl-Dickstein, A correspondence between random neural networks and statistical field theory, arXiv:1710.06570 [stat.ML]
-
[24]
S. Sonoda and N. Murata, Transport analysis of infinitely deep neural network, arXiv:1605.02832 [cs.LG]
-
[25]
E. Dyer and G. Gur-Ari, Asymptotics of Wide Networks from Feynman Diagrams, arXiv:1909.11304 [cs.LG]
Pith/arXiv arXiv 1909
-
[26]
Sho, Non-gaussian processes and neural networks at finite widths, arXiv:1910.00019 [stat.ML]
Y. Sho, Non-gaussian processes and neural networks at finite widths, arXiv:1910.00019 [stat.ML]
Pith/arXiv arXiv 1910
-
[27]
Lee, Quantum fields as deep learning, J
J.-W. Lee, Quantum fields as deep learning, J. Korean Phys. Soc.76, 684 (2020), arXiv:1708.07408 [physics.gen- ph]
Pith/arXiv arXiv 2020
-
[28]
Helias and D
M. Helias and D. Dahmen,Statistical Field Theory for Neural Networks, Vol. 970 (2020)
2020
-
[29]
H. Erbin, V. Lahoche, and D. O. Samary, Non- perturbative renormalization for the neural network- QFT correspondence, Mach. Learn. Sci. Tech.3, 015027 (2022), arXiv:2108.01403 [hep-th]
Pith/arXiv arXiv 2022
-
[30]
H. Erbin, V. Lahoche, and D. O. Samary, Renormaliza- tion in the neural network-quantum field theory corre- spondence (2022) arXiv:2212.11811 [hep-th]
Pith/arXiv arXiv 2022
-
[31]
Halverson, Building Quantum Field Theories Out of Neurons, arXiv:2112.04527 [hep-th]
J. Halverson, Building Quantum Field Theories Out of Neurons, arXiv:2112.04527 [hep-th]
-
[32]
J. Halverson, A. Maiti, and K. Stoner, Neural Networks and Quantum Field Theory, Mach. Learn. Sci. Tech.2, 035002 (2021), arXiv:2008.08601 [cs.LG]
Pith/arXiv arXiv 2021
-
[33]
R. Bondesan and M. Welling, The hintons in your neural network: a quantum field theory view of deep learning, arXiv:2103.04913 [quant-ph]
-
[34]
Segadlo, B
K. Segadlo, B. Epping, A. van Meegen, D. Dahmen, M. Kr¨ amer, and M. Helias, Unified field theoretical ap- proach to deep and recurrent neuronal networks, J. Stat. Mech.2022, 103401 (2022)
2022
-
[35]
S. Krippendorf and M. Spannowsky, A duality connecting neural network and cosmological dynamics, Mach. Learn. Sci. Tech.3, 035011 (2022), arXiv:2202.11104 [gr-qc]
Pith/arXiv arXiv 2022
-
[36]
Cohen, O
O. Cohen, O. Malka, and Z. Ringel, Learning curves for overparametrized deep neural networks: A field theory perspective, Phys. Rev. Res.3, 023034 (2021)
2021
-
[37]
A. Maiti, K. Stoner, and J. Halverson, Symmetry- via-Duality: Invariant Neural Network Densities from Parameter-Space Correlators, arXiv:2106.00694 [cs.LG]
-
[38]
J. Halverson, J. Naskar, and J. Tian, Conformal Fields from Neural Networks, arXiv:2409.12222 [hep-th]
-
[39]
Demirtas, J
M. Demirtas, J. Halverson, A. Maiti, M. D. Schwartz, and K. Stoner, Neural network field theories: non- gaussianity, actions, and locality, Mach. Learn. Sci. Tech- nol.5, 015002 (2024)
2024
-
[40]
Vanchurin, Emergent field theories from neural net- 6 works, arXiv:2411.08138 [hep-th]
V. Vanchurin, Emergent field theories from neural net- 6 works, arXiv:2411.08138 [hep-th]
-
[41]
J. N. Howard, M. S. Klinger, A. Maiti, and A. G. Staple- ton, Bayesian RG flow in neural network field theories, SciPost Phys. Core8, 027 (2025), arXiv:2405.17538 [hep- th]
Pith/arXiv arXiv 2025
-
[42]
Z. Ringel, N. Rubin, E. Mor, M. Helias, and I. Seroussi, Applications of statistical field theory in deep learning, arXiv:2502.18553 [stat.ML]
-
[43]
Lee, H.-S
D. Lee, H.-S. Lee, and J. Yi, Synaptic field theory for neural networks, Phys. Rev. D112, L031902 (2025)
2025
-
[44]
W.-C. Gan and F.-W. Shu, Holography as deep learning, Int. J. Mod. Phys. D26, 1743020 (2017), arXiv:1705.05750 [gr-qc]
Pith/arXiv arXiv 2017
-
[45]
K. Hashimoto, S. Sugishita, A. Tanaka, and A. Tomiya, Deep learning and the AdS/CFT correspondence, Phys. Rev. D98, 046019 (2018), arXiv:1802.08313 [hep-th]
Pith/arXiv arXiv 2018
-
[46]
K. Hashimoto, S. Sugishita, A. Tanaka, and A. Tomiya, Deep Learning and Holographic QCD, Phys. Rev. D98, 106014 (2018), arXiv:1809.10536 [hep-th]
Pith/arXiv arXiv 2018
-
[47]
Hashimoto, AdS/CFT correspondence as a deep Boltzmann machine, Phys
K. Hashimoto, AdS/CFT correspondence as a deep Boltzmann machine, Phys. Rev. D99, 106017 (2019), arXiv:1903.04951 [hep-th]
Pith/arXiv arXiv 2019
-
[48]
Parisi,Statistical Field Theory, Frontiers in Physics (Addison-Wesley, Redwood City, CA, 1988)
G. Parisi,Statistical Field Theory, Frontiers in Physics (Addison-Wesley, Redwood City, CA, 1988)
1988
-
[49]
T. L. H. Watkin, A. Rau, and M. Biehl, The statistical mechanics of learning a rule, Rev. Mod. Phys.65, 499 (1993)
1993
-
[50]
S. Jadon, A survey of loss functions for semantic seg- mentation, in2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biol- ogy (CIBCB)(IEEE, 2020) pp. 1–7
2020
-
[51]
L. Ciampiconi, A. Elwood, M. Leonardi, A. Mohamed, and A. Rozza, A survey and taxonomy of loss functions in machine learning, arXiv:2301.05579
-
[52]
V. L. Ginzburg and L. D. Landau, On the theory of su- perconductivity, Zh. Eksp. Teor. Fiz.20, 1064 (1950)
1950
-
[53]
K. G. Wilson and J. Kogut, The renormalization group and theϵexpansion, Physics Reports12, 75 (1974)
1974
-
[54]
Aizenman and R
M. Aizenman and R. Fern´ andez, On the critical behavior of the magnetization in high-dimensional ising models, Journal of Statistical Physics44, 393 (1986)
1986
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.