Pith. sign in

REVIEW 4 major objections 6 minor 42 references

Growing Neural Networks: Dynamic Evolution through Gradient Descent

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A neural network that grows during training can beat a static network of the same final size, and can match static accuracy with half the neurons.

desk verdict The core mechanism is simple and reproducible, but the 'consistently outperform' claim is undercut by the paper's own hyperparameter sweeps and an under-tuned static baseline. read the letter →

arxiv 2501.18012 v2 pith:TNOJFW5V submitted 2025-01-29 cs.LG cond-mat.dis-nn

Anil Radhakrishnan , John F. Lindner , Scott T. Miller , Sudeshna Sinha , William L. Ditto This is my paper · ORCID
classification cs.LGcond-mat.dis-nn
keywords growingneuralnetworksgradientdescentnetworksizeoptimizationauxiliaryweightcontrollermasksmoothtransitionfunctionspiralclassificationlosslandscape
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a neural network whose width grows during gradient-descent training can outperform a static network of the same final width, even though both end with the same number of neurons. It makes network size itself a differentiable parameter in two ways: an auxiliary weight that directly sets the number of active hidden neurons, and a controller-generated mask that smoothly switches neurons on. On nonlinear regression with Bessel functions and on spiral classification, the growing networks consistently reach lower final loss or higher accuracy than their static counterparts. The authors report a concrete size-efficiency relation: growing networks achieve roughly the same accuracy as static networks while using only half the neurons, $A_g(N) \approx A_s(2N)$. If this holds, the training trajectory itself becomes an optimization target, and automatic size control during training could reduce the compute and energy cost of large networks.

What carries the argument

The machinery is a continuous, differentiable stand-in for the discrete decision of how many neurons are active. The transition function $\psi(x)=1$ for $x<-1$, $\psi(x)=\sin^2(\pi x/2)$ for $-1\le x\le 0$, and $\psi(x)=0$ for $x>0$ turns neuron counts into smooth masks. In the auxiliary-weight algorithm, a prepended identity neuron carries weight $N=w^0_{11}$, and each hidden neuron $i$ is scaled by $\psi(i-N)$, so increasing $N$ smoothly activates new neurons; the size loss $L_{\rm size}=(N-N_{\rm target})^2$ keeps growth anchored. In the controller-mask algorithm, a controller outputs $C_1$ and the effective size is $\tilde N = N\sin^2(\pi C_1/2)$, with a masked step function equivalent to $\psi$. Both formulations put the size parameter on the same footing as weights and biases, so the same gradient descent updates all of them together.

What would settle it

A decisive check is to give the static baseline the same per-size tuning that the growing network receives: sweep the learning rate and training duration for each fixed size, train until early-stopping convergence, and compare best losses. If a tuned static network of final size $N$ matches or beats the growing network of size $N$, the reported half-neuron advantage would disappear.

Watch

Extended reading notes

Core claim

The central claim is that the path a network takes through parameter space matters as much as where it ends. A network that starts with one or a few active neurons and continuously activates more during training can beat a network initialized directly at that final size. The signature result is $A_g(N) \approx A_s(2N)$ on the spiral classification task: a growing network with $N$ hidden neurons is as accurate as a static network with $2N$ hidden neurons. The paper attributes the advantage to smaller networks having simpler loss landscapes, so the growing trajectory bypasses local minima that trap the larger static network, and it derives convergence bounds from $\beta$-smoothness and convexity of the quadratic size loss. The advantage is not claimed to be universal: it appears in an intermediate regime of training duration, size-loss coupling, and learning rate, where the network matures just as training ends; with very fast growth or very short training the static network wins.

Load-bearing premise

The comparison assumes that a static network trained from the start at the final size, with the same learning rate, initialization, and epoch count, is a fair and well-tuned baseline; the paper does not tune the static baseline per size.

Editorial extensions

If this is right

  • If the trajectory advantage is real, architecture search and fixed-size training are missing an optimization axis: the size of each layer can be tuned continuously during training instead of being fixed before training.
  • The relation $A_g(N) \approx A_s(2N)$ implies that, on the tested spiral task, a growing network can deliver the accuracy of a twice-as-large static network, roughly halving the parameters needed at a given accuracy.
  • Because growth must finish near the end of training, the method turns the training budget into a design input: the size-loss coupling and learning rate should be chosen so that growth and learning saturate simultaneously.
  • With adaptive optimizers such as Adam, the explicit size-loss coupling constant is redundant, so the same size-control behavior can be obtained without tuning $\lambda$; the paper shows this by the cancellation of $\lambda$ in the moment ratio.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test that goes beyond the paper's low-dimensional tasks is to transfer the same two algorithms to standard image benchmarks; if the $A_g(N) \approx A_s(2N)$ relation survives, the trajectory benefit is not specific to Bessel and spiral problems.
  • The paper's own hyperparameter scans imply that the best operating point is a race between growth time and training time; this suggests a testable scheduling rule, such as making the growth rate a decreasing function of residual loss, which the authors do not propose.
  • The same smooth-transition machinery could be applied to layer depth or to bidirectional size changes (growing then pruning), so the method is a template for a wider class of differentiable architecture parameters rather than only hidden-layer width.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes two gradient-based methods for growing the width of a feed-forward network during training: an auxiliary-weight scheme that identifies a single trainable weight with network size, and a controller-mask scheme that learns a scalar mask controlling how many neurons participate. The authors evaluate these methods on Bessel-function regression and spiral classification, report that growing networks "consistently outperform static networks of equivalent final size," present hyperparameter sweeps over training duration E, size-loss coupling λ, and learning rate η, and claim the scaling relation A_g(N) ≈ A_s(2N). They also provide a convergence framework based on smoothness and a descent lemma, plus an appendix arguing that λ is redundant under Adam. The paper includes a public code repository.

Significance. If the central claim were established, the paper would make a useful contribution to differentiable architecture optimization: it would show that the training trajectory itself, not just the final architecture, affects performance and that network size can be optimized by gradient descent. The methods are simple, the presentation is clear, and the authors provide code, which strengthens reproducibility. However, the headline claim of consistent superiority is not supported by the paper's own sweeps, and the static baseline is not tuned or controlled in a way that isolates the effect of growth. The A_g(N) ≈ A_s(2N) relation is an interesting empirical observation, but it rests on an undertrained static baseline, so its quantitative form is not yet established. The theoretical section states convergence claims without supplying proofs, which limits its value. Overall, the paper reports a promising phenomenon but needs substantial revision to support its claims.

major comments (4)
  1. [Section I and Figures 6-8] The statement in Section I that growing networks "can consistently outperform static networks of equivalent final size" is contradicted by the paper's own hyperparameter sweeps. Figure 7 shows R = L_g/L_s > 1 for small λ and short E, and Figure 8 shows δL > 0 (static networks better) for small and large learning rates. The claimed advantage holds only in an intermediate regime of growth speed relative to training duration. The word "consistently" must either be removed or supported by evidence across a well-defined task/regime, or the central claim should be reframed as a regime-dependent advantage.
  2. [Section IV, Figures 7-9] The static baseline is not fairly tuned. Static networks are trained with the same learning rate, epoch count, initialization, and optimizer as the growing network, with no size-adjusted learning-rate schedule or epoch budget, while growing networks receive an additional size-loss term and, in the controller-mask variant, an extra controller input. Larger static networks generally converge more slowly at fixed epoch count, so the finding in Figure 9 that growing networks reach comparable accuracy with half the neurons may reflect undertrained baselines rather than a genuine efficiency advantage. The authors should either tune the static baseline per size (e.g., by scanning learning rate and epochs) or demonstrate that the reported advantage survives such tuning.
  3. [Section IV.D, Eq. (29)] The relation A_g(N) ≈ A_s(2N) is presented as a central quantitative result, but it is derived from a single classification task (spiral data) at a single epoch budget and with the untuned baseline described above. The text itself limits it to "this classification task." As stated, the relation is not a general scaling law. To make it load-bearing, the authors need at least a second task and a demonstration that the 2× factor is robust to baseline tuning and to changes in training duration and learning rate.
  4. [Section II, Eqs. (12)-(18)] The convergence analysis is asserted rather than proved. Equation (15) claims a descent lemma with coefficient -η/2, but the proof requires β-smoothness of the full augmented loss L = L_base + λ L_size, including the piecewise ψ transition, and the text does not establish β or verify the condition η < 2/β. Equation (16) introduces a bound M without specifying the Lipschitz constant in terms of ψ and the loss gradients, and Eq. (18) mixes conditions on η without derivation. Either provide a complete proof or clearly label these as heuristic stability conditions rather than theorems.
minor comments (6)
  1. [Equations (9)-(10)] The notation for gradients is incomplete: ∂L/∂θ is shown as "L_base/∂θ" and ∂L/∂α as "L_base/∂α"; these should be ∂L_base/∂θ and ∂L_base/∂α, plus the corresponding λ terms.
  2. [Figures 4 and 5 captions] The captions refer to the "control-mask algorithm" while the rest of the text uses "controller-mask algorithm." Please unify the terminology.
  3. [Section IV.A] There is a typo: "This task serves to strengthen our result for auxiliary-weight algorithm by by using a harder problem" should read "...by using a harder problem."
  4. [Equation (23) and surrounding text] The same symbol N is used both for the maximum number of neurons and for the effective network size; the tilde notation in Eq. (23) is helpful but the text should define N_max explicitly to avoid confusion.
  5. [Section V and Reference [28]] The final sentence, "After completing this manuscript, related work [28] came to our attention," is not a substitute for an in-text discussion. The authors should briefly explain how their approach relates to Evci et al. and what the differences are.
  6. [Appendix A] The argument that λ is redundant under Adam is a first-order cancellation that ignores the effect of bias correction and the interaction between the base and size-loss gradients in the optimizer's second moment. The claim should be softened unless the authors provide a more careful treatment or empirical verification.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the growing-network results are empirical and are not defined or fitted into existence.

full rationale

I walked the paper's derivation chain: the theoretical framework (Eqs. 1-18), the two implementations (auxiliary-weight, Eqs. 19-22, and controller-mask, Eqs. 23-26), the empirical sweeps (Figs. 6-8), and the scaling relation (Eq. 29). The network size is genuinely a trainable parameter: N = w_11^0 in the auxiliary-weight scheme and a controller-generated mask in the controller-mask scheme, both updated by gradient descent. No load-bearing claim reduces to a definition or to a fitted parameter renamed as a prediction. The scaling relation A_g(N) ≈ A_s(2N) in Eq. 29 is an empirical summary of the authors' own simulations, not a held-out prediction derived from a fitted quantity, so it may be fragile or confounded by undertrained static baselines, but it is not circular. The paper contains no load-bearing self-citations: the only nearby citation, Ref. [19], is external prior work for the auxiliary-weight idea and is not used to justify the claimed performance advantage; the GitHub reference is an implementation detail. The observation that growing networks lose to static networks in parts of the hyperparameter sweeps (Figs. 6-8) undermines the word 'consistently' but is an empirical inconsistency, not a circularity. The appendix's claim about λ-invariance under Adam is mathematically questionable, but that is a correctness issue, not a circular reduction. No step in the manuscript asserts a prediction that is equivalent by construction to its inputs, so the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities or unobserved objects are introduced; the controller and auxiliary weight are standard trainable modules. The free parameters and ad hoc design choices (ψ, size loss, mask ordering) are the main hidden costs that the central claim depends on.

free parameters (4)
  • size-loss coupling λ = 0.1 (auxiliary), 1 (controller-mask per Appendix A)
    Chosen by hand; Figures 6-7 show the growth advantage exists only for intermediate λ; small λ yields R>1 (static better).
  • target size n∞ = 5 (auxiliary), 10 (controller-mask)
    Defines 'equivalent final size' for static baselines; performance comparisons depend on this choice.
  • learning rate η = 0.001 (most runs)
    Figure 8 shows the controller-mask advantage is confined to an intermediate η range; at low η the network stays small, at high η it grows fast and matches static.
  • scaling law prefactor and exponent in E ≈ 2000 λ^-0.8 = 2000, -0.8
    Fitted to the density plot in Figures 6-7; not derived from first principles and not validated out of sample.
assumptions (5)
  • domain assumption The loss L = Lbase + λLsize is β-smooth in θ and α jointly (Eq. 12).
    The transition function ψ in Eq. 6 is only C^1, not C^2, and β-smoothness is asserted with citation, not proven for this specific loss.
  • ad hoc to paper Network size can be represented by a single scalar that activates neurons in a fixed order.
    Auxiliary weight N = w0_11 and mask M(C1,n) both assume neurons are ordered and activated by index; this ordering is arbitrary and limits expressiveness.
  • ad hoc to paper The quadratic size loss Lsize = (N - Ntarget)^2 and the sinusoidal mapping Ñ = N sin²(π/2 C1) are appropriate size controls.
    Many alternative smooth size-loss forms exist; the choice is not justified beyond convenience.
  • domain assumption Adam's gradient scaling makes λ redundant (Appendix A).
    The ratio argument m/√v ~ sign(∇) treats each gradient component in isolation and ignores coupling between Lbase and Lsize terms and moment accumulation; this is heuristic.
  • standard math Convergence of gradient descent requires the descent lemma (Eq. 15) for η < 2/β.
    Standard optimization theory; cited but not rederived.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Growing Neural Networks: Dynamic Evolution through Gradient Descent." pith.science (2026). https://pith.science/paper/TNOJFW5V

@misc{pith2026250118012,
  author       = {Pith},
  title        = {Pith review of: Growing Neural Networks: Dynamic Evolution through Gradient Descent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TNOJFW5V}},
  note         = {Machine review of arXiv:2501.18012}
}
read the original abstract

In contrast to conventional artificial neural networks, which are structurally static, we present two approaches for evolving small networks into larger ones during training. The first method employs an auxiliary weight that directly controls network size, while the second uses a controller-generated mask to modulate neuron participation. Both approaches optimize network size through the same gradient-descent algorithm that updates the network's weights and biases. We evaluate these growing networks on nonlinear regression and classification tasks, where they consistently outperform static networks of equivalent final size. We then explore the hyperparameter space of these networks to find associated scaling relations relative to their static counterparts. Our results suggest that starting small and growing naturally may be preferable to simply starting large, particularly as neural networks continue to grow in size and energy consumption.

Figures

Figures reproduced from arXiv: 2501.18012 by the authors.

Figure 1
Figure 1. FIG. 1. Loss function drives network size from 0 to 5 hidden neurons via gradient descent. At each training round or epoch [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2. Controller-mask paradigm schematic for up to [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 2
Figure 2. figure 2 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figures from the paper (7 more)
Figure 3
Figure 3. Figure 3: FIG. 3. Auxiliary-weight algorithm nonlinear regression example. Training a growing network via a loss function (left) versus [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 4
Figure 4. Figure 4: FIG. 4. Control-mask algorithm nonlinear regression example, target equation 28 composite Bessel function (left) and loss [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: FIG. 5. Control-mask algorithm classification example, target spirals (left) and loss versus training epoch (right) for 2 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: FIG. 6. Performance dependence of auxiliary-weight algorithm for the equation 27 simple Bessel function nonlinear regression. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: FIG. 7. Relative Performance dependence of auxiliary-weight algorithm for the equation 27 simple Bessel function nonlinear [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: FIG. 8. Scaling of controller-mask equation 28 composite Bessel function regression versus number of training epochs [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: FIG. 9. Scaling of controller-mask spiral pattern classification. 3D plot shows that accuracy [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 27 canonical work pages

  1. [1]

    This requires careful design of differentiable transitions between network sizes

    Structural modification: One can directly mod- ify the network’s structure through continuous pa- rameters that control the effective presence or ab- sence of neurons. This requires careful design of differentiable transitions between network sizes

    work page

  2. [2]

    Both approaches require a smooth transition function ψ(x) that approximates a step function while maintain- ing differentiability

    Activity modulation: Alternatively, one can maintain a fixed maximum-size network but con- tinuously control the contribution of each neuron through differentiable scaling factors. Both approaches require a smooth transition function ψ(x) that approximates a step function while maintain- ing differentiability. We choose ψ(x) =    1, x < −1, sin2( π 2...

    work page

  3. [3]

    on” (white squares), one partially “on

    β-smoothness: For all θ1, θ2, α1, α2, ∥∇L(θ1, α1)−∇L(θ2, α2)∥ ≤β∥(θ1, α1)−(θ2, α2)∥. (12) 3 FIG. 1. Loss function drives network size from 0 to 5 hidden neurons via gradient descent. At each training round or epoch E, lines represent weights and circles biases, partially labeled at bottom, thicknesses are proportional to magnitudes, red is positive and bl...

    work page

  4. [4]

    Lower boundedness: For all θ, α, L(θ, α) ≥ Lmin > −∞. (13)

    work page

  5. [5]

    Size-loss Convexity: The size loss Lsize is convex in α due to its quadratic form Lsize(α) = (N (α) − Ntarget)2. (14) The β-smoothness property is inherited from the neu- ral network architecture with bounded activation func- tions [16], while the transition function ψ(x) ensures smoothness during size transitions. For learning rate η < 2/β, we can prove ...

    work page

  6. [6]

    The auxiliary-weight algorithm, which directly modifies network structure

    work page

  7. [7]

    The controller-mask algorithm, which modulates neuron participation. A. Auxiliary-W eight Algorithm The auxiliary-weight algorithm achieves network growth by identifying network size with a learnable pa- rameter in the network itself. Consider a feed-forward neural network with one hid- den layer. We prepend a layer 0 that consists of a sin- gle input fix...

    work page

  8. [8]

    Z. Fan, G. Chen, and K. T. Ko, Journal of Control Theory and Applications 2, 60–64 (2004)

    work page 2004

Show all 42 references
  1. [9]

    Input Layer (Layer 0): N = w0 11 = a0 1 = id(w0 11 · 1 + 0), (19) where the prepended 1 is converted to N through an identity activation with zero bias

  2. [10]

    Hidden Layer (Layer 1): a1 i = ψi−N σ(w1 i1x + b1 i ), (20) where ψ modulates activation based on relative po- sition to N

  3. [11]

    This technique of using an auxiliary weight to set neural network constraints was first used by Jin et al

    This weight serves as our size parame- ter N = w0 11, controlling the effective number of active neurons in the hidden layer. This technique of using an auxiliary weight to set neural network constraints was first used by Jin et al. in the context of physics-informed neural ne...

  4. [12]

    Output Layer (Layer 2): ˆy = NmaxX i=1 w2 1ia1 i + b2

  5. [13]

    (22) As N increases, new neurons smoothly activate and join the computation, as shown in figure 1

    (21) During training, the network size N evolves alongside other parameters through gradient descent Nt+1 = Nt − η ∂L ∂N . (22) As N increases, new neurons smoothly activate and join the computation, as shown in figure 1. B. Controller-Mask Algorithm The controller-mask algori...

  6. [14]

    A standard multilayer perceptron (MLP) with fixed maximum size

  7. [15]

    bigger is better

    A controller that generates masking values. The controller outputs a value C1 := C(1) that deter- mines the effective network size, ˜N = N sin2 π 2 C1 . (23) The output of neuron n is masked by M(C1, n) =    1, n < ⌊ ˜N ⌋, ˜N − ⌊˜N ⌋, n = ⌊ ˜N ⌋, 0, n > ⌊ ˜N ⌋, (24) which...

  8. [16]

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Ka- plan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sas- try, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray,...

    2020 arXiv

  9. [17]

    Strubell, A

    E. Strubell, A. Ganesh, and A. McCallum, arXiv:1906.02243 (2019)

    2019 arXiv

  10. [18]

    Ai and compute,

    D. Amodei and D. Hernandez, “Ai and compute,” (2018), accessed: 2025-02-14

    2018

  11. [19]

    Frankle and M

    J. Frankle and M. Carbin, arXiv:1803.03635 (2018)

    2018 arXiv

  12. [20]

    See, M.-T

    A. See, M.-T. Luong, and C. D. Manning, arXiv:1606.09274 (2016)

    2016 arXiv

  13. [21]

    M. Zhao, N. Wang, X. Jiang, X. Ma, H. Ma, G. He, K. Du, L. Ma, and T. Huang, Nature Computational Science 4, 978–990 (2024)

    2024

  14. [22]

    P. J. Gebicke-Haerter, Frontiers in Cellular Neuroscience 17 (2023), 10.3389/fncel.2023.1220030

    2023

  15. [23]

    Aggarwal and K

    C. Aggarwal and K. Subbian, ACM Comput. Surv. 47 (2014), 10.1145/2601412

    2014 doi

  16. [24]

    Tunc and L

    I. Tunc and L. B. Shaw, Physical Review E 90 (2014), 10.1103/physreve.90.022801

    2014 doi

  17. [25]

    Elsken, J

    T. Elsken, J. H. Metzen, and F. Hutter, arXiv:1808.05377 (2018)

    2018 arXiv

  18. [26]

    Blalock, J

    D. Blalock, J. J. G. Ortiz, J. Frankle, and J. Guttag, arXiv:2003.03033 (2020)

    2020 arXiv

  19. [27]

    K. O. Stanley and R. Miikkulainen, Evolutionary Com- putation 10, 99–127 (2002)

    2002

  20. [28]

    Fahlman and C

    S. Fahlman and C. Lebiere, in Advances in Neural Infor- mation Processing Systems , Vol. 2, edited by D. Touret- zky (Morgan-Kaufmann, 1989)

    1989

  21. [29]

    Bottou, F

    L. Bottou, F. E. Curtis, and J. Nocedal, arXiv:1606.04838 (2016)

    2016 arXiv

  22. [30]

    Allen-Zhu, Y

    Z. Allen-Zhu, Y. Li, and Z. Song, in Proceedings of the 36th International Conference on Machine Learn- ing, Proceedings of Machine Learning Research, Vol. 97, edited by K. Chaudhuri and R. Salakhutdinov (PMLR,

  23. [31]

    Ghadimi and G

    S. Ghadimi and G. Lan, arXiv:1309.5549 (2013)

    2013 arXiv

  24. [32]

    W. E, C. Ma, and L. Wu, arXiv:1904.04326 (2019)

    2019 arXiv

  25. [33]

    H. Jin, M. Mattheakis, and P. Protopapas, arXiv:2203.00451 (2022)

    2022 arXiv

  26. [34]

    Wolfram Research, Inc., Mathematica,

    “Wolfram Research, Inc., Mathematica,” Champaign, IL, 2024

    2024

  27. [35]

    JAX: com- posable transformations of Python+NumPy programs,

    J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. Vander- Plas, S. Wanderman-Milne, and Q. Zhang, “JAX: com- posable transformations of Python+NumPy programs,” (2018)

    2018

  28. [36]

    Kidger and C

    P. Kidger and C. Garcia, Differentiable Programming workshop at Neural Information Processing Systems 2021 (2021)

    2021

  29. [37]

    The DeepMind JAX Ecosystem,

    DeepMind, I. Babuschkin, K. Baumli, A. Bell, S. Bhu- patiraju, J. Bruce, P. Buchlovsky, D. Budden, T. Cai, A. Clark, I. Danihelka, A. Dedieu, C. Fantacci, J. God- win, C. Jones, R. Hemsley, T. Hennigan, M. Hes- sel, S. Hou, S. Kapturowski, T. Keck, I. Kemaev, M. King, M. Kunes...

    2020

  30. [38]

    Our code is available at our GitHub repository https://github.com/NonlinearArtificialIntelligenceLab/N3

  31. [39]

    S. K. Chalup and L. Wiklendt, Connection Science 19, 183–199 (2007)

    2007

  32. [40]

    Sarao Mannelli, Y

    S. Sarao Mannelli, Y. Ivashynka, A. Saxe, and L. Sagli- etti, Journal of Statistical Mechanics: Theory and Ex- periment 2024, 114001 (2024)

    2024

  33. [41]

    Varoquaux, A

    G. Varoquaux, A. S. Luccioni, and M. Whittaker, arXiv:2409.14160 (2024)

    2024 arXiv

  34. [42]

    U. Evci, B. van Merri¨ enboer, T. Unterthiner, M. Vla- dymyrov, and F. Pedregosa, arXiv:2201.05125 (2022)

    2022 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.