Pith. sign in

REVIEW 3 major objections 6 minor 60 references

Additive function approximation in the brain

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A neural network whose hidden neurons each receive $d$ random inputs learns exactly the functions of an order-$d$ additive model, giving sparsely wired brains a built-in learning advantage.

desk verdict Solid kernel derivation between sparse random features and additive models; stability advantage is overstated and Appendix C has fixable proof slips. read the letter →

arxiv 1909.02603 v2 pith:QWEQKSQH submitted 2019-09-05 cs.NE cs.LGq-bio.NCstat.ML

Kameron Decker Harris This is my paper · ORCID
classification cs.NEcs.LGq-bio.NCstat.ML MSC 68T0746E2262G08
keywords sparserandomfeaturesadditivemodelskernelmethodsreproducingHilbertspacegeneralizationstabilityneuralcircuitscurseofdimensionality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a single hidden layer of neurons with random, sparse input connectivity—each hidden unit receiving $d$ inputs chosen uniformly—induces exactly the function space of an order-$d$ additive model. With a random degree distribution, the network's kernel is a mixture of additive models of different orders. The author works this out through kernel theory: in the infinite-width limit, the inner product of sparse random features converges to a kernel that averages an order-$d$ base kernel over all $d$-element input neighborhoods. This matters because additive structure is a known cure for the curse of dimensionality in high-dimensional regression, so even simple, randomly wired brain circuits could be strong learners. The paper then claims three concrete advantages of such sparsity: better generalization from limited data, stability to noise affecting a few inputs, and computational savings.

What carries the argument

The central identity is the neighborhood-averaging kernel $k_{\mathrm{reg}}^d$ and its degree-mixture extension $k_D^{\mathrm{dist}}$: a sparse random feature with in-degree $d$ has feature value depending only on the chosen $d$ coordinates, so its two-point product, after averaging over neighborhoods and weights, is exactly the average of a $d$-dimensional kernel over all $d$-subsets. This identity carries the argument because it converts the network's wiring statistics into an explicit reproducing kernel Hilbert space, letting known results about additive models—generalization rates, stability of robust distances, and primal random-feature computation—apply to the brain-inspired network.

What would settle it

Run the paper's spike-and-slab noise regression many times and record the distribution of test $R^2$; if the sparse-feature kernel does not consistently beat naive linear regression and Huber regression on corrupted inputs, the stability claim is unsupported. A sharper check is to compute the operator norm of the learned readout map from feature perturbations to output perturbations: if $|f(x+e)-f(x)|$ is not $O(d/l)$ for sparse $e$ with $\|\alpha\|$ fixed, the Cauchy-Schwarz argument fails.

Watch

Extended reading notes

Core claim

At fixed in-degree $d$, the random feature map's limiting kernel is $$k_{\mathrm{reg}}^d(x,x')=\binom{l}{d}^{-1}\sum_{|N|=d} k_d(x_N,x'_N),$$ the average of a $d$-dimensional base kernel over all $d$-element neighborhoods of the input. Thus the learned functions are sums of terms, each depending on only $d$ coordinates—an additive model of order $d$. If the in-degrees are drawn from a distribution $D$, the kernel is $k_D^{\mathrm{dist}} = \sum_d D(d) k_{\mathrm{reg}}^d$, a weighted mixture of interaction orders. This equivalence is the paper's central claim: sparse connectivity itself, without any training of the first layer, turns a random neural network into a structured function approximator whose inductive bias is additivity.

Load-bearing premise

The load-bearing assumption is that the learned readout weights do not amplify the movement of the few features that see corrupted inputs; the paper's stability argument bounds the feature difference via Cauchy-Schwarz but does not control $\|\alpha\|$, so if a small number of corrupted coordinates can move the output significantly through the readout, the stability claim collapses.

Editorial extensions

If this is right

  • If the equivalence holds, a sparsely connected random layer trained only at the readout is provably an additive model, so its generalization error in high dimensions is governed by $d$ rather than by the input dimension $l$.
  • For $d=1$, the kernel becomes a sum over single coordinates, and for step or sign nonlinearities it reduces to robust distance kernels (normalized Hamming distance or $\ell^1$ distance), so the network denoises sparse input corruption by construction.
  • Degree distributions such as binomial connectivity yield a weighted mixture of interaction orders, so learning adapts model complexity to the data—the same inductive bias attributed to dropout.
  • Because features are evaluated in the primal, the cost scales as $O(nmd)$ instead of $O(n^2 l^d)$, making sparse random features a practical way to approximate additive kernels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is to compare the learned functions of sparse random networks against explicit order-$d$ additive models on data with known interaction orders; the kernel identity predicts the readout weights should concentrate on the true interaction subsets.
  • The stability argument would be strengthened by bounding the regression weights; a natural experiment is to measure the Lipschitz constant of the learned readout with respect to sparse perturbations and see whether it stays small as $d/l$ grows.
  • The identification with additive models suggests that sparsity and dropout are two routes to the same inductive bias; one could train a dense network with dropout and compare its effective kernel to a sparse network of the same width.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper studies single-hidden-layer networks with random, sparse input weights and characterizes the function spaces they induce. The central result is Eq. (3)-(4): when each hidden unit receives exactly d inputs chosen uniformly, the induced kernel is the average of a base d-dimensional kernel over all d-element input neighborhoods, so the RKHS is an additive model of order d; when the in-degree follows a distribution, the kernel is a mixture of additive kernels of different orders. The paper claims three advantages of this sparsity: additivity as an inductive bias that mitigates the curse of dimensionality, stability to sparse input noise, and computational and biological scalability. It also gives closed-form d=1 kernels for step and sign nonlinearities, proves a uniform kernel-approximation theorem for Lipschitz nonlinearities in Appendix C, and reports numerical experiments on additive regression and corrupted-input stability.

Significance. The equivalence in Eq. (3)-(4) is simple, original in its neuroscience framing, and likely correct; it provides a concrete bridge between sparse circuit motifs in mushroom body, cerebellum, and hippocampus and additive-model/kernel theory. The paper also ships open-source code, which is a concrete strength, and it connects the result to existing generalization bounds for additive models rather than claiming new statistical rates. If the stability and approximation claims were properly supported, the paper would be a useful and accessible contribution to both computational neuroscience and random-feature theory. However, the stability advantage is currently not established, and the Appendix C kernel-approximation proof contains an algebraic error in the stated probability bound, so the broader claims in the abstract and Section 4 need revision before the paper can be accepted.

major comments (3)
  1. [4.2] The stability claim 'any regressor f(x)=α^T φ(x) built from these features will also be stable' does not follow from the Cauchy-Schwarz argument. The bound |f(x)-f(x')| ≤ ||α||_2 ||φ(x)-φ(x')||_2 only controls the feature displacement; it provides no control on ||α||_2. For l=100, d=1, m=1000, one can choose α_i=100 for a feature whose neighborhood contains the corrupted coordinate and α_j=0 elsewhere; then only about 1% of the features are affected, yet |f(x)-f(x')| can be arbitrarily large. Even for ridge regression, ||α||_2 depends on the penalty and the data, not on sparsity alone, and Table 1 is a single simulation with no error bars and no measurement of ||α||. The claim should be restated with an explicit norm constraint on α or a proven bound on ||α|| for the specific estimator, and the statement that φ(x)≈φ(x') should be quantified in terms of the weight distribution and the Lipschitz constant of the nonlinearity.
  2. [Appendix C] The final probability bound has the wrong exponent on κ. Equation (12) applies Hoeffding's inequality with |f_i(ξ)| ≤ κ^2, which gives exp(-mϵ^2/(8κ^4)); after the covering-net optimization the exponential factor becomes exp(-mϵ^2/(4(2l+2)κ^4)), not exp(-mϵ^2/(4(2l+2)κ^2)) as stated in the theorem and in the main-text Claim. Consequently the sample-complexity statement m=Ω(lκ^2/ϵ^2 log(C/ϵ)) should be m=Ω(lκ^4/ϵ^2 log(C/ϵ)) on the given argument. In addition, equation (12) says 'for all i' where the union bound requires 'for some i' or a supremum over the anchor points, and Corollary 2's boundedness assumption '|φ(x_N)|^2 ≤ κ' is inconsistent with Theorem 1's '|φ(x)| ≤ κ'.
  3. [4.2] The closed-form examples in Eqs. (5)-(6) use step and sign nonlinearities, which are not Lipschitz and are explicitly excluded from Theorem 1 and Corollary 2. These examples are also the ones used to motivate the stability advantage and in the Table 1 experiment. The paper should either prove a separate convergence and stability statement for these discontinuous nonlinearities or clearly mark the Lipschitz theorem as inapplicable to the showcased examples; otherwise the formal guarantees and the numerical illustrations are not aligned.
minor comments (6)
  1. [Section 3] In the main-text Claim, 'κ-Lipschitz nonlinearity' is a typo: κ is the almost-sure bound on |φ|, while L is the Lipschitz constant.
  2. [Section 4.2] The phrase 'a minority of them features' should read 'a minority of the features'.
  3. [Appendix B] The name 'Bôchner' should be 'Bochner'.
  4. [Appendix C, Eq. (12)] The phrase 'for all i' in the union bound should be 'for some i' (equivalently, a supremum over the anchor points).
  5. [Abstract / Section 1] The phrase 'equivalent to an additive model' is stronger than what is proved: the equivalence is for the limiting kernel as m→∞, while finite networks are finite-dimensional approximations of the additive RKHS. Consider rewording to 'induces' or 'converges to'.
  6. [Table 1] The R² values are reported without error bars or the number of repeated trials; please state whether these are single runs and provide variability information.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the sparse-feature kernel identities are derived from the feature model by expectation, and the external generalization theorems carry the additive-model claim.

full rationale

The central result, Eqs. (3)-(4), is a direct expansion of the defining kernel k(x,x') = E[phi(x)phi(x')] under the stated generative model: a feature with degree d is supported on a uniformly random d-subset, so the expectation conditional on N is k_d(x_N,x'_N), and averaging over N gives the normalized sum over d-subsets; the degree-distribution kernel is then the mixture over d. This is a derivation, not a fitted or assumed conclusion. The identification with additive models uses this closed form and imports the known generalization bounds of Stone and of Kandasamy and Yu, which are external to the paper. The stability claim in Sec. 4.2 is supported by an explicit (though incomplete) Cauchy-Schwarz argument and a numerical experiment; the missing bound on ||alpha||_2 is a correctness gap, not circularity, because no parameter is fitted to enforce the stability conclusion. The only self-citations ([19],[20]) are used for biological background and for the claim that sparsity is optimized for dimensionality and wiring cost; they do not carry the kernel derivation or the additive-model equivalence. No equation is equal to its input by construction, no fitted quantity is renamed as a prediction, and no uniqueness result is imported from the author's prior work.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The derivation of the additive kernel is self-contained: given the sparse feature construction, Eqs (3)-(4) follow by linearity of expectation. The theoretical contributions rely on external generalization bounds and on the biological premise of random sparse connectivity; neither is fitted to make the equivalence true. Experimental hyperparameters are ancillary.

free parameters (4)
  • Gaussian weight scale sigma = 0.05*sqrt(d)*n^(1/5) in experiments
    Selected in Appendix A.1.1 to match SALSA's RBF bandwidth; not part of the theoretical kernel derivation.
  • Feature count multiplier m = 300*l = 300 times input dimension in SALSA comparison
    Experimental choice to ensure kernel approximation; theory only requires Omega(l*kappa^2/epsilon^2*log(C/epsilon)).
  • Ridge penalty lambda = Cross-validated from 5 or 7 log-spaced values
    Chosen by RidgeCV in experiments; not a theoretical parameter.
  • Polynomial test mixing constant alpha = 0.05
    Sets relative strength of linear and nonlinear terms in Appendix A.1.2; illustrative only.
assumptions (5)
  • domain assumption Connectivity in mushroom body, cerebellum, and dentate gyrus is approximately random and sparse, so the random sparse feature model is a relevant description of these circuits.
    Motivates the entire application; cited from refs [16-21], not verified by this paper.
  • standard math Hidden weights and biases are i.i.d. with finite second moments, and the induced inner product converges to a kernel.
    Standard random feature assumption used in Eq (2); underlies Eqs (3)-(4).
  • standard math For the approximation theorem, the nonlinearity h is kappa-Lipschitz, bounded, and E|phi(x)|^2 < infinity on compact X containing 0.
    Conditions of Theorem 1; step and sign nonlinearities in Eqs (5)-(6) violate Lipschitz, so the rate guarantee does not cover the closed-form examples.
  • standard math Additive models of order d have sample complexity that does not grow exponentially with input dimension l, as proved by Stone and by Kandasamy and Yu.
    External generalization theorems imported in Section 4.1; the paper does not re-derive them.
  • domain assumption The readout layer is trained while sparse random features are fixed.
    Defines the learning setting; real neurons may have different plasticity, acknowledged as a limitation in Section 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Additive function approximation in the brain." pith.science (2026). https://pith.science/paper/QWEQKSQH

@misc{pith2026190902603,
  author       = {Pith},
  title        = {Pith review of: Additive function approximation in the brain},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QWEQKSQH}},
  note         = {Machine review of arXiv:1909.02603}
}
abstract

Many biological learning systems such as the mushroom body, hippocampus, and cerebellum are built from sparsely connected networks of neurons. For a new understanding of such networks, we study the function spaces induced by sparse random features and characterize what functions may and may not be learned. A network with $d$ inputs per neuron is found to be equivalent to an additive model of order $d$, whereas with a degree distribution the network combines additive terms of different orders. We identify three specific advantages of sparsity: additive function approximation is a powerful inductive bias that limits the curse of dimensionality, sparse networks are stable to outlier noise in the inputs, and sparse random features are scalable. Thus, even simple brain architectures can be powerful function approximators. Finally, we hope that this work helps popularize kernel theories of networks among computational neuroscientists.

Figures

Figures reproduced from arXiv: 1909.02603 by the authors.

Figure 1
Figure 1. Sparse connectivity in a shallow neural network. The function shown is the sparse random [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of sparse random feature approximation to additive kernel method SALSA [ [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Performance of sparse random features of differing degree [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Kernel eigenvalue amplification while (left) varying p with σ = 6 fixed, and (right) varying σ with p = 0.03 fixed. Plotted is the ratio of eigenvalues of the kernel matrix corrupted by noise to those without any corruption, ordered from largest to smallest in magnitud…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 37 canonical work pages

  1. [1]

    Additive Approximations in High Dimensional Nonparametric Regression via the SALSA

    Kirthevasan Kandasamy and Yaoliang Yu. Additive Approximations in High Dimensional Nonparametric Regression via the SALSA . In International Conference on Machine Learning , pages 69--78, June 2016

    work page 2016

  2. [2]

    Random Features for Large - Scale Kernel Machines

    Ali Rahimi and Benjamin Recht. Random Features for Large - Scale Kernel Machines . In J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, editors, Advances in Neural Information Processing Systems 20 , pages 1177--1184. Curran Associates, Inc. , 2008

    work page 2008

  3. [3]

    Smola, Zolt \'a n L

    Alex J. Smola, Zolt \'a n L. \'O v \'a ri, and Robert C Williamson. Regularization with Dot - Product Kernels . In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13 , pages 308--314. MIT Press , 2001

    work page 2001

  4. [4]

    Breaking the Curse of Dimensionality with Convex Neural Networks

    Francis Bach. Breaking the Curse of Dimensionality with Convex Neural Networks . Journal of Machine Learning Research, 18 0 (19): 0 1--53, 2017 a

    work page 2017

  5. [5]

    Youngmin Cho and Lawrence K. Saul. Kernel Methods for Deep Learning . In Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22 , pages 342--350. Curran Associates, Inc. , 2009

    work page 2009

  6. [6]

    Youngmin Cho and Lawrence K. Saul. Analysis and Extension of Arc - Cosine Kernels for Large Margin Classification . arXiv:1112.3712 [cs], December 2011

    arXiv 2011

  7. [7]

    On the Equivalence Between Kernel Quadrature Rules and Random Feature Expansions

    Francis Bach. On the Equivalence Between Kernel Quadrature Rules and Random Feature Expansions . J. Mach. Learn. Res., 18 0 (1): 0 714--751, January 2017 b . ISSN 1532-4435

    work page 2017

  8. [8]

    Sutherland and Jeff Schneider

    Dougal J. Sutherland and Jeff Schneider. On the Error of Random Fourier Features . arXiv:1506.02785 [cs, stat], June 2015

    arXiv 2015

Show all 60 references
  1. [9]

    Optimal Rates for Random Fourier Features

    Bharath Sriperumbudur and Zoltan Szabo. Optimal Rates for Random Fourier Features . In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28 , pages 1144--1152. Curran Associates, Inc. , 2015

    2015

  2. [10]

    Generalization Properties of Learning with Random Features

    Alessandro Rudi and Lorenzo Rosasco. Generalization Properties of Learning with Random Features . In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30 , pages 3215--3225. Cu...

    2017

  3. [11]

    But How Does It Work in Theory ? Linear SVM with Random Features

    Yitong Sun, Anna Gilbert, and Ambuj Tewari. But How Does It Work in Theory ? Linear SVM with Random Features . arXiv:1809.04481 [cs, stat], September 2018

    2018 arXiv

  4. [12]

    Breaking the Curse of Dimensionality with Convex Neural Networks

    Francis Bach. Breaking the Curse of Dimensionality with Convex Neural Networks . Journal of Machine Learning Research, 18 0 (19): 0 1--53, 2017

    2017

  5. [13]

    Neural Tangent Kernel : Convergence and Generalization in Neural Networks

    Arthur Jacot, Franck Gabriel, and Cl \'e ment Hongler. Neural Tangent Kernel : Convergence and Generalization in Neural Networks . arXiv:1806.07572 [cs, math, stat], June 2018

    2018 arXiv

  6. [14]

    On the Global Convergence of Gradient Descent for Over -parameterized Models using Optimal Transport

    Lenaic Chizat and Francis Bach. On the Global Convergence of Gradient Descent for Over -parameterized Models using Optimal Transport . arXiv:1805.09545 [cs, math, stat], May 2018

    2018 arXiv

  7. [15]

    A Mean Field View of the Landscape of Two - Layers Neural Networks

    Song Mei, Andrea Montanari, and Phan-Minh Nguyen. A Mean Field View of the Landscape of Two - Layers Neural Networks . arXiv:1804.06561 [cond-mat, stat], April 2018

    2018 arXiv

  8. [16]

    Rotskoff and Eric Vanden-Eijnden

    Grant M. Rotskoff and Eric Vanden-Eijnden . Trainability and Accuracy of Neural Networks : An Interacting Particle System Approach . arXiv:1805.00915 [cond-mat, stat], May 2018

    2018 arXiv

  9. [17]

    Bandeira, and Joan Bruna

    Luca Venturi, Afonso S. Bandeira, and Joan Bruna. Spurious Valleys in Two -layer Neural Network Optimization Landscapes . arXiv:1802.06384 [cs, math, stat], February 2018

    2018 arXiv

  10. [18]

    Rosenblatt

    F. Rosenblatt. The Perceptron : A Probabilistic Model for Information Storage and Organization in the Brain . Psychological Review, 65 0 (6): 0 386--408, 1958

    1958

  11. [19]

    D. S. Broomhead and David Lowe. Radial Basis Functions , Multi - Variable Functional Interpolation and Adaptive Networks . Technical Report RSRE-MEMO-4148, Royal Signals and Radar Establishment Malvern (UK) , March 1988

    1988

  12. [20]

    Igelnik and Yoh-Han Pao

    B. Igelnik and Yoh-Han Pao. Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Transactions on Neural Networks, 6 0 (6): 0 1320--1329, November 1995. ISSN 1045-9227. doi:10.1109/72.471375

    1995 doi

  13. [21]

    Radford M. Neal. Priors for Infinite Networks . In Bayesian Learning for Neural Networks , Lecture Notes in Statistics , pages 29--53. Springer, New York, NY , 1996. ISBN 978-0-387-94724-2 978-1-4612-0745-0. doi:10.1007/978-1-4612-0745-0_2

    1996 doi

  14. [22]

    Christopher K. I. Williams. Computing with Infinite Networks . In M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems 9 , pages 295--301. MIT Press , 1997

    1997

  15. [23]

    The Extreme Learning Machine

    L. P. Wang and C. R. Wan. Comments on " The Extreme Learning Machine ". IEEE Transactions on Neural Networks, 19 0 (8): 0 1494--1495, August 2008. ISSN 1045-9227. doi:10.1109/TNN.2008.2002273

    2008

  16. [24]

    Randomness in neural networks: An overview

    Simone Scardapane and Dianhui Wang. Randomness in neural networks: An overview. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 7 0 (2): 0 e1200, 2017. ISSN 1942-4795. doi:10.1002/widm.1200

    2017 doi

  17. [25]

    Random Features for Large - Scale Kernel Machines

    Ali Rahimi and Benjamin Recht. Random Features for Large - Scale Kernel Machines . In J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, editors, Advances in Neural Information Processing Systems 20 , pages 1177--1184. Curran Associates, Inc. , 2008 a

    2008

  18. [26]

    Rahimi and B

    A. Rahimi and B. Recht. Uniform approximation of functions with random bases. In 2008 46th Annual Allerton Conference on Communication , Control , and Computing , pages 555--561, September 2008 b . doi:10.1109/ALLERTON.2008.4797607

    2008

  19. [27]

    Compressed Sensing , Sparsity , and Dimensionality in Neuronal Information Processing and Data Analysis

    Surya Ganguli and Haim Sompolinsky. Compressed Sensing , Sparsity , and Dimensionality in Neuronal Information Processing and Data Analysis . Annual Review of Neuroscience, 35 0 (1): 0 485--508, 2012. doi:10.1146/annurev-neuro-062111-150410

    2012 doi

  20. [28]

    Sophie J. C. Caron, Vanessa Ruta, L. F. Abbott, and Richard Axel. Random convergence of olfactory inputs in the Drosophila mushroom body. Nature, 497 0 (7447): 0 113--117, May 2013. ISSN 0028-0836. doi:10.1038/nature12063

    2013 doi

  21. [29]

    Sophie J. C. Caron. Brains Don 't Play Dice or Do They ? Science, 342 0 (6158): 0 574--574, November 2013. ISSN 0036-8075, 1095-9203. doi:10.1126/science.1245982

    2013 doi

  22. [30]

    Garcia, Jan-Marino Ramirez, and Eric Shea-Brown

    Kameron Decker Harris, Tatiana Dashevskiy, Joshua Mendoza, Alfredo J. Garcia, Jan-Marino Ramirez, and Eric Shea-Brown . Different roles for inhibition in the rhythm-generating respiratory network. Journal of Neurophysiology, 118 0 (4): 0 2070--2088, October 2017. ISSN 0022-307...

    2017

  23. [31]

    Ashok Litwin-Kumar , Kameron Decker Harris, Richard Axel, Haim Sompolinsky, and L. F. Abbott. Optimal Degrees of Synaptic Connectivity . Neuron, 93 0 (5): 0 1153--1164.e7, March 2017. ISSN 0896-6273. doi:10.1016/j.neuron.2017.01.030

    2017 doi

  24. [32]

    Alex Cayco-Gajic and R

    N. Alex Cayco-Gajic and R. Angus Silver. Re-evaluating Circuit Mechanisms Underlying Pattern Separation . Neuron, 101 0 (4): 0 584--602, February 2019. ISSN 08966273. doi:10.1016/j.neuron.2019.01.044

    2019 doi

  25. [33]

    Wolff and Nicholas J

    Gabriella H. Wolff and Nicholas J. Strausfeld. Genealogical correspondence of a forebrain centre implies an executive brain in the protostome deuterostome bilaterian ancestor. Philosophical Transactions of the Royal Society B: Biological Sciences, 371 0 (1685): 0 20150055, Jan...

    2016

  26. [34]

    Learning both Weights and Connections for Efficient Neural Network

    Song Han, Jeff Pool, John Tran, and William Dally. Learning both Weights and Connections for Efficient Neural Network . In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28 , pages 1135--1143. Curra...

    2015

  27. [35]

    Song Han, Huizi Mao, and William J. Dally. Deep Compression : Compressing Deep Neural Networks with Pruning , Trained Quantization and Huffman Coding . arXiv:1510.00149 [cs], October 2015 b

    2015 arXiv

  28. [36]

    Learning Structured Sparsity in Deep Neural Networks

    Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning Structured Sparsity in Deep Neural Networks . In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29 , pages 2074--2082. Curran Associ...

    2016

  29. [37]

    Convolutional Kernel Networks

    Julien Mairal, Piotr Koniusz, Zaid Harchaoui, and Cordelia Schmid. Convolutional Kernel Networks . arXiv:1406.3332 [cs, stat], June 2014

    2014 arXiv

  30. [38]

    Kernel-based Translations of Convolutional Networks

    Corinne Jones, Vincent Roulet, and Zaid Harchaoui. Kernel-based Translations of Convolutional Networks . arXiv:1903.08131 [cs, math, stat], March 2019

    1903 arXiv

  31. [39]

    Spline Models for Observational Data

    Grace Wahba. Spline Models for Observational Data . SIAM , September 1990. ISBN 978-0-89871-244-5

    1990

  32. [40]

    Francis R. Bach. Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning . In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21 , pages 105--112. Curran Associates, Inc. , 2009

    2009

  33. [41]

    Rasmussen

    David K Duvenaud, Hannes Nickisch, and Carl E. Rasmussen. Additive Gaussian Processes . In J. Shawe-Taylor , R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 24 , pages 226--234. Curran Associates, Inc. , 2011

    2011

  34. [42]

    The Elements of Statistical Learning : Data Mining , Inference , and Prediction

    Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning : Data Mining , Inference , and Prediction . Springer-Verlag New York , New York, NY , 2009. ISBN 978-0-387-84858-7. OCLC: 428882834

    2009

  35. [44]

    Charles J. Stone. The Dimensionality Reduction Principle for Generalized Additive Models . The Annals of Statistics, 14 0 (2): 0 590--606, June 1986. ISSN 0090-5364, 2168-8966. doi:10.1214/aos/1176349940

    1986

  36. [45]

    Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R

    Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580 [cs], July 2012

    2012 arXiv

  37. [46]

    Improving Neural Networks with Dropout

    Nitish Srivastava. Improving Neural Networks with Dropout . University of Toronto , 2013

    2013

  38. [47]

    Adams, and Zoubin Ghahramani

    David Duvenaud, Oren Rippel, Ryan P. Adams, and Zoubin Ghahramani. Avoiding pathologies in very deep networks. arXiv:1402.5836 [cs, stat], February 2014

    2014 arXiv

  39. [48]

    Certified Robustness to Adversarial Examples with Differential Privacy

    Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified Robustness to Adversarial Examples with Differential Privacy . arXiv:1802.03471 [cs, stat], February 2018

    2018 arXiv

  40. [49]

    Cohen, Elan Rosenfeld, and J

    Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified Adversarial Robustness via Randomized Smoothing . arXiv:1902.02918 [cs, stat], February 2019

    1902 arXiv

  41. [50]

    Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers

    Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, and Sebastien Bubeck. Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers . arXiv:1906.04584 [cs, stat], June 2019

    1906 arXiv

  42. [51]

    Fast and Robust Learning by Reinforcement Signals : Explorations in the Insect Brain

    Ram \'o n Huerta and Thomas Nowotny. Fast and Robust Learning by Reinforcement Signals : Explorations in the Insect Brain . Neural Computation, 21 0 (8): 0 2123--2151, August 2009. ISSN 0899-7667, 1530-888X. doi:10.1162/neco.2009.03-08-733

    2009 doi

  43. [52]

    Delahunt and J

    Charles B. Delahunt and J. Nathan Kutz. Putting a bug in ML : The moth olfactory network learns to read MNIST . Neural Networks, 118: 0 54--64, October 2019. ISSN 0893-6080. doi:10.1016/j.neunet.2019.05.012

    2019 doi

  44. [53]

    Delahunt and J

    Charles B. Delahunt and J. Nathan Kutz. Insect cyborgs: Bio -mimetic feature generators improve machine learning accuracy on limited data. arXiv:1808.08124 [cs, stat], August 2018

    2018 arXiv

  45. [54]

    Warden, Xiao-Jing Wang, Nathaniel D

    Mattia Rigotti, Omri Barak, Melissa R. Warden, Xiao-Jing Wang, Nathaniel D. Daw, Earl K. Miller, and Stefano Fusi. The importance of mixed selectivity in complex cognitive tasks. Nature, 497 0 (7451): 0 585--590, May 2013. ISSN 0028-0836. doi:10.1038/nature12160

    2013 doi

  46. [55]

    Sparseness and Expansion in Sensory Representations

    Baktash Babadi and Haim Sompolinsky. Sparseness and Expansion in Sensory Representations . Neuron, 83 0 (5): 0 1213--1226, September 2014. ISSN 0896-6273. doi:10.1016/j.neuron.2014.07.035

    2014 doi

  47. [56]

    On the dimensionality of odor space

    Markus Meister. On the dimensionality of odor space. eLife, 4: 0 e07865, July 2015. ISSN 2050-084X. doi:10.7554/eLife.07865

    2015 doi

  48. [57]

    Stimuli Reduce the Dimensionality of Cortical Activity

    Luca Mazzucato, Alfredo Fontanini, and Giancarlo La Camera. Stimuli Reduce the Dimensionality of Cortical Activity . Frontiers in Systems Neuroscience, 10, 2016. ISSN 1662-5137. doi:10.3389/fnsys.2016.00011

    2016

  49. [58]

    Yu, Gopal Santhanam, Stephen Ryu, Krishna Shenoy, and Surya Ganguli

    Peiran Gao, Eric Trautmann, Byron M. Yu, Gopal Santhanam, Stephen Ryu, Krishna Shenoy, and Surya Ganguli. A theory of multineuronal dimensionality, dynamics and measurement. November 2017. doi:10.1101/214262

    2017 doi

  50. [59]

    Linking Connectivity , Dynamics , and Computations in Low - Rank Recurrent Neural Networks

    Francesca Mastrogiuseppe and Srdjan Ostojic. Linking Connectivity , Dynamics , and Computations in Low - Rank Recurrent Neural Networks . Neuron, 99 0 (3): 0 609--623.e29, August 2018. ISSN 0896-6273. doi:10.1016/j.neuron.2018.07.003

    2018 doi

  51. [60]

    Farrell, Stefano Recanatesi, Guillaume Lajoie, and Eric Shea-Brown

    Matthew S. Farrell, Stefano Recanatesi, Guillaume Lajoie, and Eric Shea-Brown . Dynamic compression and expansion in a classifying recurrent network. bioRxiv, page 564476, March 2019. doi:10.1101/564476

    2019 doi

  52. [61]

    Learning Bounds for Kernel Regression Using Effective Data Dimensionality

    Tong Zhang. Learning Bounds for Kernel Regression Using Effective Data Dimensionality . Neural Computation, 17 0 (9): 0 2077--2098, September 2005. ISSN 0899-7667. doi:10.1162/0899766054323008

    2005 doi

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.