Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper shows that noise-prune, a local pruning rule using noise-driven activity covariance, preserves task performance in task-trained recurrent neural networks, beating magnitude-based pruning and matching or exceeding a non-local…

desk verdict A solid empirical validation of noise-prune on task-trained RNNs, but the missing magnitude-based sample-and-rescale control leaves the active ingredient unproven. read the letter →

arxiv 2608.05464 v1 pith:XCAHELSH submitted 2026-08-05 q-bio.NC cs.LGcs.NE

Sanjith Senthil , Rishidev Chaudhuri This is my paper · ORCID
classification q-bio.NCcs.LGcs.NE
keywords noise-prunesynapticpruningrecurrentneuralnetworksbiologically-plausiblelearningcovariance-basedimportancetask-trainedRNNscognitivetasksstochasticsampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests noise-prune, a pruning rule for recurrent networks that decides which synapses to keep using the covariance of neural activity under injected noise rather than connection strength alone. On recurrent networks trained to perform eight cognitive tasks, the rule preserves task accuracy far better than magnitude-based pruning, and matches or exceeds a non-local second-order method (recurrent OBS) at high sparsities. The authors show the rule's stochastic sampling step—retaining connections with probability proportional to their importance and strengthening survivors by the inverse probability—is what makes it work; deterministic versions of the same importance scores perform much worse. They also find the optimal amount of strengthening is lower than the theoretical 1/p rescaling predicts: capping amplification factors at modest levels improves both task retention and dynamical stability. The result matters because it offers a biologically-plausible account of how synaptic pruning could work in the brain, using only local information.

What carries the argument

The central machinery is a covariance-based importance score combined with stochastic sample-and-rescale pruning. The retention probability for a connection from neuron j to neuron i is p_ij = K w_ij (C_ii + C_jj − 2C_ij) for excitatory weights and K|w_ij|(C_ii + C_jj + 2C_ij) for inhibitory weights, where C is the covariance matrix of activity under injected noise; the covariance term C_ii + C_jj − 2C_ij penalizes connections between neurons whose activity is already highly correlated, marking the direct connection as redundant. The rule independently retains each edge with probability p_ij and, if retained, multiplies the weight by 1/p_ij, preserving the expected strength of the edge while replacing a redundant set of connections by a few strengthened survivors. The paper obtains C in two ways: empirically from noisy simulation (S-NP) and analytically from the Lyapunov equation applied to the linearized network (L-NP).

What would settle it

Run the same pruning comparison on task-trained recurrent networks with a different activation function (e.g., ReLU or gated units) or on a task with long temporal dependencies; if noise-prune's retention advantage over magnitude pruning at 70–80% sparsity disappears, the covariance-based importance criterion does not generalize beyond the tanh networks tested here.

Watch

Extended reading notes

Core claim

The central claim is that noise-prune preserves function in task-trained recurrent neural networks. For each connection, the rule computes a retention probability from the connection weight and the covariance of the two neurons' noisy activity—p_ij = K w_ij (C_ii + C_jj − 2C_ij) for excitatory connections, with the sign of the covariance term flipped for inhibitory connections—then samples whether to keep the connection with that probability and, if kept, strengthens it by 1/p_ij. Across 24 networks trained on eight cognitive tasks requiring working memory, context-dependent selection, interval estimation, and sequence production, both the simulation-based and the linearized Lyapunov-based variants of noise-prune outperform magnitude pruning at all tested sparsities (50–80%), match a one-shot recurrent version of the Optimal Brain Surgeon at 50–60%, and exceed it at 70–80%. The paper further establishes that the sampling-and-rescaling mechanism, not just the covariance importance scores, is essential: deterministic thresholding of the same scores performs much worse. Finally, the authors show that the theoretically-derived rescaling is too aggressive; capping the amplification at the 50th–60th percentile of the candidate distribution improves task retention and keeps the spectral abscissa lower, indicating a stability benefit.

Load-bearing premise

The rule assumes that the covariance of two neurons' activity under noise, or its linear approximation, correctly identifies which direct connections are redundant in a nonlinear task-trained network, and this is validated only empirically on eight tasks.

Editorial extensions

If this is right

  • A biologically-plausible local pruning rule—using only the weight of a connection and the activity of the two neurons it connects—can preserve function in recurrent networks that perform cognitive tasks.
  • The linearized variant performs nearly as well as the full simulation variant, so a closed-form covariance estimate suffices when the network can be linearized.
  • The stochastic sampling-and-rescaling step is the load-bearing component; deterministic thresholding of the same importance scores loses most of the benefit.
  • Capping rescaling at modest factors (roughly 5–10x) improves both task retention and dynamical stability, so the exact expectation-preserving rescaling suggested by theory is not optimal in nonlinear networks.
  • At high sparsity (70–80%), noise-prune exceeds one-shot recurrent OBS, indicating that a local rule can beat a non-local second-order method when both are applied in a single shot.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the covariance importance score generalizes beyond tanh networks, noise-prune could be paired with retraining or continual learning to prune recurrent networks iteratively, pushing sparsity well below the one-shot limits reported here.
  • The stability-accuracy tradeoff from capping suggests a testable biophysical prediction: homeostatic synaptic scaling should be bounded, and post-pruning synapse strength distributions should show a finite cutoff rather than the unbounded heavy tail predicted by exact expectation-preserving rescaling.
  • Because the linearized variant works almost as well, even coarse covariance estimates—for example from subsampled or noisy recordings—might drive effective pruning, which could be tested in hybrid models that estimate C from partial observations.
  • The same covariance-based score could be reinterpreted as a Hebbian/anti-Hebbian rewiring rule operating during learning, not just a one-shot pruning event, connecting noise-prune to theories of developmental synapse elimination.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript evaluates noise-prune, an unsupervised local pruning rule for recurrent networks, on task-trained continuous-time RNNs (8 Mod-Cog tasks, 24 networks, H=512). It compares two noise-prune variants (simulation-based S-NP and linearized L-NP) against magnitude pruning, random pruning, a recurrent adaptation of optimal brain surgeon, and deterministic versions of noise-prune, at 50--80% sparsity. The main findings are that noise-prune preserves task performance substantially better than magnitude pruning, is comparable to or better than recurrent-OBS at high sparsity, that the probabilistic sample-and-rescale operation is essential, and that capping the rescale factor at an intermediate quantile improves performance relative to the uncapped rule.

Significance. If the claims hold, the paper makes a useful contribution: it provides the first evaluation of noise-prune on functional, task-trained recurrent networks, with a broad suite of tasks, a strong comparison baseline in recurrent-OBS, and careful statistical practices including Holm-corrected paired tests, sign-test robustness checks, fixed evaluation batches, and reproducibility details. The work also identifies a practically important non-monotonicity in rescaling that refines the original theoretical prescription. The main caveats are that the comparison to magnitude pruning does not isolate the covariance-based importance score from the sampling-and-rescaling recipe, and that the optimal cap quantile is selected on the same data used to report significance, so the reported p-values are optimistic for the selection procedure.

major comments (3)
  1. [Results, 'Optimal rescaling is lower than predicted by theory' and Methods, 'Variance-capped rescaling'] The experimental design never includes a magnitude-based importance score combined with the same sample-and-rescale operation that noise-prune uses. As a result, the comparison between noise-prune and magnitude pruning varies two factors at once: the importance score (|w| times the covariance expression versus |w| alone) and the pruning operation (probabilistic sample-and-rescale versus deterministic top-k). The paper explicitly calls the covariance term 'the key contribution of noise-prune' in the Results, but the current data do not rule out the possibility that a probabilistic magnitude-based rule with 1/p rescaling performs as well as S-NP or L-NP. A control that applies the same sampling and rescaling to the magnitude score is needed to support the claim that noisy fluctuations, rather than the sample-and-rescale recipe alone, drive the observed advantage.
  2. [Results, 'Noise-prune preserves function...'] The cap quantile q is selected by sweeping q on the same evaluation data used to report the peak-versus-uncapped significance (L-NP q=60, P=6.53e-5; S-NP q=50, P=8.34e-6). The reported p-values therefore do not account for the fact that the best q was chosen after looking at the same data. Holm correction across the 18 pairwise comparisons is a step in the right direction, but it does not control for the maximum-selection implicit in choosing q. The claim that optimal rescaling is lower than the theoretical prediction should be validated on held-out data or with an analysis that properly adjusts for selection (for example, a split-half or cross-validated choice of q).
  3. [Results, 'Noise-prune preserves function...'] The statement that 'both variants of noise-prune significantly outperform magnitude pruning (p<10^-4 for all noise-prune to magnitude comparisons)' is not supported by the supplementary statistics. At 50% sparsity, the Holm-corrected p-value for L-NP rescale versus magnitude is 1.79e-4, which is not below 1e-4, and the deterministic L-NP variant is not significantly different from magnitude at 50% (p=0.0885) or 60% (p=1.000). The claim should be corrected or qualified to refer specifically to the rescale variants at sparsity levels of 60% and above.
minor comments (5)
  1. [Methods, noise-prune] The procedure for choosing K is described only as 'chosen to match the desired connection density in expectation'; please specify the algorithm, for example a binary search over K, and state any bounds or tolerances used.
  2. [Methods, simulation noise-prune] The text should state unambiguously whether the covariance matrix C is computed from firing rates r(t) or from voltages v(t), since Eq. (9) adds noise to rates while the calibration in Eq. (10) uses voltage variability.
  3. [Supplementary Table S2] Several rows in Supplementary Table S2 show an asterisk in only one of the two significance columns, but the meaning of a single asterisk is not explained; please clarify which test each asterisk refers to.
  4. [Methods, variance-capped rescaling] The evaluation of capped rescaling averages task retention across the 50--80% sparsity levels; it would be informative to also show the cap effect separately at each sparsity level, since the optimal cap may depend on sparsity.
  5. [General] The manuscript would benefit from a brief statement about the computational cost of S-NP versus L-NP, since the near-equivalence of the two variants is a useful practical finding but the runtime implications are not quantified.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the task-retention results are measured against external baselines on held-out task data, the rescaling ablation is a genuine within-method test, and the capped-rescale finding falsifies the authors' own prior theory; the sole self-citation (Moore & Chaudhuri 2020) is background, not evidence.

full rationale

Score 2 reflects one non-load-bearing self-citation, not circularity; none of the headline claims reduces to its inputs by construction. (1) The noise-prune importance score is not fitted to the predicted quantity: Eq. (7) builds p_ij from noise-driven activity covariances (S-NP) or a linearized Lyapunov solution (L-NP); K only sets target density, and the noise scale is calibrated to the network's natural voltage variability with a robustness check ('set overlaps by >95% as σ is changed from 0.75 to 1.5×σnat'). No parameter in p_ij is fitted to task-retention values, so the reported retention numbers are not forced by construction. (2) The 'sampling and rescaling is essential' claim is a within-method ablation: the deterministic variants use the identical covariance scores but rank edges deterministically and skip rescaling, and perform dramatically worse, so the claim does not reduce to the score definition. (3) The capped-rescale result contradicts the authors' prior theory rather than confirming it: the theoretical 1/p amplification is shown to be suboptimal (uncapped retention 0.668/0.676 vs. peak 0.691/0.730 at q=60/50). The sole self-citation, ref [28] (Moore & Chaudhuri 2020, one overlapping author), supplies the algorithm, the expectation-preserving rescaling rationale, and explicitly class-limited spectral guarantees for 'linear and rectified-linear symmetric diagonally-dominant' networks; the paper uses these as background and itself flags the gap ('suggests that it may be possible to provide weaker but more general theoretical guarantees'). The central results are self-contained: 24 networks on 8 Mod-Cog tasks, evaluated on fixed held-out batches against magnitude, random, and OBS baselines. The paper also flags its recurrent-OBS adaptation limitation ('It is possible that recurrent-OBS would perform better if allowed multiple cycles of Hessian evaluation and pruning'), a comparison-design caveat rather than a circularity. A non-circular completeness caveat: a magnitude-plus-sample-and-rescale control is never run, so the covariance term is not isolated as the active ingredient; this is a mechanism-attribution gap, not a circular reduction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on noise-prune's covariance-based importance scores (inherited from ref. 28), a Lyapunov approximation for L-NP, and a cap quantile chosen by performance sweep. No new entities are introduced.

free parameters (5)
  • K (density scale constant) = chosen per sparsity target (50-80%) to match expected retention density
    Scale factor in p_ij (Eq. 7) that sets overall pruning density; not fitted to task performance, but a hand-chosen calibration per sparsity level.
  • Rescale cap quantile q = 60 (L-NP), 50 (S-NP)
    Selected by sweeping q from 10 to 100 on the same evaluation data; directly supports the claim that optimal rescaling is lower than predicted. This is the paper's main fitted parameter.
  • Noise standard deviation sigma (S-NP) = sigma = sigma_nat, the square root of mean voltage variance under task input
    Calibration chosen from the network's natural voltage variability so injected noise matches task-driven fluctuations; a modeling choice that affects covariance estimates.
  • Stabilization shift delta (L-NP) = start 0.5, doubled until A-delta*I is Hurwitz; max 8 (deterministic) or 4 (rescale)
    Ad hoc fix because trained networks violate the Hurwitz condition required for the Lyapunov equation; changes the covariance used by L-NP.
  • Damping lambda (recurrent-OBS) = 1e-3
    Regularizes the inverse-Hessian estimate for the OBS baseline; peripheral to the main noise-prune claims.
assumptions (5)
  • domain assumption Covariance-based importance score p_ij from Eq. (1) is a valid proxy for connection redundancy in nonlinear task-trained recurrent networks.
    Inherited from linear spectral-sparsification theory (Moore and Chaudhuri 2020, ref. 28); not re-derived for nonlinear tanh CTRNNs.
  • standard math Lyapunov equation AC+CA^T = -sigma^2*I yields the stationary covariance for the linearized dynamics A = Wrec - I (after delta-shift if needed).
    Standard result for stable linear systems; used by L-NP. The paper enforces stability by shifting A.
  • domain assumption Post-fixation sequence accuracy on 8 selected Mod-Cog tasks is a meaningful measure of network function.
    The evaluation metric is task accuracy from the Mod-Cog battery; tasks that do not require recurrence were excluded by design.
  • standard math Spectral abscissa alpha(Wrec) < 1 indicates stability of the linearized origin and is a useful diagnostic for pruning damage.
    Standard linear stability criterion; motivates capping rescaling factors.
  • domain assumption One-shot OBS with a block-diagonal Hessian approximated by the damped second-moment matrix is a fair strong baseline.
    Adapted from layer-wise OBS (Dong et al. 2017); the paper notes the Hessian is not re-evaluated after each pruning step.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling." pith.science (2026). https://pith.science/paper/XCAHELSH

@misc{pith2026260805464,
  author       = {Pith},
  title        = {Pith review of: Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XCAHELSH}},
  note         = {Machine review of arXiv:2608.05464}
}
read the original abstract

The pruning of network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. In this work we evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections. Noise-prune has previously only been empirically tested on random networks without a specific computational function. We show that noise-prune preserves task-performance in task-trained recurrent neural networks, greatly outperforming a strategy that only uses the magnitude of connections and performing on par with or exceeding a non-local strategy that uses second-order information. Rather than deterministically removing connections that fall below a certain threshold importance, noise-prune samples connections to preserve based on their importance and strengthens retained connections to preserve average synaptic strength. We show that this sampling and rescaling is essential to good performance, but that the optimal empirical degree of rescaling is lower than that predicted by the original theoretical argument. Our work thus validates noise-prune as a biologically-plausible pruning rule for functional recurrent network architectures and characterizes its optimal parameter settings.

Figures

Figures reproduced from arXiv: 2608.05464 by the authors.

Figure 1
Figure 1. Task retention across pruning methods and sparsity levels. Task (post-fixation sequence) accuracy of each pruned network relative to its unpruned baseline, plotted against the percentage of recurrent weights pruned, for two varieties of noise-prune along with alternative strategies. Points show the mean over n = 24 trained networks, with stochastic pruning methods averaged across three pruning seeds within each netw… view at source ↗
Figure 2
Figure 2. Role of probabilistic sampling and strengthening retained connections in preserving task performance. Task retention of each pruned network relative to its unpruned baseline, plotted against the percentage of recurrent weights pruned, for noise-prune with sampling and rescaling (indicated as “rescale”) compared to without (indicated as “deterministic”) along with magnitude-based pruning as a baseline. Note that nois… view at source ↗
Figure 3
Figure 3. Distribution of predicted rescale amplification factors at 80% sparsity. Histogram of predicted rescale factors (i.e., distribution of 1/pi j) for the linear (L-NP) and simulation (S-NP) noise-prune rescale variants, pooled over all networks and pruning seeds (n = 72). Both distributions are heavy-tailed: most rescale factors are small, while a small number of rescale factors are very large (up to ∼108 ). Probabilis… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Spectral abscissae across pruning methods and sparsity levels. Mean spectral abscissa a(Wrec) of connectivity matrix of each pruned network, plotted against the percentage of recurrent weights pruned, for all pruning rules considered, including noise-prune with and wit…
Figure 5
Figure 5. Figure 5: Effect of capping rescale factors on task retention. Task retention of both variants of noise-prune when per-edge amplification is capped at the qth percentile of the candidate-edge amplification distribution (q = 100 corresponds to the original uncapped variant). Each…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 29 canonical work pages

  1. [1]

    Huttenlocher, P. R. & Dabholkar, A. S. Regional differences in synaptogenesis in human cerebral cortex.J. Comp. Neurol. 387, 167–178, DOI: 10.1002/(SICI)1096-9861(19971020)387:2⟨167::AID-CNE1⟩3.0.CO;2-Z (1997)

    work page doi:10.1002/(sici)1096-9861(19971020)387:2 1997

  2. [2]

    & Noguchi, J

    Kasai, H., Fukuda, M., Watanabe, S., Hayashi-Takagi, A. & Noguchi, J. Structural dynamics of dendritic spines in memory and cognition.Trends Neurosci.33, 121–129 (2010)

    work page 2010

  3. [3]

    Petanjek, Z.et al.Extraordinary neoteny of synaptic spines in the human prefrontal cortex.Proc. Natl. Acad. Sci.108, 13281–13286 (2011)

    work page 2011

  4. [4]

    C.et al.Synaptic pruning by microglia is necessary for normal brain development.Science333, 1456–1458, DOI: 10.1126/science.1202529 (2011)

    Paolicelli, R. C.et al.Synaptic pruning by microglia is necessary for normal brain development.Science333, 1456–1458, DOI: 10.1126/science.1202529 (2011)

    work page doi:10.1126/science.1202529 2011

  5. [5]

    Stein, I. S. & Zito, K. Dendritic spine elimination: molecular mechanisms and implications.The Neurosci.25, 27–47 (2019)

    work page 2019

  6. [6]

    & Gan, W.-B

    Yang, G., Pan, F. & Gan, W.-B. Stably maintained dendritic spines are associated with lifelong memories.Nature462, 920–924 (2009)

    work page 2009

  7. [7]

    Lai, C. S. W., Franke, T. F. & Gan, W.-B. Opposite effects of fear conditioning and extinction on dendritic spine remodelling. Nature483, 87–91 (2012)

    work page 2012

  8. [8]

    Core concept: How synaptic pruning shapes neural wiring during development and, possibly, in disease.Proc

    Sakai, J. Core concept: How synaptic pruning shapes neural wiring during development and, possibly, in disease.Proc. Natl. Acad. Sci.117, 16096–16099, DOI: 10.1073/pnas.2010281117 (2020)

    work page doi:10.1073/pnas.2010281117 2020

Show all 33 references
  1. [9]

    S., Davis, R., Karmiloff-Smith, A., Knowland, V

    Thomas, M. S., Davis, R., Karmiloff-Smith, A., Knowland, V . C. & Charman, T. The over-pruning hypothesis of autism. Dev. science19, 284–305 (2016)

    2016

  2. [10]

    Sekar, A.et al.Schizophrenia risk from complex variation of complement component 4.Nature530, 177–183, DOI: 10.1038/nature16549 (2016)

    2016 doi

  3. [11]

    J., Jolivet, R

    Harris, J. J., Jolivet, R. & Attwell, D. Synaptic energy use and supply.Neuron75, 762–777, DOI: 10.1016/j.neuron.2012. 08.019 (2012)

    2012 doi

  4. [12]

    LeCun, Y ., Denker, J. S. & Solla, S. A. Optimal brain damage. InAdvances in Neural Information Processing Systems, 598–605 (1990). 13.Reed, R. Pruning algorithms—a survey.IEEE transactions on Neural Networks4, 740–747 (1993)

    1990

  5. [14]

    & Stork, D

    Hassibi, B. & Stork, D. G. Second order derivatives for network pruning: Optimal Brain Surgeon. InAdvances in Neural Information Processing Systems, 164–171 (1993)

    1993

  6. [15]

    & Dally, W

    Han, S., Pool, J., Tran, J. & Dally, W. Learning both weights and connections for efficient neural network. InAdvances in Neural Information Processing Systems, 1135–1143 (2015)

    2015

  7. [16]

    & Pan, S

    Dong, X., Chen, S. & Pan, S. Learning to prune deep neural networks via layer-wise optimal brain surgeon. InAdvances in Neural Information Processing Systems, 4857–4867 (2017)

    2017

  8. [17]

    & Legenstein, R

    Bellec, G., Kappel, D., Maass, W. & Legenstein, R. A. Deep rewiring: Training very sparse deep networks. In6th International Conference on Learning Representations, ICLR 2018, V ancouver , BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings(OpenReview.net, 2018)

    2018

  9. [18]

    & Carbin, M

    Frankle, J. & Carbin, M. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019(OpenReview.net, 2019)

    2019

  10. [19]

    & Elsen, E

    Narang, S., Diamos, G., Sengupta, S. & Elsen, E. Exploring sparsity in recurrent neural networks. In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings (OpenReview.net, 2017)

    2017

  11. [20]

    & Torr, P

    Lee, N., Ajanthan, T. & Torr, P. H. S. Snip: single-shot network pruning based on connection sensitivity. In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019(OpenReview.net, 2019). 13/19

    2019

  12. [21]

    & Rus, D

    Baykal, C., Liebenwein, L., Gilitschenski, I., Feldman, D. & Rus, D. Data-dependent coresets for compressing neural networks with applications to generalization bounds. In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019(O...

    2019

  13. [22]

    J., Frankle, J

    Blalock, D., Gonzalez Ortiz, J. J., Frankle, J. & Guttag, J. What is the state of neural network pruning?Proc. machine learning systems2, 129–146 (2020)

    2020

  14. [23]

    & Shi, J

    Cheng, H., Zhang, M. & Shi, J. Q. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations.IEEE Transactions on Pattern Analysis Mach. Intell.46, 10558–10578, DOI: 10.1109/TPAMI.2024. 3447085 (2024)

    2024 doi

  15. [24]

    & Alistarh, D

    Frantar, E. & Alistarh, D. SparseGPT: Massive language models can be accurately pruned in one-shot. In Krause, A. et al.(eds.)Proceedings of the 40th International Conference on Machine Learning, vol. 202 ofProceedings of Machine Learning Research, 10323–10337 (PMLR, 2023)

    2023

  16. [25]

    & Shen, G

    Han, B., Zhao, F., Zeng, Y . & Shen, G. Developmental plasticity-inspired adaptive pruning for deep spiking and artificial neural networks.IEEE Transactions on Pattern Analysis Mach. Intell.47, 240–251, DOI: 10.1109/TPAMI.2024.3467268 (2025)

    2025

  17. [26]

    & Brea, J

    Gerstner, W., Lehmann, M., Liakoni, V ., Corneil, D. & Brea, J. Eligibility traces and plasticity on behavioral time scales: Experimental support of neohebbian three-factor learning rules.Front. Neural Circuits12, 53, DOI: 10.3389/fncir.2018. 00053 (2018)

    2018 doi

  18. [27]

    P., Santoro, A., Marris, L., Akerman, C

    Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J. & Hinton, G. Backpropagation and the brain.Nat. Rev. Neurosci. 21, 335–346, DOI: 10.1038/s41583-020-0277-3 (2020)

    2020 doi

  19. [28]

    & Chaudhuri, R

    Moore, E. & Chaudhuri, R. Using noise to probe recurrent neural network structure and prune synapses. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. & Lin, H. (eds.)Advances in Neural Information Processing Systems, vol. 33, 14046–14057 (Curran Associates, Inc., 2020)

    2020

  20. [29]

    R., Joglekar, M

    Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X.-J. Task representations in neural networks trained to perform many cognitive tasks.Nat. Neurosci.22, 297–306 (2019)

    2019

  21. [30]

    Khona, M., Chandra, S., Ma, J. J. & Fiete, I. R. Winning the lottery with neural connectivity constraints: Faster learning across cognitive tasks with spatially constrained sparse RNNs.Neural Comput.35, 1850–1869, DOI: 10.1162/neco a 01613 (2023). 31.Yang, G. R. & Wang, X.-J. ...

    2023 doi

  22. [32]

    & Mizuseki, K

    Buzs´aki, G. & Mizuseki, K. The log-dynamic brain: how skewed distributions affect network operations.Nat. reviews neuroscience15, 264–278 (2014)

    2014

  23. [33]

    Turrigiano, G. G. The dialectic of Hebb and homeostasis.Philos. Transactions Royal Soc. B: Biol. Sci.372, 20160258 (2017). 34.Holtmaat, A. J.et al.Transient and persistent dendritic spines in the neocortex in vivo.Neuron45, 279–291 (2005)

    2017

  24. [35]

    Spielman, D. A. & Srivastava, N. Graph sparsification by effective resistances.SIAM J. on Comput.40, 1913–1926 (2011). 36.Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980(2017)

    2011 arXiv

  25. [37]

    In Wallach, H.et al.(eds.)Advances in Neural Information Processing Systems, vol

    Paszke, A.et al.Pytorch: An imperative style, high-performance deep learning library. In Wallach, H.et al.(eds.)Advances in Neural Information Processing Systems, vol. 32 (Curran Associates, Inc., 2019). Funding This study is based upon work supported by the Air Force Office o...

    2019

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.