Pith. sign in

REVIEW 4 major objections 4 minor 46 references

Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A reinforcement learning agent with a CNN frozen at random initialization solves Pong through 1–3 of 64 readout neurons, with sparsity scaling to task complexity.

desk verdict The frozen-random-CNN sparsity phenomenon is probably real, but the headline numbers overstate it; worth refereeing after the authors fix the abstracts and add a threshold sensitivity analysis. read the letter →

arxiv 2607.26059 v1 pith:J3I6O3QA submitted 2026-05-15 cs.LG cs.AIcs.NE

Scott M. Norton This is my paper
classification cs.LGcs.AIcs.NE
keywords emergentsparsityfrozenrandomfeaturesdeepreinforcementlearninginformationbottleneckactiveneuroncountparticipationratiotaskcomplexityprojection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that when a deep reinforcement learning agent's convolutional feature extractor is frozen at random initialization, gradient descent on the trainable readout spontaneously discovers extremely sparse representations: deterministic Pong is solved through only 1–3 of 64 neurons in the first fully-connected layer, with 5–11 under sticky-action stochasticity. The sparsity is not engineered—no pruning, regularization, or sparse-inducing objective—and the number of active neurons grows with the task's estimated complexity across four Atari games. Ablations show the active set is both necessary and sufficient: removing it crashes reward to random play across two independent policy-gradient implementations, while keeping only it preserves performance. Longitudinal checkpoints show the active set locks in by 15–30M steps while reward turns positive 35–105M steps later, ruling out the idea that neurons died because the policy converged. A sympathetic reader would care because, if right, a frozen random projection plus ordinary gradient descent exposes the effective dimensionality of a task that end-to-end training obscures.

What carries the argument

The load-bearing object is a frozen, randomly initialized convolutional stack—three conv layers producing 3,136 features—followed by trainable fully-connected layers (FC1: 3,136→64; FC2: 64→64) and policy/value heads; formally, this is a reservoir/random-feature architecture. The central observable is 'active-neuron count': FC1 neurons whose mean post-ReLU activation exceeds 0.01, complemented by PCA dimensionality and the participation ratio of FC2-output activations. Active-count measurement carries the task-complexity claim; the ablation protocol (zeroing active versus inactive neurons) carries the causal-necessity claim; and the longitudinal checkpoint sweep carries the temporal-commitme

What would settle it

Re-run the active-neuron analysis with thresholds ranging from 0.001 to 0.1; if Pong's 1–3 'active' neurons balloon to 50–64 under a slightly looser threshold, the sparsity is a measurement artifact. Alternatively, find a same-recipe frozen Pong seed that reaches expert reward with zero active FC1 neurons, which would mean the ablation signature is not necessary for competence.

Watch

Extended reading notes

Core claim

The central claim is that a fixed random projection followed by gradient descent acts as a compressive measurement device: the number of FC1 neurons a proximal-policy-optimization agent actually uses is governed by the task's intrinsic dimensionality, not by network width or a fixed fraction of capacity. Concretely, frozen-CNN agents activate 1–3 neurons for deterministic Pong, 19–26 for Breakout, and roughly 42 for Space Invaders, while trainable-CNN controls activate 55–64 under matched conditions. Within a single game at matched hyperparameters, three same-recipe Pong seeds produce 5, 7, and 11 active neurons, and the 5-neuron seed plateaus below expert reward while the others reach it—co

Load-bearing premise

The cross-game scaling claim rests on hand-assigned estimates of how many state variables each game uses (Pong ~5, Breakout ~15–25, Space Invaders ~40+); if those ordinal complexity numbers are wrong, the claim that active-neuron count tracks task complexity loses its quantitative grounding.

Editorial extensions

If this is right

  • Frozen-random-feature agents can be used as a no-penalty probe: the number of active readout neurons estimates how many independent state variables a task's policy actually uses.
  • Width-scaling results imply that increasing FC width does not change the absolute number of neurons recruited, so sparse bottlenecks reflect task structure rather than a fixed fraction of capacity.
  • Representational compression precedes competence: because the active set locks long before reward improves, downstream learning works within an immutable feature subspace, suggesting training interventions should focus on the readout rather than the features.
  • Frozen and trainable CNNs can reach comparable reward with structurally different bottlenecks—narrow-but-diverse versus broad-but-redundant—so active count alone is not a health metric; participation ratio is a complementary indicator.
  • Deployment is cheap: only the trained readout (about 9,400 parameters for a 3-neuron Pong solution) needs to be stored, since the random seed regenerates the frozen projection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same frozen-projection-plus-emergent-sparse-readout recipe could serve as a diagnostic for intrinsic dimensionality in any high-dimensional prediction setting with few underlying factors—genomics, imaging, or sensor arrays—where the paper offers this as conjecture but does not test it.
  • Beyond the paper: if early active-set lock generalizes, a direct testable extension is to rewire only the trainable head after lock (e.g., replace FC2 with a wider MLP) and ask whether reward ceiling lifts; the paper's capacity-lag pattern predicts it would not for a fixed projection.
  • Beyond the paper: the 0.01 activation threshold is a measurement choice; a threshold sweep (0.001 to 0.1) would reveal whether the 1–3-neuron count is a stable plateau or a knife-edge, which the current manuscript does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper reports that deep RL agents trained with a frozen, randomly initialized CNN feature extractor spontaneously learn sparse fully-connected representations without any explicit sparsity objective. It claims that in the first fully-connected layer (FC1, 3,136->64), Pong agents use as few as 1-3 of 64 neurons, and that the number of active FC1 neurons scales with estimated task complexity across Pong, Freeway, Breakout, and Space Invaders. Ablations show that removing the active neurons collapses performance to random play, and a longitudinal sweep indicates that the active set locks early in training, well before reward becomes positive. The paper also contrasts frozen and trainable CNNs in Breakout using active count and participation ratio.

Significance. The phenomenon is interesting and, if confirmed, would connect reservoir computing, random feature methods, and implicit sparsity in RL in a way that is potentially useful for diagnosing task complexity and designing efficient readouts. The ablation protocol is a genuine strength: demonstrating necessity of the active set with a clean intervention across two PPO implementations and multiple games is the kind of evidence that separates a real effect from a threshold artifact. The paper is also transparent about small sample sizes and the lack of a proof. However, the central quantitative claims rest on a single activation threshold and hand-assigned complexity estimates, and there is an internal inconsistency between the headline numbers and the full results table. These issues materially affect the strength of the conclusions as currently stated.

major comments (4)
  1. [Abstract/Table 1 vs. Appendix C, Table 7] The headline number for deterministic Pong is internally inconsistent. The abstract and Table 1 state 1-3 active FC1 neurons, but Appendix C, Table 7 lists five SB3 frozen deterministic seeds with active counts 3, 4, 6, 8, 8 (range 3-8); the 1-2 values come only from SF seeds. Please reconcile the reported ranges and either present the full distribution or clearly separate SB3 and SF results. Because the sparsity claim and the cross-game scaling in Figure 2 depend on these counts, this discrepancy needs a precise correction.
  2. [Section 3.4 (active-neuron definition), Fig. 2/Table 1] All quantitative sparsity results depend on the threshold that classifies a neuron as active if its mean post-ReLU activation over 1,000 frames exceeds 0.01. The paper provides no histogram of per-neuron mean activations and no sensitivity analysis. If many neurons have activations just below 0.01, lowering the threshold to, say, 0.001 could materially increase the reported counts and potentially alter the ordering in Table 1 and Figure 2. The FC1-Keep ablation shows that sub-threshold neurons are not needed for performance, which supports a causal bottleneck, but it does not establish the quantitative active-count values used for the complexity-scaling claim. Please add a threshold sweep (e.g., 0.001, 0.005, 0.01, 0.05, 0.1) and report whether the headline ranges and cross-game monotonicity are stable.
  3. [Section 3.3, Table 16 (task-complexity estimates)] The x-axis of Figure 2 is based on hand-set estimates of 'policy-relevant state variables' (Pong ~5, Freeway ~10-15, Breakout ~15-25, Space Invaders ~40+). The paper notes that substituting AtariARI labeled counts would change the slope but preserve rank order, but no such alternative is plotted or quantified. Because the claim that active count 'tracks task complexity' is central, please provide a robustness check with at least one independent complexity measure (e.g., AtariARI counts, a learned state-dimension estimator, or a simple proxy like the number of moving objects). If the monotonic relation only holds under one particular ordinal assignment, the claim needs to be weakened.
  4. [Section 6.1 / Appendix M (mechanism and "emergent sparsity" wording)] The phrase 'emergent sparsity without any sparsity-inducing objective' is strong. ReLU itself naturally produces exact zeros, and the 0.01 threshold treats near-zero mean activations as inactive. The paper's own Appendix N.6 shows that an L1-regularized linear probe needs ~1,000 features, while the ReLU network uses ~3, so the nonlinearity is doing important work. The ablation evidence supports the causal claim, but the 'spontaneous emergence' framing should be tempered unless the threshold-robustness analysis shows that sparsity is not merely a tail of the ReLU output distribution. I recommend explicitly stating this caveat in the main text.
minor comments (4)
  1. [Figure 2] The 'Frozen trend' line is described as a qualitative visual guide, not a fitted regression. The word 'trend' may mislead readers into thinking there is a fitted quantitative relationship. Consider labeling it 'illustrative' or removing it from the figure.
  2. [Table 4] The 'Final PR' column for c10 is listed as 13.31 in Table 4, while Table 2 reports 13.73 for the same seed. If these come from different sweeps or checkpoint timings, please clarify; otherwise unify the values.
  3. [Abstract / Section 4.3] The within-game scaling claim is appropriately hedged as 'suggestive' with n=3 in the main text, but the abstract states 'three identical Pong seeds produce 5, 7, and 11 active neurons' without noting the wide error bars or the binary nature of the reward split (+14 vs. +18). A short caveat in the abstract would improve precision.
  4. [Section 3.4 / Appendix N.4] The text says activation statistics are computed from 1,000 frames sampled during evaluation, but elsewhere it says 50-100 evaluation episodes. Please clarify how many episodes and how many frames per episode are used for the activation sample, since this affects the reliability of the 0.01 threshold.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the sparsity results are empirical and independently ablated, not derived from fitted quantities.

full rationale

The paper's central claims are empirical observations with independent validation, not reductions to fitted quantities. The active-neuron count is defined by an explicit threshold ('A neuron is classified as “active” if its mean post-ReLU activation exceeds 0.01', Section 3.4), but this is a measurement convention rather than a parameter fitted to the target conclusion. The causal role of the threshold-defined active set is independently established by ablations: FC1-Remove crashes performance to random play across two PPO implementations and four games, and FC1-Keep preserves near-baseline performance, showing that the threshold is not just relabeling a smooth activation tail. The cross-game complexity axis is admittedly heuristic ('grounded in domain analysis and broadly consistent with AtariARI semantic RAM annotations', Section 3.3), but the paper explicitly labels the frozen trend as 'a qualitative visual guide, not a fitted regression' and reports the within-game, same-recipe seed comparison as a separate controlled test. Width-scaling, cross-framework replication, and the longitudinal lock-before-reward sweep are all external empirical checks that do not collapse into the inputs. Candidate theoretical mechanisms are explicitly described as unproven and future work. There are no self-citations, no imported uniqueness theorems, and no fitted parameter renamed as a prediction. The threshold and state-variable estimates are legitimate robustness/validity concerns, but they do not make the derivation circular.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities or theoretical constructs. Its load-bearing free parameters are the activation threshold, the heuristic complexity axis, the progressive sticky curriculum, and the post-hoc PR band. The main axioms are the validity of these measurement choices and the heuristic random-projection mechanism, both acknowledged by the authors as unproven or approximate.

free parameters (4)
  • FC1 active-neuron threshold = 0.01 mean post-ReLU activation
    A neuron is counted as active only if its mean activation exceeds 0.01 (Section 3.4). No sensitivity analysis is provided, and this choice directly determines all reported active counts.
  • Task-complexity x-axis estimates = Pong ~5; Breakout ~15–25; Space Invaders ~40+; Freeway ~10–15
    Hand-assigned state-variable counts used to show cross-game scaling (Section 3.3, Table 16). The paper admits they are heuristic and only broadly consistent with AtariARI labels.
  • Progressive sticky curriculum schedule = p_sticky interpolates linearly from 0 at reward −20 to 0.25 at reward 0
    This curriculum was load-bearing for stochastic frozen Pong (Appendix B reports only 3/~20 seeds passed screening) and for Breakout progressive-sticky training. Without it the frozen sticky results largely do not obtain.
  • FC2-output PR expert/stuck band = empty band [12.6, 17.7]
    The claim that PR separates expert from stuck trainable Pong seeds is derived from n=9 seeds with a gap discovered post hoc (Section 4.4, Figure 3B). The paper correctly labels it suggestive, but it is still a data-fitted threshold.
assumptions (4)
  • standard math Random projections preserve task-relevant geometric structure (Johnson–Lindenstrauss regime)
    Invoked in Section 6.1 to argue the frozen random CNN retains enough structure for low-dimensional tasks. Quantitative applicability to RL pixel inputs and ReLU readouts is not proven.
  • domain assumption Hand-assigned state-variable estimates measure intrinsic task complexity
    Section 3.3 and Table 16. The cross-game scaling conclusion depends on this ordinal complexity axis.
  • ad hoc to paper Mean post-ReLU activation > 0.01 identifies functionally active neurons
    Section 3.4 defines the central metric with this threshold. No threshold sweep is reported, and all downstream claims use this single value.
  • domain assumption Gradient descent on a frozen random ReLU projection discovers the projection's usable task-relevant directions
    Section 6.1 and Appendix M present this as a heuristic interpretation; the paper explicitly states the precise implicit bias 'remains mathematically unproven.'

how reviews work

0 comments
Cite this review

Pith. "Pith review of Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/J3I6O3QA

@misc{pith2026260726059,
  author       = {Pith},
  title        = {Pith review of: Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J3I6O3QA}},
  note         = {Machine review of arXiv:2607.26059}
}
abstract

We report a striking phenomenon: deep reinforcement learning agents trained with frozen, randomly initialized CNN feature extractors spontaneously develop extremely sparse fully-connected representations, without any sparsity-inducing objective. In the first fully-connected layer (FC1, $3{,}136 \to 64$), agents compress task-relevant information through as few as 1-3 neurons out of 64 for deterministic Pong (5-11 for stochastic Pong), while trainable CNNs activate 55-64 neurons under matched conditions. We establish four principal findings. First, FC1 sparsity scales with task complexity: 1-11 for Pong, 19-26 for Breakout, and $\sim$42 for Space Invaders. Width-scaling confirms this reflects task structure rather than a fixed capacity fraction. Second, within-game scaling emerges: three identical Pong seeds produce 5, 7, and 11 active neurons. The 5-neuron seed plateaus at $+14$ reward, while the others reach expert performance ($+18.4$, $+18.7$), suggesting the random projection's usable dimensionality bounds achievable performance. Third, ablation confirms necessity: removing these active neurons crashes performance across two PPO implementations and four games. Fourth, the information bottleneck commits early: a sweep shows the active set locks by 15-30M steps, while reward turns positive 35-105M steps later. A complementary finding in Breakout shows frozen and trainable CNNs reach competitive rewards via structurally different bottlenecks: frozen agents use 17-25 active neurons (participation ratio $\sim$10-14), while trainable agents use 51 (participation ratio $\sim$3.6). Finally, wherever input dimensionality dwarfs intrinsic task dimensionality, gradient descent on a frozen random projection may reveal the effective rank of the underlying problem without explicit sparsity machinery.

Figures

Figures reproduced from arXiv: 2607.26059 by the authors.

Figure 1
Figure 1. Frozen random CNN architecture and the cross-game sparsity it produces. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The frozen-CNN bottleneck size tracks task complexity, while trainable-CNN size does not. Active [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Ablation universality and a candidate cross-framework PR threshold. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The information bottleneck locks before performance emerges, with lag inversely proportional [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 5 linked inside Pith

  1. [1]

    Loss of plasticity in continual deep reinforcement learning

    Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, and Rene Vidal. Loss of plasticity in continual deep reinforcement learning. In Conference on Lifelong Learning Agents, 2023

    2023

  2. [2]

    Database-friendly random projections: J ohnson-- L indenstrauss with binary coins

    Dimitris Achlioptas. Database-friendly random projections: J ohnson-- L indenstrauss with binary coins. Journal of Computer and System Sciences, 66 0 (4): 0 671--687, 2003

    2003

  3. [3]

    Intrinsic dimensionality explains the effectiveness of language model fine-tuning

    Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. Intrinsic dimensionality explains the effectiveness of language model fine-tuning. arXiv preprint arXiv:2012.13255, 2020

    arXiv 2012

  4. [4]

    Unsupervised state representation learning in A tari

    Ankesh Anand, Evan Racah, Sherjil Ozair, Yoshua Bengio, Marc-Alexandre C \^o t \'e , and R Devon Hjelm. Unsupervised state representation learning in A tari. In Advances in Neural Information Processing Systems, 2019

    2019

  5. [5]

    SGD with large step sizes learns sparse features

    Maksym Andriushchenko, Aditya Varre, Loucas Pillaud-Vivien, and Nicolas Flammarion. SGD with large step sizes learns sparse features. In International Conference on Machine Learning, 2023

    2023

  6. [6]

    DiffuserCam : lensless single-exposure 3D imaging

    Nick Antipa, Grace Kuo, Reinhard Heckel, Ben Mildenhall, Emrah Bostan, Ren Ng, and Laura Waller. DiffuserCam : lensless single-exposure 3D imaging. Optica, 5 0 (1): 0 1--9, 2018

    2018

  7. [7]

    Random projection in dimensionality reduction: applications to image and text data

    Ella Bingham and Heikki Mannila. Random projection in dimensionality reduction: applications to image and text data. In Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.\ 245--250, 2001

    2001

  8. [8]

    Robust uncertainty principles

    Emmanuel J Cand \`e s, Justin K Romberg, and Terence Tao. Robust uncertainty principles. IEEE Transactions on Information Theory, 52 0 (2): 0 489--509, 2006

    2006

Show all 46 references
  1. [9]

    Reinforcement learning with convolutional reservoir computing

    Hanten Chang and Katsuya Futagami. Reinforcement learning with convolutional reservoir computing. Applied Intelligence, 2020

    2020

  2. [10]

    What makes freezing layers effective? arXiv preprint, 2025

    Martin Coil et al. What makes freezing layers effective? arXiv preprint, 2025

    2025

  3. [11]

    Playing A tari with six neurons

    Giuseppe Cuccu, Julian Togelius, and Philippe Cudre-Mauroux. Playing A tari with six neurons. In International Conference on Autonomous Agents and MultiAgent Systems, 2019

    2019

  4. [12]

    An elementary proof of a theorem of J ohnson and L indenstrauss

    Sanjoy Dasgupta and Anupam Gupta. An elementary proof of a theorem of J ohnson and L indenstrauss. Random Structures & Algorithms, 22 0 (1): 0 60--65, 2003

    2003

  5. [13]

    The interplay between sparsity and training in deep reinforcement learning

    Julio Davelouis et al. The interplay between sparsity and training in deep reinforcement learning. arXiv preprint, 2025

    2025

  6. [14]

    Loss of plasticity in deep continual learning

    Shibhansh Dohare, J Fernando Hernandez-Garcia, et al. Loss of plasticity in deep continual learning. Nature, 632: 0 768--774, 2024

    2024

  7. [15]

    Compressed sensing

    David L Donoho. Compressed sensing. IEEE Transactions on Information Theory, 52 0 (4): 0 1289--1306, 2006

    2006

  8. [16]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carlin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019

    2019

  9. [17]

    Why random pruning is all we need to start sparse

    Advait Gadhikar, Sohom Mukherjee, and Rebekka Burkholz. Why random pruning is all we need to start sparse. In International Conference on Machine Learning, 2024

    2024

  10. [18]

    Weight agnostic neural networks

    Adam Gaier and David Ha. Weight agnostic neural networks. In Advances in Neural Information Processing Systems, 2019

    2019

  11. [19]

    Implicit regularization in matrix factorization

    Suriya Gunasekar, Blake Woodworth, Srinadh Bhojanapalli, Behnam Neyshabur, and Nathan Srebro. Implicit regularization in matrix factorization. Advances in Neural Information Processing Systems, 2018

    2018

  12. [20]

    Learning both weights and connections for efficient neural networks

    Song Han, Jeff Pool, John Tran, and William J Dally. Learning both weights and connections for efficient neural networks. In Advances in Neural Information Processing Systems, 2015

    2015

  13. [21]

    Delving deep into rectifiers

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers. IEEE International Conference on Computer Vision, 2015

    2015

  14. [22]

    LoRA : Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA : Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  15. [23]

    The ``echo state'' approach to analysing and training recurrent neural networks

    Herbert Jaeger. The ``echo state'' approach to analysing and training recurrent neural networks. GMD Technical Report, 148, 2001

    2001

  16. [24]

    Extensions of L ipschitz mappings into a H ilbert space

    William B Johnson and Joram Lindenstrauss. Extensions of L ipschitz mappings into a H ilbert space. Contemporary Mathematics, 26: 0 189--206, 1984

    1984

  17. [25]

    Implicit under-parameterization inhibits data-efficient deep reinforcement learning

    Aviral Kumar, Rishabh Agarwal, Dibya Ghosh, and Sergey Levine. Implicit under-parameterization inhibits data-efficient deep reinforcement learning. arXiv preprint arXiv:2010.14498, 2021

    2010 arXiv

  18. [26]

    Optimal brain damage

    Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. In Advances in Neural Information Processing Systems, 1990

    1990

  19. [27]

    Measuring the intrinsic dimension of objective landscapes

    Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimension of objective landscapes. In International Conference on Learning Representations, 2018

    2018

  20. [28]

    Understanding plasticity in neural networks

    Clare Lyle, Zeyu Zheng, Evgenii Nikishin, et al. Understanding plasticity in neural networks. In International Conference on Machine Learning, 2023

    2023

  21. [29]

    Sparsity for free: Overcoming the limitations of dense scaling in deep reinforcement learning

    Yiqin Ma, Jonathan Viquerat, and Elie Hachem. Sparsity for free: Overcoming the limitations of dense scaling in deep reinforcement learning. arXiv preprint, 2025

    2025

  22. [30]

    Real-time computing without stable states

    Wolfgang Maass, Thomas Natschl \"a ger, and Henry Markram. Real-time computing without stable states. Neural Computation, 14 0 (11): 0 2531--2560, 2002

    2002

  23. [31]

    Revisiting the A rcade L earning E nvironment: Evaluation protocols and open problems for general agents

    Marlos C Machado, Marc G Bellemare, Erik Talvitie, Joel Veness, Matthew Hausknecht, and Michael Bowling. Revisiting the A rcade L earning E nvironment: Evaluation protocols and open problems for general agents. Journal of Artificial Intelligence Research, 61: 0 523--562, 2018

    2018

  24. [32]

    Proving the lottery ticket hypothesis: Pruning is all you need

    Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, and Ohad Shamir. Proving the lottery ticket hypothesis: Pruning is all you need. In International Conference on Machine Learning, 2020

    2020

  25. [33]

    Human-level control through deep reinforcement learning

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. Human-level control through deep reinforcement learning. Nature, 518 0 (7540): 0 529--533, 2015

    2015

  26. [34]

    No representation, no trust: Connecting representation, collapse, and trust issues in PPO

    Skander Moalla et al. No representation, no trust: Connecting representation, collapse, and trust issues in PPO . arXiv preprint, 2024

    2024

  27. [35]

    Pruning convolutional neural networks for resource efficient inference

    Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440, 2017

    2017 arXiv

  28. [36]

    The primacy bias in deep reinforcement learning

    Evgenii Nikishin, Max Schwarzer, Pierluca D'Oro, Pierre-Luc Bacon, and Aaron Courville. The primacy bias in deep reinforcement learning. In International Conference on Machine Learning, 2022

    2022

  29. [37]

    In value-based deep reinforcement learning, a pruned network is a good network

    Johan Obando-Ceron, Aaron Courville, and Pablo Samuel Castro. In value-based deep reinforcement learning, a pruned network is a good network. In International Conference on Machine Learning, 2024

    2024

  30. [38]

    Sample factory: Egocentric 3d control from pixels at 100000 FPS with asynchronous reinforcement learning

    Aleksei Petrenko, Zhehui Huang, Tushar Kumar, Gaurav Sukhatme, and Vladlen Koltun. Sample factory: Egocentric 3d control from pixels at 100000 FPS with asynchronous reinforcement learning. In International Conference on Machine Learning, 2020

    2020

  31. [39]

    Stable- B aselines3: Reliable reinforcement learning implementations

    Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable- B aselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22 0 (268): 0 1--8, 2021

    2021

  32. [40]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems, 2007

    2007

  33. [41]

    What's hidden in a randomly weighted neural network? In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

    Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari. What's hidden in a randomly weighted neural network? In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

    2020

  34. [42]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  35. [43]

    Dormant neuron phenomenon in deep reinforcement learning

    Ghada Sokar, Rishabh Agarwal, Pablo Samuel Castro, and Utku Evci. Dormant neuron phenomenon in deep reinforcement learning. In International Conference on Machine Learning, 2023

    2023

  36. [44]

    Regression shrinkage and selection via the lasso

    Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B, 58 0 (1): 0 267--288, 1996

    1996

  37. [45]

    Kernel and rich regimes in overparameterized models

    Blake Woodworth, Suriya Gunasekar, Jason D Lee, et al. Kernel and rich regimes in overparameterized models. In Conference on Learning Theory, 2020

    2020

  38. [46]

    Pre-trained visual features for visual RL

    Jesse Zhang et al. Pre-trained visual features for visual RL . arXiv preprint, 2025

    2025

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.