Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read rlpyt claims a single PyTorch code base implements all three deep RL algorithm families and reproduces record-setting Atari results from R2D2 on a single workstation.

desk verdict A useful, honest software paper whose R2D2 reproduction headline is plausible but not statistically established; the codebase itself is the real contribution. read the letter →

arxiv 1909.01500 v2 pith:Z3CJXXTG submitted 2019-09-03 cs.LG cs.AI

Adam Stooke , Pieter Abbeel This is my paper · ORCID
classification cs.LGcs.AI
keywords rlpytdeepreinforcementlearningPyTorchrecurrentexperiencereplayparallelsamplingsharedmemorynamedarraytupleAtaribenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the three main families of model-free deep reinforcement learning—deep Q-learning, policy gradients, and Q-value policy gradients—share enough machinery to live in one code base, and that a well-designed shared infrastructure can serve all of them without sacrificing throughput. It introduces rlpyt, a modular repository that implements algorithms from all three families and claims to reproduce record-setting Atari learning curves from R2D2, a benchmark previously requiring distributed compute. On a single workstation with 24 CPUs and 3 GPUs, rlpyt reaches over 16,000 steps per second, making this class of experiment accessible to labs without clusters. If the claims hold, researchers gain a single starting point for comparing and extending algorithms across the field.

What carries the argument

The load-bearing mechanism is the shared sampling-optimization stack built on system shared memory. It offers serial, parallel-CPU, parallel-GPU, and alternating-GPU sampling; synchronous multi-GPU optimization with all-reduced gradients; and an asynchronous mode where sampling and optimization run in separate processes connected by a shared-memory replay buffer protected by a read-write lock, with memory-copier processes and a double buffer so sampling never blocks. Recurrent agents are supported through CuDNN-compatible state handling, and the replay buffer can use n-step returns, sequence replay, prioritized replay via sum tree, and frame-based storage. This machinery is what lets a non-distributed setup sustain the data rates R2D1 needs.

What would settle it

Run rlpyt's R2D1 on the full Atari suite with the corrected 5-step TD initial priorities and the original replay-ratio counting, using at least three seeds per game; if median scores on several games fall clearly outside the published R2D2 learning curves—as Gravitar already does—the reproduction claim is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that rlpyt is a comprehensive, high-throughput research code base implementing deep Q-learning (DQN and variants including Rainbow minus Noisy Nets and recurrent R2D2-like agents), policy gradients (A2C, PPO), and Q-function policy gradients (DDPG, TD3, SAC) on one shared, optimized infrastructure in a single repository. The architecture uses system shared memory for inter-process communication, with four sampling configurations, synchronous multi-GPU optimization, and an asynchronous sampling-optimization mode tied together by a shared-memory replay buffer. The headline evidence is the R2D1 run, a non-distributed R2D2 reproduction that reached 8 billion steps and 1 million updates in under 138 hours, with learning curves that surpass previous algorithms on several Atari games. The paper also introduces namedarraytuple, a data structure for writing into collections of arrays with a single indexed assignment.

Load-bearing premise

The benchmark claim rests on the assumption that rlpyt's R2D1 configuration—a replay ratio of 0.67 by the original authors' counting, uncorrected new-sample prioritization in several games, and one seed per game—is close enough to the original R2D2 setup to count as a reproduction.

Editorial extensions

If this is right

  • Researchers can run recurrent experience-replay benchmarks like R2D1 on one workstation, at over 16,000 steps per second, without spinning up distributed infrastructure.
  • A single repository can support direct comparisons among DQN-family, policy-gradient, and Q-function policy-gradient algorithms, since all share the same sampler, agent, and logger interfaces.
  • The asynchronous mode lets sampling and optimization run continuously, with the replay ratio throttled; increasing the replay ratio is listed as a direction for faster learning using multi-GPU optimization.
  • The namedarraytuple structure means multi-modal observations need not be flattened; fields like camera images and joint angles can keep separate identities while sharing batch and time slicing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to raise the replay ratio from its counted 0.67 closer to the original 0.8 and apply the corrected 5-step TD priorities, which could close the gap on games like Gravitar where the reported run plateaued below 6,000.
  • If the reproduction claim generalizes, the practical threshold for deep RL research drops: experiments that once required a distributed cluster can be iterated on a single node, changing which groups can pursue recurrent off-policy methods.
  • Beyond the paper, the single-node design could be extended to multi-node scaling; the paper itself notes its components could form building blocks for a distributed framework, and such an extension might approach R2D2's originally quoted throughput.
  • The namedarraytuple design may be adopted outside RL wherever heterogeneous arrays share leading dimensions, such as multimodal supervised learning pipelines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents rlpyt, a PyTorch-based deep reinforcement learning code base that implements three algorithm families (deep Q-learning, policy gradients, and Q-function policy gradients) on a shared infrastructure. It describes the parallel sampling and optimization schemes, asynchronous replay-buffer operation, a new 'namedarraytuple' data structure, and provides learning curves on MuJoCo and Atari environments. The headline claim is that rlpyt reproduces record-setting R2D2 Atari results without distributed compute, reaching over 16,000 steps per second on a single workstation with 24 CPUs and 3 GPUs. The paper is written as a white paper / software release note, with implementation details and relation to prior work.

Significance. rlpyt is a substantial, publicly available software artifact. Its modular design, support for recurrent agents, and coverage of widely used algorithms (A2C, PPO, DQN variants, DDPG, TD3, SAC) make it a potentially valuable resource for the RL community. The paper's credibility rests on two pillars: the code repository itself, which is machine-checkable, and the reported learning curves, which for the non-R2D2 benchmarks appear to follow expected behavior with multiple seeds. However, the headline R2D2 reproduction claim is not statistically supported: it uses one seed per game, the configuration deviates from the original R2D2 in several acknowledged ways, and one game (Gravitar) fails to reach the claimed level. If the authors either provide multi-seed corrected runs or substantially soften the reproduction claim, the paper would be appropriate as a software contribution. The namedarraytuple concept is a minor but potentially useful idea, though its novelty and performance are not evaluated.

major comments (3)
  1. [§3.2, Figure 7 and footnotes 4–5] The central claim that rlpyt 'reproduces record-setting results' from R2D2 is not supported by the evidence presented. Figure 7 shows a single seed per game, which is insufficient for Atari due to high score variance. More importantly, footnotes 4 and 5 report material configuration deviations: swapped replay priority coefficients, 1-step TD errors for prioritization of new samples (as opposed to R2D2's 5-step), use of only half of the 80-step training segment for priority computation, and a replay ratio of 0.67 by the original authors' counting rather than roughly 0.8. These differences directly affect the learning dynamics of a prioritization-based recurrent agent. The paper also concedes that Gravitar plateaued below 6,000 and that 'work to remedy this continues.' Consequently, the data cannot establish that rlpyt's R2D1 is a faithful reproduction of R2D2. The authors should either provide multi-seed results with corrected configuration, or revise the claim to something like 'single-run learning curves that resemble R2D2 on several games, with a known remaining gap included.'
  2. [§3.2, R2D1 paragraph and Figure 8] The throughput claim of 'over 16,000 steps per second' is based on a single run (Amidar) with no description of how steps are counted (environment steps vs. frames), whether evaluation and logging time are included, or whether the reported rate is sustained or peak. The comparison with R2D2's 66,000 steps per second under a different hardware configuration (256 CPUs, 1 GPU) also lacks a clear normalization. As the paper's 'high-throughput' characterization is a key selling point, this measurement should be described more precisely and ideally reported over multiple runs with error bars.
  3. [§5, Related Work] The statement that other code bases 'are not optimized to the extent of rlpyt' is made without a quantitative basis. The paper does not report comparative timing or resource-utilization benchmarks against Baselines, Dopamine, or RLlib. Since the related-work section is short and the paper's contribution is explicitly infrastructural, adding at least a table or prose summary of comparative throughput on a common benchmark would make the claim checkable rather than rhetorical.
minor comments (5)
  1. [Figure 2 caption] The caption contains a typo: 'DistribuedDataParallel' should be 'DistributedDataParallel'.
  2. [§3.1] The text says 'Mujoco 2 tasks' and footnote 2 says 'mujoco200'; please use the standard spelling 'MuJoCo' and clarify the exact version (e.g., MuJoCo 2.0).
  3. [§4] The namedarraytuple is presented as a 'new data structure,' but the paper does not compare it with existing tuple- or array-based structures (e.g., NumPy structured arrays or named tuples) in terms of API ergonomics or performance. A short paragraph or code example showing a potential use case would strengthen this section; currently it is more of an implementation note than a standalone contribution.
  4. [§3.2] The acronym 'R2D1' is introduced without explicit definition; consider 'non-distributed R2D2 (R2D1)' on first use to avoid confusion.
  5. [References] Reference [6] (R2D2) is cited as an arXiv preprint without a year; provide the full citation (e.g., the ICLR 2019 version or the arXiv identifier with year 2018).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are checked against external published benchmarks and a public repository, not derived from its own assumptions.

full rationale

This is a software-infrastructure and benchmarking paper, not a theoretical derivation. The central claim that rlpyt reproduces record-setting R2D2 results is validated by comparing learning curves against the external R2D2 paper and standard Atari benchmarks, and the repository itself is the artifact under discussion. No quantity is defined in terms of another, no fitted parameter is renamed as a prediction, and no load-bearing argument reduces to a self-citation. Self-citations to the authors' accel_rl work are used for ancestry and design rationale only; even if that prior scaling experience were disputed, the implemented code and the externally referenced learning curves stand independently. The paper's own footnotes 4 and 5 acknowledge configuration deviations from R2D2: swapped replay priority coefficients, 1-step TD errors for new-sample priorities, half-segment priority computation, a replay ratio of 0.67 by the original authors' counting, a single seed per game, and a Gravitar run that plateaued below 6,000. These are significant evidence limitations for the benchmark claim, but they concern experimental fidelity and statistical power, not circularity. The derivation chain, such as it is, is self-contained against external benchmarks, so the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

No free parameters: hyperparameters come from prior publications and hardware or replay-ratio choices are operational, not fitted constants. The central claim depends on four domain assumptions: benchmark comparability, hyperparameter transfer, synchronous all-reduce equivalence, and CuDNN recurrence correctness. The only introduced artifact is namedarraytuple, an implemented software structure with external evidence in the repository.

assumptions (4)
  • domain assumption The OpenAI Gym Atari and Mujoco environments provide fair, comparable testbeds across implementations.
    Section 3 compares learning curves on these environments against published values without validating environment versions, wrappers, or benchmark protocols.
  • domain assumption Published hyperparameters transfer to rlpyt's implementations.
    Section 3.1 says the same published hyperparameters are used across all environments, with no sensitivity analysis to rule out framework-specific tuning.
  • domain assumption Synchronous multi-process gradient all-reduce in PyTorch DistributedDataParallel preserves the semantics of a single-process training run.
    Section 2.2 relies on this equivalence for parallel-GPU and multi-CPU optimization; no isolation experiment is provided.
  • domain assumption CuDNN recurrent layers faithfully implement the recurrent agent math used in R2D2 and R2D1.
    Section 6.3 delegates recurrent training to the CuDNN interface, implicitly assuming it matches the original R2D2 recurrent update.
invented entities (1)
  • namedarraytuple data structure independent evidence
    purpose: A container for collections of numpy arrays or torch tensors that supports indexed or sliced read and write access.
    It is implemented and documented in the repository at rlpyt/utils/collections.py, so anyone can run it; it is not a postulated physical entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch." pith.science (2026). https://pith.science/paper/Z3CJXXTG

@misc{pith2026190901500,
  author       = {Pith},
  title        = {Pith review of: rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z3CJXXTG}},
  note         = {Machine review of arXiv:1909.01500}
}
read the original abstract

Since the recent advent of deep reinforcement learning for game play and simulated robotic control, a multitude of new algorithms have flourished. Most are model-free algorithms which can be categorized into three families: deep Q-learning, policy gradients, and Q-value policy gradients. These have developed along separate lines of research, such that few, if any, code bases incorporate all three kinds. Yet these algorithms share a great depth of common deep reinforcement learning machinery. We are pleased to share rlpyt, which implements all three algorithm families on top of a shared, optimized infrastructure, in a single repository. It contains modular implementations of many common deep RL algorithms in Python using PyTorch, a leading deep learning library. rlpyt is designed as a high-throughput code base for small- to medium-scale research in deep RL. This white paper summarizes its features, algorithms implemented, and relation to prior work, and concludes with detailed implementation and usage notes. rlpyt is available at https://github.com/astooke/rlpyt.

Figures

Figures reproduced from arXiv: 1909.01500 by the authors.

Figure 1
Figure 1. Environment interaction sampling schemes. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Synchronous multi-process reinforcement learning. Each python process runs a copy of the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Asynchronous sampling/optimization mode. Separate python processes run optimization [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Continuous control in Mujoco by RL algorithms–DDPG (settings from [ [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Policy gradient algorithms–A2C (feed-forward), A2C-LSTM (1-frame observation), A2C [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: DQN plus variants–Categorical, Prioritized-Dueling-Double, Rainbow minus Noisy Nets, [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Reproduction of R2D2 learning curves in rlpyt, a single seed each. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: The same learning curve for the game Amidar over three horizontal axes: environment steps, model updates, and wall-clock time, for rlpyt’s R2D1 implementation run in asynchronous sampling mode using 24 CPU cores and 3 GPUs. 4Most curves used 1-step TD errors for priori…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Tilted Quantile Gradient Updates for Quantile-Constrained Reinforcement Learning

    cs.LG 2024-12 reject novelty 5.0 of 10

    TQPO estimates gradients of quantile safety constraints directly through sampling and adds a tilted update to the Lagrange multiplier, improving return while satisfying the constraints.

  2. Towards Fault Tolerance in Multi-Agent Reinforcement Learning

    cs.LG 2024-11 conditional novelty 4.0 of 10

    A fault-tolerant MARL method using attention in actor and critic networks plus per-module prioritized experience replay improves team performance when agents suddenly fail.

Reference graph

Works this paper leans on

36 extracted references · 10 canonical work pages · cited by 2 Pith papers

  1. [1]

    Playing atari with deep reinforcement learning

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013

    arXiv 2013

  2. [2]

    Trust region policy optimization

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International conference on machine learning, pages 1889–1897, 2015

    2015

  3. [3]

    Automatic differentiation in PyTorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. In NIPS Autodiff Workshop, 2017

    2017

  4. [4]

    Alphastar

    DeepMind. Alphastar. https://deepmind.com/blog/article/ alphastar-mastering-real-time-strategy-game-starcraft-ii , 2019

    work page 2019

  5. [5]

    Openai five

    OpenAI. Openai five. https://blog.openai.com/openai-five/, 2018

    2018

  6. [6]

    Recurrent experience replay in distributed reinforcement learning

    Steven Kapturowski, Georg Ostrovski, John Quan, Remi Munos, and Will Dabney. Recurrent experience replay in distributed reinforcement learning. 2018

    work page 2018

  7. [7]

    Openai gym, 2016

    Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, 2016

    2016

  8. [8]

    Asynchronous methods for deep reinforcement learning

    V olodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lilli- crap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928–1937, 2016

    1928

Show all 36 references
  1. [9]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  2. [10]

    Deep reinforcement learning with double q-learning

    Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q-learning. In Thirtieth AAAI conference on artificial intelligence, 2016

    2016

  3. [11]

    Dueling network architectures for deep reinforcement learning

    Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Van Hasselt, Marc Lanctot, and Nando De Freitas. Dueling network architectures for deep reinforcement learning. arXiv preprint arXiv:1511.06581, 2015

    2015 arXiv

  4. [12]

    A distributional perspective on reinforce- ment learning

    Marc G Bellemare, Will Dabney, and Rémi Munos. A distributional perspective on reinforce- ment learning. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 449–458. JMLR. org, 2017

    2017

  5. [13]

    Rainbow: Combining improvements in deep reinforcement learning

    Matteo Hessel, Joseph Modayil, Hado Van Hasselt, Tom Schaul, Georg Ostrovski, Will Dab- ney, Dan Horgan, Bilal Piot, Mohammad Azar, and David Silver. Rainbow: Combining improvements in deep reinforcement learning. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018

    2018

  6. [14]

    Distributed prioritized experience replay

    Dan Horgan, John Quan, David Budden, Gabriel Barth-Maron, Matteo Hessel, Hado Van Hasselt, and David Silver. Distributed prioritized experience replay. arXiv preprint arXiv:1803.00933, 2018

    2018 arXiv

  7. [15]

    Implicit quantile networks for distributional reinforcement learning

    Will Dabney, Georg Ostrovski, David Silver, and Rémi Munos. Implicit quantile networks for distributional reinforcement learning. arXiv preprint arXiv:1806.06923, 2018. 10

    2018 arXiv

  8. [16]

    Continuous deep q-learning with model-based acceleration

    Shixiang Gu, Timothy Lillicrap, Ilya Sutskever, and Sergey Levine. Continuous deep q-learning with model-based acceleration. In International Conference on Machine Learning , pages 2829–2838, 2016

    2016

  9. [17]

    Addressing function approximation error in actor-critic methods

    Scott Fujimoto, Herke van Hoof, and David Meger. Addressing function approximation error in actor-critic methods. arXiv preprint arXiv:1802.09477, 2018

    2018 arXiv

  10. [18]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv preprint arXiv:1801.01290, 2018

    2018 arXiv

  11. [19]

    Soft actor-critic algorithms and applications

    Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Soft actor-critic algorithms and applications. CoRR, abs/1812.05905, 2018. URL http://arxiv.org/abs/ 1812.05905

    2018 arXiv

  12. [20]

    Distributed distributional deterministic policy gradients

    Gabriel Barth-Maron, Matthew W Hoffman, David Budden, Will Dabney, Dan Horgan, Alistair Muldal, Nicolas Heess, and Timothy Lillicrap. Distributed distributional deterministic policy gradients. arXiv preprint arXiv:1804.08617, 2018

    2018 arXiv

  13. [21]

    Prioritized experience replay

    Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv:1511.05952, 2015

    2015 arXiv

  14. [22]

    The arcade learning environment: An evaluation platform for general agents

    Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253–279, 2013

    2013

  15. [23]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE, 2012

    2012

  16. [24]

    Accelerated methods for deep reinforcement learning

    Adam Stooke and Pieter Abbeel. Accelerated methods for deep reinforcement learning. arXiv preprint arXiv:1803.02811, 2018

    2018 arXiv

  17. [25]

    Openai spinning up

    Joshua Achiam. Openai spinning up. GitHub, GitHub repository, 2018

    2018

  18. [26]

    Theano: new features and speed improvements

    Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian Goodfellow, Arnaud Bergeron, Nicolas Bouchard, David Warde-Farley, and Yoshua Bengio. Theano: new features and speed improvements. arXiv preprint arXiv:1211.5590, 2012

    2012 arXiv

  19. [27]

    An empirical model of large-batch training

    Sam McCandlish, Jared Kaplan, Dario Amodei, and OpenAI Dota Team. An empirical model of large-batch training. arXiv preprint arXiv:1812.06162, 2018

    2018 arXiv

  20. [28]

    Benchmarking deep reinforcement learning for continuous control

    Yan Duan, Xi Chen, Rein Houthooft, John Schulman, and Pieter Abbeel. Benchmarking deep reinforcement learning for continuous control. In International Conference on Machine Learning, pages 1329–1338, 2016

    2016

  21. [29]

    Openai baselines

    Prafulla Dhariwal, Christopher Hesse, Oleg Klimov, Alex Nichol, Matthias Plappert, Alec Radford, John Schulman, Szymon Sidor, Yuhuai Wu, and Peter Zhokhov. Openai baselines. GitHub, GitHub repository, 2017

    2017

  22. [30]

    Dopamine: A research framework for deep reinforcement learning

    Pablo Samuel Castro, Subhodeep Moitra, Carles Gelada, Saurabh Kumar, and Marc G Belle- mare. Dopamine: A research framework for deep reinforcement learning. arXiv preprint arXiv:1812.06110, 2018

    2018 arXiv

  23. [31]

    Tensorflow: A system for large-scale machine learning

    Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: A system for large-scale machine learning. In 12th{USENIX} Symposium on Operating Systems Design and Implementati...

    2016

  24. [32]

    Ray rllib: A composable and scalable reinforcement learning library

    Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Joseph Gonzalez, Ken Goldberg, and Ion Stoica. Ray rllib: A composable and scalable reinforcement learning library. arXiv preprint arXiv:1712.09381, 2017. 11

    2017 arXiv

  25. [33]

    Ray: A distributed framework for emerging{AI} applications

    Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan, et al. Ray: A distributed framework for emerging{AI} applications. In 13th{USENIX} Symposium on Operating Systems Design and...

    2018

  26. [34]

    Horizon: Facebook’s open source applied reinforcement learning platform

    Jason Gauci, Edoardo Conti, Yitao Liang, Kittipat Virochsiri, Yuchen He, Zachary Kaden, Vivek Narayanan, and Xiaohui Ye. Horizon: Facebook’s open source applied reinforcement learning platform. arXiv preprint arXiv:1811.00260, 2018

    2018 arXiv

  27. [35]

    Hogwild: A lock-free approach to parallelizing stochastic gradient descent

    Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems, pages 693–701, 2011

    2011

  28. [36]

    cudnn: Efficient primitives for deep learning

    Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. cudnn: Efficient primitives for deep learning. arXiv preprint arXiv:1410.0759, 2014. 12

    2014 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.