Pith. sign in

REVIEW 3 major objections 4 minor 39 references

Rethinking the Suitability of Reinforcement Learning Algorithms Under Practical Transfer Constraints

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sample efficiency alone can mislead RL algorithm choice for transfer; wall-clock time and domain-randomization robustness matter.

desk verdict Solid empirical benchmark with a genuinely new DR-coverage comparison, but the wall-clock headline rests on an unvalidated per-interaction time conversion that needs direct timing before it is trusted. read the letter →

arxiv 2607.17326 v1 pith:EJBZHNIF submitted 2026-07-19 cs.LG cs.RO

Hany Hamed , Abhishek Naik , Colin Bellinger , A. Rupam Mahmood This is my paper · ORCID
classification cs.LGcs.RO
keywords reinforcementlearningtransferdomainrandomizationwall-clockevaluationsampleefficiencyPPOSACTD-MPC2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that for transfer-oriented reinforcement learning, choosing an algorithm by sample efficiency alone can mislead practitioners. It compares PPO, SAC, and TD-MPC2 under two evaluation axes—interactions and wall-clock time—and finds that massively parallel PPO often produces a usable policy sooner in wall-clock terms even though it consumes more interactions. The paper also varies domain randomization coverage across five levels and reports that all three algorithm families benefit similarly; none is systematically more robust. The authors conclude that algorithm suitability depends on the evaluation axis and on how the training distribution is constructed, not just on sample efficiency.

What carries the argument

The paper's central device is the separation of evaluation axes: interaction-based evaluation (returns versus environment samples) versus time-based evaluation (returns versus wall-clock time), with wall-clock time reconstructed by converting training steps using an averaged elapsed time per interaction. Transfer is formalized with a contextual Markov decision process in which training and evaluation draw context parameters (mass, friction, damping, etc.) from different distributions, and domain randomization is treated as shaping the training context distribution. This setup lets the authors study how the support of the training distribution relates to the evaluation contexts, and how that

What would settle it

Directly measure end-to-end wall-clock training time for PPO, SAC, and TD-MPC2 on the same tasks and hardware, then compare time-to-threshold (e.g., reaching 80% of final return) on cheetah-run and walker-walk; if SAC or TD-MPC2 reaches the threshold before PPO in the majority of tasks, the claimed wall-clock advantage fails. Alternatively, training SAC and TD-MPC2 with 2,048 parallel environments and observing their time-to-policy drop below PPO's would show that the result stems from configuration rather than algorithm family.

Watch

Extended reading notes

Core claim

Under the configurations studied—PPO with 2,048 parallel environments versus SAC and TD-MPC2 in their standard single-environment setups—the sample-inefficient PPO frequently reaches strong performance earlier on a wall-clock axis, even though SAC and TD-MPC2 remain more sample-efficient on an interaction axis. This wall-clock advantage comes from PPO's ability to generate interactions rapidly through massive parallelism, not from more efficient use of each interaction. In transfer experiments with domain randomization, the paper finds no consistent advantage for any algorithmic paradigm: PPO, SAC, and TD-MPC2 can all benefit from randomized training, and the effect depends on the task, the

Load-bearing premise

The wall-clock rankings rest on an assumed conversion from interaction counts to time using an averaged per-interaction elapsed time over an unspecified training window, rather than on direct wall-clock measurement; if that conversion is biased across algorithms with very different per-step compute, the paper's headline result is not established.

Editorial extensions

If this is right

  • Transfer-oriented RL evaluations should report time-to-policy alongside sample efficiency, because the algorithm ranking can reverse between the two axes.
  • The common use of PPO in sim-to-real pipelines may be explained by its ability to exploit massive parallel simulation, not by superior sample efficiency.
  • Domain randomization can be applied to off-policy and model-based algorithms (SAC and TD-MPC2) without expecting a systematic incompatibility.
  • Increasing domain randomization coverage does not guarantee better zero-shot transfer; the relationship between training support and evaluation support matters more than coverage alone.
  • Standard interaction-based benchmark rankings may not predict which algorithm performs best under realistic wall-clock constraints in transfer workflows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The wall-clock ranking is tied to PPO's 2048-environment configuration; running SAC and TD-MPC2 with comparable parallelism—an active research direction the authors cite—could narrow or reverse the gap, so the result may be configuration-dependent rather than paradigm-dependent.
  • The conversion from training steps to wall-clock time averages per-interaction elapsed time over an unspecified window and may miss fixed costs such as model-based planning overhead, replay sampling, or logging; direct end-to-end timing would strengthen or weaken the practical-efficiency claim.
  • If domain randomization effects are truly paradigm-agnostic, then the choice of randomization schedule and coverage may be a more important design lever for sim-to-real robustness than the choice among these algorithm families.
  • The zero-shot sim-to-sim protocol could be extended to sim-to-real, where wall-clock includes real-environment interaction and per-step time is far higher; such an extension could change the relative ranking more dramatically than the sim-to-sim results suggest.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper compares PPO, SAC, and TD-MPC2 on 18 continuous-control tasks under two evaluation axes — interaction-based sample efficiency and wall-clock training time — and across five domain-randomization (DR) training distributions with zero-shot transfer to nominal-centered and shifted evaluation regimes. The central claims are (i) that sample-inefficient PPO can nevertheless produce a performant policy faster in wall-clock time than SAC and TD-MPC2 under its massively parallel configuration, and (ii) that DR does not systematically favor any of the three paradigms, its effect being task-, algorithm-, coverage-, and regime-dependent. The body carefully hedges the wall-clock claim to the configurations studied, and the supplementary material provides full learning curves and transfer box plots.

Significance. If the results hold, the paper is a valuable methodological contribution: it demonstrates that algorithm rankings can invert under time-based budgets and that transfer-oriented evaluation should jointly consider sample efficiency, wall-clock cost, and robustness under dynamics mismatch. The study's scale is a strength: 18 tasks, five DR levels, two evaluation regimes, and standard reference implementations for all three algorithms. The reproduction of the conventional sample-efficiency ranking (SAC and TD-MPC2 ahead of PPO) under interaction-based evaluation is clean and gives credibility to the experimental setup. However, the headline wall-clock result rests on a wall-clock conversion procedure that is underspecified and unvalidated, and the transfer conclusions are based on best-of-seed summaries without variance accounting. These issues are load-bearing for the paper's two main insights.

major comments (3)
  1. [Appendix A / §4.1 / Fig. 1] The wall-clock axis for all time-based curves is not directly measured. Appendix A states only that 'we averaged the elapsed time per environment interaction over a representative training window and used it to convert training steps into wall-clock time.' The window is not defined, no direct timing validation is reported, and per-algorithm fixed costs (replay warmup, TD-MPC2 planning, periodic evaluation, checkpointing) and nonstationarity of per-step time are not accounted for. Since the ranking in Fig. 1 and the paper's first insight are built entirely on this conversion, this is a load-bearing measurement assumption. The authors should either report direct wall-clock timestamps or provide itemized per-algorithm timing and a sensitivity analysis of the chosen window.
  2. [§4.3 / Appendix A / Figs. 3–4] The zero-shot transfer results and the wall-clock curves are reported using a post-hoc best-of-seed selection. Appendix A says 'we select the best-performing run among the training seeds,' and Fig. 1's dark curves are similarly obtained by ranking runs by the sum of returns over the full recorded curve. Best-of-seed selection inflates expected performance and can favor algorithms with higher run-to-run variance; no standard-error or individual-seed summary is provided for the transfer comparisons. This weakens the claim that DR 'affects all three algorithms in a similar way,' because the apparent similarity may partly reflect selection across highly variable runs. Please report all seeds or a summary statistic that does not depend on hindsight selection.
  3. [Abstract / §4.3] The claim that 'domain randomization affects all three algorithms in a similar way' is not operationalized. The body itself concludes more cautiously that the effect is 'task-, algorithm-, coverage-, and regime-dependent' and that broader coverage does not consistently improve transfer. The box plots in Figs. 3 and 4 show overlapping distributions with no statistical comparisons, effect sizes, or hypothesis tests. As stated, the abstract overstates the evidence. The authors should either provide a quantitative definition of 'similar way' (e.g., interaction effects in a regression, paired comparisons across tasks) or soften the claim to match the descriptive evidence.
minor comments (4)
  1. [Abstract] Typo: 'an insufficient criteria' should be 'an insufficient criterion.' Also, the abstract's statement that 'the sample-inefficient PPO algorithm can produce a performant policy faster' should carry the configuration qualifier used in §4.1 ('under its commonly adopted massively parallel configuration relative to the SAC and TD-MPC2 default configurations considered in this study').
  2. [Appendix A] The term 'representative training window' is undefined. Even if the conversion method is retained, the authors must specify the window (e.g., which training steps, how many interactions, how the average was computed) and justify that it is representative for all three algorithms.
  3. [Fig. B.3 caption] The caption says 'log-scale' and 'for an extended period of time,' but the x-axis appears to be linear seconds ranging from 0 to 30000. This inconsistency should be corrected.
  4. [Related Work] Minor grammatical issue: 'Narendra et al. (2025) noted diminishing stability of TD-MPC2 when utilized with a higher-degree of parallelism' — 'utilized with' should likely be 'utilized at' or 'under.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's insights are empirical comparisons based on measured returns and logged interaction counts; the only self-citation is a plotting convention, and the wall-clock conversion is a validity concern, not a circularity.

full rationale

Both central claims are empirical, not derived from assumptions that contain the conclusions. Returns under fixed evaluation contexts, training-step counts, and domain-randomization conditions are measured and compared directly; no parameter is fitted and then relabeled as an independent prediction. The only author-overlapping citation that appears in the argument, Tanaka & Mahmood (2026), is used in Section 4.1 solely to justify plotting individual runs instead of pointwise aggregates; it does not determine any ranking or numerical result. The Appendix A wall-clock construction ('we averaged the elapsed time per environment interaction over a representative training window and used it to convert training steps into wall-clock time') is a measurement/aggregation assumption whose bias could affect the time-based ranking, but it is not circular: wall-clock time is not defined in terms of the paper's conclusions, and the conversion is an arithmetic scaling of logged interaction counts rather than a fitted quantity being presented as an independent finding. The paper also discloses the main configuration confound in its limitations. Therefore, no load-bearing step reduces to its own inputs; the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claims rest on hand-chosen experimental scaffolding rather than derivations: the four DR coverage ranges and two evaluation regimes (Table A.1) define what "coverage" and "in/out-of-support" mean; the 15-environment fixed evaluation sets delimit the statistical evidence; the wall-clock axis depends on an unspecified conversion window; and the best-of-seed convention shapes the transfer summaries. No new entities are introduced; the CMDP context variable is standard modeling vocabulary.

free parameters (3)
  • Domain-randomization coverage ranges (narrow/moderate/broad/extensive) = Mass [0.85,1.15]/[0.80,1.20]/[0.70,1.30]/[0.50,1.80]; Damping similar; Table A.1
    Hand-chosen multiplicative ranges define what "coverage" means; all DR conclusions in §4.3 are relative to these arbitrary boundaries.
  • Evaluation-regime ranges (Nominal-Centered, Shifted) = Mass nominal [0.85,1.15], shifted [0.70,0.85]∪[1.15,1.30]; etc. (Table A.1)
    Hand-chosen ranges determine the Table 1 labels (in-support vs out-of-support), so the transfer taxonomy is an artifact of these boundaries.
  • Wall-clock conversion window
    Appendix A averages per-interaction elapsed time over an unspecified "representative training window" to convert steps to wall-clock time; the resulting time axis depends on this unstated choice.
assumptions (5)
  • domain assumption CMDP formulation: transfer as generalization across context distributions p_train vs p_eval (Section 3.2)
    Standard domain-randomization modeling; conclusions about transfer are framed entirely within this parameterization (multiplicative physical-parameter scale factors).
  • domain assumption 15 fixed evaluation environments per regime characterize zero-shot transfer
    Appendix A: "we construct fixed test sets by sampling 15 environments for each of the two evaluation regimes." All transfer box plots summarize only 15 contexts.
  • domain assumption CleanRL SAC and official TD-MPC2 "standard configurations" fairly represent off-policy and model-based paradigms
    Appendix A and Conclusions: SAC/TD-MPC2 run single-environment defaults while PPO runs 2,048 Brax environments; the comparison is configuration-level, a limitation the authors disclose.
  • ad hoc to paper Per-interaction elapsed time averaged over a representative window estimates wall-clock time
    Appendix A; unexamined measurement premise underpinning Insight 1; window is unspecified and no direct timing validation is provided.
  • ad hoc to paper Best-of-seed selection is an acceptable summary for zero-shot transfer performance
    Appendix A: "we select the best-performing run among the training seeds." This suppresses across-seed variance that the paper elsewhere (Tanaka & Mahmood 2026) argues is important.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking the Suitability of Reinforcement Learning Algorithms Under Practical Transfer Constraints." pith.science (2026). https://pith.science/paper/EJBZHNIF

@misc{pith2026260717326,
  author       = {Pith},
  title        = {Pith review of: Rethinking the Suitability of Reinforcement Learning Algorithms Under Practical Transfer Constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EJBZHNIF}},
  note         = {Machine review of arXiv:2607.17326}
}
read the original abstract

Transfer-oriented reinforcement learning requires evaluating algorithms along dimensions that go beyond standard sample efficiency. We focus on two dimensions: practical efficiency, which asks whether conclusions about algorithm suitability change under wall-clock rather than interaction-based budgets, and robustness under dynamics mismatch, which asks how different learning paradigms respond to variability in the training distribution induced by domain randomization. We provide two insights to reinforcement-learning practitioners. First, comparing the sample efficiency of different algorithms is often an insufficient criterion in transfer-oriented settings. The wall-clock time required to train a decent policy is an important consideration for practitioners, and we find that the sample-inefficient PPO algorithm can produce a performant policy faster than relatively more sample-efficient algorithms such as SAC and TD-MPC2, validating the common understanding of massively parallel training paradigms. Second, domain randomization can help different kinds of algorithms learn robust policies. In particular, although PPO, SAC, and TD-MPC2 represent different RL paradigms - on-policy, off-policy, and model-based learning and planning, respectively - we find that domain randomization affects all three algorithms in a similar way. To the best of our knowledge, this is the first controlled comparison of the effect of domain-randomization coverage on PPO, SAC, and TD-MPC2 under the same transfer protocol. Taken together, these two insights highlight the importance of evaluating RL algorithms not only by sample efficiency, but also by practical considerations such as training time and the algorithms' ability to produce usable policies.

Figures

Figures reproduced from arXiv: 2607.17326 by the authors.

Figure 1
Figure 1. Wall-clock learning curves. Performance is plotted against wall-clock training time on a logarithmic scale. Light curves show all six seeds. The dark curve shows the best-performing run, obtained by ranking runs according to the sum of returns over the full recorded learning curve. transfer-oriented workflows, DR is commonly used to introduce controlled variability in environ￾ment dynamics during training, with the … view at source ↗
Figure 2
Figure 2. Interaction-based learning curves. Performance is plotted against the environment in￾teractions on a logarithmic scale, for the same training runs shown in [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Zero-shot transfer under Nominal-Centered evaluation. Each box summarizes perfor￾mance across 15 fixed evaluation environments, with the return in each environment averaged over 10 episodes. The center line denotes the median, the box spans the first and third quartiles, and the whiskers extend to the most extreme observations within 1.5 times the interquartile range. Points beyond the whiskers are shown individuall… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Zero-shot transfer under Shifted evaluation. Each box summarizes performance across 15 fixed evaluation environments, with the return in each environment averaged over 10 episodes. These observations show that covering the evaluation distribution is not sufficient to e…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 6 linked inside Pith

  1. [1]

    and Barto, Andrew G

    Sutton, Richard S. and Barto, Andrew G. , title =

  2. [2]

    R. S. Sutton and D. McAllester and S. Singh and Y. Mansour , title =. Advances in Neural Information Processing Systems , volume =

  3. [3]

    R. J. Williams , title =. Machine Learning , volume =

  4. [4]

    Robotics: Science and Systems , year =

    Sadeghi, Fereshteh and Levine, Sergey , title =. Robotics: Science and Systems , year =

  5. [5]

    IEEE International Conference on Robotics and Automation (ICRA) , pages =

    Peng, Xue Bin and Andrychowicz, Marcin and Zaremba, Wojciech and Abbeel, Pieter , title =. IEEE International Conference on Robotics and Automation (ICRA) , pages =

  6. [6]

    arXiv preprint arXiv:1707.06347 , year =

    John Schulman and Filip Wolski and Prafulla Dhariwal and Alec Radford and Oleg Klimov , title =. arXiv preprint arXiv:1707.06347 , year =

    arXiv

  7. [7]

    Haarnoja and A

    T. Haarnoja and A. Zhou and P. Abbeel and S. Levine , title =. International Conference on Machine Learning , pages =

  8. [8]

    International Conference on Learning Representations , year =

    Nicklas Hansen and Hao Su and Xiaolong Wang , title =. International Conference on Learning Representations , year =

Show all 39 references
  1. [9]

    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages =

    Josh Tobin and Rachel Fong and Alex Ray and Jonas Schneider and Wojciech Zaremba and Pieter Abbeel , title =. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages =

  2. [10]

    Robotics: Science and Systems , year =

    Tan, Jie and Zhang, Tingnan and Coumans, Erwin and Iscen, Atil and Bai, Yunfei and Hafner, Danijar and Bohez, Steven and Vanhoucke, Vincent , title =. Robotics: Science and Systems , year =

  3. [11]

    Software Impacts , volume =

    Saran Tunyasuvunakool and Alistair Muldal and Yotam Doron and Siqi Liu and Steven Bohez and Josh Merel and Tom Erez and Timothy Lillicrap and Nicolas Heess and Yuval Tassa , title =. Software Impacts , volume =. 2020 , doi =

    2020

  4. [12]

    2025 , month = feb, url =

    Raffin, Antonin , title =. 2025 , month = feb, url =

    2025

  5. [13]

    arXiv preprint arXiv:2512.01996 , year =

    Seo, Younggyo and Sferrazza, Carmelo and Chen, Juyue and Shi, Guanya and Duan, Rocky and Abbeel, Pieter , title =. arXiv preprint arXiv:2512.01996 , year =

  6. [14]

    arXiv preprint arXiv:1910.07113 , year =

    Akkaya, Ilge and Andrychowicz, Marcin and Chociej, Maciek and Litwin, Mateusz and McGrew, Bob and Petron, Arthur and Paino, Alex and Plappert, Matthias and Powell, Glenn and Ribas, Raphael and others , title =. arXiv preprint arXiv:1910.07113 , year =

    1910 arXiv

  7. [15]

    Conference on Robot Learning , pages =

    Mehta, Bhairav and Diaz, Manfred and Golemo, Florian and Pal, Christopher J and Paull, Liam , title =. Conference on Robot Learning , pages =

  8. [16]

    International Conference on Learning Representations , year =

    Chen, Xiaoyu and Hu, Jiachen and Jin, Chi and Li, Lihong and Wang, Liwei , title =. International Conference on Learning Representations , year =

  9. [17]

    International Conference on Machine Learning , pages =

    Hansen, Nicklas and Su, Hao and Wang, Xiaolong , title =. International Conference on Machine Learning , pages =

  10. [18]

    Conference on Robot Learning , pages =

    Rudin, Nikita and Hoeller, David and Reist, Philipp and Hutter, Marco , title =. Conference on Robot Learning , pages =

  11. [19]

    Frontiers in Robotics and AI , volume =

    Rizzardo, Carlo and Chen, Fei and Caldwell, Darwin , title =. Frontiers in Robotics and AI , volume =

  12. [20]

    International Conference on Learning Representations , year =

    Yin, Patrick and Westenbroek, Tyler and Cheng, Ching-An and Kolobov, Andrey and Gupta, Abhishek , title =. International Conference on Learning Representations , year =

  13. [21]

    International Conference on Machine Learning , pages =

    Li, Zechu and Chen, Tao and Hong, Zhang-Wei and Ajay, Anurag and Agrawal, Pulkit , title =. International Conference on Machine Learning , pages =

  14. [22]

    arXiv preprint arXiv:2601.21363 , year =

    Huang, Weidong and Li, Zhehan and Liu, Hangxin and Hou, Biao and Su, Yao and Zhang, Jingwen , title =. arXiv preprint arXiv:2601.21363 , year =

  15. [23]

    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages =

    Narendra, Aditya and Makarov, Dmitry and Panov, Aleksandr , title =. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages =

  16. [24]

    Araújo , title =

    Shengyi Huang and Rousslan Fernand Julien Dossa and Chang Ye and Jeff Braga and Dipam Chakraborty and Kinal Mehta and João G.M. Araújo , title =. Journal of Machine Learning Research , volume =. 2022 , url =

    2022

  17. [25]

    Journal of Machine Learning Research , volume =

    Antonin Raffin and Ashley Hill and Adam Gleave and Anssi Kanervisto and Maximilian Ernestus and Noah Dormann , title =. Journal of Machine Learning Research , volume =. 2021 , url =

    2021

  18. [26]

    AAAI Conference on Artificial Intelligence , volume =

    Henderson, Peter and Islam, Riashat and Bachman, Philip and Pineau, Joelle and Precup, Doina and Meger, David , title =. AAAI Conference on Artificial Intelligence , volume =

  19. [27]

    arXiv preprint arXiv:1507.04296 , year =

    Arun Nair and Praveen Srinivasan and Sam Blackwell and Cagdas Alcicek and Rory Fearon and Alessandro De Maria and Vedavyas Panneershelvam and Mustafa Suleyman and Charles Beattie and Stig Petersen and Shane Legg and Volodymyr Mnih and Koray Kavukcuoglu and David Silver , title...

    arXiv

  20. [28]

    Asynchronous Methods for Deep Reinforcement Learning , booktitle =

    Volodymyr Mnih and Adri. Asynchronous Methods for Deep Reinforcement Learning , booktitle =

  21. [29]

    Nicolas Heess and Dhruva TB and Srinivasan Sriram and Jay Lemmon and Josh Merel and Greg Wayne and Yuval Tassa and Tom Erez and Ziyu Wang and S. M. Ali Eslami and Martin A. Riedmiller and David Silver , title =. arXiv preprint arXiv:1707.02286 , year =

    arXiv

  22. [30]

    Distributed Prioritized Experience Replay , booktitle =

    Dan Horgan and John Quan and David Budden and Gabriel Barth. Distributed Prioritized Experience Replay , booktitle =

  23. [31]

    Kahrs and Carmelo Sferrazza and Yuval Tassa and Pieter Abbeel , title =

    Kevin Zakka and Baruch Tabanpour and Qiayuan Liao and Mustafa Haiderbhai and Samuel Holt and Jing Yuan Luo and Arthur Allshire and Erik Frey and Koushil Sreenath and Lueder A. Kahrs and Carmelo Sferrazza and Yuval Tassa and Pieter Abbeel , title =. arXiv preprint arXiv:2502.08...

    arXiv

  24. [32]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto , title =

  25. [33]

    International Conference on Machine Learning , pages =

    Wan, Yi and Naik, Abhishek and Sutton, Richard S , title =. International Conference on Machine Learning , pages =

  26. [34]

    Reinforcement Learning Journal , year =

    Adamczyk, Jacob and Makarenko, Volodymyr and Tiomkin, Stas and Kulkarni, Rahul V , title =. Reinforcement Learning Journal , year =

  27. [35]

    Advances in Neural Information Processing Systems , volume =

    Vasan, Gautham and Elsayed, Mohamed and Azimi, Seyed Alireza and He, Jiamin and Shahriar, Fahim and Bellinger, Colin and White, Martha and Mahmood, Rupam , title =. Advances in Neural Information Processing Systems , volume =

  28. [36]

    Advances in Neural Information Processing Systems Datasets and Benchmarks Track , year =

    Freeman, C Daniel and Frey, Erik and Raichuk, Anton and Girgin, Sertan and Mordatch, Igor and Bachem, Olivier , title =. Advances in Neural Information Processing Systems Datasets and Benchmarks Track , year =

  29. [37]

    arXiv preprint arXiv:2606.06746 , year =

    Tanaka, Haruto and Mahmood, A Rupam , title =. arXiv preprint arXiv:2606.06746 , year =

    arXiv

  30. [38]

    IEEE International Conference on Robotics and Automation (ICRA) , pages =

    Handa, Ankur and Allshire, Arthur and Makoviychuk, Viktor and Petrenko, Aleksei and Singh, Ritvik and Liu, Jingzhou and Makoviichuk, Denys and Van Wyk, Karl and Zhurkevich, Alexander and Sundaralingam, Balakumar and others , title =. IEEE International Conference on Robotics a...

  31. [39]

    The International Journal of Robotics Research , volume =

    Andrychowicz, Marcin and Baker, Bowen and Chociej, Maciek and Jozefowicz, Rafal and McGrew, Bob and Pachocki, Jakub and Petron, Arthur and Plappert, Matthias and Powell, Glenn and Ray, Alex and others , title =. The International Journal of Robotics Research , volume =

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.