Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

Real-Time Reinforcement Learning for Dynamic Tasks with a Parallel Soft Robot

T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A soft robot learns to balance a puck in a single continuous hardware run, with no prior data, in as little as 15 minutes—and keeps working after half its actuators are buckled or cut.

desk verdict A credible single-shot RL hardware demo on a soft Stewart platform, but the damage-robustness headline needs the numbers behind 'nearly identical' before it can stand. read the letter →

arxiv 2509.19525 v1 pith:WLGFEMFO submitted 2025-09-23 cs.RO

James Avtges , Jake Ketchum , Millicent Schlafly , Helena Young , Taekyoung Kim , Allison Pinosky , Ryan L. Truby , Todd D. Murphey This is my paper · ORCID
classification cs.RO
keywords single-shotreinforcementlearningsoftroboticscurriculumhandedshearingauxeticStewartplatformdamageadaptationMaximumDiffusionRL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether reinforcement learning can master a dynamic control task on a soft robot entirely during one real-time hardware deployment, without simulation, resets, or prior data. The authors demonstrate that it can: a six-degree-of-freedom parallel soft robot learns to balance a sliding puck at the center and at arbitrary target points, with training times as short as 15 minutes. A curriculum that starts near a known equilibrium and expands outward makes learning reliable for off-center setpoints. The strongest demonstration is adaptation: when half of the soft actuators are buckled or physically cut mid-training, MaxDiff RL recovers and balances with performance nearly identical to the intact platform.

What carries the argument

The load-bearing element is the combination of two structures: (1) a curriculum-learning schedule that samples balancing setpoints within a radius that starts small around the platform center and expands during the first half of training, preventing the puck from becoming stuck in a corner and thereby keeping data informative; and (2) a rigid-body Stewart inverse-kinematics model, L = ||RP - B + T||, that converts desired roll and pitch into six strut lengths, reducing the RL action space from six motors to two commanded rotations. Maximum Diffusion RL supplies the exploration pressure that lets the policy adapt to buckled or broken actuators mid-episode.

What would settle it

Repeat the single-shot balancing protocol with the action space expanded to all six motor commands (no inverse-kinematics reduction). If learning no longer completes in 15 minutes or the buckled/breaking recovery disappears, then the geometric model reduction—not RL alone—is carrying the result. Alternatively, run the same protocol on a parallel soft platform with a different actuator type that fatigues faster; if the damage-adaptation result fails, the durability of HSAs is a hidden prerequisite.

Watch

Extended reading notes

Core claim

The central claim is that single-shot reinforcement learning—one continuous, non-episodic deployment with no resets—is a viable route to closed-loop control of soft robots, despite their nonlinearity, hysteresis, and changing dynamics. Using a deformable Stewart platform built from motorized handed shearing auxetic (HSA) struts, the authors show that model-based RL (NN-MPPI and Maximum Diffusion) learns center balancing in under 15 minutes and arbitrary-point balancing in about 2.75 hours, with Maximum Diffusion outperforming both the model-based NN-MPPI and the model-free SAC. In a single episode, MaxDiff recovers from having three of six actuators buckled or broken with bolt cutters, achie

Load-bearing premise

The fastest results depend on the rigid-body Stewart inverse kinematics with measured linear HSA extension rates preserving enough control authority once actuators buckle or break; if that approximation fails, the claimed damage robustness may only hold for this specific platform and task.

Editorial extensions

If this is right

  • Soft robots can be trained on hardware without simulation, closing the sim-to-real gap for dynamic tasks.
  • Damage tolerance can emerge within a single training episode: breaking or buckling actuators does not require resetting or reinitializing the policy.
  • Model-based RL (NN-MPPI and MaxDiff) learns balancing much faster than the model-free SAC, suggesting sample efficiency matters for soft actuators with limited lifespan.
  • The curriculum-over-setpoints procedure is necessary for arbitrary-point balancing; without it, the system gets stuck in absorbing states.
  • The inverse-kinematics dimension reduction is what makes 15-minute training feasible, at the cost of relying on an approximate rigid model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the results transfer, single-shot hardware RL could replace sim-to-real pipelines for other underactuated or compliant mechanisms, since the policy learns the plant's actual dynamics each time.
  • The curriculum idea generalizes beyond balancing: any task with an absorbing 'stuck' state and a known nearby stable condition could use the same expanding-neighborhood schedule.
  • The model-reduction step is the likely bottleneck for generality; a different platform geometry or a task needing all six degrees of freedom would remove the 15-minute guarantee, so the method's claims are tightly coupled to the Stewart inverse kinematics.
  • A natural testable extension is to let the platform learn without the model reduction (six-dimensional actions) to quantify how much of the speed is due to the geometry prior.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper reports on-hardware, single-shot reinforcement learning (RL) for a soft Stewart platform actuated by six motorized handed shearing auxetics (HSAs). The authors benchmark MaxDiff, NN-MPPI, and SAC on center and arbitrary-point balancing, introduce a curriculum over expanding setpoint neighborhoods for the arbitrary-point task, and claim that MaxDiff adapts to mid-episode buckling or breaking of half of the actuators with performance "nearly identical" to the intact platform. Training is entirely on hardware with no simulation, and center balancing is reported in under 15 minutes. The stated contributions are reliable single-shot learning, the curriculum procedure, and a demonstration of damage adaptation.

Significance. If the damage-robustness and curriculum claims are substantiated, this is a notable step for real-world RL on soft hardware: it shows that dynamic control policies can be learned in a single continuous hardware deployment without simulation or resets. The intact-condition benchmarking is credible, using named algorithms, five seeds per condition, and tabulated errors in Table III. However, the two headline claims—curriculum necessity and post-damage equivalence—are currently supported only qualitatively, with the damage claim resting on a figure without numerical or statistical backup. The paper would be substantially stronger if these gaps are filled with quantitative ablations and summary statistics.

major comments (2)
  1. [Section IV.C, Fig. 7] The central damage-robustness claim—"near-identical" and "indistinguishable" performance after buckling or breaking half the HSAs—is not supported by quantitative evidence. Table III provides means and standard deviations for intact center and arbitrary balancing, but no comparable summary is given for the buckled or broken conditions. Fig. 7 is cited, but without per-condition MSEs, standard deviations, effect sizes, or a statistical equivalence test, n=5 seeds cannot establish "nearly identical" performance. Also, because the perturbation occurs halfway through a continuous episode, final evaluation alone cannot distinguish active adaptation from pre-existing robustness of the already-learned policy. Report before/after-damage evaluations or learning curves around the switch, and provide a numerical table with a statistical comparison (e.g., two one-sided tests or confidence intervals)
  2. [Section IV.B, Algorithm 1] The curriculum is presented as necessary: the text states that without it the task is "impossible to accomplish consistently, or at all" and that the puck "frequently becomes stuck in a corner." This is a load-bearing claim for one of the three contributions, yet no ablation is shown. No experiment compares the proposed expanding-neighborhood curriculum against training with uniform sampling over the full platform or with setpoints fixed far from center, using the same algorithm, step budget, and seeds. Provide such a comparison, reporting success rates, fraction of training time spent with the puck stuck, and evaluation MSE, to substantiate that the curriculum is essential rather than merely helpful.
minor comments (4)
  1. [Section IV.B] Typo: "Model-free RL is generally less sample efficient than model-free approaches" should read "model-based approaches."
  2. [Section III.D] Reward weights (a=250, b=24, c=50) and the MaxDiff temperature annealing schedule are stated without justification or sensitivity analysis. At least a brief rationale or a reference to a sensitivity study would help the reader assess the robustness of the reported comparisons.
  3. [Section III.C] The claim of "no prior data" should be qualified. The geometric model L=||RP−B+T|| and the measured HSA extension rates in Table I are prior system knowledge, though not task-specific data. "No prior task-specific data" is more precise.
  4. [Section IV.C, Fig. 7] The experimental protocol for the damage experiments is underspecified in the text. State the total episode duration, the point at which the damage is introduced, and the evaluation protocol (number of trials, trial length, and metric) explicitly in Section IV.C rather than only in the figure caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: central results are measured on hardware and benchmarked against independent baselines.

full rationale

The paper's central claims—single-shot RL training in ~15 minutes, arbitrary-point balancing via curriculum learning, and adaptation to buckled/broken actuators—are empirical demonstrations supported by live hardware experiments and comparisons against NN-MPPI and SAC (Figs. 2–4, Table III). The inverse kinematics model (Section III.C) is an input assumption, not a quantity derived from the target results, and its parameters are measured directly from the actuators. The damage-robustness claim rests on Fig. 7, which is not reproduced with numerical summary statistics; this is a verifiability/reporting gap, not a definitional equivalence. MaxDiff is cited from the authors' prior work [30] for its exploration properties and single-shot generalization motivation, but the paper's outcome is not derived from that citation—it is measured and benchmarked. Self-citations in the related work ([11], [20], [30]) are contextual and not load-bearing. No equation or fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no ansatz is smuggled in via citation. Therefore the derivation chain is self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical or theoretical entities are introduced. MaxDiff and HSA actuators are prior published work; the platform is an assembly of existing components. The central free parameters are the hand-set reward weights, the curriculum schedule, and MaxDiff's exploration temperature.

free parameters (5)
  • Reward weights a, b, c = a=250, b=24, c=50
    Hand-chosen scalar weights in Eq. 2 balancing position, velocity, and action penalties; they define the learned behavior and are not learned or derived.
  • Curriculum parameters lambda0 and gamma = not reported in text
    Algorithm 1 defines the curriculum radius as min(step_count*gamma+lambda0,1)*R, but values of lambda0 and gamma are omitted; the expanding-neighborhood schedule is hand-tuned and load-bearing for arbitrary-point balancing.
  • Curriculum timing schedule = 900 steps/setpoint, 50,000-step curriculum length, setpoints updated every 60 s
    These schedule choices (Table II) determine whether the puck avoids corner-stuck states; no sensitivity analysis is reported.
  • MaxDiff temperature alpha and annealing schedule = alpha=0.1, decays after first half of training
    Alpha is a hand-set exploration parameter; the annealing details are not fully specified.
  • Action smoothing factor = 0.3 center balancing, 0.7 arbitrary balancing
    Weighted average with the previous action command; chosen by hand and differs between tasks, affecting the effective control frequency.
assumptions (5)
  • domain assumption Rigid Stewart platform inverse kinematics L=||RP-B+T|| maps desired roll/pitch to six strut lengths (Section III.C).
    The physical platform lacks ball joints and is made of compliant HSAs; the paper assumes this rigid model is a sufficient action parameterization so RL only needs to learn residuals.
  • domain assumption HSA extension is linear in motor rotation, with measured rates of 12.3 mm/rad for a fresh actuator and 13.8 mm/rad for a worn one (Table I).
    Used to convert computed strut lengths into motor commands; wear changes the rate and the paper does not model this nonlinearity.
  • domain assumption Apriltag tracking and Hough-circle puck detection provide accurate state information at about 60 Hz.
    The observation vector Eq. 1 depends on these measurements; no quantitative accuracy or calibration error analysis is given.
  • ad hoc to paper The hand-shaped reward in Eqs. 2-5 is a learnable, single-peaked signal for balancing.
    The reward is constructed for this task; no comparison to alternative reward designs is reported.
  • ad hoc to paper The curriculum over expanding setpoint neighborhoods prevents absorbing corner states.
    This is the paper's method contribution, but it is asserted rather than supported by a no-curriculum ablation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Real-Time Reinforcement Learning for Dynamic Tasks with a Parallel Soft Robot." pith.science (2026). https://pith.science/paper/WLGFEMFO

@misc{pith2026250919525,
  author       = {Pith},
  title        = {Pith review of: Real-Time Reinforcement Learning for Dynamic Tasks with a Parallel Soft Robot},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WLGFEMFO}},
  note         = {Machine review of arXiv:2509.19525}
}
read the original abstract

Closed-loop control remains an open challenge in soft robotics. The nonlinear responses of soft actuators under dynamic loading conditions limit the use of analytic models for soft robot control. Traditional methods of controlling soft robots underutilize their configuration spaces to avoid nonlinearity, hysteresis, large deformations, and the risk of actuator damage. Furthermore, episodic data-driven control approaches such as reinforcement learning (RL) are traditionally limited by sample efficiency and inconsistency across initializations. In this work, we demonstrate RL for reliably learning control policies for dynamic balancing tasks in real-time single-shot hardware deployments. We use a deformable Stewart platform constructed using parallel, 3D-printed soft actuators based on motorized handed shearing auxetic (HSA) structures. By introducing a curriculum learning approach based on expanding neighborhoods of a known equilibrium, we achieve reliable single-deployment balancing at arbitrary coordinates. In addition to benchmarking the performance of model-based and model-free methods, we demonstrate that in a single deployment, Maximum Diffusion RL is capable of learning dynamic balancing after half of the actuators are effectively disabled, by inducing buckling and by breaking actuators with bolt cutters. Training occurs with no prior data, in as fast as 15 minutes, with performance nearly identical to the fully-intact platform. Single-shot learning on hardware facilitates soft robotic systems reliably learning in the real world and will enable more diverse and capable soft robots.

Figures

Figures reproduced from arXiv: 2509.19525 by the authors.

Figure 1
Figure 1. Learning dynamic tasks in a parallel soft robot. The experimental platform: a 6-DoF parallel soft robot. A puck is placed on the platform; the learning objective is to balance the puck at arbitrary equilibria. Dynamic balancing is achieved despite the nonlinear behavior of the soft actuators, including buckling and breaking the HSAs. An additional challenge with learning a balancing task in single-shot episodes is t… view at source ↗
Figure 2
Figure 2. Evaluation trajectories for center balancing. We evaluate five seeds for each model with the puck starting from each corner of the platform with no initial tilt. We display representative evaluation trajectories, with each color representing a different seed. NN-MPPI and MaxDiff consistently balance at the center, while certain seeds of SAC fail to achieve the task entirely. Without explicitly supplying path-depende… view at source ↗
Figure 3
Figure 3. Single-shot center balancing. We report average training and evaluation performance for each algorithm across five seeds. a: Average position rewards during training. We trained SAC for 5x the duration in order to achieve the task. b: Average position rewards during evaluations. c: The average arc length of each evaluation trajectory—circuitous or oscillatory paths result in a high arc length. 1) Model-Predictive Pa… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Balancing at arbitrary points on the platform. The scale for mean-squared-error (MSE) is logarithmic. MaxDiff has an average MSE that is 38% better than NN-MPPI and 61% better than SAC. We also found that MaxDiff was more consistent, with an MSE std. deviation 40% smal…
Figure 5
Figure 5. Figure 5: While the platform is still formally controllable due [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 5
Figure 5. Figure 5: Learning with buckled actuators. Buckling of HSAs occurs when a subset of the actuators are extended a significant range past their neighbors. When buckled, HSAs have reduced stiffness and approach an effective singularity—further rotation has little to no effect on th…
Figure 6
Figure 6. Figure 6: Learning with broken actuators. Handed Shearing Auxetics are able to suffer significant breakages of their living hinges and still retain partial functionality. Despite cutting the major helices of half of the HSAs during training, single-shot RL is able to achieve nea…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Damage Adaptation in Seconds for Architected Materials

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    LEAP enables real-time proprioceptive adaptation to unseen damage in a 6DoF soft wrist using HSA actuators by combining latent damage representations with a robust ensemble method, with conditions identified for linea...

Reference graph

Works this paper leans on

51 extracted references · 2 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Design, fabrication and control of soft robots,

    D. Rus and M. T. Tolley, “Design, fabrication and control of soft robots,”Nature, vol. 521, no. 7553, pp. 467–475, 2015

    2015

  2. [2]

    Soft robotics: Review of fluid-driven intrinsically soft devices; manufacturing, sensing, control, and appli- cations in human-robot interaction,

    P. Polygerinos, N. Correll,et al., “Soft robotics: Review of fluid-driven intrinsically soft devices; manufacturing, sensing, control, and appli- cations in human-robot interaction,”Advanced Engineering Materials, vol. 19, no. 12, p. 1700016, 2017

    2017

  3. [3]

    Modeling and control of soft robots using the koopman operator and model predictive control,

    D. Bruder, B. Gillespie,et al., “Modeling and control of soft robots using the koopman operator and model predictive control,”Robotics: Science and Systems (RSS), 2019

    2019

  4. [4]

    Model-based control of soft robots: A survey of the state of the art and open challenges,

    C. Della Santina, C. Duriez,et al., “Model-based control of soft robots: A survey of the state of the art and open challenges,”IEEE Control Systems Magazine, vol. 43, no. 3, pp. 30–65, 2023

    2023

  5. [5]

    Model-based reinforcement learning for closed-loop dynamic control of soft robotic manipulators,

    T. G. Thuruthel, E. Falotico,et al., “Model-based reinforcement learning for closed-loop dynamic control of soft robotic manipulators,” IEEE Transactions on Robotics, vol. 35, pp. 127–134, 2 2019

    2019

  6. [6]

    Review of machine learning methods in soft robotics,

    D. Kim, S. H. Kim,et al., “Review of machine learning methods in soft robotics,”PLoS ONE, vol. 16, 2 2021

    2021

  7. [7]

    Reinforcement learning enables real- time planning and control of agile maneuvers for soft robot arms,

    R. Jitosho, T. G. W. Lum,et al., “Reinforcement learning enables real- time planning and control of agile maneuvers for soft robot arms,” Conference on Robot Learning (CoRL), pp. 1131–1153, 2023

    2023

  8. [8]

    Closed-loop dynamic control of a soft manipulator using deep reinforcement learning,

    A. Centurelli, L. Arleo,et al., “Closed-loop dynamic control of a soft manipulator using deep reinforcement learning,”IEEE Robotics and Automation Letters, vol. 7, pp. 4741–4748, 4 2022

    2022

Show all 51 references
  1. [9]

    Model-free reinforcement learning with ensemble for a soft continuum robot arm,

    R. Morimoto, S. Nishikawa,et al., “Model-free reinforcement learning with ensemble for a soft continuum robot arm,”IEEE International Conference on Soft Robotics (RoboSoft), pp. 141–148, 4 2021

    2021

  2. [10]

    Pushing with soft robotic arms via deep reinforcement learning,

    C. Alessi, D. Bianchi,et al., “Pushing with soft robotic arms via deep reinforcement learning,”Advanced Intelligent Systems, vol. 6, no. 8, p. 2300899, 2024

    2024

  3. [11]

    Automated gait generation for walking, soft robotic quadrupeds,

    J. Ketchum, S. Schiffer,et al., “Automated gait generation for walking, soft robotic quadrupeds,”IEEE International Conference on Intelligent Robots and Systems (IROS), pp. 10245–10251, 2023

    2023

  4. [12]

    A platform with six degrees of freedom,

    D. Stewart, “A platform with six degrees of freedom,”Proceedings of the Institution of Mechanical Engineers, vol. 180, no. 1, pp. 371–386, 1965

    1965

  5. [13]

    Handedness in shearing auxetics creates rigid and compliant structures,

    J. I. Lipton, R. MacCurdy,et al., “Handedness in shearing auxetics creates rigid and compliant structures,”Science, vol. 360, no. 6389, pp. 632–635, 2018

    2018

  6. [14]

    A flexible, architected soft robotic actuator for motorized extensional motion,

    T. Kim, P. Kaarthik,et al., “A flexible, architected soft robotic actuator for motorized extensional motion,”Advanced Intelligent Systems, p. 2300866, 2024

    2024

  7. [15]

    Challenges of real-world reinforcement learning,

    G. Dulac-Arnold, D. Mankowitz,et al., “Challenges of real-world reinforcement learning,”Mach Learn, no. 110, pp. 2419––2468, 2021

    2021

  8. [16]

    How to train your robot with deep reinforce- ment learning: lessons we have learned,

    J. Ibarz, J. Tan,et al., “How to train your robot with deep reinforce- ment learning: lessons we have learned,”The International Journal of Robotics Research, vol. 40, no. 4-5, pp. 698–721, 2021

    2021

  9. [17]

    Hardware-in-the-loop soft robotic testing framework using an actor-critic deep reinforcement learning algorithm,

    J. Marquez, C. Sullivan,et al., “Hardware-in-the-loop soft robotic testing framework using an actor-critic deep reinforcement learning algorithm,”IEEE Robotics and Automation Letters, vol. 8, no. 9, pp. 6076–6082, 2023

    2023

  10. [18]

    Smooth exploration for robotic reinforce- ment learning,

    A. Raffin, J. Kober,et al., “Smooth exploration for robotic reinforce- ment learning,”Conference on Robot Learning (CoRL), pp. 1634– 1644, 2022

    2022

  11. [19]

    Daydreamer: World models for physical robot learning,

    P. Wu, A. Escontrela,et al., “Daydreamer: World models for physical robot learning,”Conference on Robot Learning (CoRL), pp. 2226– 2240, 2023

    2023

  12. [20]

    Physical state exploration for reinforcement learning from scratch,

    A. Pinosky, T. A. Berrueta,et al., “Physical state exploration for reinforcement learning from scratch,”IEEE International Conference on Automation Science and Engineering (CASE), 2025

    2025

  13. [21]

    Comparison of modern control methods for soft robots,

    M. Grube, J. C. Wieck,et al., “Comparison of modern control methods for soft robots,”Sensors, vol. 22, no. 23, 2022

    2022

  14. [22]

    Simple, low-hysteresis, foldable, fabric pneumatic artificial muscle,

    N. D. Naclerio and E. W. Hawkes, “Simple, low-hysteresis, foldable, fabric pneumatic artificial muscle,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3406–3413, 2020

    2020

  15. [23]

    Dynamic task space control enables soft manipulators to perform real-world tasks,

    O. Fischer, Y . Toshimitsu,et al., “Dynamic task space control enables soft manipulators to perform real-world tasks,”Advanced Intelligent Systems, vol. 5, no. 1, p. 2200024, 2023

    2023

  16. [24]

    Motorized, untethered soft robots via 3d printed auxetics,

    P. Kaarthik, F. L. Sanchez,et al., “Motorized, untethered soft robots via 3d printed auxetics,”Soft Matter, vol. 18, no. 43, pp. 8229–8237, 2022

    2022

  17. [25]

    A recipe for electrically-driven soft robots via 3d printed handed shearing auxetics,

    R. L. Truby, L. Chin,et al., “A recipe for electrically-driven soft robots via 3d printed handed shearing auxetics,”IEEE Robotics and Automation Letters, vol. 6, pp. 795–802, 4 2021

    2021

  18. [26]

    Electrically controllable materials for soft, bioinspired machines,

    A. L. Evenchik, A. Q. Kane,et al., “Electrically controllable materials for soft, bioinspired machines,”Annual Review of Materials Research, vol. 53, no. V olume 53, 2023, pp. 225–251, 2023

    2023

  19. [27]

    Vision-based sensing for electrically- driven soft actuators,

    A. Zhang, R. L. Truby,et al., “Vision-based sensing for electrically- driven soft actuators,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11509–11516, 2022

    2022

  20. [28]

    Fluidic innervation sensorizes structures from a single build material,

    R. L. Truby, L. Chin,et al., “Fluidic innervation sensorizes structures from a single build material,”Science Advances, vol. 8, no. 31, p. eabq4385, 2022

    2022

  21. [29]

    One-shot reinforcement learning for robot navigation with interactive replay,

    J. Bruce, N. S ¨underhauf,et al., “One-shot reinforcement learning for robot navigation with interactive replay,”ArXiv, vol. abs/1711.10137, 2017

    2017 arXiv

  22. [30]

    Maximum diffusion reinforcement learning,

    T. A. Berrueta, A. Pinosky,et al., “Maximum diffusion reinforcement learning,”Nature Machine Intelligence, vol. 6, pp. 504–514, 5 2024

    2024

  23. [31]

    Reset-free guided policy search: Efficient deep reinforcement learning with stochastic initial states,

    W. Montgomery, A. Ajay,et al., “Reset-free guided policy search: Efficient deep reinforcement learning with stochastic initial states,” IEEE International Conference on Robotics and Automation (ICRA), pp. 3373–3380, 2017

    2017

  24. [32]

    Single episode policy transfer in rein- forcement learning,

    J. Yang, B. Petersen,et al., “Single episode policy transfer in rein- forcement learning,”ArXiv, vol. abs/1910.07719, 2019

    1910 arXiv

  25. [33]

    Grow your limits: Continuous improvement with real-world rl for robotic locomotion,

    L. Smith, Y . Cao,et al., “Grow your limits: Continuous improvement with real-world rl for robotic locomotion,”IEEE International Con- ference on Robotics and Automation (ICRA), pp. 10829–10836, 2024

    2024

  26. [34]

    You only live once: Single-life reinforce- ment learning,

    A. Chen, A. Sharma,et al., “You only live once: Single-life reinforce- ment learning,”Advances in Neural Information Processing Systems (NeurIPS), vol. 35, pp. 14784–14797, 2022

    2022

  27. [35]

    Leave no trace: Learning to reset for safe and autonomous reinforcement learning,

    B. Eysenbach, S. Gu,et al., “Leave no trace: Learning to reset for safe and autonomous reinforcement learning,”ArXiv, 2017

    2017

  28. [36]

    Continual reinforcement learning in 3d non-stationary environments,

    V . Lomonaco, K. Desai,et al., “Continual reinforcement learning in 3d non-stationary environments,”IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 248–249, 2020

    2020

  29. [37]

    Automating reinforcement learning with example-based resets,

    J. Kim, J. Hyeon Park,et al., “Automating reinforcement learning with example-based resets,”IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6606–6613, 2022

    2022

  30. [38]

    Don’t start from scratch: Leveraging prior data to automate robotic reinforcement learning,

    H. R. Walke, J. H. Yang,et al., “Don’t start from scratch: Leveraging prior data to automate robotic reinforcement learning,”Conference on Robot Learning (CoRL), 2022

    2022

  31. [39]

    Reset-free reinforcement learning via multi- task learning: Learning dexterous manipulation behaviors without human intervention,

    A. Gupta, J. Yu,et al., “Reset-free reinforcement learning via multi- task learning: Learning dexterous manipulation behaviors without human intervention,”IEEE International Conference on Robotics and Automation (ICRA), pp. 6664–6671, 2021

    2021

  32. [40]

    Self-improving robots: End-to-end autonomous visuomotor reinforcement learning,

    A. Sharma, A. M. Ahmed,et al., “Self-improving robots: End-to-end autonomous visuomotor reinforcement learning,”Conference on Robot Learning (CoRL), 2023

    2023

  33. [41]

    Rtmba: A real-time model-based rein- forcement learning architecture for robot control,

    T. Hester, M. Quinlan,et al., “Rtmba: A real-time model-based rein- forcement learning architecture for robot control,”IEEE International Conference on Robotics and Automation (ICRA), pp. 85–90, 2012

    2012

  34. [42]

    Curriculum learning,

    Y . Bengio, J. Louradour,et al., “Curriculum learning,” inProceedings of the 26th annual international conference on machine learning, pp. 41–48, 2009

    2009

  35. [43]

    Curriculum learning for reinforcement learning domains: A framework and survey,

    S. Narvekar, B. Peng,et al., “Curriculum learning for reinforcement learning domains: A framework and survey,”Journal of Machine Learning Research, vol. 21, no. 181, pp. 1–50, 2020

    2020

  36. [44]

    A survey on curriculum learning,

    X. Wang, Y . Chen,et al., “A survey on curriculum learning,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 9, pp. 4555–4576, 2021

    2021

  37. [45]

    Training and tracking in robotics.,

    O. G. Selfridge, R. S. Sutton,et al., “Training and tracking in robotics.,” inInternational Joint Conference on Artificial Intelligence (IJCAI), pp. 670–672, 1985

    1985

  38. [46]

    Neural network learning control of robot manipulators using gradually increasing task difficulty,

    T. D. Sanger, “Neural network learning control of robot manipulators using gradually increasing task difficulty,”IEEE Transactions on Robotics and Automation, vol. 10, no. 3, pp. 323–333, 1994

    1994

  39. [47]

    Stewart py,

    Y . Cheah, “Stewart py,” https://github.com/Yeok-c/Stewart Py. 2022

    2022

  40. [48]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou,et al., “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,”Interna- tional Conference on Machine Learning, pp. 1861–1870, 2018

    2018

  41. [49]

    Information theoretic mpc for model-based reinforcement learning,

    G. Williams, N. Wagener,et al., “Information theoretic mpc for model-based reinforcement learning,”IEEE International Conference on Robotics and automation (ICRA), pp. 1714–1721, 2017

    2017

  42. [50]

    Aggressive driving with model predic- tive path integral control,

    G. Williams, P. Drews,et al., “Aggressive driving with model predic- tive path integral control,”IEEE International Conference on Robotics and Automation (ICRA), pp. 1433–1440, 2016

    2016

  43. [51]

    Model-based reinforcement learning: A survey,

    T. M. Moerland, J. Broekens,et al., “Model-based reinforcement learning: A survey,”Foundations and Trends in Machine Learning, vol. 16, no. 1, pp. 1–118, 2023

    2023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.