Pith. sign in

REVIEW 4 major objections 6 minor 24 references

Hysteresis-Aware Neural Network Modeling and Whole-Body Reinforcement Learning Control of Soft Robots

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hysteresis-aware neural modeling plus whole-body PPO achieves sub-millimeter tracking on a real soft robot.

desk verdict A useful engineering pipeline with a simple hysteresis feature and real-world demos, undermined by an internally inconsistent headline number and an under-tested quasi-static modeling assumption. read the letter →

arxiv 2504.13582 v2 pith:OPOKN3OQ submitted 2025-04-18 cs.RO cs.LG

Zongyuan Chen , Yan Xia , Jiayuan Liu , Jijia Liu , Wenhao Tang , Jiayu Chen , Feng Gao , Longfei Ma
show 5 more authors
Hongen Liao Yu Wang Chao Yu Boyu Zhang Fei Xing
This is my paper · ORCID
classification cs.ROcs.LG
keywords softroboticshysteresismodelingwhole-bodyneuralnetworkdynamicsreinforcementlearningcontrolPPOtrajectorytrackingsurgicallaserablation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a soft surgical robot can be modeled and controlled without analytical material models by making hysteresis a first-class input to a neural network. It adds the direction of pressure change to each chamber's pressure value, trains a multilayer perceptron on whole-body key-point coordinates, and uses that model as the environment for PPO policy training. The claimed payoff is an 84.95% reduction in prediction MSE compared with pressure-only inputs and real-world trajectory tracking errors between 0.126 and 0.250 mm. If these numbers hold, the approach offers a practical data-driven pipeline for whole-body soft-robot control in tasks such as surgical laser ablation.

What carries the argument

The central object is the Hysteresis-aware Whole-Body Neural Network (HAW-NN): an MLP whose six-dimensional input is per-chamber pressure plus a signed direction flag, and whose output is the 3D coordinates of n key feature points along the body, with the continuous shape reconstructed by B-spline fitting. The direction flag is the mechanism for hysteresis, because it distinguishes pressurization from depressurization paths that reach the same pressure. The model is trained with a motion-range-weighted MSE that prevents the fixed end's tiny motions from being swamped by the free end's large deflections. It then serves as the deterministic transition function $s_{t+1} = T_{nn}(s_t, a_t)$ inside a 64-environment parallel PPO simulator, letting the policy learn whole-body control without a physical robot in the loop.

What would settle it

Run the trained policy on a pressure trajectory that alternates direction rapidly or varies sinusoidally at 0.5-2 Hz while recording motion-capture positions; if the hysteresis-aware network's predictions diverge from measured positions by more than the reported 0.126-0.250 mm, the quasi-static direction-sign model does not cover the closed-loop regime. A simpler check is to hold pressure constant after a fast ramp and look for creep; visible drift over seconds would violate the training assumption.

Watch

Extended reading notes

Core claim

The paper's central claim is that the hysteresis of a soft pneumatic robot can be captured in a static neural network simply by including the direction of pressure change (rising, +1, or falling, -1) alongside the current pressure values in each of the three chambers. Trained on 13,824 quasi-static samples of key feature points along the body, this six-input model reaches a test MSE of 0.0693, an 84.95% reduction from the 0.3947 MSE of the pressure-only baseline. The same network is used as the transition function in a 64-environment PPO simulator, and the resulting policy, when deployed on the physical robot, tracks a circular trajectory with 0.250 mm mean error and a square trajectory with 0.126 mm mean error. A whole-body laser-pointing variant of the task is demonstrated on a phantom liver ablation with two coordinated robots.

Load-bearing premise

The load-bearing premise is that a static map from pressure values plus a binary direction sign, learned from data taken after a 3-second stabilization delay, predicts the robot's shape well enough while the controller is running, so that effects like creep, pressure lag, and vibration are negligible at tracking speed.

Editorial extensions

If this is right

  • A whole-body state representation makes tasks that depend on global body shape, such as aiming a laser mounted in the robot's central channel, trainable in the same framework.
  • Because the model is trained from measured key-point data, it applies to soft robots with nonuniform materials or attached tools, where PCC and Cosserat assumptions break down.
  • Training the policy in a parallel neural-network simulator avoids collecting millions of real-world interactions while still transferring to hardware.
  • The reported real-world tracking errors of 0.250 mm (circle) and 0.126 mm (square) indicate the sim-to-real gap is small for the tested trajectories.
  • Coordinated multi-robot tasks, such as endoscope viewing combined with laser ablation, can be executed with the same learned-control pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would replace the binary direction flag with a short pressure-history window or pressure rate; if performance holds, the flag was already capturing the dominant memory effect, and if it improves, rate dependence matters.
  • The real-world evaluation covers two trajectories and end-effector error; body-level errors at the internal key points on untrained trajectories would test the whole-body claim more strictly.
  • The same direction-sign input could be transferred to other actuators with path-dependent behavior, such as tendon-driven or shape-memory-alloy soft robots, at near-zero modeling cost.
  • The 3-second settle time in data collection suggests the model is a quasi-static map; a testable extension is active data collection during motion to capture viscoelastic transients.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper designs a three-chamber, fiber-reinforced pneumatic soft robot for laparoscopic surgery and proposes a hysteresis-aware whole-body neural network (HAW-NN/HWB-NN) that predicts the coordinates of key feature points along the robot body from current chamber pressures plus a binary direction-of-change input. This network is used as the transition model in a parallel PPO training environment, and the resulting policy is deployed on the real robot for circle and square trajectory tracking. The manuscript also reports a two-robot phantom liver ablation demonstration. The headline results are an 84.95% reduction in prediction MSE for the 6D direction-augmented model over a 3D pressure-only model, and real-world tracking errors of 0.126-0.250 mm.

Significance. If the results are confirmed, the paper offers a practically relevant data-driven modeling and whole-body RL control pipeline for soft robots, with a real-world deployment benchmark and a surgical phantom demonstration. The strengths are a head-to-head comparison across six architectures on the same test set, a real-world deployment that provides an external check on the simulator, and a whole-body key-point representation that is useful for tasks beyond endpoint control. The central risks are that the learned transition model is static and quasi-static, the real-world evaluation lacks repeated trials and an appropriate control ablation, and the headline percentage reduction is not consistent with the numbers in the text. These issues are local and correctable, so I view the manuscript as requiring major revision rather than rejection.

major comments (4)
  1. [Abstract and Section V-A] The reported 84.95% MSE reduction is not supported by the numbers given in Section V-A. With the 3D pressure-only model reaching a minimum MSE of 0.3947 and the 6D direction-augmented model reaching 0.0693, the reduction is 1 - 0.0693/0.3947 = 0.824, i.e., about 82.4%. Please correct the abstract and text, or state explicitly which metric and formula produce 84.95%.
  2. [Sections III-B and IV-A] The transition function in Eq. (9), st+1 = Tnn(st, at), is a static feedforward MLP trained on quasi-static samples: each data point follows a 3-second stabilization delay and then 1 second of averaged recording, and the input is only current pressures plus a binary direction sign. The model contains no pressure-rate, history, velocity, or time dependence, and no control-loop frequency is reported. At the control rate, rate-dependent viscoelasticity and transients are therefore unmodeled; the paper's conclusion explicitly lists LSTM as future work, acknowledging this gap. This is load-bearing for the sim-to-real claim because the simulated tracking errors in Table II are produced by the same network that forms the RL environment, not by an independent dynamic check. Please provide evidence of model accuracy at the control sampling rate, or augment the model with rate/temporal information.
  3. [Section V-C and Table II] The real-world tracking results are reported as single average errors per trajectory, with no number of trials, no standard deviation or error bars, and no statistical comparison. The motion capture system is described only as 'millimeter-level', which is comparable to the claimed sub-millimeter errors. Please report trial counts, variances, and the measurement resolution/accuracy, and state whether the reported values are per-point means over the trajectory.
  4. [Section V-C] There is no control ablation using the pressure-only model. The modeling comparison in Fig. 5 shows that the 6D input achieves lower test MSE, but the deployed policy evaluation is never compared against a policy trained in the pressure-only environment. Without such an ablation, the trajectory-tracking performance cannot be attributed to hysteresis-awareness. Please add a pressure-only control baseline in simulation and, ideally, on the real robot.
minor comments (6)
  1. [Throughout] The nomenclature is inconsistent: the model is called HAW-NN in the abstract and contributions but HWB-NN in Sections III, IV, and V. Please unify.
  2. [Section III-A] The number of key feature points n is never specified. The text says it is chosen according to resolution requirements and Fig. 6 shows five points, but the value used in modeling and control should be stated.
  3. [Fig. 5] The y-axis of Fig. 5 is labeled only 'MSE'. Since the text cites 0.3947 and 0.0693, please provide units (mm^2) and indicate whether this is an endpoint-only or whole-body error.
  4. [Table II] The caption says 'MSEs' but the listed values are given in mm, not mm^2. Please make the units consistent; the quoted numbers appear to be mean Euclidean errors rather than MSEs.
  5. [Contributions and Sections IV-V] The first contribution bullet claims a 'sim-to-real-to-sim process', but no second sim step is described anywhere in Sections IV or V. Only sim-to-real deployment is shown. Please either add the second sim step or remove the claim.
  6. [Section V-D] The phrase 'average Eulerian error' should be 'average Euclidean error'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's central claims are validated by an independent held-out test set and by real-world deployment of the learned policy.

full rationale

The paper's two main empirical claims are (1) the hysteresis-aware 6D-input model achieves lower test MSE than a 3D pressure-only model, and (2) a PPO policy trained in the HAW-NN simulator achieves 0.126–0.250 mm tracking error on the real robot. Both are anchored to external data: the MSE comparison is made on a separately collected 1,000-sample test set ('validation and test sets containing 1000 samples each were collected separately, with distributions different from the training set'), and the real-world tracking numbers are measured on the physical robot with an OptiTrack motion-capture system. The RL transition function st+1 = Tnn(st, at) is a fitted neural-network model, so the simulation tracking errors in Table II are not independent physical predictions; however, the paper does not present those simulation numbers as the validation of the method. The sim-to-real deployment is the actual test, and it does not reduce to the fitted model by construction. The direction-of-pressure input is defined from the sign of pressure change in both data collection and the RL environment, which is a modeling choice, not a self-referential definition of the target output. No load-bearing self-citation, uniqueness theorem, or ansatz-smuggling step is present; the only self-citations are to general prior work on soft actuators and data-driven modeling, and they are not used to justify the central claims. The LSTM future-work statement is a limitation about temporal generalization, not evidence of circularity. Overall, the derivation chain is self-contained with independent empirical validation.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the assumption that a static, direction-labeled feedforward map can serve as the true transition function for RL training, plus the representational adequacy of key feature points and B-spline reconstruction. No new physical entities are posited; the key feature point representation and B-spline reconstruction are computational modeling choices. The neural network weights are fitted and not released, which is the main unaccounted free parameter.

free parameters (3)
  • MLP weights (4 hidden layers, 128 neurons each) = not released
    The transition model used as the RL simulator is the fitted MLP; its weights are optimized on 13,824 samples and are neither published nor checkable, so the central control claim rests on an unauditable fitted function.
  • Direction labels d_i = 1 or -1
    The hysteresis feature is a binary sign of pressure change; this is a hand-chosen proxy for pressure history, not a learned or physically derived quantity, and it omits rate and full-path information.
  • Quasi-static stabilization delay = 3 s
    Data are recorded 3 seconds after a pressure setpoint under quasi-static conditions; this choice defines what the model sees and excludes transient dynamics that may matter during closed-loop control.
assumptions (3)
  • domain assumption Soft robot behavior during control is a deterministic static map from current pressure and direction sign to pose, with no latent temporal state.
    Section III-B collects data after a 3-second stabilization delay, and Section IV-A defines the transition as st+1 = Tnn(st, at). The entire RL approach assumes the robot's pose is a function of current pressure and direction sign.
  • domain assumption Motion-capture marker-plane centers accurately represent whole-body keypoints, and B-spline reconstruction between keypoints is sufficient for control.
    Section III-B fits circles to at least 3 markers per plane; the true body may deform non-circularly, and B-spline interpolation is a geometric assumption about the shape between keypoints.
  • domain assumption PPO trained in the fitted neural-network simulator transfers to the physical robot without domain randomization or explicit dynamics randomization.
    Section IV-C and Section V-C rely on the accuracy of the fitted model; no sim-to-real transfer technique such as domain randomization is used, so transfer rests entirely on model fidelity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hysteresis-Aware Neural Network Modeling and Whole-Body Reinforcement Learning Control of Soft Robots." pith.science (2026). https://pith.science/paper/OPOKN3OQ

@misc{pith2026250413582,
  author       = {Pith},
  title        = {Pith review of: Hysteresis-Aware Neural Network Modeling and Whole-Body Reinforcement Learning Control of Soft Robots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OPOKN3OQ}},
  note         = {Machine review of arXiv:2504.13582}
}
read the original abstract

Soft robots exhibit inherent compliance and safety, which makes them particularly suitable for applications requiring direct physical interaction with humans, such as surgical procedures. However, their nonlinear and hysteretic behavior, resulting from the properties of soft materials, presents substantial challenges for accurate modeling and control. In this study, we present a soft robotic system designed for surgical applications and propose a hysteresis-aware whole-body neural network model that accurately captures and predicts the soft robot's whole-body motion, including its hysteretic behavior. Building upon the high-precision dynamic model, we construct a highly parallel simulation environment for soft robot control and apply an on-policy reinforcement learning algorithm to efficiently train whole-body motion control strategies. Based on the trained control policy, we developed a soft robotic system for surgical applications and validated it through phantom-based laser ablation experiments in a physical environment. The results demonstrate that the hysteresis-aware modeling reduces the Mean Squared Error (MSE) by 84.95 percent compared to traditional modeling methods. The deployed control algorithm achieved a trajectory tracking error ranging from 0.126 to 0.250 mm on the real soft robot, highlighting its precision in real-world conditions. The proposed method showed strong performance in phantom-based surgical experiments and demonstrates its potential for complex scenarios, including future real-world clinical applications.

Figures

Figures reproduced from arXiv: 2504.13582 by the authors.

Figure 1
Figure 1. The proposed framework combining hysteresis-aware [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Manufacturing process of a fiber-constrained three-cavity pneumatic soft robot. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Schematic diagram of the Hysteresis-aware Whole [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Schematic of the data acquisition system designed to [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Comparison of kinematic modeling performance [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Simulation result of square trajectory tracking by the [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Tracking results of the soft robot on (a) circular [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Laser pointing experiment to validate whole-body [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: (a) Overview of the multiple soft robot system for laparoscopic surgery, demonstrating coordinated control and task [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 22 canonical work pages

  1. [1]

    Controlling Soft Robotic Arms Using Continual Learning,

    F. Pique, H. T. Kalidindi, L. Fruzzetti, C. Laschi, A. Menciassi, and E. Falotico, “Controlling Soft Robotic Arms Using Continual Learning,” IEEE Robotics and Automation Letters , vol. 7, pp. 5469– 5476, Apr. 2022

    work page 2022

  2. [2]

    Deep Learning Methods in Soft Robotics: Architectures and Applications,

    T. ˇCakurda, M. Trojanov ´a, P. Pomin, and A. Ho ˇsovsk´y, “Deep Learning Methods in Soft Robotics: Architectures and Applications,” Advanced Intelligent Systems , p. 2400576, Nov. 2024

    work page 2024

  3. [3]

    Self-healing soft pneumatic robots,

    S. Terryn, J. Brancart, D. Lefeber, G. Van Assche, and B. Vander- borght, “Self-healing soft pneumatic robots,” Science Robotics, vol. 2, no. 9, p. eaan4268, 2017

    work page 2017

  4. [4]

    Vacuum-powered soft pneumatic twisting actuators to empower new capabilities for soft robots,

    Z. Jiao, C. Ji, J. Zou, H. Yang, and M. Pan, “Vacuum-powered soft pneumatic twisting actuators to empower new capabilities for soft robots,” Advanced Materials Technologies , vol. 4, no. 1, p. 1800429, 2019

    work page 2019

  5. [5]

    Deep- Learning-Based Compliant Motion Control of a Pneumatically-Driven Robotic Catheter,

    D. Wu, X. T. Ha, Y . Zhang, M. Ourak, G. Borghesan, K. Niu, F. Trauzettel, J. Dankelman, A. Menciassi, and E. V . Poorten, “Deep- Learning-Based Compliant Motion Control of a Pneumatically-Driven Robotic Catheter,” IEEE Robotics and Automation Letters , vol. 7, pp. 8853–8860, Oct. 2022

    work page 2022

  6. [6]

    Laser Endoscopic Manipula- tor Using Spring-Reinforced Multi-DoF Soft Actuator,

    B. Zhang, P. Yang, X. Gu, and H. Liao, “Laser Endoscopic Manipula- tor Using Spring-Reinforced Multi-DoF Soft Actuator,” IEEE Robotics and Automation Letters , vol. 6, pp. 7736–7743, Oct. 2021

    work page 2021

  7. [7]

    Design and kinematic modeling of constant curvature continuum robots: A review,

    R. J. Webster III and B. A. Jones, “Design and kinematic modeling of constant curvature continuum robots: A review,” The International Journal of Robotics Research , vol. 29, no. 13, pp. 1661–1683, 2010

    2010

  8. [8]

    Modeling of fluidic pre- stressed composite actuators with application to soft robotic grippers,

    Y . Zhou, L. M. Headings, and M. J. Dapino, “Modeling of fluidic pre- stressed composite actuators with application to soft robotic grippers,” IEEE Transactions on Robotics , vol. 38, no. 4, pp. 2166–2178, 2022

    work page 2022

Show all 24 references
  1. [9]

    Practical kinematics for real-time implementation of continuum robots,

    B. A. Jones and I. D. Walker, “Practical kinematics for real-time implementation of continuum robots,” IEEE Transactions on Robotics, vol. 22, no. 6, pp. 1087–1099, 2006

    2006

  2. [10]

    RL-Based Adaptive Controller for High Precision Reaching in a Soft Robot Arm,

    M. S. Nazeer, C. Laschi, and E. Falotico, “RL-Based Adaptive Controller for High Precision Reaching in a Soft Robot Arm,” IEEE Transactions on Robotics , vol. 40, pp. 2498–2512, 2024

    2024

  3. [11]

    Vision-Based Online Key Point Estimation of De- formable Robots,

    H. Zheng, S. Pinzello, B. G. Cangan, T. J. K. Buchner, and R. K. Katzschmann, “Vision-Based Online Key Point Estimation of De- formable Robots,” Advanced Intelligent Systems , vol. 6, p. 2400105, Oct. 2024

    2024

  4. [12]

    Controlling Soft Robotic Arms Using Hybrid Modelling and Reinforcement Learning,

    G. Lou, C. Wang, Z. Xu, J. Liang, and Y . Zhou, “Controlling Soft Robotic Arms Using Hybrid Modelling and Reinforcement Learning,” IEEE Robotics and Automation Letters , vol. 9, pp. 7070–7077, Aug. 2024

    2024

  5. [13]

    Model-less feedback control for soft manipu- lators with jacobian adaptation,

    Y .-Y . Wu and N. Tan, “Model-less feedback control for soft manipu- lators with jacobian adaptation,” in 2020 International Symposium on Autonomous Systems (ISAS) , pp. 217–222, IEEE, 2020

    2020

  6. [14]

    A novel hysteresis mod- elling method with improved generalization capability for pneumatic artificial muscles,

    Y . Zhang, J. Gao, H. Yang, and L. Hao, “A novel hysteresis mod- elling method with improved generalization capability for pneumatic artificial muscles,” Smart Materials and Structures , vol. 28, p. 105014, Oct. 2019

    2019

  7. [15]

    Data-Driven Methods Applied to Soft Robot Modeling and Control: A Review,

    Z. Chen, F. Renda, A. Le Gall, L. Mocellin, M. Bernabei, T. Dangel, G. Ciuti, M. Cianchetti, and C. Stefanini, “Data-Driven Methods Applied to Soft Robot Modeling and Control: A Review,” IEEE Trans- actions on Automation Science and Engineering , vol. 22, pp. 2241– 2256, 2025

    2025

  8. [16]

    Closed-Loop Dynamic Control of a Soft Manipulator Using Deep Reinforcement Learning,

    A. Centurelli, L. Arleo, A. Rizzo, S. Tolu, C. Laschi, and E. Falotico, “Closed-Loop Dynamic Control of a Soft Manipulator Using Deep Reinforcement Learning,” IEEE Robotics and Automation Letters , vol. 7, pp. 4741–4748, Apr. 2022

    2022

  9. [17]

    Open loop position control of soft continuum arm using deep reinforcement learning,

    S. Satheeshbabu, N. K. Uppalapati, G. Chowdhary, and G. Krishnan, “Open loop position control of soft continuum arm using deep reinforcement learning,” in 2019 International Conference on Robotics and Automation (ICRA) , pp. 5133–5139, IEEE, 2019

    2019

  10. [18]

    Model- Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators,

    T. G. Thuruthel, E. Falotico, F. Renda, and C. Laschi, “Model- Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators,” IEEE Transactions on Robotics , vol. 35, pp. 124–134, Feb. 2019

    2019

  11. [19]

    A Cable-Actuated Soft Manipulator for Dexterous Grasping Based on Deep Reinforcement Learning,

    K. Zhou, B. Mao, Y . Zhang, Y . Chen, Y . Xiang, Z. Yu, H. Hao, W. Tang, Y . Li, H. Liu, X. Wang, X. Wang, and J. Qu, “A Cable-Actuated Soft Manipulator for Dexterous Grasping Based on Deep Reinforcement Learning,” Advanced Intelligent Systems , vol. 6, p. 2400112, Oct. 2024

    2024

  12. [20]

    SoMoGym: A Toolkit for Developing and Evaluating Controllers and Reinforcement Learning Algorithms for Soft Robots,

    M. A. Graule, T. P. McCarthy, C. B. Teeple, J. Werfel, and R. J. Wood, “SoMoGym: A Toolkit for Developing and Evaluating Controllers and Reinforcement Learning Algorithms for Soft Robots,” IEEE Robotics and Automation Letters , vol. 7, pp. 4071–4078, Apr. 2022

    2022

  13. [21]

    Reinforcement Learning Controllers for Soft Robots Using Learned Environments,

    U. Berdica, M. Jackson, N. E. Veronese, J. Foerster, and P. Maiolino, “Reinforcement Learning Controllers for Soft Robots Using Learned Environments,” in 2024 IEEE 7th International Conference on Soft Robotics (RoboSoft), (San Diego, CA, USA), pp. 933–939, IEEE, Apr. 2024

    2024

  14. [22]

    Markov decision processes,

    M. L. Puterman, “Markov decision processes,” Handbooks in opera- tions research and management science , vol. 2, pp. 331–434, 1990

    1990

  15. [23]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  16. [24]

    Trust region policy optimization,

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International conference on machine learning, pp. 1889–1897, PMLR, 2015

    2015

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.