Pith. sign in

REVIEW 4 major objections 6 minor 26 references

DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Object affordance priors, injected as functional grasp candidates and reward shaping, raise success rates of high-dimensional dexterous manipulation policies by an average of 15.4 percentage points in simulation.

desk verdict The voting-based grasp classifier is genuinely new, but the experiments never isolate the affordance prior from dense reward shaping, so the 15.4% claim is not yet supportable. read the letter →

arxiv 2505.14819 v1 pith:TGUNK5LD submitted 2025-05-20 cs.RO

Lei Zhang , Soumya Mondal , Zhenshan Bing , Kaixin Bai , Diwen Zheng , Zhaopeng Chen , Alois Christian Knoll , Jianwei Zhang This is my paper
classification cs.RO
keywords dexterousmanipulationobjectaffordancereinforcementlearningfunctionalgraspinggraspposegenerationrewardshapingmulti-fingeredhandsimulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a reinforcement-learning agent controlling a multi-fingered robotic hand learns dexterous manipulation faster and more reliably when the object's functional regions—its affordances, the action possibilities offered by object parts—are fed into the loop as constraints and reward signals. The proposed pipeline generates affordance-aligned grasp candidates from an object affordance map, filters them by motion feasibility, and adds an affordance-aware reward that pulls fingertips toward semantically meaningful contact regions. Across three simulated tasks (cube grasp-lift, jug handle grasp-lift, hammer reorientation), this guidance raises average task success by 15.4 percentage points over the same learners without such guidance (PPO and SAC), with the largest gains where the agent must choose the right contact point. The paper's positive reading is that semantic priors about what an object is for can substitute for much of the blind exploration that makes high-dimensional hand control hard.

What carries the argument

The load-bearing object is the object affordance map $M_{\mathrm{Aff}}\in\mathbb{R}^P$, a per-surface-point label field declaring functional categories such as handle, wrap, support, and pour. Two mechanisms carry the argument: a voting-based grasp classifier (Eqs. 4–6) that reads, for each contact point, the $N$ nearest affordance labels and aggregates finger-level votes into a grasp-level category; and the affordance-aware reward term $R_{\mathrm{affordance}}$ above, plus a motion-feasibility filter that discards grasp candidates unreachable without collision before they enter policy training. These components convert the affordance map into both a constraint on the action space and a shaping signal on the reward, which is what lets the policy focus on semantically meaningful contact regions.

What would settle it

Take the jug and hammer tasks and rerun the same training with affordance maps whose functional labels are partially corrupted or removed (for instance, delete the handle region from the jug's map). The central claim predicts success rates fall substantially; if they stay flat, then the affordance guidance is not what drives the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that a hand-policy learner told which object regions afford which actions—via an object affordance map $M_{\mathrm{Aff}}$, functional grasp candidates generated from it, and the affordance-aware reward term $R_{\mathrm{affordance}}=\exp(-\alpha\sum_i\|p_{\mathrm{tip}}^i-p_{\mathrm{afford}}\|^2)$—outperforms the same learner without that knowledge. Grasp candidates are classified by a weighted voting scheme that reads each finger's nearest affordance labels, so categories such as handle grasp, wrap grasp, support, and pour stay aligned with the functional map. Concretely, with affordance guidance overall success rises from 55.3% to 73.2% for PPO and from 51.8% to 68.3% for SAC on cube grasp-lift; from 37.9% to 65.6% and from 28.3% to 57.9% on jug handle grasp-lift; and from 68.6% to 69.3% and from 75.0% to 75.1% on hammer use. The authors attribute these gains to constrained exploration and faster convergence, and report average training-time reductions of about 4.6% for PPO and 6.8% for SAC.

Load-bearing premise

The pipeline presumes every object comes with a correct and complete affordance map that labels its functional regions, so the generated grasp candidates and the fingertip reward target are trustworthy.

Editorial extensions

If this is right

  • Affordance guidance improves final success for both PPO and SAC on all three tasks, with the largest absolute gain (27.7 points for PPO grasp success) in the handle-grasp task.
  • Training converges faster with affordance cues: average wall-clock training time falls about 4.6% for PPO and 6.8% for SAC across the task suite.
  • The gains are task-dependent: small in the dynamic hammer-reorientation task (about 0.7 to 0.1 points) and large in tasks that require selecting semantically meaningful contact points.
  • Affordance priors appear to compensate for PPO's limited exploration in structured contact tasks, while SAC's entropy-driven exploration already covers more of the space and therefore benefits less.
  • The same reward schema instantiates across grasping, lifting, and in-hand reorientation, suggesting the affordance constraint transfers across object geometries and task structures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never varies affordance-map quality; a natural extension is to corrupt or delete functional labels (for example, remove the jug handle) and measure how success degrades. If success stays high, the affordance map is not actually load-bearing; if it collapses, the assumed completeness of the map is confirmed as critical.
  • Because the voting classifier operates on contact-semantic labels and affordance labels rather than on joint angles, the same pipeline may transfer to other multi-fingered hand morphologies simply by regenerating grasp candidates; this is an unstated implication of the method's design.
  • The reported failure mode with mesh penetration in a microwave-door task suggests that affordance guidance alone does not enforce physical plausibility; a testable extension is to add a penetration-aware penalty and check whether the learned policies stop exploiting the door.
  • If the simulated gains reproduce on hardware, the largest practical payoff may be in sim-to-real transfer, since functional-region constraints are physics-independent priors; that transfer claim is not in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DORA, an object affordance-guided reinforcement learning framework for dexterous manipulation with a multi-fingered hand. The method generates functional grasp candidates from affordance maps using a voting-based classifier, filters them with motion-feasibility checks, and feeds them as state information and reward shaping into PPO/SAC training. Experiments in Isaac Gym on three tasks (cube grasp-and-lift, jug handle grasp-and-lift, hammer reorientation) report that affordance guidance improves average success rate by 15.4% over affordance-agnostic baselines and reduces training time.

Significance. If the causal attribution to affordance semantics is established, DORA would be a valuable demonstration that high-level semantic priors can be injected into high-dimensional dexterous RL in simulation. The paper has concrete strengths: standard and reproducible simulation infrastructure, evaluations with two RL algorithms and three seeds, sub-task success metrics, training-time reporting, and an honest limitations section. However, the experimental design currently does not isolate the affordance prior from generic dense reward shaping and added state features, so the significance is conditional on additional control experiments.

major comments (4)
  1. [Section III-C, III-D, Table II] The 'with affordance' condition differs from the 'without' baselines in several simultaneous ways: an additional dense proximity reward (Eq. 10), functional grasp candidates in the state (Section III-A), motion-feasibility filtering (Eq. 7), and sub-task transitions. The reported 15.4% average improvement therefore cannot be uniquely attributed to the semantic content of the affordance map; a dense reward toward any task-relevant reference point, or any added candidate-conditioned state, could plausibly produce comparable gains. Please add ablations that change one component at a time, in particular (a) replace p_afford with a non-semantic reference such as the object centroid while keeping Eq. (10) intact, and (b) remove the grasp-candidate state input while retaining the other modifications, and report the resulting success rates.
  2. [Table II, Section IV-D1] The claim that affordance guidance 'consistently improves success rates across all tasks' is not supported at the per-task level for Task 3: PPO changes from 68.6 to 69.3 and SAC from 75.0 to 75.1, both well within the reported standard deviations, and no statistical significance tests are provided. The aggregate 15.4% figure is dominated by Task 2. Please report per-task confidence intervals or paired statistical tests across the three seeds, and either temper the universal improvement claim or explain the mechanism that makes Task 3 qualitatively different.
  3. [Section III-B, Eqs. (4)-(6)] The voting-based functional grasp classifier is central to generating the candidates that later act as constraints and reward anchors, but the manuscript provides no validation of the classifier itself. I request a quantitative evaluation of classification accuracy against the known affordance labels of the generated grasps, a confusion matrix across grasp categories, and a sensitivity analysis with respect to the neighborhood size N and the finger weights w_fm. Without this, the reader cannot assess how much of the downstream RL gain depends on the quality of this intermediate module.
  4. [Section III-A, Fig. 3] The affordance maps are assumed given and manually specified for the three objects, and the paper does not examine robustness to imperfect maps, such as missing or mislabeled handle regions. Since the abstract and introduction emphasize generalization, please add a sensitivity study that corrupts or removes a fraction of affordance labels, or explicitly scope the claims to perfectly annotated maps and discuss the implications for real-world deployment.
minor comments (6)
  1. [Section III-D, Eq. (11)] The term z_feasible_candidate is not defined; please specify the reference frame and how the feasible candidate height is computed for each task.
  2. [Section III-C, Eq. (7)] The text contains an unrendered LaTeX token, 'mathcalG', in the sentence introducing the candidate set; please fix this typo.
  3. [Section III-D, Eq. (13)] The orientation reward formula -2*arcsin(2*<r_current, r_target>^2 - 1) appears to evaluate to -π when the orientations are perfectly aligned; please verify the intended sign and scaling, and clarify whether a negative reward at alignment is intentional.
  4. [Section IV-C, Table III] Please state whether the success-rate curves are averaged over seeds before or after Gaussian smoothing, and report the number of runs and variance for the training-time numbers in Table III.
  5. [Section IV-B] The paper reports only the batch size and epoch count; for reproducibility, please provide the remaining hyperparameters for PPO and SAC, such as learning rates, network sizes, entropy coefficients, and discount factors.
  6. [Abstract / Section IV-D1] The abstract says '27.7% improvement in grasp success rate' but Section IV-D1 reports the gain for jug grasping using the overall success rate from Table II; please make the metric terminology consistent between the abstract, introduction, and results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the reported 15.4% gain is an empirical simulation outcome, not a quantity forced by the paper's equations or by a self-citation chain.

full rationale

The paper's central claim—that the affordance-augmented pipeline improves task success rates by an average of 15.4%—is an empirical measurement from Isaac Gym rollouts, not a value derived from the paper's own equations. Table II compares PPO and SAC with and without the full affordance-augmented pipeline, and the numbers are simulation outcomes averaged over three seeds. The affordance-aware reward (Eq. 10) and the voting-based grasp classifier (Eqs. 4–6) are hand-specified components of the treated condition; no parameter of these components is fitted to the reported success rates, and no reported success value is obtained by substituting a fitted constant back into an equation. The only notable self-citation is the grasp-synthesis method [3], used to generate functional grasp candidates. This is a component of the pipeline, but the empirical comparison does not reduce to [3]: the baseline and treated policies are trained in the same simulator with the same task success criteria, and the improvement is measured rather than implied by the candidate-generation procedure. The absence of an ablation isolating the semantic content of the affordance map—e.g., replacing pafford with a non-semantic reference point or removing the grasp-candidate state input—is a potential confounding-factor issue in the experimental design, but it is a validity threat, not a circular derivation. Accordingly, no step in the paper's derivation chain reduces, by construction or by self-citation, to its own inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. The free parameters listed above are hand-chosen and mostly unreported. The main assumptions are the availability of correct affordance maps, the reliability of the prior grasp synthesis pipeline, and the fidelity of the simulator for the reported tasks.

free parameters (3)
  • Reward weights wa, wt, wp = not reported
    Task-specific weighting coefficients in Eq. 9 are chosen by hand to balance the affordance, task, and penalty terms. Their values are never given in the paper, yet they directly control the trade-off that produces the reported success rates.
  • Affordance sensitivity alpha = not reported
    Alpha in Eq. 10 controls the spatial sensitivity of the affordance reward. It is introduced ad hoc and its value is not specified, so the reader cannot reproduce the reward shaping.
  • Per-finger voting weights wfm = not reported
    The weighted voting in Eq. 6 assigns higher weights to fingers that 'play a more critical role,' but no procedure to compute these weights is given. They are effectively free parameters of the classifier.
assumptions (3)
  • domain assumption Each object is associated with a correct and complete object affordance map Ao.
    Stated in Section III-A: 'We assume each object is associated with an object affordance map Ao.' The entire grasp candidate generation and reward depend on this map being accurate.
  • domain assumption The grasp generation method from the authors' prior work [3] produces high-quality, physically plausible grasp poses.
    Section III-B relies on 'our previous work [3]' for the data and optimization used to propose grasp candidates. If that method produces poor candidates, the voting classifier and the RL constraints inherit the error.
  • domain assumption The Isaac Gym physics simulation is sufficiently accurate for the three tasks, with no significant mesh penetration artifacts.
    The experiments are entirely in Isaac Gym. The appendix reports that mesh penetration severely degraded performance on a more complex task, implying the simulation is not always faithful; the paper assumes the three tasks are unaffected.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation." pith.science (2026). https://pith.science/paper/TGUNK5LD

@misc{pith2026250514819,
  author       = {Pith},
  title        = {Pith review of: DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TGUNK5LD}},
  note         = {Machine review of arXiv:2505.14819}
}
read the original abstract

Dexterous robotic manipulation remains a longstanding challenge in robotics due to the high dimensionality of control spaces and the semantic complexity of object interaction. In this paper, we propose an object affordance-guided reinforcement learning framework that enables a multi-fingered robotic hand to learn human-like manipulation strategies more efficiently. By leveraging object affordance maps, our approach generates semantically meaningful grasp pose candidates that serve as both policy constraints and priors during training. We introduce a voting-based grasp classification mechanism to ensure functional alignment between grasp configurations and object affordance regions. Furthermore, we incorporate these constraints into a generalizable RL pipeline and design a reward function that unifies affordance-awareness with task-specific objectives. Experimental results across three manipulation tasks - cube grasping, jug grasping and lifting, and hammer use - demonstrate that our affordance-guided approach improves task success rates by an average of 15.4% compared to baselines. These findings highlight the critical role of object affordance priors in enhancing sample efficiency and learning generalizable, semantically grounded manipulation policies. For more details, please visit our project website https://sites.google.com/view/dora-manip.

Figures

Figures reproduced from arXiv: 2505.14819 by the authors.

Figure 1
Figure 1. Overview of the proposed object affordance-guided [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the object affordance-guided reinforcement learning pipeline for various dexterous manipulation tasks. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Examples of object affordances used in the task. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance comparison across three tasks based on optimal policy. Task 1: Cube grasping and lifting, Task 2: Jug [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Scene of task 4 for opening microwave and picking, [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 18 canonical work pages

  1. [1]

    Learning dexterous in-hand manipulation,

    O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. Mc- Grew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray et al. , “Learning dexterous in-hand manipulation,” The International Journal of Robotics Research , vol. 39, no. 1, pp. 3–20, 2020

    work page 2020

  2. [2]

    Learning dexterous grasping with object-centric visual affordances,

    P. Mandikal and K. Grauman, “Learning dexterous grasping with object-centric visual affordances,” in 2021 IEEE international con- ference on robotics and automation (ICRA) . IEEE, 2021, pp. 6169– 6176

    work page 2021

  3. [3]

    Multi-fingered robotic hand grasping in cluttered environ- ments through hand-object contact semantic mapping,

    L. Zhang, K. Bai, G. Huang, Z. Bing, Z. Chen, A. Knoll, and J. Zhang, “Multi-fingered robotic hand grasping in cluttered environ- ments through hand-object contact semantic mapping,” arXiv preprint arXiv:2404.08844, 2024

    arXiv 2024

  4. [4]

    Learning generalizable dexterous manipulation from human grasp affordance,

    Y .-H. Wu, J. Wang, and X. Wang, “Learning generalizable dexterous manipulation from human grasp affordance,” in Conference on Robot Learning. PMLR, 2023, pp. 618–629

    work page 2023

  5. [5]

    The ecological approach to visual perception,

    J. J. Gibson, “The ecological approach to visual perception,” 1979

    work page 1979

  6. [6]

    Affordancenet: An end-to-end deep learning approach for object affordance detection,

    T.-T. Do, A. Nguyen, and I. Reid, “Affordancenet: An end-to-end deep learning approach for object affordance detection,” in 2018 IEEE international conference on robotics and automation (ICRA) . IEEE, 2018, pp. 5882–5889

    work page 2018

  7. [7]

    Towards human-level bimanual dexterous manipulation with reinforcement learning,

    Y . Chen, T. Wu, S. Wang, X. Feng, J. Jiang, S. M. McAleer, Y . Geng, H. Dong, Z. Lu, S.-C. Zhu, and Y . Yang, “Towards human-level bimanual dexterous manipulation with reinforcement learning,” 2022. [Online]. Available: https://arxiv.org/abs/2206.08686

    arXiv 2022

  8. [8]

    Sequential dexterity: Chaining dexterous policies for long-horizon manipulation,

    Y . Chen, C. Wang, L. Fei-Fei, and C. K. Liu, “Sequential dexterity: Chaining dexterous policies for long-horizon manipulation,” arXiv preprint arXiv:2309.00987, 2023

    arXiv 2023

Show all 26 references
  1. [9]

    Bi-dexhands: Towards human-level bimanual dexterous manipulation,

    Y . Chen, Y . Geng, F. Zhong, J. Ji, J. Jiang, Z. Lu, H. Dong, and Y . Yang, “Bi-dexhands: Towards human-level bimanual dexterous manipulation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 2804–2818, 2023

    2023

  2. [10]

    Object-based affordances detection with convolutional neural net- works and dense conditional random fields,

    A. Nguyen, D. Kanoulas, D. G. Caldwell, and N. G. Tsagarakis, “Object-based affordances detection with convolutional neural net- works and dense conditional random fields,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2017, pp. 5908–5915

    2017

  3. [11]

    Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching,

    A. Zeng, S. Song, K.-T. Yu, E. Donlon, F. R. Hogan, M. Bauza, D. Ma, O. Taylor, M. Liu, E. Romo et al. , “Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching,” The International Journal of Robotics Research , vol. 4...

    2022

  4. [12]

    Tooleenet: Tool affordance 6d pose estimation,

    Y . Wang, L. Zhang, Y . Tu, H. Zhang, K. Bai, Z. Chen, and J. Zhang, “Tooleenet: Tool affordance 6d pose estimation,” in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2024, pp. 10 519–10 526

    2024

  5. [13]

    Affordance- guided reinforcement learning via visual prompting,

    O. Y . Lee, A. Xie, K. Fang, K. Pertsch, and C. Finn, “Affordance- guided reinforcement learning via visual prompting,” arXiv preprint arXiv:2407.10341, 2024

    2024 arXiv

  6. [14]

    Affordance learning from play for sample-efficient policy learning,

    J. Borja-Diaz, O. Mees, G. Kalweit, L. Hermann, J. Boedecker, and W. Burgard, “Affordance learning from play for sample-efficient policy learning,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 6372–6378

    2022

  7. [15]

    Learn- ing object affordances: from sensory–motor coordination to imitation,

    L. Montesano, M. Lopes, A. Bernardino, and J. Santos-Victor, “Learn- ing object affordances: from sensory–motor coordination to imitation,” Ieee transactions on robotics , vol. 24, no. 1, pp. 15–26, 2008

    2008

  8. [16]

    Grasping affor- dances with the other’s hand: a tms study,

    P. Cardellicchio, C. Sinigaglia, and M. Costantini, “Grasping affor- dances with the other’s hand: a tms study,” Social cognitive and affective neuroscience, vol. 8, no. 4, pp. 455–459, 2013

    2013

  9. [17]

    Cross-category functional grasp transfer,

    R. Wu, T. Zhu, X. Lin, and Y . Sun, “Cross-category functional grasp transfer,” IEEE Robotics and Automation Letters , 2024

    2024

  10. [18]

    Task-oriented grasping with point cloud representation of objects,

    A. Patankar, K. Phi, D. Mahalingam, N. Chakraborty, and I. Ramakr- ishnan, “Task-oriented grasping with point cloud representation of objects,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 6853–6860

    2023

  11. [19]

    Constrained policy op- timization,

    J. Achiam, D. Held, A. Tamar, and P. Abbeel, “Constrained policy op- timization,” in International conference on machine learning . PMLR, 2017, pp. 22–31

    2017

  12. [20]

    Safe exploration in finite markov decision processes with gaussian processes,

    M. Turchetta, F. Berkenkamp, and A. Krause, “Safe exploration in finite markov decision processes with gaussian processes,” Advances in neural information processing systems , vol. 29, 2016

    2016

  13. [21]

    Safe model-based reinforcement learning with stability guarantees,

    F. Berkenkamp, M. Turchetta, A. Schoellig, and A. Krause, “Safe model-based reinforcement learning with stability guarantees,” Ad- vances in neural information processing systems , vol. 30, 2017

    2017

  14. [22]

    Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations,

    S. Song, A. Zeng, J. Lee, and T. Funkhouser, “Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations,” IEEE Robotics and Automation Letters , vol. 5, no. 3, pp. 4978–4985, 2020

    2020

  15. [23]

    Multisensory five-finger dexterous hand: The dlr/hit hand ii,

    H. Liu, K. Wu, P. Meusel, N. Seitz, G. Hirzinger, M. Jin, Y . Liu, S. Fan, T. Lan, and Z. Chen, “Multisensory five-finger dexterous hand: The dlr/hit hand ii,” in 2008 IEEE/RSJ international conference on intelligent robots and systems . IEEE, 2008, pp. 3692–3697

    2008

  16. [24]

    Isaac gym: High performance gpu-based physics simulation for robot learning,

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Mack- lin, D. Hoeller, N. Rudin, A. Allshire, A. Handa et al. , “Isaac gym: High performance gpu-based physics simulation for robot learning,” arXiv preprint arXiv:2108.10470 , 2021

    2021 arXiv

  17. [25]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  18. [26]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” in International conference on machine learning . Pmlr, 2018, pp. 1861–1870. APPENDIX A. Results of Training Time Table III rep...

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.