Pith. sign in

REVIEW 43 references

Language models can supply the task spaces and difficulty orderings that let automatic curriculum learning solve long-horizon robot manipulation from sparse rewards alone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 20:25 UTC pith:GR35KMI4

load-bearing objection Clean systems result: LLM meta-specs let sparse-reward ACL nearly match a human curriculum on five LIBERO tasks, but the 'sparse beats dense' headline rests on a brittle internal baseline.

arxiv 2607.23515 v1 pith:GR35KMI4 submitted 2026-07-26 cs.RO

LEACL: LLM-Enhanced Automatic Curriculum Learning for Reinforcement Learning in Long-Horizon Manipulation Tasks

Faraz Heravi , James Ouyang , Zifan Xu , Arjun Kumar , Yoonchang Sung , Peter Stone This is my paper
classification cs.RO
keywords automatic curriculum learninglarge language modelslong-horizon manipulationsparse rewardstask decompositionPDDLreinforcement learningLIBERO
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long-horizon robot manipulation is hard for reinforcement learning because success signals arrive only at the very end. Automatic curriculum learning can help by training on easier variants first, but it normally needs humans to hand-craft the space of task variants and a difficulty measure. This paper shows that a large language model can generate both the subtask breakdown and those missing specifications, after which a standard curriculum algorithm learns each subtask from sparse completion rewards only. On five multi-step LIBERO manipulation tasks the resulting method reaches higher final success rates than a strong baseline that uses carefully engineered dense rewards, and approaches a fully human-designed curriculum. The practical payoff is that complex manipulation skills can be acquired with far less manual reward and curriculum engineering.

Core claim

When an LLM first decomposes a long-horizon manipulation goal into PDDL subtasks and then emits a parameterized task space plus an ordered difficulty sequence for each subtask, ordinary automatic curriculum learning can train a single policy to high asymptotic success using only sparse task-completion rewards, outperforming human-designed dense rewards on most of the evaluated tasks.

What carries the argument

LEACL’s three-stage pipeline: LLM task decomposition into PDDL subtasks, LLM meta-task generation of control-parameter spaces and difficulty-ordered instances, and plug-and-play automatic curriculum learning that samples those instances under sparse rewards.

Load-bearing premise

The full set of predicates that describe objects, fixtures and goals must already be supplied to the language model; the method does not invent new predicates on its own.

What would settle it

Retrain the five LIBERO+ tasks with the identical LLM pipeline but replace the generated task spaces by random or empty parameterizations; if success rates then collapse to the near-zero levels of the sparse-reward and no-ACL baselines, the claim that the LLM-generated specifications are doing the essential work is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Dense reward design can be removed from many multi-step manipulation pipelines without sacrificing final success rate.
  • Existing automatic curriculum algorithms become usable on new robot tasks once an LLM supplies the missing parameter space and difficulty measure.
  • The same three-stage pattern can be applied to any domain that already possesses a predicate vocabulary and a sparse completion signal.
  • Human effort shifts from writing reward functions to curating the predicate grammar and the initial natural-language goal.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the predicate-vocabulary bottleneck can be lifted by letting the LLM propose new predicates, the method would become domain-agnostic rather than library-dependent.
  • The same LLM-generated task spaces could be reused across different base RL algorithms or even imitation-learning curricula, not only PPO-plus-ADR.
  • Performance gaps that remain versus the human curriculum likely trace to the quality of the difficulty ordering rather than the decomposition itself, suggesting a cheap human-in-the-loop re-ranking step.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Circularity Check

0 steps flagged

No circularity: empirical systems paper whose success rates are measured against external sparse task predicates, not quantities defined from fitted inputs.

full rationale

LEACL is an engineering/empirical RL framework paper. Its central claim is comparative asymptotic success rates of LLM-generated task decompositions + ACL under sparse rewards versus baselines (sparse-only, LEACL w/o ACL, hand-crafted dense LEAGUE, human curriculum) on five LIBERO+ manipulation tasks (Table II). There is no derivation chain in which a quantity is defined or fitted from data and then re-presented as a first-principles prediction. Task success is evaluated by external environment predicates (goal literals in PDDL / sparse R=1 on SG). LLM outputs (subtask PDDL, parameter spaces, difficulty orderings) are used as inputs to standard ACL algorithms (ADR); they are not circularly re-derived from the reported success rates. Self-citations (LIBERO, TeachMyAgent, ALP-GMM, ADR, LEAGUE) supply baselines and tooling, not load-bearing uniqueness theorems or ansatzes that force the result. Weaknesses of the dense-reward baseline (author-designed, high variance) are comparison-validity concerns, not circularity. Score 0 is the correct honest finding.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central empirical claim rests on standard RL/ACL machinery, a fixed domain predicate grammar, LLM common-sense competence, and several engineering choices (PPO hyper-parameters, ADR as the sole ACL backend, ChatGPT-4o-mini). No new physical entities; free parameters are ordinary ML hyper-parameters and prompt design.

free parameters (5)
  • PPO learning rate = 0.0002
    Set to 0.0002 after informal tuning on short-horizon tasks; held fixed across all methods.
  • PPO rollout length (n_steps) = 512
    Set to 512 after informal tuning for training stability.
  • Actor-critic MLP width = 128-128
    Two hidden layers of 128 units chosen by authors; not swept in the paper.
  • Training budget per task = ≥500k env steps (task-dependent)
    Chosen so all baselines reach success-rate plateaus (minimum 500k steps); affects asymptotic comparison fairness.
  • LLM choice and temperature/prompt wording = gpt-4o-mini (default decoding unspecified)
    ChatGPT 4o-mini with hand-written bi-level prompts and in-context examples; generation stochasticity not quantified.
axioms (5)
  • domain assumption Full observability; tasks are finite-horizon goal-augmented MDPs with sparse R∈{0,1} and early termination on success.
    Stated in Sec. III; standard for the LIBERO-style setting.
  • domain assumption The complete predicate vocabulary of LIBERO+ is provided a priori to the LLM; the LLM only selects and parameterizes from that set.
    Sec. IV-A and listed as major limitation in Sec. VII; load-bearing for decomposition and meta-task generation.
  • domain assumption LLM web-scale knowledge yields valid PDDL subtasks and useful continuous task-parameter spaces / difficulty orderings in zero-shot fashion for these kitchen tasks.
    Core methodological bet of stages 1–2; supported only by the five-task empirical outcome.
  • ad hoc to paper Active Domain Randomization (ADR) is a sufficient ACL backend when given LLM-generated parameter spaces.
    Only ACL algorithm evaluated (Sec. V-C); Table I lists many alternatives not tried.
  • ad hoc to paper Human-expert dense rewards for LEAGUE are a fair upper-bound surrogate for automated dense-reward generators.
    Stated in Sec. V-C; used to claim superiority over dense-reward approaches without running Text2Reward/Eureka-style generators.
invented entities (2)
  • LEACL three-stage pipeline (LLM decomposer + LLM meta-task generator + plug-and-play ACL) no independent evidence
    purpose: Organize sparse-reward long-horizon manipulation learning without dense rewards.
    Framework name and interface; not a physical entity. Independent evidence is the empirical table only.
  • LIBERO+ parameterized predicates independent evidence
    purpose: Enable continuous task-parameter spaces (e.g. open(object, ?o)) for curriculum generation.
    Engineering extension of LIBERO; code link provided. Independent usefulness outside this paper is plausible but not separately validated here.

pith-pipeline@v1.2.0-grok45-kimik3 · 18120 in / 3463 out tokens · 68345 ms · 2026-07-30T20:25:51.921697+00:00 · methodology

0 comments
read the original abstract

Long-horizon manipulation tasks pose significant challenges for reinforcement learning due to sparse reward signals and long horizons. Automatic curriculum learning (ACL) has been proposed to tackle these challenges by progressively training agents on a sequence of tasks, from easier to more difficult. However, the success of ACL depends heavily on task-dependent specifications-such as well-defined task parameter spaces and difficulty measures-which are often manually crafted and difficult to generalize across diverse tasks. Recent advances in large language models (LLMs) offer a promising alternative by enabling the decomposition of complex tasks into meaningful subtasks using the LLMs' web-scale common-sense knowledge. This decomposition can provide a natural curriculum structure for efficient learning of long-horizon tasks. However, existing LLM-based methods typically rely on hand-designed dense reward functions to learn each subtask, which can introduce bias and still requires significant human supervision. In this work, we propose LLM-enhanced automatic curriculum learning (LEACL), a framework that integrates LLMs and ACL to address these limitations. Specifically, LLMs are used to both decompose tasks into subtasks and to generate task-dependent specifications for each subtask. These specifications are then used by ACL algorithms to guide learning using only sparse reward signals, eliminating the need for dense reward design. We evaluate LEACL on five long-horizon manipulation tasks from the LIBERO benchmark. LEACL achieves better asymptotic performance in terms of the success rates compared to human-designed dense rewards.

Figures

Figures reproduced from arXiv: 2607.23515 by Arjun Kumar, Faraz Heravi, James Ouyang, Peter Stone, Yoonchang Sung, Zifan Xu.

Figure 1
Figure 1. Figure 1: LEACL stages: (1) LLM-based task decomposition; (2) LLM-based meta-task generation; (3) Automatic CL. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An example PDDL specification of the task [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The prompts used for LLM-based task decomposition (left) and meta-task generation (right). [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Front camera views of the five evaluation tasks. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 15 linked inside Pith

  1. [1]

    Multi-goal reinforcement learning: Challenging robotics environ- ments and request for research,

    M. Plappert, M. Andrychowicz, A. Ray, B. McGrew, B. Baker, G. Powell, J. Schneider, J. Tobin, M. Chociej, P. Welinderet al., “Multi-goal reinforcement learning: Challenging robotics environ- ments and request for research,”arXiv preprint arXiv:1802.09464, 2018

    Pith/arXiv arXiv 2018

  2. [2]

    Learning dexterous in-hand manipulation,

    O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. Mc- Grew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Rayet al., “Learning dexterous in-hand manipulation,”The International Journal of Robotics Research, vol. 39, no. 1, pp. 3–20, 2020

    2020

  3. [3]

    Deep reinforcement learning for robotics: A survey of real-world successes,

    C. Tang, B. Abbatematteo, J. Hu, R. Chandra, R. Mart ´ın-Mart´ın, and P. Stone, “Deep reinforcement learning for robotics: A survey of real-world successes,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 8, 2024

    2024

  4. [4]

    Automatic curriculum learning for deep rl: A short survey,

    R. Portelas, C. Colas, L. Weng, K. Hofmann, and P.-Y . Oudeyer, “Automatic curriculum learning for deep rl: A short survey,”arXiv preprint arXiv:2003.04664, 2020

    Pith/arXiv arXiv 2003

  5. [5]

    Curriculum learning for reinforcement learning domains: A framework and survey,

    S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, and P. Stone, “Curriculum learning for reinforcement learning domains: A framework and survey,”Journal of Machine Learning Research, vol. 21, no. 181, pp. 1–50, 2020

    2020

  6. [6]

    Teachmya- gent: a benchmark for automatic curriculum learning in deep rl,

    C. Romac, R. Portelas, K. Hofmann, and P.-Y . Oudeyer, “Teachmya- gent: a benchmark for automatic curriculum learning in deep rl,” in International Conference on Machine Learning. PMLR, 2021, pp. 9052–9063

    2021

  7. [7]

    Adaptive procedu- ral task generation for hard-exploration problems,

    K. Fang, Y . Zhu, S. Savarese, and L. Fei-Fei, “Adaptive procedu- ral task generation for hard-exploration problems,”arXiv preprint arXiv:2007.00350, 2020

    Pith/arXiv arXiv 2007

  8. [8]

    Integrated task and motion planning,

    C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, T. Silver, L. P. Kael- bling, and T. Lozano-P ´erez, “Integrated task and motion planning,” Annual review of control, robotics, and autonomous systems, vol. 4, no. 1, pp. 265–293, 2021

    2021

  9. [9]

    League: Guided skill learning and abstraction for long-horizon manipulation,

    S. Cheng and D. Xu, “League: Guided skill learning and abstraction for long-horizon manipulation,”IEEE Robotics and Automation Let- ters, vol. 8, no. 10, pp. 6451–6458, 2023

    2023

  10. [10]

    Llm+ p: Empowering large language models with optimal planning proficiency,

    B. Liu, Y . Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone, “Llm+ p: Empowering large language models with optimal planning proficiency,”arXiv preprint arXiv:2304.11477, 2023

    Pith/arXiv arXiv 2023

  11. [11]

    League++: Empowering continual robot learning through guided skill acquisition with large language models,

    Z. Li, K. Yu, S. Cheng, and D. Xu, “League++: Empowering continual robot learning through guided skill acquisition with large language models,” inICLR 2024 Workshop on Large Language Model (LLM) Agents, 2024

    2024

  12. [12]

    Language to rewards for robotic skill synthesis,

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.- T. L. Chiang, T. Erez, L. Hasenclever, J. Humpliket al., “Language to rewards for robotic skill synthesis,”arXiv preprint arXiv:2306.08647, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    Eureka: Human- level reward design via coding large language models,

    Y . J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Ja- yaraman, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human- level reward design via coding large language models,”arXiv preprint arXiv:2310.12931, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Dreureka: Language model guided sim-to-real transfer. arxiv 2024,

    Y . Ma, W. Liang, H. Wang, S. Wang, Y . Zhu, L. Fan, O. Bastani, and D. Jayaraman, “Dreureka: Language model guided sim-to-real transfer. arxiv 2024,”arXiv preprint arXiv:2406.01967

    Pith/arXiv arXiv 2024

  15. [15]

    Do as i can, not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausmanet al., “Do as i can, not as i say: Grounding language in robotic affordances,”arXiv preprint arXiv:2204.01691, 2022

    Pith/arXiv arXiv 2022

  16. [16]

    Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning,

    K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suenderhauf, “Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning,”arXiv preprint arXiv:2307.06135, 2023

    Pith/arXiv arXiv 2023

  17. [17]

    Progprompt: Generating situated robot task plans using large language models,

    I. Singh, V . Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situated robot task plans using large language models,” in2023 IEEE Interna- tional Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 11 523–11 530

    2023

  18. [18]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,

    W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” inInternational conference on machine learning. PMLR, 2022, pp. 9118–9147

    2022

  19. [19]

    Code as policies: Language model programs for em- bodied control,

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for em- bodied control,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 9493–9500

    2023

  20. [20]

    Text2reward: Reward shaping with language models for reinforcement learning,

    T. Xie, S. Zhao, C. H. Wu, Y . Liu, Q. Luo, V . Zhong, Y . Yang, and T. Yu, “Text2reward: Reward shaping with language models for reinforcement learning,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  21. [21]

    Towards autonomous reinforcement learning for real-world robotic manipulation with large language models,

    N. Turcato, M. Iovino, A. Synodinos, A. Dalla Libera, R. Carli, and P. Falco, “Towards autonomous reinforcement learning for real-world robotic manipulation with large language models,”IEEE Robotics and Automation Letters, 2025

    2025

  22. [22]

    Curricullm: Automatic task curricula design for learning complex robot skills using large language models,

    K. Ryu, Q. Liao, Z. Li, P. Delgosha, K. Sreenath, and N. Mehr, “Curricullm: Automatic task curricula design for learning complex robot skills using large language models,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 4470–4477

    2025

  23. [23]

    A real-to-sim-to-real approach to robotic manipulation with vlm-generated iterative keypoint rewards,

    S. Patel, X. Yin, W. Huang, S. Garg, H. Nayyeri, L. Fei-Fei, S. Lazebnik, and Y . Li, “A real-to-sim-to-real approach to robotic manipulation with vlm-generated iterative keypoint rewards,”arXiv preprint arXiv:2502.08643, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    V oyager: An open-ended embodied agent with large language models, 2023,

    G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. Fan, and A. Anandkumar, “V oyager: An open-ended embodied agent with large language models, 2023,”URL https://arxiv. org/abs/2305.16291, 2023

    Pith/arXiv arXiv 2023

  25. [25]

    Eurekaverse: Environment curriculum generation via large language models,

    W. Liang, S. Wang, H.-J. Wang, O. Bastani, D. Jayaraman, and Y . J. Ma, “Eurekaverse: Environment curriculum generation via large language models,”arXiv preprint arXiv:2411.01775, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Reinforcement learning for classical planning: Viewing heuristics as dense reward generators,

    C. Gehring, M. Asai, R. Chitnis, T. Silver, L. Kaelbling, S. Sohrabi, and M. Katz, “Reinforcement learning for classical planning: Viewing heuristics as dense reward generators,” inProceedings of the Inter- national Conference on Automated Planning and Scheduling, vol. 32, 2022, pp. 588–596

    2022

  27. [27]

    Pddl— the planning domain definition language,

    C. Aeronautiques, A. Howe, C. Knoblock, I. D. McDermott, A. Ram, M. Veloso, D. Weld, D. W. Sri, A. Barrett, D. Christiansonet al., “Pddl— the planning domain definition language,”Technical Report, Tech. Rep., 1998

    1998

  28. [28]

    Au- tomatically correcting large language models: Surveying the landscape of diverse self-correction strategies,

    L. Pan, M. Saxon, W. Xu, D. Nathani, X. Wang, and W. Y . Wang, “Au- tomatically correcting large language models: Surveying the landscape of diverse self-correction strategies,”arXiv preprint arXiv:2308.03188, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    Position: Llms can’t plan, but can help planning in llm-modulo frameworks,

    S. Kambhampati, K. Valmeekam, L. Guan, M. Verma, K. Stechly, S. Bhambri, L. P. Saldyt, and A. B. Murthy, “Position: Llms can’t plan, but can help planning in llm-modulo frameworks,” inForty-first International Conference on Machine Learning, 2024

    2024

  30. [30]

    Teacher algo- rithms for curriculum learning of deep rl in continuously parameterized environments,

    R. Portelas, C. Colas, K. Hofmann, and P.-Y . Oudeyer, “Teacher algo- rithms for curriculum learning of deep rl in continuously parameterized environments,” inConference on Robot Learning. PMLR, 2020, pp. 835–853

    2020

  31. [31]

    Prioritized level replay,

    M. Jiang, E. Grefenstette, and T. Rockt ¨aschel, “Prioritized level replay,” inInternational Conference on Machine Learning. PMLR, 2021, pp. 4940–4950

    2021

  32. [32]

    Active domain randomization,

    B. Mehta, M. Diaz, F. Golemo, C. J. Pal, and L. Paull, “Active domain randomization,” inConference on Robot Learning. PMLR, 2020, pp. 1162–1176

    2020

  33. [33]

    Self-organization of early vocal development in infants and machines: the role of intrinsic motivation,

    C. Moulin-Frier, S. M. Nguyen, and P.-Y . Oudeyer, “Self-organization of early vocal development in infants and machines: the role of intrinsic motivation,”Frontiers in psychology, vol. 4, p. 1006, 2014

    2014

  34. [34]

    Automatic goal gen- eration for reinforcement learning agents,

    C. Florensa, D. Held, X. Geng, and P. Abbeel, “Automatic goal gen- eration for reinforcement learning agents,” inInternational conference on machine learning. PMLR, 2018, pp. 1515–1528

    2018

  35. [35]

    R-iac: Robust intrinsically motivated exploration and active learning,

    A. Baranes and P.-Y . Oudeyer, “R-iac: Robust intrinsically motivated exploration and active learning,”IEEE Transactions on Autonomous Mental Development, vol. 1, no. 3, pp. 155–169, 2009

    2009

  36. [36]

    Self-paced deep reinforcement learning,

    P. Klink, C. D’Eramo, J. R. Peters, and J. Pajarinen, “Self-paced deep reinforcement learning,”Advances in Neural Information Processing Systems, vol. 33, pp. 9216–9227, 2020

    2020

  37. [37]

    Automated curricula through setter-solver inter- actions,

    S. Racaniere, A. K. Lampinen, A. Santoro, D. P. Reichert, V . Firoiu, and T. P. Lillicrap, “Automated curricula through setter-solver inter- actions,”arXiv preprint arXiv:1909.12892, 2019

    Pith/arXiv arXiv 1909

  38. [38]

    Libero: Benchmarking knowledge transfer for lifelong robot learning,

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone, “Libero: Benchmarking knowledge transfer for lifelong robot learning,”Ad- vances in Neural Information Processing Systems, vol. 36, 2024

    2024

  39. [39]

    Mujoco: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control,” in2012 IEEE/RSJ international conference on intelligent robots and systems. IEEE, 2012, pp. 5026–5033

    2012

  40. [40]

    Curriculum learning in reinforcement learning for multi-step manipulation tasks,

    F. Bar, “Curriculum learning in reinforcement learning for multi-step manipulation tasks,” Master’s thesis, University X, 2025, (redacted for blind review)

    2025

  41. [41]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  42. [42]

    Stable-baselines3: Reliable reinforcement learning implementations,

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforcement learning implementations,”Journal of Machine Learning Research, vol. 22, no. 268, pp. 1–8, 2021. [Online]. Available: http://jmlr.org/papers/ v22/20-1364.html

    2021

  43. [43]

    Gpt-4o mini: Advancing cost-efficient intelligence,

    OpenAI, “Gpt-4o mini: Advancing cost-efficient intelligence,” https:// openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/, 2024, accessed: 2025-04-03

    2024