HDDLGym: A Tool for Studying Multi-Agent Hierarchical Problems Defined in HDDL with OpenAI Gym

Julie A. Shah; Ngoc La; Ruaridh Mon-Williams

REVIEW 3 major objections 7 minor 29 references

HDDLGym: A Tool for Studying Multi-Agent Hierarchical Problems Defined in HDDL with OpenAI Gym

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HDDLGym converts HDDL hierarchical-planning domains into OpenAI Gym environments, letting RL policies drive multi-agent hierarchical plans.

desk verdict A useful first bridge from HDDL to Gym, with real code, but the task-completion semantics and thin validation need work before I'd trust the multi-agent claims. read the letter →

arxiv 2505.22597 v1 pith:AQNNQG3I submitted 2025-05-28 cs.AI cs.LGcs.MA

Ngoc La , Ruaridh Mon-Williams , Julie A. Shah This is my paper · ORCID

classification cs.AIcs.LGcs.MA

keywords hierarchicalplanningHDDLOpenAIGymreinforcementlearningmulti-agentsystemsHTNbenchmarkenvironmentsPPO

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HDDLGym is a Python tool that automatically turns HDDL domain and problem files into OpenAI Gym environments, so that reinforcement-learning policies can be trained to drive hierarchical planning. The paper's central claim is that this conversion can be done with modest additions to standard HDDL: an agent type hierarchy, explicit effects on compound tasks, a 'none' action, and a distinction between agent actions and environment actions. These additions let a centralized planner build each agent's action hierarchy one operator at a time, with an RL policy deciding which grounded operators to try next. If the approach holds, researchers can take well-documented hierarchical-planning benchmarks from the IPC-HTN competitions, add multi-agent collaborations, and study RL in long-horizon, hierarchical multi-agent settings without rewriting environment code.

What carries the argument

The central mechanism is the HDDLGym Planner (Algorithm 1) together with the agent-centric HDDL extension. Given a list of agents, a policy, and a deterministic flag, the planner repeatedly collects the valid operators available to each agent, prunes joint operator combinations whose effects conflict with other agents' preconditions, and uses the policy's probability distribution to pick one combination; it then extends each agent's action hierarchy until every hierarchy ends in a primitive action or the 'none' action. The supporting representational choice is the lifted-operator-with-objects encoding, which keeps the action space far smaller than the full set of grounded operators in domains like Transport and Overcooked.

What would settle it

Run a hand-built HDDL domain in which a method achieves a goal through a predicate omitted from the task's :effect list; if HDDLGym declares the task complete as soon as the listed effect holds, the completion check is unsound, and if it never completes, the check is incomplete. In the Overcooked domain, for example, removing the served predicate from the deliver task's effect while keeping it in the method would produce a visible loop or premature termination within one episode.

Watch

Extended reading notes

Core claim

HDDLGym establishes that HDDL's hierarchical task networks can be lifted into the Gym step/reset/observation API while preserving multi-agent collaboration. The key move is to define an agent-centric planning domain in which agent types are explicit, every primitive action is either an agent action or an automatic environment action, and every compound task carries an :effect clause. Task completion is then decided by comparing the current world state against the declared effects, and the planner removes finished tasks from each agent's hierarchy. The RL policy outputs probabilities over lifted operators and objects, the planner instantiates those as grounded primitive actions, and execution proceeds through a standard Gym step function.

Load-bearing premise

The whole pipeline stands on the assumption that a compound task is complete exactly when the current world state matches the task's declared :effect list, so any completion condition that is not written into the effect list will be ignored by the planner.

Editorial extensions

If this is right

Existing IPC-HTN benchmarks such as Transport, Rover, and Satellite can be converted into Gym environments with only small domain-file edits, making them reusable as RL benchmarks.
Multi-agent collaboration can be added to a domain by writing one composite task (for example, transfer-package in Transport) and a corresponding method, which HDDLGym will treat as part of the action-hierarchy search.
Default PPO policies can be trained, evaluated, and deployed in the generated environments, with metrics for plan complexity, plan difficulty, loss, success rate, planning time, and steps.
The lifting and dynamic-predicate trade-off keeps observation and action spaces tractable, at the cost of policies that are tied to the specific problem's objects and static conditions.
Success rates reported in the paper drop sharply as the number of agents grows (for example, three-agent Transport and four-agent Rover reach 0 percent within 100 steps), signalling where hierarchical RL training needs better exploration or reward shaping.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: The declared-effects completion check makes task authoring itself part of the modeling burden; a natural stress test is to see whether HDDLGym's automatic effect-generation tool ever produces effects that are necessary but not sufficient, and whether a domain written without effects, as standard HDDL allows, can still be handled correctly.
Editorial inference: The planner's belief-about-other-agents-hierarchies mechanism suggests a path to genuinely decentralized hierarchical execution, where agents plan with partial or stale beliefs; the current design is effectively a shared-state approximation of that setting.
Editorial inference: By converting IPC-HTN domains into Gym environments, HDDLGym creates the possibility of comparing classical HTN planners with RL-guided planners on identical domain files, which could quantify when learned action selection beats hand-coded decomposition.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

HDDLGym is a real tool paper: it's the first system I know that auto-generates OpenAI Gym environments from HDDL, with a multi-agent extension and a planner that uses RL policy outputs to select hierarchical decompositions. Code and demos are public, and the Transport/Overcooked walk-throughs suggest the thing actually runs. The lifted-vs-grounded space analysis (Table 1) is a genuinely useful contribution for anyone building RL-for-HTN benchmarks.

The main soft spot is the task-completion semantics. Standard HDDL compound tasks don't have effects; HDDLGym requires explicit task effects and uses them (via U) to decide when a task is done. The paper states this plainly in Section 4, but it is a real semantic change with no soundness argument. If an effect is achieved early, the task gets removed and subtasks can be skipped; if the effect set is incomplete, tasks never clear and the planner can loop. The 'autonomous task effect generation' in the codebase might mitigate this, but it isn't described or validated in the paper. This should be a documented limitation at least, and ideally handled with a check that all methods for a task have been fully decomposed before marking it complete.

Validation is thinner than the claims. The only RL result is one PPO loss plot for Transport; there are no baselines against PDDLGym, pyRDDLGym, or flat RL, and the 'collaborative planning' feature is shown as a hierarchy diagram, not a trained collaborative policy that outperforms non-collaborative planning. The difficulty tables (Table 3) are useful for benchmarking but are random-exploration statistics, not evidence of RL effectiveness.

That said, I'd send this to review. The tool fills a real gap, the code is there, and the problems are fixable in revision: clarify the task-effect semantics, add a soundness note or a safety check, and run at least one comparison that shows the RL-assisted planner beats a random or flat baseline. The paper is honest about its other limitations (deterministic, discrete), which I take as a good sign.

Referee Report

3 major / 7 minor

Summary. The paper presents HDDLGym, a Python tool that converts HDDL hierarchical planning domain and problem files into OpenAI Gym environments for reinforcement learning. It introduces an agent-centric extension to HDDL, requires tasks to declare explicit effects, supports multi-agent scenarios via a centralized planner with optional decentralized execution, and provides evaluation metrics and visualization. The tool is demonstrated on the IPC-HTN Transport domain and a custom Overcooked domain, with quantitative planning-difficulty tables and training curves for PPO. The authors position HDDLGym as the first system bridging HDDL-based hierarchical planning with the Gym RL interface while supporting multi-agent configurations.

Significance. If the underlying semantics are sound, HDDLGym is a potentially valuable tool for the AI planning and RL communities: it ships open-source code, tutorials, and initial quantitative planning-difficulty metrics, and it directly targets multi-agent hierarchical problems, which is a growing area. The paper's central semantic extension, however, is not validated. The requirement that tasks include explicit effects and that task completion be checked against the world state (Sections 4 and 5.2) is a nonstandard modification of HDDL; no formal correctness argument or empirical validation is given to show that declared effects are equivalent to successful method execution. The multi-agent transition semantics are also underspecified in Algorithm 1. These issues are load-bearing for the central claim that HDDLGym automatically generates correct Gym environments from HDDL, and they require substantial work. The paper otherwise demonstrates a real working system with reproducible code and reasonable evaluation scaffolding.

major comments (3)

[Section 5.5, Algorithm 1] The task completion semantics based on declared task effects are a nonstandard and potentially unsound modification of HDDL. In standard HDDL, a compound task is accomplished by decomposing it through a method into primitive actions; a task does not itself carry an effect. HDDLGym instead checks the current world state against the task's declared effect and removes the task when the effect holds, as described in Section 4 ('to ensure task completion, HDDLGym checks the current world state against the defined task effects') and in the definition of the update function U in Section 5.2. Because the effect set is not derived from the methods, two failure modes arise: a false positive, where an effect is already true in the initial state or becomes true before all subtasks have executed, causing U to delete the task and skip required subtasks; and a false negative, where a method achieves its intended outcome through side effects not listed in the declared effect, causing the task to never be removed and the planner to loop or fail. The paper claims the codebase includes 'autonomous task effect generation' but provides no formal validation that the generated effects are exactly equivalent to the completion of all methods for a task. Since the planner and the environment's correctness depend on U, the central claim that HDDLGym generates faithful HDDL-based environments is not currently supported. Please provide a soundness condition or an empirical validation against the IPC-HTN domains listed in Table 2, or revise the semantics to remove tasks only upon successful method decomposition.
[Section 5.5, Algorithm 1] The joint operator combination validity conditions are insufficient for correct multi-agent execution. The algorithm prunes combinations only for two conditions: no agent performing multiple different actions, and no action's effects conflicting with another action's preconditions. It does not specify how simultaneous actions are executed by the step function when two agents' actions have interacting effects or when preconditions must be evaluated against the pre-state or post-state of a joint action. Section 5.1 states only that 'after executing agents' actions' environment actions are applied, leaving the concurrency semantics undefined. In domains like Transport Collab, where agents transfer packages, the order of effect application can determine whether a plan is valid, and the current specification does not guarantee that the planner's chosen joint combination is executable as intended. A precise definition of the transition function for multiple simultaneous agent actions is needed, including how conflicting effects are resolved.
[Sections 1 and 4] The claim of 'automatically generates' Gym environments overstates the actual workflow. The paper requires users to manually add agent type declarations, augment action parameters with agents, add explicit task effects, and include a 'none' action in every domain. While Section 4 mentions an interactive platform with 'autonomous task effect generation' and 'agent parameter augmentation,' no details or correctness guarantees are provided, and the modifications change the semantics of the original HDDL files. This is not merely a presentation issue; it affects the scope of the contribution. Either provide a fully automated transformation pipeline with validation, or clearly scope the tool as a semi-automatic framework that requires domain-specific manual modifications, and state what guarantees hold after those modifications.

minor comments (7)

[Section 4] In the task definition example, the parameter is '?dest' but the effect references '?destination'; this appears to be a typo.
[Algorithm 1] Lines 8–14 are confusing: the while loop condition 'while Oa not empty do' uses an uninitialized Oa and appears to have the intended logic inverted. The loop should continue until a non-empty valid operator set is found, so the condition should likely be 'while Oa is empty'. Please clarify the pseudocode.
[Table 2] The 'Hidden' entry in the Agent-centric column is undefined. Add a footnote explaining what 'hidden' means, e.g., an agent is present but is not an explicit parameter in all actions.
[Section 5.3] The observation space includes 'other agents' previous primitive actions,' but the precise encoding of this feature is not specified. Please provide a formal definition.
[Section 5.4] The paper reports using PPO but does not give hyperparameters (learning rate, network architecture, training steps, clipping parameters). Provide these details for reproducibility.
[Section 6.3 and Table 3] Success rates in Table 3 are reported without the number of episodes or seeds; typical RL evaluations use multiple seeds with standard deviations. Please specify the number of runs or add confidence intervals.
[Section 5.5] The distinction between centralized and decentralized planning is not fully formalized: in decentralized mode, the belief set B is mentioned but no update rule is given beyond function U. Add a description of how beliefs are updated and how the planner is invoked.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: HDDLGym's claims are software-functionality claims; the task-effect completion rule is an unvalidated semantic modification, not a circular reduction.

full rationale

HDDLGym makes no first-principles mathematical predictions; its central claims (parsing HDDL, exposing a Gym API, enabling RL-guided hierarchical planning) are implemented software behaviors. The only candidate for a self-definitional step is the task-completion check of Section 4: 'HDDLGym checks the current world state against the defined task effects. Thus, task definitions must include explicit effects.' That is a nonstandard semantic modification and a genuine soundness limitation, because declared effects need not be equivalent to successful method decomposition, but it is not circular: the planner's objective is exactly to make those declared effects true, and the paper does not fit a parameter and then re-predict it. Likewise, the demonstrations use HDDLGym's own planner and metrics; self-evaluation of a software artifact is a validation limitation, not a derivation from its own conclusion. The only self-citation (Favier et al. 2025, which includes La and Shah) appears in the future-work paragraph on LLM-based domain learning and is not load-bearing for any central claim. Under the hard rule that non-load-bearing self-citation is not circularity, it does not raise the score. No imported uniqueness theorem, ansatz smuggled by citation, or renaming of a known result is present. Score 0: no significant circularity; the task-effect semantics and self-evaluation should be read as correctness and robustness risks, not circular reasoning.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

Most of the framework's machinery is software engineering built on HDDL and Gym. The genuine assumptions are deterministic semantics, the effect-based task completion check, and the agent-type designation rule. No fitted parameters or invented physical entities appear in the paper.

assumptions (4)

domain assumption Standard HDDL syntax and semantics (Höller et al. 2020) are accepted as the formal foundation for hierarchical tasks and methods.
Section 2.1 defines the planning domain and problem using Höller et al.; the tool builds on this formalism without re-verifying it.
domain assumption The environment transition is deterministic and fully specified by HDDL action effects.
Section 7 states HDDLGym assumes a deterministic transition function and predictable action effects; this limits application to stochastic environments.
ad hoc to paper Task completion can be determined by checking declared task effects against the current world state; therefore every task must include an explicit effect.
Section 4 introduces the requirement that task definitions must include explicit effects so HDDLGym can check completion; this is a modification beyond standard HDDL and may not capture all completion semantics.
ad hoc to paper Declaring a type as an agent and adding agent parameters to actions faithfully captures multi-agent interaction in the domain.
Section 4 'Agent type hierarchy ta' describes this agent-centric extension, and Section 5.5 relies on it for centralized and decentralized planning.

how reviews work

0 comments

Cite this review

Pith. "Pith review of HDDLGym: A Tool for Studying Multi-Agent Hierarchical Problems Defined in HDDL with OpenAI Gym." pith.science (2026). https://pith.science/paper/AQNNQG3I

@misc{pith2026250522597,
  author       = {Pith},
  title        = {Pith review of: HDDLGym: A Tool for Studying Multi-Agent Hierarchical Problems Defined in HDDL with OpenAI Gym},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AQNNQG3I}},
  note         = {Machine review of arXiv:2505.22597}
}

read the original abstract

In recent years, reinforcement learning (RL) methods have been widely tested using tools like OpenAI Gym, though many tasks in these environments could also benefit from hierarchical planning. However, there is a lack of a tool that enables seamless integration of hierarchical planning with RL. Hierarchical Domain Definition Language (HDDL), used in classical planning, introduces a structured approach well-suited for model-based RL to address this gap. To bridge this integration, we introduce HDDLGym, a Python-based tool that automatically generates OpenAI Gym environments from HDDL domains and problems. HDDLGym serves as a link between RL and hierarchical planning, supporting multi-agent scenarios and enabling collaborative planning among agents. This paper provides an overview of HDDLGym's design and implementation, highlighting the challenges and design choices involved in integrating HDDL with the Gym interface, and applying RL policies to support hierarchical planning. We also provide detailed instructions and demonstrations for using the HDDLGym framework, including how to work with existing HDDL domains and problems from International Planning Competitions, exemplified by the Transport domain. Additionally, we offer guidance on creating new HDDL domains for multi-agent scenarios and demonstrate the practical use of HDDLGym in the Overcooked domain. By leveraging the advantages of HDDL and Gym, HDDLGym aims to be a valuable tool for studying RL in hierarchical planning, particularly in multi-agent contexts.

Figures

Figures reproduced from arXiv: 2505.22597 by the authors.

**Figure 2.** HDDLGym high-level architecture. Outputs of RL policy help HDDLGym Planner update the action hierarchy of each agent. Then, primitive actions are extracted from the hierarchies, and applied to the environment. Domain Source Agent-centric Collab Transport IPC-HTN Yes No Transport Collab. Modified IPC-HTN Yes Yes Overcooked Ours Yes Yes Rover IPC-HTN Yes No Satellite IPC-HTN Yes No Depots IPC-HTN Hidden No Minecraft-… view at source ↗

**Figure 3.** HTNs of the Overcooked domain. 6.3 Evaluation Metrics Complexity and Difficulty HDDLGym provides a range of metrics to evaluate RL models within hierarchical planning contexts, with particular focus on plan complexity and planning difficulty. Plan complexity can be assessed by examining the dimensionality of the elements defining a problem, as summarized in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Training Dynamics Analysis. Figures A, B, and C show the training losses for the Transport domain with 1, 2, and 3 agents, highlighting longer convergence times as the number of agents increases. Figures D and E display the PPO policy’s training progression for the 1-a…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 25 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Brockman, G.; Cheung, V.; Pettersson, L.; Schneider, J.; Schulman, J.; Tang, J.; and Zaremba, W. 2016. OpenAI Gym. arXiv preprint arXiv:1606.01540

arXiv 2016
[4]

C.; and Bordini, R

Cardoso, R. C.; and Bordini, R. H. 2017. A Multi-Agent Extension of a Hierarchical Task Network Planning Formalism. Advances in Distributed Computing and Artificial Intelligence Journal, 6(2): 5--17

work page 2017
[5]

K.; Griffiths, T.; Seshia, S.; Abbeel, P.; and Dragan, A

Carroll, M.; Shah, R.; Ho, M. K.; Griffiths, T.; Seshia, S.; Abbeel, P.; and Dragan, A. 2019. On the Utility of Learning About Humans for Human- AI Coordination. In Advances in Neural Information Processing Systems (NeurIPS)

work page 2019
[6]

Ducho n , F.; Babinec, A.; Kajan, M.; Be n o, P.; Florek, M.; Fico, T.; and Juri s ica, L. 2014. Path Planning with Modified A Star Algorithm for a Mobile Robot. Procedia Engineering, 96: 59--69

work page 2014
[7]

A.; and Nau, D

Erol, K.; Hendler, J. A.; and Nau, D. S. 1994. UMCP: A Sound and Complete Procedure for Hierarchical Task-network Planning. In Proceedings of the 2nd International Conference on Artificial Intelligence Planning Systems (AIPS)

work page 1994
[8]

Favier, A.; Verma, P.; La, N.; and Shah, J. A. 2025. Leveraging LLMs for Collaborative Human-AI Decision Making. In Proceedings of the AAAI 2025 Spring Symposium on Current and Future Varieties of Human-AI Collaboration

work page 2025

Show all 29 references

[9]

N.; Hiatt, L

Fine-Morris, M.; Hsiao, V.; Smith, L. N.; Hiatt, L. M.; and Roberts, M. 2024. Leveraging LLMs for Generating Document-Informed Hierarchical Planning Models: A Proposal. In AAAI 2025 Workshop on Planning in the Era of LLMs (LM4Plan)

2024
[10]

Goel, S.; Wei, Y.; Lymperopoulos, P.; Chur \'a , K.; Scheutz, M.; and Sinapov, J. 2024. NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Syst...

2024
[11]

Grand, M.; Pellier, D.; and Fiorino, H. 2022. An Accurate HDDL Domain Learning Algorithm from Partial and Noisy Observations. In Proceedings of the IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI)

2022
[12]

H \"o ller, D.; Behnke, G.; Bercher, P.; Biundo, S.; Fiorino, H.; Pellier, D.; and Alford, R. 2020. HDDL: An Extension to PDDL for Expressing Hierarchical Planning Problems. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI)

2020
[13]

IPC 2023 HTN Tracks. 2023. International Planning Competition 2023 HTN Tracks. Available at https://ipc2023-htn.github.io/

2023
[14]

Kovacs, D. L. 2012. A Multi-Agent Extension of PDDL3.1 . In Proceedings of the ICAPS 2012 Workshop on the International Planning Competition (WS--IPC)

2012
[15]

Lamanna, L.; Saetti, A.; Serafini, L.; Gerevini, A.; and Traverso, P. 2021. Online Learning of Action Models for PDDL Planning. In Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI)

2021
[16]

Li, N.; Cushing, W.; Kambhampati, S.; and Yoon, S. 2014. Learning Probabilistic Hierarchical Task Networks as Probabilistic Context-Free Grammars to Capture User Preferences. ACM Transactions on Intelligent Systems and Technology, 5(2)

2014
[17]

Li, R.; Roberts, M.; Fine-Morris, M.; and Nau, D. 2022. Teaching an HTN Learner. In Proceedings of the 5th ICAPS Workshop on Hierarchical Planning (HPlan)

2022
[18]

Liu, M.; Sivakumar, K.; Omidshafiei, S.; Amato, C.; and How, J. P. 2017. Learning for Multi-Robot Cooperation in Partially Observable Stochastic Environments with Macro-Actions. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2017
[19]

S.; and Wilkins, D

McDermott, D.; Ghallab, M.; Howe, A.; Knoblock, C.; Ram, A.; Veloso, M.; Weld, D. S.; and Wilkins, D. 1998. PDDL -- T he Planning Domain Definition Language. Technical Report CVC TR-98-003/DCS TR-1165, Yale Center for Computational Vision and Control

1998
[20]

Ng, J. H. A.; and Petrick, R. P. A. 2019. Incremental Learning of Planning Actions in Model-Based Reinforcement Learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI)

2019
[21]

Sanner, S. 2010. Relational Dynamic Influence Diagram Language ( RDDL ): Language Description. Unpublished ms. Australian National University, 32: 27

2010
[22]

Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347

2017 arXiv
[23]

Silver, T.; and Chitnis, R. 2020. PDDLGym : Gym Environments from PDDL Problems. In ICAPS 2020 Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL)

2020
[24]

Taitler, A.; Gimelfarb, M.; Jeong, J.; Gopalakrishnan, S.; Mladenov, M.; Liu, X.; and Sanner, S. 2023. pyRDDLGym: From RDDL to Gym Environments. In ICAPS 2023 Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL)

2023
[25]

R.; and Srivastava, S

Verma, P.; Marpally, S. R.; and Srivastava, S. 2021. Asking the Right Questions: Learning Interpretable Action Models Through Query Answering. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI)

2021
[26]

R.; and Srivastava, S

Verma, P.; Marpally, S. R.; and Srivastava, S. 2022. Discovering User-Interpretable Capabilities of Black-Box Planning Agents. In Proceedings of the 19th International Conference on Principles of Knowledge Representation and Reasoning

2022
[27]

A.; Wang, R

Wu, S. A.; Wang, R. E.; Evans, J. A.; Tenenbaum, J. B.; Parkes, D. C.; and Kleiman-Weiner, M. 2021. Too Many Cooks: Bayesian Inference for Coordinating Multi-Agent Collaboration. Topics in Cognitive Science, 13(2): 414--432

2021
[28]

Xiao, Y.; Hoffman, J.; and Amato, C. 2020. Macro-Action-Based Deep Multi-Agent Reinforcement Learning. In Proceedings of the 3rd Conference on Robot Learning (CoRL)

2020
[29]

H.; Mu \ n oz-Avila, H.; and Yang, Q

Zhuo, H. H.; Mu \ n oz-Avila, H.; and Yang, Q. 2014. Learning Hierarchical Task Network Domains from Partially Observed Plan Traces. Artificial Intelligence, 212: 134--157

2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Brockman, G.; Cheung, V.; Pettersson, L.; Schneider, J.; Schulman, J.; Tang, J.; and Zaremba, W. 2016. OpenAI Gym. arXiv preprint arXiv:1606.01540

arXiv 2016

[4] [4]

C.; and Bordini, R

Cardoso, R. C.; and Bordini, R. H. 2017. A Multi-Agent Extension of a Hierarchical Task Network Planning Formalism. Advances in Distributed Computing and Artificial Intelligence Journal, 6(2): 5--17

work page 2017

[5] [5]

K.; Griffiths, T.; Seshia, S.; Abbeel, P.; and Dragan, A

Carroll, M.; Shah, R.; Ho, M. K.; Griffiths, T.; Seshia, S.; Abbeel, P.; and Dragan, A. 2019. On the Utility of Learning About Humans for Human- AI Coordination. In Advances in Neural Information Processing Systems (NeurIPS)

work page 2019

[6] [6]

Ducho n , F.; Babinec, A.; Kajan, M.; Be n o, P.; Florek, M.; Fico, T.; and Juri s ica, L. 2014. Path Planning with Modified A Star Algorithm for a Mobile Robot. Procedia Engineering, 96: 59--69

work page 2014

[7] [7]

A.; and Nau, D

Erol, K.; Hendler, J. A.; and Nau, D. S. 1994. UMCP: A Sound and Complete Procedure for Hierarchical Task-network Planning. In Proceedings of the 2nd International Conference on Artificial Intelligence Planning Systems (AIPS)

work page 1994

[8] [8]

Favier, A.; Verma, P.; La, N.; and Shah, J. A. 2025. Leveraging LLMs for Collaborative Human-AI Decision Making. In Proceedings of the AAAI 2025 Spring Symposium on Current and Future Varieties of Human-AI Collaboration

work page 2025