pith. sign in

arxiv: 2605.07707 · v2 · pith:Z7AHPVV2new · submitted 2026-05-08 · 💻 cs.AI

Hierarchical Task Network Planning with LLM-Generated Heuristics

Felipe Meneguzzi , Alexandre Buchweitz , Augusto B. Corr\^ea , Victor Scherer Putrich , Andr\'e Grahl Pereira This is my paper

Pith reviewed 2026-05-11 01:49 UTC · model grok-4.3

classification 💻 cs.AI
keywords hierarchical task network planninglarge language modelssearch heuristicsautomated planningbenchmark domains
0
0 comments X

The pith

LLM-generated heuristics for HTN planning nearly match the coverage of the top specialized planner while cutting search effort on 83 percent of problems.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines whether large language models can produce effective search heuristics for hierarchical task network planning by prompting them with domain details. It tests this approach on six standard total-order HTN benchmarks using the Pytrich planner and measures performance against domain-independent baselines plus the PANDA planner. A sympathetic reader would care because such heuristics could let planners handle complex hierarchical problems with far less hand-engineered knowledge and lower computational cost. Results indicate the LLM versions reach nearly the same number of solved problems as the strongest existing system while speeding up search in most cases.

Core claim

LLM-generated heuristics nearly match the coverage of the best available HTN planner, while substantially reducing search effort on 83% of shared problems.

What carries the argument

Domain-specific prompting of LLMs to generate heuristics that guide task decomposition and search in the Pytrich HTN planner.

Load-bearing premise

That prompting LLMs with domain information produces genuinely useful heuristics that generalize beyond the six tested benchmarks rather than overfitting to them.

What would settle it

Running the same LLM-generated heuristics on new HTN domains outside the original six and finding that coverage drops below PANDA levels or search effort increases on most problems.

Figures

Figures reproduced from arXiv: 2605.07707 by Alexandre Buchweitz, Andr\'e Grahl Pereira, Augusto B. Corr\^ea, Felipe Meneguzzi, Victor Scherer Putrich.

Figure 1
Figure 1. Figure 1: Comparison of LLM and PANDA RCFF on search efficiency and algorithm effects: (a) per-problem expanded nodes (scatter), (b) cumulative coverage versus node-expansion budget (cactus plot), and (c) median expanded nodes by search algorithm for each LLM model. In (a), points below the diagonal indicate an LLM advantage. 5.2 Search Efficiency and Plan Quality On the 125 problems solved by both the LLM virtual b… view at source ↗
Figure 2
Figure 2. Figure 2: Coverage results for LLM vs. PANDA RCFF: per-domain bar chart (a) and per-model heatmap (b). E.2 PANDA RCLMCut This subsection mirrors the RCFF comparison for completeness. Since PANDA RCLMCut solves fewer problems than PANDA RCFF, the shared instance set is smaller in several domains. E.3 Plan Length by Domain [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Win/loss breakdown by domain: LLM vs. PANDA RCFF on expanded nodes (a) and solution size (b). Each bar segment counts shared instances where LLM expands fewer (win), equal (tie), or more (loss) nodes or actions. Barman-BDI Blocksworld-GTOHP Depots Robot Rover-GTOHP Towers Domain 0 20 40 60 80 Average Improvement (%) Average Improvement: LLM over Panda (rc2ff) (when LLM wins) Expanded Nodes Solution Size (a… view at source ↗
Figure 4
Figure 4. Figure 4: Search efficiency detail: LLM vs. PANDA RCFF. Panel (a) shows mean improvement on LLM wins; panel (b) shows the full distribution of node counts per domain. so differences in plan length across planners with different internal search strategies are unsurprising (see also Section 5.2). E.4 Median Expanded Nodes by Model and Algorithm [PITH_FULL_IMAGE:figures/full_fig_p018_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Model ranking and cumulative coverage: LLM vs. PANDA RCFF. Panel (a) compares median search effort per LLM model against the two PANDA variants; panel (b) shows how coverage accumulates as the node budget grows. 10 1 10 2 10 3 10 4 10 5 10 6 Panda (rc2lmc) Expanded Nodes 10 1 10 2 10 3 10 4 10 5 10 6 LLM Expanded Nodes Expanded Nodes: LLM vs Panda (rc2lmc) (Below diagonal = LLM better) Barman-BDI Blockswor… view at source ↗
Figure 6
Figure 6. Figure 6: Per-problem scatter plots: LLM vs. PANDA RCLMCut on expanded nodes (a) and solution size (b). Points below the diagonal indicate LLM advantage. Model interface. The Model object exposes grounded planning objects including facts, operators, abstract_tasks, decompositions, and the goal bitset goals. Node interface. Each HTNNode provides: • node.state: integer bitset encoding the current state; • node.task_ne… view at source ↗
Figure 7
Figure 7. Figure 7: Coverage and search-efficiency summary: LLM vs. PA [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Per-problem scatter plots: LLM vs. TDG on expanded nodes (a) and solution size (b). Points below the diagonal indicate LLM advantage. Barman-BDI Blocksworld-GTOHP Depots Robot Rover-GTOHP Towers Domain 0 5 10 15 20 25 30 Problems Solved 18 23 23 11 27 15 20 30 27 12 27 15 Coverage: Problems Solved per Domain Pytrich (TDG) LLM (any model) (a) Coverage by domain. LLM covers 131 problems; TDG covers only 117,… view at source ↗
Figure 9
Figure 9. Figure 9: Coverage and search-efficiency summary: LLM vs. TDG. [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
read the original abstract

HTN planning is a variation of classical planning where, instead of searching for a linear sequence of actions, an algorithm decomposes higher-level tasks using a method library until only executable actions remain. On one hand, this allows one to introduce domain knowledge that can speed up the search for a solution through the method library. On the other hand, it creates challenges that go beyond those of classical state-space search. While recent research produced a number of heuristics and novel algorithms that speed up HTN planning, these heuristics are not yet as informative as those available in classical planning algorithms. We investigate whether large language models (LLMs) can generate effective search heuristics for HTN planning, extending the methodology of Corr\^ea, Pereira, and Seipp (2025) from classical to hierarchical planning. Using the Pytrich planner on six standard total-order HTN benchmark domains, we evaluate heuristics generated by nine LLMs under domain-specific prompting and compare them against the TDG and LMCount domain-independent baselines and the PANDA planner. Our results show that LLM-generated heuristics nearly match the coverage of the best available HTN planner, while substantially reducing search effort on 83% of shared problems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper extends LLM-based heuristic generation from classical planning to Hierarchical Task Network (HTN) planning. It uses domain-specific prompting with nine LLMs to produce heuristics for the Pytrich planner, evaluated on six standard total-order HTN benchmark domains. These are compared against domain-independent baselines (TDG, LMCount) and the state-of-the-art PANDA planner. The central empirical claim is that LLM heuristics nearly match PANDA's coverage while reducing search effort on 83% of shared problems.

Significance. If the results prove robust, this would be a meaningful contribution by showing that LLMs can produce informative, domain-aware heuristics for HTN planning—an area where heuristic quality has lagged behind classical planning. The work provides a direct empirical head-to-head on fixed benchmarks against independent baselines and extends a prior methodology, offering a practical path to more efficient hierarchical planning without hand-crafted heuristics.

major comments (2)
  1. [Experimental setup] Experimental setup (likely §4 or §5): No exact prompting templates, no ablation removing method-library or task-decomposition details from prompts, and no evaluation on held-out domains are reported. This directly undermines the claim that the heuristics are 'genuinely useful and generalizable' rather than benefiting from benchmark leakage, which is load-bearing for the coverage and effort-reduction results.
  2. [Results] Results section: The 83% effort-reduction figure and 'nearly match PANDA coverage' claim are presented without per-domain breakdowns, variance measures, statistical tests, or definition of the effort metric (nodes expanded, time, etc.). Without these, it is impossible to verify whether improvements are consistent or concentrated in a subset of the six domains.
minor comments (2)
  1. [Introduction] The abstract and introduction could more explicitly state the precise differences from Corrêa et al. (2025) in the prompting and HTN-specific adaptations.
  2. [Methods] The nine LLMs are mentioned but not identified by version, size, or access method; adding this would improve reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback. We appreciate the opportunity to clarify our methodology and strengthen the presentation of results. We address each major comment below and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [Experimental setup] Experimental setup (likely §4 or §5): No exact prompting templates, no ablation removing method-library or task-decomposition details from prompts, and no evaluation on held-out domains are reported. This directly undermines the claim that the heuristics are 'genuinely useful and generalizable' rather than benefiting from benchmark leakage, which is load-bearing for the coverage and effort-reduction results.

    Authors: We agree that exact prompting templates are essential for reproducibility and will add them verbatim to an appendix in the revised manuscript. Our domain-specific prompts are intentionally constructed to include method-library and task-decomposition information because these elements are core to HTN planning; we will expand the methodology section to explain this design rationale and contrast it with the domain-independent baselines (TDG and LMCount). We did not perform explicit ablations in the current study, but the consistent outperformance over those baselines provides indirect evidence of the value of the hierarchical details. For held-out domains, the six benchmarks are the established standard set in the HTN literature and exhibit diversity in structure and size; we will add an explicit discussion of potential benchmark leakage risks and list evaluation on held-out domains as future work. revision: partial

  2. Referee: [Results] Results section: The 83% effort-reduction figure and 'nearly match PANDA coverage' claim are presented without per-domain breakdowns, variance measures, statistical tests, or definition of the effort metric (nodes expanded, time, etc.). Without these, it is impossible to verify whether improvements are consistent or concentrated in a subset of the six domains.

    Authors: We apologize for the insufficient detail in the original presentation. The effort metric is the number of nodes expanded (with runtime reported as a secondary measure). In the revised results section we will (1) explicitly define the metric, (2) add a table with per-domain coverage and effort-reduction percentages, (3) report variance (standard deviation across problems within each domain), and (4) include statistical significance tests (Wilcoxon signed-rank tests) comparing LLM heuristics against the baselines. These additions will show that the aggregate 83% figure is not driven by a small subset of domains. revision: yes

Circularity Check

0 steps flagged

Minor self-citation to prior methodology extension; results remain independent empirical evaluation

full rationale

The paper extends the prompting methodology from Corrêa, Pereira, and Seipp (2025) (with author overlap) to HTN planning but makes no load-bearing use of that citation for its central claims. Instead, it reports new head-to-head coverage and search-effort results on six fixed total-order HTN benchmarks against independent baselines (TDG, LMCount, PANDA). No equations, fitted parameters, or predictions reduce to inputs by construction; the evaluation is falsifiable via the stated metrics on standard domains. This qualifies as one minor self-citation that is not load-bearing, yielding a low circularity score.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Empirical evaluation paper; central claim rests on standard planning assumptions and LLM capabilities rather than new axioms or fitted parameters.

axioms (1)
  • domain assumption Total-order HTN planning benchmarks are representative of practical hierarchical planning problems
    Invoked when generalizing results from the six standard domains to broader HTN use.

pith-pipeline@v0.9.0 · 5526 in / 1032 out tokens · 34031 ms · 2026-05-11T01:49:41.166014+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.