REVIEW 4 major objections 4 minor 29 references
A Generative Partially Specified Finite State Machine Approach to Complex Behaviour Planning
T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read The paper argues that finite state machines are a more reliable intermediate representation than behaviour trees for LLM-generated robot plans, and presents a working generative FSM stack that outperforms a leading BT-based system.
desk verdict Useful systems paper with a novel generative FSM stack, but the headline FSM-vs-BT comparison is confounded and the abstract overstates the results. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Partially Specified Finite State Machine: a directed graph of robot capabilities as states, with success/failure events triggering transitions, encoded as an XML execution plan. 'Partially specified' means the graph need not enumerate every contingency; unconnected events can be treated as 'don't care' or routed to a fallback such as 'regenerate.' This is what lets the LLM produce a plan that is executable yet tolerant of unknowns. The supporting machinery is the three-part stack: an FSM engine (Fabric) that parses and validates plans and converts them into runtime connections; a capability system that separates starting a behaviour from triggering it, adds event-ba
What would settle it
Take one navigation task, one LLM, and two fixed prompt templates that are identical except that one asks for a behaviour-tree plan and the other for a finite-state-machine plan, using the same capability names and one-shot examples, then validate both with the same checker over many runs. If success rates converge, the claimed FSM advantage is an artifact of the systems compared, not of the representation.
Extended reading notes
Core claim
The central claim is that a Generative Partially Specified Finite State Machine (GPSFSM) — an FSM whose transitions may be left unspecified, with states and events described in a text format — lets a large language model compose robot behaviours more reliably than the hierarchical, continuously ticking Behaviour Tree representation. The paper argues that FSMs' explicit states and event-triggered transitions align more naturally with natural-language task descriptions, so generation requires less prompt engineering. It supports this with a stack—an FSM engine that parses, validates, and executes plans; a capability registry extended with asynchronous events and runtime parameters; and a promp
Load-bearing premise
The comparison is assumed to isolate the effect of FSM versus BT representation, even though the two systems differ in prompts, capability descriptions, context length, execution engine, and fine-tuning; if any of those drive the gap, the central claim about representation weakens.
Editorial extensions
If this is right
- If FSM representations are as easy for LLMs to generate as the experiments suggest, zero-shot behaviour planning becomes practical for cloud-language-model-equipped robots, reducing reliance on fine-tuned local models.
- The event-triggered, partially specified graph means a planner does not have to pre-enumerate every failure branch; it can delegate contingencies to a recovery or regeneration mechanism, which is exactly what long-horizon tasks require.
- Standardised semantic capability descriptions, combined with runtime loading and unloading of behaviours, allow the same generated plan to run on different robot platforms and to conserve resources by keeping inactive behaviours unloaded.
- The demonstrated complex behaviours (navigation plus speech, perception, and repetition) suggest the same representation can scaffold multi-modal tasks, not just waypoint following.
- A direct corollary is that future generative planners can choose between BT and FSM target representations based on task structure, using FSM when event-driven sequencing is natural and BT when hierarchical fallbacks dominate.
Reading between the lines
- The real driver of the zero-shot gains may be 'partial specification' rather than FSMs per se: admitting unknown transitions lets the LLM stop doing exhaustive contingency planning, and a BT target may penalise incompleteness by design. A controlled study that varies only the amount of unspecified structure would separate these.
- The same state-machine serialisation could be applied outside robotics, e.g., to LLM-generated workflow or protocol descriptions, wherever a text plan must be validated against a known set of capabilities before execution.
- A cheap test of the representation hypothesis: ask the same LLM to translate a fixed set of natural-language tasks into FSM and BT formats with matched vocabulary and then score plan validity; if FSM wins without an execution environment, the advantage is linguistic rather than architectural.
- The paper's local-model failures point to prompt-length limits rather than representational limits; extending the FSM prompt to be token-efficient (e.g., via abbreviations or retrieval) might close the gap on resource-constrained robots.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Generative Partially Specified Finite State Machine (GPSFSM) architecture for robot behaviour planning, comprising three ROS2 components: Fabric (an FSM engine that parses, validates and executes XML behaviour plans with Sequential, Recovery, Parallel-Any and Parallel-All control structures), an extension of Capabilities2 with start-trigger separation, an event subsystem and parameter injection, and PromptTools for interfacing local/cloud LLMs. The authors compare plan-generation success rates and latency against BTGenBot on five navigation tasks using local and GPT models, and demonstrate three complex behaviours on a Turtlebot4. The central claim is that GPSFSM's simpler state-based semantics make LLM generation more reliable than behaviour trees, especially in zero-shot settings.
Significance. If the central comparison were controlled, the paper would make a useful contribution: the open-source release of a complete generative-FSM stack, the event-driven execution model, and the standardised capability descriptions are solid engineering contributions. The reported zero-shot GPT success rates are striking and deserve a careful follow-up. However, the experimental evidence as written does not isolate the representation (FSM vs BT) from many other simultaneously varied factors, and the paper's own local-model results contradict the abstract's 'consistently higher' claim. The system itself appears coherent, but the headline comparative claim is not currently supported.
major comments (4)
- [Section IV.A.1–IV.A.2 and Section V] The benchmark cannot support the attribution in the conclusion that 'Fabric’s simpler state-based semantics facilitate more reliable LLM reasoning'. The two systems differ simultaneously in at least four ways: (i) prompt content (Fabric's prompt is restricted to Nav2 waypoints, while BTGenBot's oneshot prompt is modified to add recovery functionality the original lacked); (ii) output schema and execution engine (Fabric XML on the Fabric engine vs BehaviorTree.CPP XML); (iii) capability layer (Capabilities2 with parameter injection vs BTGenBot's ad-hoc descriptions); and (iv) model adaptation (BTGenBot's fine-tuned local models are used, while Fabric receives no fine-tuning). Any observed success-rate difference could be caused by any of these factors. The statement in IV.A.2 that the results 'highlight that the modular prompting approach... is successful' is a non-sequitur unless these f
- [Abstract and Section IV.A.2] The claim of 'consistently higher plan-generation success rates' is contradicted by the paper's own local-model results. Section IV.A.2 states: 'In local models, BTGenBot performs better (13% success, 35% partial, 52% failed) than Fabric (10% success, 22% partial, 68% failed).' Table I also shows runs where BTGenBot outperforms Fabric (e.g., Llamachat oneshot: 17/27/6 vs 0/0/50). The abstract and conclusion should be qualified to GPT models, or the claim to 'consistently higher' should be removed.
- [Table I and Section IV.A.1] The treatment of corrupted outputs is inconsistent. Section IV.A.1 says Failed categories include corrupted plans, but the Table I footnote says 'corrupted results excluded' and N can vary. For Codellama base/finetuned oneshot, Fabric has N=0 with no timing, which can only mean every run was corrupted and discarded. In the Llamachat oneshot rows, Fabric has N=50 with S/P/F = 0/0/50, while the text says 'both codellama and llamachat returned the prompt example' and 'These responses are indicated as corrupted using x' — yet the table shows no x and counts those runs as Failed rather than excluding them. This asymmetry changes the aggregated S/P/F and timing values. The paper must define a single treatment for corrupted outputs, report the number of runs before exclusion, and apply the same rule to both systems and all rows.
- [Equations (1)–(3) and Table I] No uncertainty quantification or significance testing is provided for the central comparisons. The counts are large enough to allow standard binomial confidence intervals or a chi-square test, and the latency comparisons involve substantial standard deviations (e.g., Codellama base zero-shot: 23.04±9.21 vs 26.44±13.55; GPT-5 oneshot: 16.15±8.47 vs 9.66±3.67). Without such tests, statements like 'comparable or lower planning latency' and 'consistently higher' are not supported by the data as reported.
minor comments (4)
- [Section III (Methodology)] The paper states 'While the quantifiable impact of these design choices has yet to be formally evaluated' regarding dynamic loading/unloading and resource goals. This is a useful limitation disclosure, but it should also appear in the conclusions or future-work section to avoid overclaiming these design goals as demonstrated benefits.
- [Table I] The column header 'S/P/Fixed' appears to be a typo; it should be 'S/P/F'. In addition, the footnote mentions 'corrupted (x) rows' but no x symbols appear in the table, which contributes to the inconsistency noted in the major comments.
- [Section IV.A.1] The text says '10 iterations per task in zero-shot and one-shot prompt modes, across 7 LLM types', but the table lists six model rows (Codellama base/finetuned, Llamachat base/finetuned, GPT-4o, GPT-4.1, GPT-5). Clarify whether the 7th type is a model version of a different backbone or whether the count includes both base and finetuned as separate types.
- [Section IV.B] The complex-behaviour experiments are demonstrations without a comparison system, which is acknowledged. It would be helpful to state explicitly what would have been a fair baseline and what validation was performed beyond 'the robot executed the plan'.
Circularity Check
No significant circularity: the central claim is an external empirical comparison, and self-citations are infrastructural rather than load-bearing.
full rationale
The paper's core claim is an empirical benchmark comparison against the external BTGenBot system (Section IV.A), with success rates measured by human evaluators on generated plans. None of the compared quantities are defined in terms of the claimed conclusion, and no equation reduces to its own inputs or relabels a fitted parameter as a prediction. The only notable self-citation is [16], the authors' prior Capabilities2 package, used as a software base layer and extended in Section III; this is a reproducible system component, not an argument that FSMs outperform BTs. The paper explicitly notes that some design-goal impacts 'have yet to be formally evaluated' and that the complex-behaviour experiment is a capability showcase rather than a comparative evaluation, which are honest limitations rather than circular moves. The benchmark comparison is confounded by simultaneous changes in prompts, output schemas, execution engines, capability layers, and fine-tuning, so the attribution of performance differences to 'simpler state-based semantics' is under-supported; however, that is an experimental-soundness concern, not circularity. Accordingly, no circular step is identified and the score is 0.
Assumptions & free parameters
assumptions (5)
- domain assumption Valid generated plan (syntactic+semantic correctness) is a meaningful proxy for behaviour-planning quality.
- domain assumption The benchmark comparison controls all variables except the FSM/BT representation.
- domain assumption Capabilities2's semantic capability descriptions improve LLM plan generation.
- domain assumption Human evaluator classification of plans is reliable and unbiased.
- domain assumption Navigation tasks are representative of complex long-horizon behaviour planning.
Cite this review
Pith. "Pith review of A Generative Partially Specified Finite State Machine Approach to Complex Behaviour Planning." pith.science (2026). https://pith.science/paper/G6HN33LA
@misc{pith2026260715674,
author = {Pith},
title = {Pith review of: A Generative Partially Specified Finite State Machine Approach to Complex Behaviour Planning},
year = {2026},
howpublished = {\url{https://pith.science/paper/G6HN33LA}},
note = {Machine review of arXiv:2607.15674}
}
read the original abstract
Autonomous robots operating in dynamic environments require behaviour planning systems that combine reactivity, interpretability, and adaptability. While Large Language Models have been successfully integrated with Behaviour Trees for dynamic replanning, Finite State Machines, despite their widespread adoption and computational efficiency, remain unexplored for generative approaches. We propose a Generative Partially Specified Finite State Machine (GPSFSM) neurosymbolic architecture that utilises the symbolic and semantic structure of FSMs, including states and event-triggered transitions, to implement Behaviour Planning. This paper introduces the first GPSFSM framework for robotics, featuring Fabric, an FSM engine that parses, validates, and executes behaviour plans that contain Sequential, Recovery, Parallel-Any, and Parallel-All control structures. We extend the Capabilities2 package in ROS2 with an asynchronous event system for behaviour chaining and runtime parameter injection for configurable execution, addressing the ad-hoc function representations that limit current generative systems. PromptTools provides a unified ROS 2 interface to local and cloud LLMs, with prompt buffering, enabling dynamic asynchronous composition of task and context information. Together, these components enable standardised semantic capability descriptions for robot-agnostic development. Experimental evaluation on navigation tasks demonstrates that our GPSFSM approach achieves consistently higher plan-generation success rates than the state-of-the-art BTGenBot system, particularly excelling in zero-shot scenarios where BTs typically struggle, while maintaining comparable or lower planning latency to frontier LLMs. We also demonstrate that our system can generate complex behaviours. We release an open-source ROS2 stack that makes generative FSM planning practical and reproducible for robotic systems.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Rein- forcement Learning,
C. Chen, Y . Liu, S. Kreiss, and A. Alahi, “Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Rein- forcement Learning,” in2019 International Conference on Robotics and Automation (ICRA), (Montreal, QC, Canada), pp. 6015–6022, IEEE, May 2019
2019
-
[2]
Decentralized structural-rnn for robot crowd navigation with deep reinforcement learning,
S. Liu, P. Chang, W. Liang, N. Chakraborty, and K. Driggs-Campbell, “Decentralized structural-rnn for robot crowd navigation with deep reinforcement learning,” in2021 IEEE International Conference on Robotics and Automation (ICRA), p. 3517–3524, IEEE Press, 2021
2021
-
[3]
Rt-1: Robotics transformer for real-world control at scale,
A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. Joshi, R. Julian, D. Kalash- nikov, Y . Kuang, I. Leal, K.-H. Lee, S. Levine, Y . Lu, U. Malla, D. Manjunath, I. Mordatch, O. Nachum, C. Parada, J. Peralta, E. Perez, K. Pertsch, J....
arXiv 2022
-
[4]
Rt- 2: Vision-language-action models transfer web knowledge to robotic control,
B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, Q. Vuong, V . Vanhoucke, H. Tran, R. Soricut, A. Singh, J. Singh, P. Sermanet, P. R. Sanketi, G. Salazar, M. S. Ryoo, K. Reymann, K. Rao, K. Pertsch, I. Mordatch, H. Michalewski, Y . Lu, S. Levine, L. Lee, T.-W. E. Lee, I. Leal, Y . Kuang, D. Kalashnikov, R. Julia...
2023
-
[5]
Safevla: Towards safety alignment of vision-language-action model via constrained learning,
B. Zhang, Y . Zhang, J. Ji, Y . Lei, J. Dai, Y . Chen, and Y . Yang, “Safevla: Towards safety alignment of vision-language-action model via constrained learning,” 2025
2025
-
[6]
Improving pre-trained vision-language-action policies with model- based search,
C. Neary, O. G. Younis, A. Kuramshin, O. Aslan, and G. Berseth, “Improving pre-trained vision-language-action policies with model- based search,” 2025
2025
-
[7]
A survey of behavior trees in robotics and ai,
M. Iovino, E. Scukins, J. Styrud, P. ¨Ogren, and C. Smith, “A survey of behavior trees in robotics and ai,”Robotics and Autonomous Systems, vol. 154, p. 104096, 2022
2022
-
[8]
Learning behavior trees with genetic programming in unpredictable environments,
M. Iovino, J. Styrud, P. Falco, and C. Smith, “Learning behavior trees with genetic programming in unpredictable environments,” in2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 4591–4597, 2021
2021
Show all 29 references
-
[9]
Do as i can, not as i say: Grounding language in robotic affordances,
b. ichter, A. Brohan, Y . Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian, D. Kalashnikov, S. Levine, Y . Lu, C. Parada, K. Rao, P. Sermanet, A. T. Toshev, V . Vanhoucke, F. Xia, T. Xiao, P. Xu, M. Yan, N. Brown, M. Ahn, O. Cortes, N. Si...
2023
-
[10]
Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning,
K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suen- derhauf, “Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning,” inProceedings of The 7th Conference on Robot Learning(J. Tan, M. Toussaint, and K. Darvish, eds.), v...
2023
-
[11]
Llm-brain: Ai-driven fast generation of robot behaviour tree based on large language model,
A. Lykov and D. Tsetserukou, “Llm-brain: Ai-driven fast generation of robot behaviour tree based on large language model,” in2024 2nd International Conference on Foundation and Large Language Models (FLLM), pp. 392–397, 2024
2024
-
[12]
BTGenBot: Behavior Tree Generation for Robotic Tasks with Lightweight LLMs,
R. A. Izzo, G. Bardaro, and M. Matteucci, “BTGenBot: Behavior Tree Generation for Robotic Tasks with Lightweight LLMs,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 9684–9690, Oct. 2024. ISSN: 2153-0866
2024
-
[13]
LLM-BT: Performing Robotic Adaptive Tasks based on Large Language Models and Behav- ior Trees,
H. Zhou, Y . Lin, L. Yan, J. Zhu, and H. Min, “LLM-BT: Performing Robotic Adaptive Tasks based on Large Language Models and Behav- ior Trees,” in2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 16655–16661, May 2024
2024
-
[14]
On the implementation of behaviour trees in robotics,
M. Colledanchise and L. Natale, “On the implementation of behaviour trees in robotics,”IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 5929–5936, 2021
2021
-
[15]
ROS Topics: Capabilities [ROS Topics],
W. Woodall, M. Liebhardt, D. Stonier, and J. Binney, “ROS Topics: Capabilities [ROS Topics],”IEEE Robotics & Automation Magazine, vol. 21, pp. 14–15, Dec. 2014
2014
-
[16]
Capabilities2 for ROS2: Advanced Skill-Based Control for Human- Robot Interaction,
M. Pritchard, K. Ratnayake, B. Gamage, M. Jayasuriya, and D. Herath, “Capabilities2 for ROS2: Advanced Skill-Based Control for Human- Robot Interaction,” in2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 1067–1071, Mar. 2025
2025
-
[17]
Comparison between behavior trees and finite state ma- chines,
M. Iovino, J. F ¨orster, P. Falco, J. Jen Chung, R. Siegwart, and C. Smith, “Comparison between behavior trees and finite state ma- chines,”IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 21098–21117, 2025
2025
-
[18]
ROS 2-Based Flexible Behavior Engine for Flexible Navigation,
J. M. Zutell, D. C. Conner, and P. Schillinger, “ROS 2-Based Flexible Behavior Engine for Flexible Navigation,” inSoutheastCon 2022, (Mobile, AL, USA), pp. 674–681, IEEE, Mar. 2022
2022
-
[19]
Yasmin: Yet another state machine library for ros 2,
M. ´A. G. Santamarta, F. J. Rodr ´ıguez-Lera, C. F. Llamas, F. Mart ´ın, and V . M. Olivera, “Yasmin: Yet another state machine library for ros 2,” inROBOT, 2022
2022
-
[20]
Code as Policies: Language Model Programs for Embodied Control,
J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as Policies: Language Model Programs for Embodied Control,” May 2023. arXiv:2209.07753 [cs]
2023 arXiv
-
[21]
Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance,
J. Zhang, J. Zhang, K. Pertsch, Z. Liu, X. Ren, M. Chang, S.-H. Sun, and J. J. Lim, “Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance,” Aug. 2023
2023
-
[22]
Au- tomatic behavior tree expansion with llms for robotic manipulation,
J. Styrud, M. Iovino, M. Norrl ¨of, M. Bj ¨orkman, and C. Smith, “Au- tomatic behavior tree expansion with llms for robotic manipulation,” in2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 1225–1232, 2025
2025
-
[23]
Llm-as-bt-planner: Leveraging llms for behavior tree generation in robot task planning,
J. Ao, F. Wu, Y . Wu, A. Swiki, and S. Haddadin, “Llm-as-bt-planner: Leveraging llms for behavior tree generation in robot task planning,” in2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 1233–1239, 2025
2025
-
[24]
VLM-driven Behavior Tree for Context-aware Task Planning,
N. Wake, A. Kanehira, J. Takamatsu, K. Sasabuchi, and K. Ikeuchi, “VLM-driven Behavior Tree for Context-aware Task Planning,” Jan
-
[25]
Look before you leap: Using seri- alized state machine for language conditioned robotic manipulation,
T. Mu, Y . Liu, and M. Armand, “Look before you leap: Using seri- alized state machine for language conditioned robotic manipulation,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8096–8102, 2025
2025
-
[26]
Unleashing the power of llm to infer state machine from the protocol implementation,
H. Wei, L. Chen, Z. Du, Y . Wu, H. Huang, Y . Liu, G. Cheng, F. Xu, L. Wang, and B. Mao, “Unleashing the power of llm to infer state machine from the protocol implementation,” in2025 IEEE/ACM 33rd International Symposium on Quality of Service (IWQoS), pp. 1–10, 2025
2025
-
[27]
Can large language models help developers with robotic finite state machine modifica- tion?,
X. Gan, Y . R. Song, N. Walker, and M. Cakmak, “Can large language models help developers with robotic finite state machine modifica- tion?,”ArXiv, vol. abs/2412.05625, 2024
2024 arXiv
-
[28]
Behavior trees and state machines in robotics applications,
R. Ghzouli, T. Berger, E. B. Johnsen, A. Wasowski, and S. Dragule, “Behavior trees and state machines in robotics applications,”IEEE Transactions on Software Engineering, vol. 49, no. 9, pp. 4243–4267, 2023
2023
-
[2025]
arXiv:2501.03968 [cs]
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.