Pith. sign in

REVIEW 3 major objections 4 minor 37 references

PIANIST: Learning Partially Observable World Models with LLMs for Multi-Agent Decision Making

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A large language model can write a working world model for a game from its rules alone, and plan with it as well as the true simulator.

desk verdict Useful decomposition idea, but the experiments do not validate the partial-observability world-model claim. read the letter →

arxiv 2411.15998 v1 pith:VL32APO7 submitted 2024-11-24 cs.AI cs.LGcs.MA

Jonathan Light , Sixue Xing , Yuanzhe Liu , Weiqin Chen , Min Cai , Xiusi Chen , Guanzhi Wang , Wei Cheng
show 2 more authors
Yisong Yue Ziniu Hu
This is my paper · ORCID
classification cs.AIcs.LGcs.MA
keywords PIANISTlargelanguagemodelsworldMonteCarlotreesearchpartialobservabilitymulti-agentdecisionmakingzero-shotgenerationPOMDP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a large language model can build a usable world model for a multi-agent game from just a natural-language rulebook and an observation format, with no domain-specific training and no hand-written simulator. It proposes PIANIST, a decomposition of a partially observable world model into seven Python-coded components, and couples it with Monte Carlo tree search to plan in GOPS and a text version of Taboo. The reported experiments indicate that the LLM-generated world model plans about as well as the ground-truth simulator, and better than asking the LLM directly for actions, although it does not beat human Taboo players as the clue-giver. If true, this would let agents acquire planning-capable simulators for new games or tasks by prompting, instead of programming them.

What carries the argument

The load-bearing object is the seven-component decomposition, especially the information realization function $I: I \to S$ that maps an information set to a concrete hidden state so MCTS can simulate transitions between hidden states rather than between observation sets. The transition and reward are combined into one function to reduce LLM errors; the action function uses the LLM to propose the top $k$ actions, including text actions; and UCT values are averaged over the information set weighted by visit counts so search does not rely on hidden information. The generated Python code is checked by execution and repaired with a reflexion-style loop.

What would settle it

Open Appendix C and inspect the generated GOPS CustomInformationPrior: if the hidden state it returns contains player_1_hand copied from the information set, the planner is being handed the opponent's full hand, which would falsify the partial-observability claim. A cleaner test is to compare PIANIST's win rate against a version of the realization function that samples opponent hands uniformly from those consistent with public observations; if the rates differ, the original was exploiting leaked information.

Watch

Extended reading notes

Core claim

The central claim is that the bottleneck in using LLMs for planning is not world knowledge but the format in which that knowledge is extracted. PIANIST asks the LLM to write seven components, including an information set representation, a hidden state, an actor list, an action enumerator, a combined transition-and-reward function, an information partition function, and an information realization function, all inheriting from a fixed template, and then runs MCTS over the generated simulator. The experiments compare this LLM-generated world model against the true game engine and against an LLM used directly as a policy; the generated model matches the true engine's planning quality and beats direct LLM action selection in both games. The paper interprets this as evidence that LLMs can supply accurate, executable world models zero-shot for partially observable multi-agent decision making.

Load-bearing premise

The claim depends on the generated information realization function never injecting private information into the planner's hidden state; the GOPS code in Appendix C appears to do exactly that by copying the opponent's hand into the hidden state, which would turn the experiments into full-information planning.

Editorial extensions

If this is right

  • An agent can be dropped into a new game or task described in natural language and receive a searchable world model in the same session, without collecting data or training a dynamics model.
  • Model-based planning becomes available in settings where no true simulator exists, because the LLM-written transition, reward, and action functions are executable code.
  • Replacing a hand-coded simulator with an LLM-generated one does not degrade MCTS planning quality in the two tested games, suggesting the extracted world model is capturing the relevant rules.
  • Language-action games with an effectively infinite action space become searchable by letting the LLM propose a small action set and the search algorithm assign probabilities, rather than letting the LLM commit to a single action.
  • The modularity of the decomposition lets each component be tested and regenerated independently, and the reflexion loop repairs failing components without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Appendix C GOPS realization function is used as shown, the planner's hidden state contains player 1's full hand, so the GOPS results would demonstrate full-information planning rather than partial-observability planning; a leakage-free version that samples opponent hands consistent with public observations would be the real test.
  • The same prompting scheme could in principle produce world models for negotiation, resource allocation, or embodied task planning, with the main risk shifting from simulator availability to the LLM's code-generation correctness.
  • A natural extension would be to have the LLM generate a prior distribution over hidden states for the realization function, so partial information is handled by sampling instead of by a single deterministic mapping; this would also make the partial-observability claim directly testable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes PIANIST, a framework that uses an LLM to generate the components of a partially observable multi-agent world model as Python code, starting from only a natural language game description and a provided representation of observations. The seven components are information sets, hidden states, actors, action enumeration, transition-reward, information partition, and information realization. The generated model is combined with Monte Carlo Tree Search using UCT, information-set averaging, and a value heuristic. Experiments are conducted on GOPS (6- and 12-card) and a text version of Taboo, comparing against ground-truth models, direct LLM policies, and humans. The paper claims that the LLM-generated world model performs similarly to ground-truth models and enables effective zero-shot planning.

Significance. If the central claim were established, the contribution would be significant: LLMs could produce executable world models for novel partially observable multi-agent games without hand-coded simulators or training, and the modular decomposition is a practical and reasonably motivated design. The paper also usefully demonstrates the integration of LLM-generated components with MCTS and provides concrete code examples. However, the current empirical evidence does not establish the claim. The main comparison confounds world-model accuracy with a different value heuristic between the two agents, and the appendix's GOPS information-realization example appears to leak the full hidden state into the planner's information set. The framework itself is interesting, but the paper as written does not validate the partial-observability world-model generation that is its stated contribution.

major comments (3)
  1. [Section 4, Table 1] The comparison in Table 1 is confounded. The text states that the ground-truth agent uses 'a random-rollout value heuristic' while PIANIST uses 'an LLM-generated value heuristic'. Therefore the near-identical win rates of PIANIST and the ground-truth agent cannot be attributed to the accuracy of the generated world model; they may reflect the value heuristic instead. The conclusion that 'PIANIST performs similarly to ground-truth models, indicating that the LLM can generate an accurate world model' is not supported by this experimental design. To support the claim, both agents should use the same value heuristic (for example, both random rollouts or both the LLM heuristic), or the contribution of the world model should be isolated.
  2. [Appendix C, CustomInformationPrior (GOPS)] The 'Example LLM Generated Information Realization Function (GOPS)' copies player_0_hand, player_1_hand, prize_deck, and all other hidden-state fields directly from the information_set argument. For this code to execute without an AttributeError, the planner's information set must contain the opponent's full hand and the prize deck, i.e., effectively the full hidden state. This contradicts the partial-observability formulation in Section 2.1 and the information-set averaging mechanism in Section 3.2, since there is no hidden information left to protect. Consequently, the GOPS results in Table 1 do not validate a partially observable world model; they validate at most a full-information version of the game. If the appendix code is only illustrative rather than the actual setup, the paper must document the real information-set representation for GOPS and show that the information realization function samples private cards without leaking them.
  3. [Appendix C, CustomForwardTransitor (GOPS)] The example forward dynamics model does not correctly implement the GOPS rules described in Section 2.2. In the tie case, the code increments contested_points by 1, treating it as a count of tied rounds rather than carrying over the value of the contested prize card(s), and it does not record the current prize card in played_prize_cards for later resolution. As a result, the example generated transition-reward function is not an accurate world model for GOPS. If this code is representative of the models used in the experiments, the 'accurate world model' claim is further undermined; if it is not representative, the actual executed code should be included so that readers can verify correctness.
minor comments (4)
  1. [Sections 2.2 and D] The text refers to the proposed method as 'OMEGA ZERO' in several places (e.g., 'We evaluate the performance of OMEGA ZERO compared to other algorithms' and 'Our proposed method, OMEGA ZERO'), which appears to be leftover from a different draft and should be corrected to 'PIANIST' throughout.
  2. [Section 3.1 and Appendix B] The 'reflexion approach' for correcting generated code is mentioned but not specified: the paper should report the number of reflection iterations, the success rate of code generation, and whether the reported experiments use the first generated version or a corrected version. Without this information, the 'zero-shot' claim is ambiguous and the results are difficult to reproduce.
  3. [Section 4] Key MCTS hyperparameters are not reported, including the simulation budget, the UCT exploration constant C, the discount factor gamma, and the action enumeration budget k. These parameters are free in the method and can materially affect win rates, so their absence limits reproducibility.
  4. [Tables 1 and 2] The Taboo experiments use only 15 games, and the reported confidence intervals overlap in several comparisons (e.g., 60.0 percent versus 53.3 percent in Table 1). No significance tests are reported, so claims of 'performs similarly' or 'strong performance' are not statistically substantiated for that game.

Circularity Check

1 steps flagged · score 6.0 of 10

The GOPS information realization function copies the opponent's hand and prize deck directly out of the information set, making the 'partially observable world model' an identity mapping by construction.

  1. self definitional [Appendix C, 'Example LLM Generated Information Realization Function (GOPS)'; Section 4, Table 1]
    "class CustomInformationPrior(InformationPrior): ... player_0_hand = information_set.player_0_hand, player_1_hand = information_set.player_1_hand, prize_deck = information_set.prize_deck, ... return HiddenState(player_0_hand=player_0_hand, player_1_hand=player_1_hand, prize_deck=prize_deck, ...)"

    The information realization function I is defined in Section 3.1 as mapping information sets to their most likely hidden states. In the only concrete GOPS example, every HiddenState field is copied verbatim from an identically named information_set field, including the opponent's hand and the prize deck. For this code to execute, the planner's information set must already contain the full hidden state. The information partition P is therefore the identity, and the realization function is a copy operation rather than an inference over partial observations.

full rationale

The paper's core pipeline — prompting an LLM to generate PIANIST components, executing that code inside MCTS, and comparing against ground-truth models — is not circular in the usual statistical sense: there is no evidence that win-rate numbers were used to fit the generated code, and no load-bearing self-citation appears in the derivation chain. The circularity is concentrated in the claim that PIANIST learns a partially observable world model. In Appendix C, the generated GOPS information realization function sets player_1_hand and prize_deck directly from the information set, so the hidden state equals the observation; the partial-observability machinery is never exercised. The Taboo experiments do not repair this gap, because the agent plays as the clue-giver (Section 4), who legitimately sees the clue word and taboo words, leaving no private hidden state to realize. The comparison against ground-truth models also confounds the LLM-generated value heuristic with the LLM-generated transition model, though this is an experimental-design issue rather than a circular step. Overall, one central claim — accurate partially observable world-model generation — reduces by construction in the presented evidence, while the code-generation and full-information gameplay results retain independent content; hence a partial circularity score of 6.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the correctness of LLM-generated code and on evaluation assumptions. No fitted parameters are disclosed; the listed hyperparameters are not reported and could affect outcomes. The approach introduces no new physical entities.

free parameters (4)
  • UCT exploration constant C = not reported
    C in Equation 1 controls the exploration-exploitation tradeoff; no value or tuning procedure is given.
  • discount factor gamma = not reported
    gamma in Equations 1 and 3 balances immediate and future rewards; value is not stated.
  • MCTS simulation budget = not reported
    The number of MCTS iterations N used in experiments is not reported, and it affects result quality.
  • action enumeration budget k = not reported
    The action function returns top k most likely actions (Section 3.1); k is not specified for either game.
assumptions (4)
  • domain assumption The LLM includes correct rules and strategic knowledge for GOPS and Taboo, so a zero-shot prompt can produce an executable world model.
    The method relies on the LLM knowing these specific games; stated in Sections 1 and 4.
  • domain assumption The provided Python parent template classes and game descriptions are sufficient to generate correct code with no manual fixes beyond the described reflexion loop.
    Section 3.1 and Appendix E assume a reflexion step corrects errors; no failure rates or manual intervention details are given.
  • standard math MCTS with UCT converges to good policies in these partially observable games when the model is accurate.
    Background Section 2 and Appendix A rely on standard MCTS convergence properties, which are assumed without novel proof.
  • domain assumption Simulating simultaneous actions as sequential hidden actions with an environment actor preserves the game's information structure.
    System prompts in Appendix E instruct the LLM to model simultaneous actions as sequential with partial observation; no proof is given that this transformation is faithful for GOPS.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PIANIST: Learning Partially Observable World Models with LLMs for Multi-Agent Decision Making." pith.science (2026). https://pith.science/paper/VL32APO7

@misc{pith2026241115998,
  author       = {Pith},
  title        = {Pith review of: PIANIST: Learning Partially Observable World Models with LLMs for Multi-Agent Decision Making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VL32APO7}},
  note         = {Machine review of arXiv:2411.15998}
}
read the original abstract

Effective extraction of the world knowledge in LLMs for complex decision-making tasks remains a challenge. We propose a framework PIANIST for decomposing the world model into seven intuitive components conducive to zero-shot LLM generation. Given only the natural language description of the game and how input observations are formatted, our method can generate a working world model for fast and efficient MCTS simulation. We show that our method works well on two different games that challenge the planning and decision making skills of the agent for both language and non-language based action taking, without any training on domain-specific training data or explicitly defined world model.

Figures

Figures reproduced from arXiv: 2411.15998 by the authors.

Figure 1
Figure 1. Overview of PIANIST. Start￾ing with just the game description, the LLM generates a complete multi-agent, partial information world model, which can then be used for planning via search. Recent studies have shown how LLMs, trained on massive amounts of online data, can be used as a world model to conduct planning [1, 2]. However, using LLMs as world models have not been as well explored in multi-agent, partial inform… view at source ↗
Figure 2
Figure 2. Integrating PIANIST components with MCTS. The realization function samples a hidden state for simulation, while the transition, action, and partition functions are used to expand new states. States are selected based on UCT values, aggregated across information sets for partial information. Though the diagram shows values for a single player, in practice, values for all players are inferred and updated simultaneousl… view at source ↗
Figure 3
Figure 3. Directed generation graph for PIANIST. We display the sequential generation order for the various components of PIANIST, with dependencies shown by directed arrows. Generating and testing objects in this order minimizes the probability of execution failure. The initial information set representation is given by the environment to allow an unified interface with the environment. Modularization also means we can test … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 6 canonical work pages

  1. [1]

    Reasoning with language model is planning with world model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhit- ing Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992, 2023

    arXiv 2023

  2. [2]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in Neural Information Processing Systems, 36, 2024

    2024

  3. [3]

    Large language models as commonsense knowledge for large-scale task planning

    Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as commonsense knowledge for large-scale task planning. Advances in Neural Information Processing Systems, 36, 2024

    work page 2024

  4. [4]

    Monte carlo sampling for regret minimization in extensive games

    Marc Lanctot, Kevin Waugh, Martin Zinkevich, and Michael Bowling. Monte carlo sampling for regret minimization in extensive games. Advances in neural information processing systems, 22, 2009

    work page 2009

  5. [5]

    Goofspiel—the game of pure strategy

    Sheldon M Ross. Goofspiel—the game of pure strategy. Journal of Applied Probability , 8(3):621–625, 1971

    work page 1971

  6. [6]

    Self-refine: Iterative refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36, 2024

    2024

  7. [7]

    Reflexion: Language agents with verbal reinforcement learning.Advances in Neural Information Processing Systems, 36, 2024

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning.Advances in Neural Information Processing Systems, 36, 2024

    2024

  8. [8]

    Language agents with reinforcement learning for strategic play in the werewolf game

    Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. Language agents with reinforcement learning for strategic play in the werewolf game. arXiv preprint arXiv:2310.18940, 2023

    arXiv 2023

Show all 37 references
  1. [10]

    Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jare...

    2021 arXiv

  2. [11]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff...

    2020

  3. [12]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1–140:67, 2020

    2020

  4. [13]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. a...

    2022 arXiv

  5. [14]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vino...

    2022 arXiv

  6. [15]

    Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent...

    arXiv

  7. [16]

    Kola: Carefully benchmarking world knowledge of large language models

    Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, et al. Kola: Carefully benchmarking world knowledge of large language models. arXiv preprint arXiv:2306.09296, 2023

    2023 arXiv

  8. [17]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):1–26, 2024

    2024

  9. [18]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2023

    2023 arXiv

  10. [19]

    V oyager: An open-ended embodied agent with large language models

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. V oyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023

    2023 arXiv

  11. [20]

    Inner monologue: Embodied reasoning through planning with language models

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Noah Brown, Tomas Jackson, Linda Luu, Sergey Levine, Karol Hausman, and Brian Ichter. Inner monologue: Embodied reasoning th...

    2022 arXiv

  12. [21]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442, 2023. 6

    2023 arXiv

  13. [22]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2024

    2024

  14. [23]

    Large language models as tool makers

    Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers. arXiv preprint arXiv:2305.17126, 2023

    2023 arXiv

  15. [24]

    Chain of thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022

    2022 arXiv

  16. [25]

    Graph of thoughts: Solving elaborate problems with large language models

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Confe...

    2024

  17. [26]

    Language models, agent models, and world models: The law for machine reasoning and planning

    Zhiting Hu and Tianmin Shu. Language models, agent models, and world models: The law for machine reasoning and planning. arXiv preprint arXiv:2312.05230, 2023

    2023 arXiv

  18. [27]

    Leveraging pre-trained large language models to construct and utilize world models for model-based task planning

    Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati. Leveraging pre-trained large language models to construct and utilize world models for model-based task planning. Advances in Neural Information Processing Systems, 36:79081–79094, 2023

    2023

  19. [28]

    Alphazero-like tree-search can guide large language model decoding and training

    Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. Alphazero-like tree-search can guide large language model decoding and training. arXiv preprint arXiv:2309.17179, 2023

    2023 arXiv

  20. [29]

    Clin: A continually learning language agent for rapid task adaptation and generalization

    Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, and Peter Clark. Clin: A continually learning language agent for rapid task adaptation and generalization. arXiv preprint arXiv:2310.10134, 2023

    arXiv

  21. [30]

    Expel: Llm agents are experiential learners

    Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19632–19642, 2024

    2024

  22. [31]

    Eureka: Human-level reward design via coding large language models

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931, 2023

    arXiv

  23. [32]

    Language to rewards for robotic skill synthesis

    Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, et al. Language to rewards for robotic skill synthesis. arXiv preprint arXiv:2306.08647, 2023

    2023 arXiv

  24. [33]

    Mastering the game of go without human knowledge

    David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017

    2017

  25. [34]

    Mastering atari, go, chess and shogi by planning with a learned model

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Si- mon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604–609, 2020

    2020

  26. [35]

    Human-level play in the game of diplomacy by combining language models with strategic reasoning

    Meta Fundamental AI Research Diplomacy Team (FAIR)†, Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, et al. Human-level play in the game of diplomacy by combining language models with strategic rea...

    2022

  27. [36]

    Llm- deliberation: Evaluating llms with interactive multi-agent negotiation games

    Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, and Mario Fritz. Llm- deliberation: Evaluating llms with interactive multi-agent negotiation games. arXiv preprint arXiv:2309.17234, 2023

    2023 arXiv

  28. [37]

    Improving language model negotiation with self-play and in-context learning from ai feedback

    Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. Improving language model negotiation with self-play and in-context learning from ai feedback. arXiv preprint arXiv:2305.10142, 2023

    2023 arXiv

  29. [38]

    State␣must␣be␣an␣instance␣of␣HiddenState

    Noam Brown and Tuomas Sandholm. Superhuman ai for multiplayer poker. Science, 365(6456):885–890, 2019. 7 A MCTS Details The Monte Carlo Tree Search (MCTS) process, illustrated in Figure 2, simulates possible future game states by expanding nodes in a search tree. Each node cor...

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.