Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Acting

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM that builds, corrects, and acts on a causal graph of its environment outperforms strong RL agents on the 22-task Crafter benchmark.

desk verdict A well-structured LLM+SCM+RL pipeline with promising Crafter results, but the adapting stage's causal-intervention test is formally wrong, leaving the paper's central claim unproven. read the letter →

arxiv 2505.24710 v1 pith:OJ5PZDK4 submitted 2025-05-30 cs.LG cs.AIcs.CL

Wei Chen , Jiahao Zhang , Haipeng Zhu , Boyan Xu , Zhifeng Hao , Keli Zhang , Junjian Ye , Ruichu Cai This is my paper
classification cs.LGcs.AIcs.CL
keywords causal-awareLLMstructuralcausalmodelinterventionreinforcementlearninggoal-conditionedpolicyCrafterLLM-guideddecision-makinggraphupdating
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that giving a large language model an explicit graph of which factors in an environment cause which other factors, and letting that graph be corrected by the environment's own feedback, makes an LLM-guided decision-maker more effective. The proposal is Causal-aware LLMs, a loop with three phases: the model learns a structural causal model from observations, adapts it by treating actions as interventions and checking how other variables respond, and acts by generating sub-goals under the updated graph for a reinforcement-learning agent. The experimental case is the open-world game Crafter, where the method scores 33.6 ± 0.02 at 5 million steps, above the 28.2 ± 1.8 of the strongest LLM-assisted baseline and far above pure RL baselines at 1 million steps. Ablations reported in the paper show that removing either the learning phase or the adapting phase lowers the 1-million-step score below the full method, supporting the claim that both stages carry weight. A sympathetic reading is that structured causal knowledge, rather than next-token prediction alone, is what lets the agent plan resource chains and unlock deep achievements.

What carries the argument

The machinery is the environment's structural causal model, represented as a causal matrix M with M[i][j]=1 when variable i directly causes variable j, first generated by few-shot prompting an LLM from observations. The adapting stage then uses the do-operator from causal inference: the agent executes an action on v_i^t in a separate verification environment and checks whether p($v_j^{{t+1}}$|do(v_i^t)) differs from p($v_j^{{t+1}}$|v_i^t); a difference is taken as evidence of a causal edge, and the matrix is updated. The acting stage consumes this matrix by having an LLM propose causally ordered sub-goals, a policy $\pi(a_t | s_t, \text{goal}, G)$ acting under those goals, and a dense reward term from cosine similarity between goal and observation embeddings. The whole loop learning, adapting, and acting repeats over training epochs, so the graph and the policy co-evolve.

What would settle it

Hold the LLM, seeds, and acting stage fixed, and replace the learned causal matrix with a set of random matrices of the same size; if the 5M-step score stays near 33.6, the causal content is not doing the work. In a synthetic environment with a known hidden confounder between two variables, the adapting stage's equality test should be checked for whether it rewrites the causal edge between them incorrectly.

Watch

Extended reading notes

Core claim

The paper's central claim is that a structural causal model of the environment, written down as a causal graph and maintained through an iterative learning-adapting-acting loop, is a usable knowledge representation for LLM-driven decision-making: it corrects the LLM's hallucinated relations, organizes sub-goals in causal order, and gives the RL agent a reason to take actions such as finding a tree before chopping it. In the Crafter environment the method unlocks rare achievements such as 'make iron pickaxe' faster, reaches a score of 33.6 ± 0.02 at 5M steps, exceeds AdaRefiner's 5M score of 28.2 ± 1.8, and at 1M steps beats PPO(ResNet), DreamerV3, ReAct, and Reflexion while remaining below human experts and the prior-informed SPRING baseline. The ablation study is the load-bearing evidence: removing the learning stage drops the score from 18.9 to 14.89, and removing the adapting stage drops it to 14.67, so the authors conclude that both causal-graph construction and intervention-based updating contribute beyond the RL backbone.

Load-bearing premise

The adapting stage rests on the assumption that running a chosen action in a separate validation environment tells the difference between real causal links and coincidental ones, even when hidden factors are present.

Editorial extensions

If this is right

  • If the central claim holds, any environment whose dynamics admit a causal graph becomes a candidate for the same loop, since the LLM proposes relations and an intervention-style probe corrects them.
  • The earlier unlock of deep achievements, such as 'make iron pickaxe' around 0.2M steps sooner than PPO, implies the method accelerates long-horizon skill acquisition rather than only raising the final score.
  • Because the ablation removing the adapting stage drops the score more than removing the learning stage, keeping the causal graph current from environment feedback is a necessary part of the reported improvement.
  • The 5M-step score surpassing the prior-informed SPRING's 1M-step score implies the loop's benefit grows as training continues and the causal graph converges.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reader might test whether the performance gain comes from the intervention test itself or simply from goal-conditioned exploration with dense semantic rewards, by keeping all other components and replacing the equality-based update with a fixed no-update rule.
  • The adapting stage assumes interventions are unconfounded; in environments with hidden state variables, the same loop could be paired with conditional-independence testing to decide whether a proposed edge is real before trusting the LLM's graph.
  • The modular structure suggests a transfer recipe for other domains: let an LLM propose candidate causal relations, let the environment falsify them, and use only the surviving structure to constrain task decomposition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Causal-aware LLMs, a three-stage framework for decision-making: (1) learning, in which an LLM extracts an initial structural causal model from environment observations; (2) adapting, in which the causal graph is updated by comparing observed transitions after an action against a do-intervention criterion; and (3) acting, in which the updated graph is used to generate sub-goals and to shape an RL policy with a semantic-alignment reward. The method is evaluated on 22 Crafter tasks, reporting a score of 33.6 ± 0.02 at 5M steps and an ablation showing that removing the adapting stage lowers the 1M-step score from 18.9 to 14.67.

Significance. If the method were sound, it would make a useful contribution by combining LLM-extracted causal structure with RL for open-world tasks, and the released code and reproducible experimental protocol are assets. The empirical numbers, if reproduced, would be meaningful for the Crafter benchmark. However, the core adaptation mechanism is formally incorrect as stated, and the reported gains are not yet convincingly attributed to causal modeling because of uncontrolled baselines and missing variance in the ablation. The central claim about causal updating therefore needs substantial reworking before the results can be taken as evidence for the proposed mechanism.

major comments (4)
  1. [§5.3 (Adapting: Updating Causal Knowledge from Environment Feedback)] The criterion for deciding whether an edge exists is formally wrong. The paper states that if p(v_j^{t+1} | do(v_i^t)) = p(v_j^{t+1} | v_i^t), then 'there is no causal relationship' between the two variables. In a Markovian SCM with a true direct cause X → Y and independent noise, P(y | do(x)) equals P(y | x) by construction of the intervened distribution, so the proposed test would delete a true edge. Conversely, with confounding X ← U → Y, P(y | do(x)) ≠ P(y | x) can hold even when no direct edge X → Y exists, so the test would insert a false edge. The ablation in Table 2 attributes +4.2 points to the adapting stage, and the introduction claims that this stage corrects LLM hallucination, but the correctness of the update rule is the load-bearing premise; as written, the mechanism is unsupported.
  2. [§5.3 (Adapting: Updating Causal Knowledge from Environment Feedback)] Executing an RL action a_t is not equivalent to performing a do-intervention on a state variable v_i^t. In the POMDP formulation of Section 4, actions affect the transition P(s' | s, a) and can change multiple variables simultaneously; 'chop tree' does not force the state variable 'tree' to a specified value while cutting all other incoming causal arrows. Therefore the observed change in v_j^{t+1} after a_t cannot be interpreted as P(v_j^{t+1} | do(v_i^t)). The paper needs either a genuine randomized manipulation of the candidate variable in the verification environment or an explicit and justified mapping from actions to interventions; neither is provided.
  3. [Table 2 and §6.4] The ablation study reports no variance or number of seeds for 'Ours', 'Ours w/o learning', and 'Ours w/o adapting', while the baseline PPO (ResNet) is reported with ±1.66. The difference between 'Ours w/o adapting' (14.67) and 'Ours w/o learning' (14.89) is small (0.22), and the difference between 'Ours w/o adapting' and PPO (ResNet) (15.6) is within the PPO standard deviation. Without repeated-run statistics, the claim that the adapting stage is responsible for the improvement—and specifically that it corrects hallucinations rather than adding noise—is not statistically supported.
  4. [Table 1 and §6.3] The baseline comparisons are not controlled enough to isolate the causal component. The proposed method uses Meta-Llama-3-8B-Instruct and an additional semantic-alignment reward (cosine similarity between goal and observation embeddings), while ReAct, Reflexion, and AdaRefiner use different LLM backbones and reward mechanisms, and the RL baselines are trained without the semantic reward. The 'Ours w/o adapting' ablation still includes both the LLM and the semantic reward, so it does not separate the effect of causal knowledge from the effect of the added dense reward signal. A comparison with an ablation that removes the semantic reward, or a baseline with the same reward but no causal graph, is needed to support the conclusion that causal structure is the driver of the gains.
minor comments (5)
  1. [§6.4] The text refers to 'PPO (Restnet)' and 'PPO (Restnet)' in the ablation discussion; the table and Section 6.2 use 'PPO (Resnet)', so the spelling should be consistent.
  2. [§5.3] The construction of the 'valid environment' is described only informally. It should state how the verification environment is instantiated for each candidate relation, how many trials are used to estimate the two probabilities, and how the equality test is made operational (e.g., threshold on a distance or a statistical test).
  3. [§6.3 and Figure 5] The success-rate curves in Figure 5 are shown in log scale but the axis labels and units are not fully specified; please clarify whether the y-axis is log-transformed raw counts or log-transformed percentages.
  4. [Footnote 1] The code URL contains a space ('Causal-aware LLMs') that will break the link; the repository path should be given as a single URL string.
  5. [Equation (1)] The score formula is written with an outer 'exp(...) - 1' but the text says the geometric mean is computed; please make explicit whether the 1 is added to each success rate before the log or whether the formula is the standard geometric-mean-based score from the Crafter benchmark.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the causal graph is updated by external environment feedback and evaluated against external benchmarks; the only self-citation is background.

full rationale

The claimed derivation chain is an iterative pipeline, not a formal reduction. The learning stage obtains an initial causal graph by LLM prompting; the adapting stage updates it by executing actions in an independently constructed verification environment and checking whether observed change follows; the acting stage feeds the updated graph into goal generation and RL policy optimization. At no point is a parameter fitted to the reported score, nor is a target quantity reused as an input: the score is the geometric mean of 22 achievement success rates computed in Crafter, and the comparison baselines (DreamerV3, AdaRefiner, SPRING, etc.) are external. The only author-overlapping citation is [Cai et al., 2024] in Section 3.1 as a general reference for SCM graph representation; it is not used to justify any load-bearing claim. The adapting-stage causal test p(v_j^{t+1}|do(v_i^t)) = p(v_j^{t+1}|v_i^t) is questionable causal inference (it can mis-identify edges in the presence of confounding or unconfoundedness), but that is a correctness concern, not circularity: the equality is not defined to be equivalent to 'no edge' and the later score is not defined in terms of it. Therefore no circular step is exhibited.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The framework relies on four domain assumptions beyond the standard POMDP and SCM definitions: the intervention equality, the sufficiency of LLM prior extraction, the treatment of RL actions as do-interventions, and the neutrality of semantic reward shaping. No fitting parameters are disclosed beyond an unspecified semantic reward weight.

free parameters (1)
  • semantic alignment reward weight = not reported
    Section 5.4 augments the reward with cosine similarity between goal and observation embeddings; the coefficient for this term is not given, and it directly affects how much causal guidance influences policy learning.
assumptions (4)
  • domain assumption The equality p(v_j^{t+1}|do(v_i^t)) = p(v_j^{t+1}|v_i^t) identifies absence of a causal edge.
    Section 5.3 uses this rule to accept or reject relations; it is false in standard SCMs, where this equality can hold even with a direct causal edge when there is no confounding.
  • domain assumption The LLM's few-shot extraction produces a causal graph that is a useful starting point and can be corrected by the adapting stage.
    Section 5.2 relies on in-context learning to construct G from observations; no direct validation of the extracted graph is provided before it guides the agent.
  • domain assumption An RL action a_t can be modeled as do(v_i^t) on a state variable.
    Section 5.3 applies the action as an intervention on v_i^t, but in the POMDP formulation of Section 4 actions are not state variables; this conflation is required for the adapting stage.
  • domain assumption The semantic alignment reward term preserves the true objective and only accelerates learning.
    Section 5.4 adds the cosine-similarity term; the paper provides no analysis of how this term changes the optimal policy or how its weight is chosen.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Acting." pith.science (2026). https://pith.science/paper/OJ5PZDK4

@misc{pith2026250524710,
  author       = {Pith},
  title        = {Pith review of: Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Acting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OJ5PZDK4}},
  note         = {Machine review of arXiv:2505.24710}
}
read the original abstract

Large language models (LLMs) have shown great potential in decision-making due to the vast amount of knowledge stored within the models. However, these pre-trained models are prone to lack reasoning abilities and are difficult to adapt to new environments, further hindering their application to complex real-world tasks. To address these challenges, inspired by the human cognitive process, we propose Causal-aware LLMs, which integrate the structural causal model (SCM) into the decision-making process to model, update, and utilize structured knowledge of the environment in a ``learning-adapting-acting" paradigm. Specifically, in the learning stage, we first utilize an LLM to extract the environment-specific causal entities and their causal relations to initialize a structured causal model of the environment. Subsequently,in the adapting stage, we update the structured causal model through external feedback about the environment, via an idea of causal intervention. Finally, in the acting stage, Causal-aware LLMs exploit structured causal knowledge for more efficient policy-making through the reinforcement learning agent. The above processes are performed iteratively to learn causal knowledge, ultimately enabling the causal-aware LLMs to achieve a more accurate understanding of the environment and make more efficient decisions. Experimental results across 22 diverse tasks within the open-world game ``Crafter" validate the effectiveness of our proposed method.

Figures

Figures reproduced from arXiv: 2505.24710 by the authors.

Figure 1
Figure 1. Comparison of Existing Approaches and Our Method: [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our framework consists of three stages. In the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Causal intervention 4 Problem Formulation In this work, we aim to leverage causality to enhance the ability of LLM to understand the environment for effec￾tive decision-making. Given the inherent uncertainty and partial observability in real-world environments, we utilize a Partially Observable Markov Process (POMDP) [Sondik, 1971], which characterizes the interaction dynamics between an agent and the environment. F… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: An example of learning causal knowledge from LLM. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Success rates of obtaining 22 achievements in log scale. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Comparative score of Causal-aware LLMs and PPO [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning

    cs.AI 2026-07 conditional novelty 5.0 of 10

    On a new three-game spatial benchmark, larger Qwen3 models with thinking mode and multi-step planning achieve higher win rates, while small models struggle to localize and causal prompt hints give only marginal, model...

  2. Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem

    cs.AI 2025-06 conditional novelty 4.0 of 10

    The paper argues that AI-assisted peer review is an urgent priority and that its success depends on collecting richer, structured peer review process data.

Reference graph

Works this paper leans on

35 extracted references · 21 canonical work pages · cited by 2 Pith papers

  1. [1]

    Cause and effect: Can large language models truly understand causality? In Proceedings of the AAAI Symposium Series , volume 4, pages 2--9, 2024

    Swagata Ashwani, Kshiteesh Hegde, Nishith Reddy Mannuru, Dushyant Singh Sengar, Mayank Jindal, Krishna Chaitanya Rao Kathala, Dishant Banga, Vinija Jain, and Aman Chadha. Cause and effect: Can large language models truly understand causality? In Proceedings of the AAAI Symposium Series , volume 4, pages 2--9, 2024

    work page 2024

  2. [2]

    Vision-language models as a source of rewards

    Kate Baumli, Satinder Singh, Feryal Behbahani, Harris Chan, Gheorghe Comanici, Sebastian Flennerhag, Maxime Gazeau, Kristian Holsheimer, Dan Horgan, Michael Laskin, et al. Vision-language models as a source of rewards. In Second Agent Learning in Open-Endedness Workshop , 2023

    work page 2023

  3. [3]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877--1901, 2020

    1901

  4. [4]

    Granger causal representation learning for groups of time series

    Ruichu Cai, Yunjin Wu, Xiaokai Huang, Wei Chen, Tom ZJ Fu, and Zhifeng Hao. Granger causal representation learning for groups of time series. Science China Information Sciences , 67(5):152103, 2024

    work page 2024

  5. [5]

    Sirui Chen, Mengying Xu, Kun Wang, Xingyu Zeng, Rui Zhao, Shengjie Zhao, and Chaochao Lu. CLEAR : Can language models really understand causal graphs? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 6247--6265, Miami, Florida, USA, November 2024. Association for C...

    work page 2024

  6. [6]

    Guiding pretraining in reinforcement learning with large language models

    Yuqing Du, Olivia Watkins, Zihan Wang, C \'e dric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in reinforcement learning with large language models. In International Conference on Machine Learning , pages 8657--8677. PMLR, 2023

    work page 2023

  7. [7]

    Is C hat GPT a good causal reasoner? a comprehensive evaluation

    Jinglong Gao, Xiao Ding, Bing Qin, and Ting Liu. Is C hat GPT a good causal reasoner? a comprehensive evaluation. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 11111--11126, Singapore, December 2023. Association for Computational Linguistics

    work page 2023

  8. [8]

    Mastering atari with discrete world models

    Danijar Hafner, Timothy P Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. In International Conference on Learning Representations , 2020

    work page 2020

Show all 35 references
  1. [9]

    Mastering diverse domains through world models

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104 , 2023

    2023 arXiv

  2. [10]

    Benchmarking the spectrum of agent capabilities

    Danijar Hafner. Benchmarking the spectrum of agent capabilities. In Deep RL Workshop NeurIPS 2021 , 2021

    2021

  3. [11]

    Benchmarking the spectrum of agent capabilities

    Danijar Hafner. Benchmarking the spectrum of agent capabilities. In International Conference on Learning Representations , 2022

    2022

  4. [12]

    Rainbow: Combining improvements in deep reinforcement learning

    Matteo Hessel, Joseph Modayil, Hado Van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, and David Silver. Rainbow: Combining improvements in deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence ...

    2018

  5. [13]

    Thinking, fast and slow

    Daniel Kahneman. Thinking, fast and slow. Farrar, Straus and Giroux , 2011

    2011

  6. [14]

    Deep reinforcement learning for autonomous driving: A survey

    B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A Al Sallab, Senthil Yogamani, and Patrick P \'e rez. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems , 23(6):4909--4926, 2021

    2021

  7. [15]

    Reinforcement learning in robotics: A survey

    Jens Kober, J Andrew Bagnell, and Jan Peters. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research , 32(11):1238--1274, 2013

    2013

  8. [16]

    Continuous control with deep reinforcement learning

    Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 , 2015

    2015 arXiv

  9. [17]

    Goal-conditioned reinforcement learning: Problems and solutions

    Minghuan Liu, Menghui Zhu, and Weinan Zhang. Goal-conditioned reinforcement learning: Problems and solutions. In Lud De Raedt, editor, Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22 , pages 5502--5511, 7 2022. Survey Track

    2022

  10. [18]

    Playing atari with deep reinforcement learning

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 , 2013

    2013 arXiv

  11. [19]

    Asynchronous methods for deep reinforcement learning

    Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning , pages 1928--1937. PMLR, 2016

    1928

  12. [20]

    Discovering hierarchical achievements in reinforcement learning via contrastive learning

    Seungyong Moon, Junyoung Yeom, Bumsoo Park, and Hyun Oh Song. Discovering hierarchical achievements in reinforcement learning via contrastive learning. Advances in Neural Information Processing Systems , 36, 2024

    2024

  13. [21]

    The book of why: the new science of cause and effect

    Judea Pearl and Dana Mackenzie. The book of why: the new science of cause and effect . Basic books, 2018

    2018

  14. [22]

    Causality

    Judea Pearl. Causality . Cambridge university press, 2009

    2009

  15. [23]

    Trust region policy optimization

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International conference on machine learning , pages 1889--1897. PMLR, 2015

    2015

  16. [24]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 , 2017

    2017 arXiv

  17. [25]

    A survey of deep reinforcement learning in video games

    Kun Shao, Zhentao Tang, Yuanheng Zhu, Nannan Li, and Dongbin Zhao. A survey of deep reinforcement learning in video games. arXiv preprint arXiv:1912.10944 , 2019

    1912 arXiv

  18. [26]

    Reflexion: an autonomous agent with dynamic memory and self-reflection

    Noah Shinn, Beck Labash, and Ashwin Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366 , 2(5):9, 2023

    2023 arXiv

  19. [27]

    The optimal control of partially observable Markov processes

    Edward Jay Sondik. The optimal control of partially observable Markov processes . Stanford University, 1971

    1971

  20. [28]

    Adaplanner: adaptive planning from feedback with language models

    Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, and Chao Zhang. Adaplanner: adaptive planning from feedback with language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems , pages 58202--58245, 2023

    2023

  21. [29]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023

    2023 arXiv

  22. [30]

    Learning from delayed rewards

    Christopher John Cornish Hellaby Watkins. Learning from delayed rewards. 1989

    1989

  23. [31]

    Spring: Studying papers and reasoning to play games

    Yue Wu, So Yeon Min, Shrimai Prabhumoye, Yonatan Bisk, Russ R Salakhutdinov, Amos Azaria, Tom M Mitchell, and Yuanzhi Li. Spring: Studying papers and reasoning to play games. Advances in Neural Information Processing Systems , 36, 2024

    2024

  24. [32]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR) , 2023

    2023

  25. [33]

    Adarefiner: Refining decisions of language models with adaptive feedback

    Wanpeng Zhang and Zongqing Lu. Adarefiner: Refining decisions of language models with adaptive feedback. In Findings of the Association for Computational Linguistics: NAACL 2024 , pages 782--799, 2024

    2024

  26. [34]

    Causal graph discovery with retrieval-augmented generation based large language models

    Yuzhe Zhang, Yipeng Zhang, Yidong Gan, Lina Yao, and Chen Wang. Causal graph discovery with retrieval-augmented generation based large language models. arXiv preprint arXiv:2402.15301 , 2024

    2024 arXiv

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.