Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

The paper claims RL fine-tuning lets a 14-billion-parameter model predict MOBA macro-actions at 90.91% accuracy, ahead of Deepseek-R1's 86.67%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A reinforcement-learning pipeline for predicting macro-actions in Honor of Kings improves action prediction accuracy, but the method is imitation of human replay labels, not the claimed environmental interaction.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A clean behavioral-cloning pipeline with an overblown 'procedural knowledge' framing; worth a referee's time only if the authors reframe or test in a live environment. the 4 major comments →

arxiv 2508.21365 v1 pith:Z6OBPZHY submitted 2025-08-29 cs.AI

Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models

Yi Liao , Yu Gu , Yuan Sui , Zining Zhu , Yifan Lu , Guohua Tang , Zhongqian Sun , Wei Yang This is my paper
classification cs.AI
keywords large language modelsreinforcement learningprocedural knowledgemacro-action predictionMOBA gamesGRPOinterpretable decision-makingHonor of Kings
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Think-in-Games (TiG) tries to show that an LLM can move from knowing facts about a game to knowing how to steer it, by turning strategic decisions into a language task. Given a JSON-encoded game state, the model must output the best macro-action plus a natural-language reasoning trace; supervised fine-tuning on distilled reasoning is followed by GRPO reinforcement learning with a rule-based reward that checks whether the predicted action matches the recorded human macro-action. On Honor of Kings, a Qwen-3-14B model trained this way reaches 90.91% macro-action prediction accuracy, beating Deepseek-R1's 86.67% and outperforming all untrained baselines. The authors read this as evidence that procedural understanding can be injected into smaller models without sacrificing general reasoning, and that the resulting decisions come with step-by-step explanations. The reason to care is the implied trade-off: much smaller, interpretable models might handle strategic decision-making in constrained action spaces as well as models an order of magnitude larger.

Core claim

On its own terms, TiG's central claim is that declarative and procedural knowledge can be bridged by reformulating RL-based decision-making as a language modeling task: the LLM generates a language-guided policy (a set of macro-actions and a reasoning chain), which is then refined iteratively through online reinforcement learning. Training is multi-stage: SFT on reasoning distilled from Deepseek-R1, then GRPO with a rule-based binary reward that assigns 1 if the predicted action equals the replay-derived ground-truth action and 0 otherwise. The reported result is that Qwen-3-14B + SFT + GRPO reaches 90.91% on macro-action prediction, outperforming Deepseek-R1 (86.67%) and Qwen-2.5-32B + GRPO

What carries the argument

The central object is the macro-action space: a fixed set of 40 high-level team objectives (push top lane, secure dragon, defend base, etc.) that abstracts away low-level control. The policy model maps a JSON-encoded game state to a subset of these actions plus a <think> reasoning chain. The training engine is GRPO: for each game-state prompt, a group of completions is generated, each scored by a rule-based verifier comparing the predicted action to the replay-derived ground truth; rewards are normalized within the group into advantages, and the policy is updated with a clipped surrogate loss plus KL regularization to a reference policy. The macro-action abstraction and the binary reward tog

Load-bearing premise

The central claim rests on treating a match between the model's predicted macro-action and a human player's recorded macro-action as the reward signal; if those human labels are imperfect, not the right abstraction, or not predictive of winning outcomes, the model may learn to imitate labeling artifacts rather than true game strategy.

What would settle it

Put the trained policy into live Honor of Kings matches (or a high-fidelity simulator) and compare win rate, tower damage, and objective control against the base LLM and against random macro-action baselines; if higher macro-action prediction accuracy does not translate to better in-game outcomes, the claim that TiG instills procedural knowledge is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A 14-billion-parameter model can match or beat a 671-billion-parameter general reasoning model on constrained strategic decision tasks, suggesting parameter count is not the binding constraint once domain-specific RL is applied.
  • GRPO with rule-based rewards is the main driver of improvement: Qwen-2.5-32B jumps from 66.67% to 86.84% with GRPO alone, and GRPO alone beats SFT alone on Qwen2.5-14B.
  • Trained models keep their general language abilities: scores on MMLU, CEval, BBH, and IfEval remain stable or improve slightly, so domain-specific RL does not cause catastrophic forgetting.
  • Because decisions come with natural-language reasoning, the resulting policy is inspectable by human players, unlike black-box RL agents.
  • The framework is presented as transferable beyond MOBAs to other interactive environments where language can serve as the policy interface.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The implemented reward is agreement with recorded human macro-actions, not outcome-based game reward, so TiG as described is closer to RL-assisted behavioral cloning than to learning from win/loss consequences; a live-game test is needed to confirm the procedural-knowledge claim.
  • Performance likely depends on the expert-designed 40-action taxonomy and priority relabeling; porting TiG to a new game means rebuilding this abstraction, and part of the reported gain may come from this human prior rather than from RL itself.
  • The reasoning traces may be post-hoc rationalizations; to test faithfulness, one could ablate parts of the game state and check whether the model's stated reasons change only when the relevant feature is removed.
  • If the accuracy result transfers, a practical route emerges: take a moderately sized instruction-tuned model, distill reasoning from a large reasoner into it, then refine on replay-labeled decisions, obtaining deployable game assistants at a fraction of the inference cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Think in Games (TiG), a framework to train LLMs for macro-level strategic decision-making in the MOBA game Honor of Kings. The method combines SFT on DeepSeek-R1 distilled outputs with GRPO reinforcement learning, where the reward is a binary match between the predicted macro-action and a human replay label. The authors claim that TiG enables LLMs to acquire procedural knowledge through direct interaction with game environments, and that a 14B model can surpass Deepseek-R1 on action prediction while preserving general capabilities. The paper reports action-prediction accuracy, a TiG-QA generalization evaluation, general NLP benchmark results, and qualitative case studies.

Significance. If the central claim were supported, the paper would be significant: it would show that a relatively small, interpretable LLM can match or exceed a much larger reasoning model on game-strategy tasks through environmental interaction, with potential applications to explainable game agents. The paper has several strengths: a clearly specified multi-stage training pipeline, evaluation on general LLM benchmarks showing that domain training does not degrade broad capabilities, a concrete error taxonomy, and qualitative case studies. However, the core claim of learning from environmental feedback is not supported by the method as written, and one of the paper's own tables directly contradicts a headline comparison. These issues are load-bearing for the paper's stated contribution.

major comments (4)
  1. [§3.2, Eq. (4)] The reward is rt = 1 iff the predicted macro-action equals A*_t obtained from replay data. No action is executed in a live environment, no state transition is observed, and no win/loss or outcome-based reward is used. This is supervised label matching, not reinforcement learning from environmental feedback. The abstract, Figure 1 caption, and §7 repeatedly claim 'direct interaction with game environments' and 'environmental feedback', but the method as written does not interact with an environment at all. A concrete test would be to deploy the policy in a live HoK environment or to evaluate it with an outcome-based metric such as win rate; without that, the procedural-knowledge claim is unsupported.
  2. [§4.4, Table 3] The text states that TiG 'achieves results comparable to Deepseek-R1' on TiG-QA, but Table 3 shows Deepseek-R1 ahead on both question types: strong-related correct responses (score 2) are 57.14% for Deepseek-R1 vs 40.00% for Qwen-2.5-32B + GRPO, and weak-related correct responses are 79.63% vs 37.78%. This directly contradicts the 'comparable' claim. The paper's explanation that Deepseek-R1 excels because the questions rely on web knowledge rather than game interaction also undercuts the paper's own argument that TiG's value comes from game-grounded procedural understanding.
  3. [§4.3, Table 2a vs. §3.2, Eq. (4)] The main action-prediction evaluation measures accuracy against the same human replay labels A*_t used to compute the training reward in Eq. (4). The reported accuracy is therefore the fitted training objective, not an independent measure of game competence. The TiG-QA evaluation is not circular, but as noted above it does not support the claim that TiG surpasses Deepseek-R1. An independent evaluation—e.g., expert human judgment on held-out states or a live-game outcome metric—is needed before the central claim can be accepted.
  4. [§4.1, §6] The abstract and conclusion claim that TiG achieves 'competitive performance with dramatically lower data and computational demands compared to conventional RL methods.' However, the experiments compare only against LLM baselines (Qwen variants, Deepseek-R1, QwQ-32B); no conventional RL agent (e.g., PPO- or DQN-based MOBA agents) is included. This claim is therefore unsupported by the reported experiments.
minor comments (5)
  1. [§4.2] Model names are inconsistent: 'Qwen2.5-14B' and 'Qwen-2.5-14B' are used interchangeably. Please standardize.
  2. [Table 3] The table columns are labeled '0', '1', '2' with the caption 'Numbers indicate count (percentage) of correct responses.' It is unclear whether entries are counts, percentages, or both; sample sizes per row should be stated.
  3. [Table 2a] QwQ-32B appears in the table but is not listed in the baselines in §4.1 and no citation or access link is provided.
  4. [Figure 1 caption] The caption says the model is updated using a 'rule-based verifier' in the 'game environment.' Since the verifier is Eq. (4), matching a replay label, the caption should not imply live environmental feedback.
  5. [§7] The Limitations section states that TiG 'still requires a substantial amount of environment interaction.' This is inconsistent with the method description in §3, where no environment interaction occurs. Either the limitation should be reworded or the method description should be corrected.

Circularity Check

2 steps flagged

The main action-prediction accuracy and the GRPO reward are the same binary match against human replay labels, so the headline improvement is a fit to the training objective, not independent evidence of procedural game knowledge.

specific steps
  1. fitted input called prediction [Section 3.2, Eq. (4); Section 4.3, Table 2a]
    "Formally, given a predicted action ˆAt at time step t and the corresponding ground truth action A∗t obtained from the replay data, the reward rt is defined as: rt = {1, if ˆAt = A∗t , 0, otherwise. (4)"

    This is the reward optimized by GRPO. The headline result in Table 2a is 'Accuracy (%)' on the Action Prediction Task, evaluated as the same binary match against the same replay-derived ground-truth actions. The model is trained to maximize exactly this quantity, so the reported gains (e.g., Qwen-3-14B reaching 90.91%) measure fit to the training signal rather than independent decision quality or environmental feedback.

  2. renaming known result [Abstract; Section 3.2]
    "LLMs generate language-guided policies, which are refined iteratively through online reinforcement learning based on environmental feedback."

    The 'environmental feedback' is, by Eq. (4), only equality with a human macro-action label from replay data (relabeled by a priority hierarchy). No live state transition, win/loss, or outcome-based reward is used. The paper's central claim that TiG acquires procedural knowledge through direct interaction with game environments is thus a renaming of supervised label matching / behavioral cloning as environmental RL.

full rationale

The central circularity is that the training reward and the main evaluation metric are identical: both are binary matches between the predicted macro-action and the human replay label A*_t. Therefore the action-prediction results in Table 2a are a direct measure of how well the model fits the human-label distribution it was optimized on, not an independent test of procedural competence. The paper's framing of this as 'environmental feedback' from 'direct interaction' is unsupported by the method as written. The general benchmarks (MMLU, BBH, IfEval, etc.) provide independent evidence that the training does not destroy general LLM capabilities, which prevents a higher score, but they do not test game decision quality. Table 3 further shows the trained model underperforms Deepseek-R1 on game-state QA, undercutting the broader procedural-understanding claim. Overall, partial circularity (score 6): the headline prediction reduces by construction to the training objective.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 1 invented entities

The method rests on several hand-chosen design elements: the 40-action space, the relabeling windows, the priority hierarchy, and the binary match-to-human-label reward. None of these are derived from first principles or externally validated. The central claim about procedural knowledge depends on the unverified assumption that imitating human macro-decisions is equivalent to learning how to play the game.

free parameters (6)
  • Macro-action space size K=40 = 40
    Hand-defined taxonomy of team objectives (Table 5); not derived from data or theory.
  • Relabeling window L_fill = not reported
    Section 3.1.2: backward-fill window length chosen by the authors; numerical value not given.
  • Relabeling window L_overwrite = not reported
    Section 3.1.2: overwrite window length chosen by the authors; numerical value not given.
  • Priority hierarchy for action labels = expert-defined ordering
    Table 5 hierarchy rests on expert human knowledge, not measured outcome data.
  • GRPO group size G, clip epsilon, KL coefficient beta = not reported
    Section 3.2 defines the loss but does not quote the hyperparameter values used.
  • Number of RL steps per model = 160, 360, 480, 600, 2000
    Section 4.2 varies training steps by model due to compute constraints; no stopping criterion is given.
axioms (5)
  • domain assumption Human replay macro-actions are a reliable proxy for optimal strategy
    Section 3.1 uses recorded player actions as ground truth for reward and evaluation.
  • domain assumption Matching human action labels confers procedural understanding
    Section 3.2 equates reward 1 with correct strategic behavior.
  • domain assumption The JSON game state contains all decision-relevant information
    Section 2.2 restricts to visible info from primary player perspective.
  • ad hoc to paper The expert-defined priority hierarchy correctly orders action importance
    Section 3.1.2: used in relabeling to overwrite lower-priority labels.
  • standard math GRPO's group-relative normalization is suitable for this reward distribution
    Section 3.2 and Appendix C assume GRPO stabilizes training; no formal justification beyond the original paper.
invented entities (1)
  • Macro-level action space A (40 actions) no independent evidence
    purpose: Abstraction to let the LLM reason about team objectives instead of low-level controls
    The 40-action ontology is constructed by the authors from expert knowledge; no external validation shows these actions cover optimal strategies.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models." pith.science (2026). https://pith.science/paper/Z6OBPZHY

@misc{pith2026250821365,
  author       = {Pith},
  title        = {Pith review of: Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z6OBPZHY}},
  note         = {Machine review of arXiv:2508.21365}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large language models (LLMs) excel at complex reasoning tasks such as mathematics and coding, yet they frequently struggle with simple interactive tasks that young children perform effortlessly. This discrepancy highlights a critical gap between declarative knowledge (knowing about something) and procedural knowledge (knowing how to do something). Although traditional reinforcement learning (RL) agents can acquire procedural knowledge through environmental interaction, they often operate as black boxes and require substantial training data. In contrast, LLMs possess extensive world knowledge and reasoning capabilities, but are unable to effectively convert this static knowledge into dynamic decision-making in interactive settings. To address this challenge, we propose Think in Games (TiG), a novel framework that empowers LLMs to develop procedural understanding through direct interaction with game environments, while retaining their inherent reasoning and explanatory abilities. Specifically, TiG reformulates RL-based decision-making as a language modeling task: LLMs generate language-guided policies, which are refined iteratively through online reinforcement learning based on environmental feedback. Our experimental results show that TiG successfully bridges the gap between declarative and procedural knowledge, achieving competitive performance with dramatically lower data and computational demands compared to conventional RL methods. Moreover, TiG provides step-by-step natural language explanations for its decisions, greatly improving transparency and interpretability in complex interactive tasks.

Figures

Figures reproduced from arXiv: 2508.21365 by Guohua Tang, Wei Yang, Yifan Lu, Yi Liao, Yuan Sui, Yu Gu, Zhongqian Sun, Zining Zhu.

Figure 1
Figure 1. Figure 1: Demonstration of GRPO training with Game State. refers to trained models, and refers to frozen models. Given the current game state, the model is asked to predict the proper action, and provide the thinking process as the analysis of why consider this action. We then compare the predicted action with ground-truth values using a rule-based verifier to update the policy model. This process enables the model … view at source ↗
Figure 2
Figure 2. Figure 2: (left) Action Prediction Task, (right) Distribution of the Error Cases across different models. The definition of error cases can be found in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Demonstration of Rewards & Response Length change during the RL training process. Generalization of TiG. To verify TiG’s generalizability on other tasks, we propose another dataset collected from real gameplay in question-answering format (denoted as TiG-QA). In the TiG-QA task, the model is given the game state and an open-ended user question, and is asked to generate a comprehensive answer grounded in th… view at source ↗
Figure 4
Figure 4. Figure 4: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language. As illustrated in the [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language. 5. Related Work Game Understanding of LLMs. While large language models (LLMs) excel at language-based reasoning, effectively applying them to games remains challenging. This difficulty stems from their reliance on static pre-t… view at source ↗
Figure 7
Figure 7. Figure 7: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language. natural strengths in reasoning and explanation. Role of RL in LLMs. Recent advances in LLMs have highlighted the crucial role of RL in aligning model outputs with human preferences (Sui et al., 2025, Jin et al., 2025). While pr… view at source ↗
Figure 9
Figure 9. Figure 9: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language. More recently, Group Relative Policy Optimization (GRPO) (Shao et al., 2024) has emerged as a flexible alternative for obtaining reward signals. Unlike PPO, GRPO does not strictly require a reward model; instead, it can incorp… view at source ↗
Figure 11
Figure 11. Figure 11: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language. to generate interpretable, language-guided policies that are refined via online feedback. Our experiments demonstrate that TiG not only bridges the gap between knowing about and knowing how to do, but also achieves competitive… view at source ↗
Figure 13
Figure 13. Figure 13: One of the cases of TiG. <think> </think> refers to the thinking process of model output, and <result> </result> refers to the model guidance to the main player in natural language. complex or real-time environments, may restrict policy performance. • Domain Generalization: Our current experiments are primarily conducted within digital game en￾vironments. The generalizability of TiG to other interactive d… view at source ↗
Figure 14
Figure 14. Figure 14: Demonstration of JSON object for each game state. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SyncPlan: Long-Horizon LLM Coordination with Explicit Synchronization and Adaptive Correction

    cs.RO 2026-08 conditional novelty 6.0

    A plan-execute-correct framework with explicit wait primitives and a learned plan-staleness detector enables LLM-based multi-agent coordination that is both faster and more adaptive than repeated-invocation baselines.

  2. Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application

    cs.CL 2026-06 unverdicted novelty 5.0

    This survey categorizes agentic environments for LLMs by eight attributes and domains, introduces symbolic and neural synthesis paradigms with evaluation, and outlines four agent evolution pathways plus three environm...

Reference graph

Works this paper leans on

42 extracted references · 14 canonical work pages · cited by 2 Pith papers · 2 internal anchors

  1. [1]

    Cause and Effect: Can Large Language Models Truly Understand Causality?

    Swagata Ashwani, Kshiteesh Hegde, Nishith Reddy Mannuru, Mayank Jindal, Dushyant Singh Sengar, Krishna Chaitanya Rao Kathala, Dishant Banga, Vinija Jain, and Aman Chadha. Cause and effect: Can large language models truly understand causality? Proceedings of the AAAI Symposium Series, 2024. doi:10.48550/arXiv.2402.18139

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2402.18139 2024

  2. [2]

    M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47: 0 253–279, June 2013. ISSN 1076-9757. doi:10.1613/jair.3912. URL http://dx.doi.org/10.1613/jair.3912

    doi:10.1613/jair.3912 2013

  3. [3]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948

    Pith/arXiv arXiv 2025

  4. [4]

    Mapeval: A map-based evaluation of geo-spatial reasoning in foundation models

    Mahir Labib Dihan, Md Tanvir Hassan, Md Tanvir Parvez, Md Hasebul Hasan, Md Almash Alam, Muhammad Aamir Cheema, Mohammed Eunus Ali, and Md Rizwan Parvez. Mapeval: A map-based evaluation of geo-spatial reasoning in foundation models. arXiv preprint arXiv: 2501.00316, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Bayeschess: A computer chess program based on bayesian networks

    Antonio Fernández and Antonio Salmerón. Bayeschess: A computer chess program based on bayesian networks. Pattern Recognition Letters, 29 0 (8): 0 1154--1159, 2008. ISSN 0167-8655. doi:https://doi.org/10.1016/j.patrec.2007.06.013. URL https://www.sciencedirect.com/science/article/pii/S0167865507002127. Pattern Recognition in Interdisciplinary Perception an...

    work page doi:10.1016/j.patrec.2007.06.013 2008

  6. [6]

    Font and Tobias Mahlmann

    Jose M. Font and Tobias Mahlmann. Dota2 bot competition. IEEE Transactions on Games, 11 0 (3): 0 285–289, September 2019. ISSN 2475-1510. doi:10.1109/tg.2018.2834566. URL http://dx.doi.org/10.1109/TG.2018.2834566

    work page arXiv 2019

  7. [7]

    Enabling self-improving agents to learn at test time with human-in-the-loop guidance

    Yufei He, Ruoyu Li, Alex Chen, Yue Liu, Yulin Chen, Yuan Sui, Cheng Chen, Yi Zhu, Luca Luo, Frank Yang, and Bryan Hooi. Enabling self-improving agents to learn at test time with human-in-the-loop guidance. arXiv preprint arXiv: 2507.17131, 2025

    work page arXiv 2025

  8. [8]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021. URL https://openreview.net/forum?id=d7KBjmI3GmQ

    2021

  9. [9]

    Openrlhf: An easy-to-use, scalable and high-performance rlhf framework

    Jian Hu, Xibin Wu, Zilin Zhu, Xianyu, Weixun Wang, Dehao Zhang, and Yu Cao. Openrlhf: An easy-to-use, scalable and high-performance rlhf framework. arXiv preprint arXiv:2405.11143, 2024 a

    Pith/arXiv arXiv 2024

  10. [10]

    A survey on large language model-based game agents

    Sihao Hu, Tiansheng Huang, Gaowen Liu, Ramana Rao Kompella, Fatih Ilhan, Selim Furkan Tekin, Yichang Xu, Zachary Yahn, and Ling Liu. A survey on large language model-based game agents. arXiv preprint arXiv: 2404.02039, 2024 b

    Pith/arXiv arXiv 2024

  11. [11]

    Pokellmon: A human-parity agent for pokemon battles with large language models, 2024 c

    Sihao Hu, Tiansheng Huang, and Ling Liu. Pokellmon: A human-parity agent for pokemon battles with large language models, 2024 c . URL https://arxiv.org/abs/2402.01118

    Pith/arXiv arXiv 2024

  12. [12]

    C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models

    Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, jiayi lei, Yao Fu, Maosong Sun, and Junxian He. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural In...

    work page 2023

  13. [13]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv: 2503.09516, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Mind the GAP: Glimpse-based Active Perception improves generalization and sample efficiency of visual reasoning

    Oleh Kolner, Thomas Ortner, Stanisław Woźniak, and Angeliki Pantazi. Mind the gap: Glimpse-based active perception improves generalization and sample efficiency of visual reasoning. arXiv preprint arXiv: 2409.20213, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  15. [15]

    School chinese benchmark, 2018

    lanhin . School chinese benchmark, 2018. URL https://github.com/lanhin/SchoolChinese

    work page 2018

  16. [16]

    Codei/o: Condensing reasoning patterns via code input-output prediction

    Junlong Li, Daya Guo, Dejian Yang, Runxin Xu, Yu Wu, and Junxian He. Codei/o: Condensing reasoning patterns via code input-output prediction. arXiv preprint arXiv: 2502.07316, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Simpo: Simple preference optimization with a reference-free reward

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37: 0 124198--124235, 2024

    2024

  18. [18]

    Playing atari with deep reinforcement learning

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv: 1312.5602, 2013

    Pith/arXiv arXiv 2013

  19. [19]

    Creating pro-level AI for a real-time fighting game using deep reinforcement learning

    Inseok Oh, Seungeun Rho, Sangbin Moon, Seongho Son, Hyoil Lee, and Jinyun Chung. Creating pro-level AI for a real-time fighting game using deep reinforcement learning. IEEE Trans. Games , 14 0 (2): 0 212--220, 2022. doi:10.1109/TG.2021.3049539. URL https://doi.org/10.1109/TG.2021.3049539

    work page arXiv 2022

  20. [20]

    Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, et al

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, et al. Training language models to follow instructions with human feedback. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Ann...

    work page 2022

  21. [21]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference ...

    2023

  22. [22]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347, 2017

    Pith/arXiv arXiv 2017

  23. [23]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300

    Pith/arXiv arXiv 2024

  24. [24]

    Megatron-lm: Training multi-billion parameter language models using model parallelism

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv: 1909.08053, 2019

    Pith/arXiv arXiv 1909

  25. [25]

    Mastering the game of go with deep neural networks and tree search

    David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529 0 (7587): 0 484--489, 2016

    2016

  26. [26]

    Bayes' bluff: Opponent modelling in poker

    Finnegan Southey, Michael P Bowling, Bryce Larson, Carmelo Piccione, Neil Burch, Darse Billings, and Chris Rayner. Bayes' bluff: Opponent modelling in poker. arXiv preprint arXiv:1207.1411, 2012

    Pith/arXiv arXiv 2012

  27. [27]

    Brown, Adam Santoro, Aditya Gupta, et al

    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Trans. Mach. Learn. Res., 2023, 2023. URL https://openreview.net/forum?id=uyTL5Bvosj

    work page 2023

  28. [28]

    Meta-reasoner: Dynamic guidance for optimized inference-time reasoning in large language models, 2025

    Yuan Sui, Yufei He, Tri Cao, Simeng Han, Yulin Chen, and Bryan Hooi. Meta-reasoner: Dynamic guidance for optimized inference-time reasoning in large language models, 2025. URL https://arxiv.org/abs/2502.19918

    Pith/arXiv arXiv 2025

  29. [29]

    Le, Ed H

    Mirac Suzgun, Nathan Scales, Nathanael Sch \" a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. In Anna Rogers, Jordan L. Boyd - Graber, and Naoaki Okazaki, editors, Findings of the Association for Computational...

    doi:10.18653/v1 2023

  30. [30]

    Charactereval: A chinese benchmark for role-playing conversational agent evaluation

    Quan Tu, Shilong Fan, Zihang Tian, and Rui Yan. Charactereval: A chinese benchmark for role-playing conversational agent evaluation. arXiv preprint arXiv: 2401.01275, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Starcraft ii: A new challenge for reinforcement learning, 2017

    Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, et al. Starcraft ii: A new challenge for reinforcement learning, 2017. URL https://arxiv.org/abs/1708.04782

    Pith/arXiv arXiv 2017

  32. [32]

    Voyager: An open-ended embodied agent with large language models

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi (Jim) Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. Trans. Mach. Learn. Res., 2023. doi:10.48550/arXiv.2305.16291

    Pith/arXiv arXiv doi:10.48550/arxiv.2305.16291 2023

  33. [33]

    Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents, 2024

    Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents, 2024. URL https://arxiv.org/abs/2302.01560

    Pith/arXiv arXiv 2024

  34. [34]

    Mind's eye of llms: Visualization-of-thought elicits spatial reasoning in large language models

    Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, and Furu Wei. Mind's eye of llms: Visualization-of-thought elicits spatial reasoning in large language models. arXiv preprint arXiv: 2404.03622, 2024

    Pith/arXiv arXiv 2024

  35. [35]

    Agents play thousands of 3d video games

    Zhongwen Xu, Xianliang Wang, Siyi Li, Tao Yu, Liang Wang, Qiang Fu, and Wei Yang. Agents play thousands of 3d video games. arXiv preprint arXiv: 2503.13356, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    Policy-to-language: Train llms to explain decisions with flow-matching generated rewards

    Xinyi Yang, Liang Zeng, Heng Dong, Chao Yu, Xiaoran Wu, Huazhong Yang, Yu Wang, Milind Tambe, and Tonghan Wang. Policy-to-language: Train llms to explain decisions with flow-matching generated rewards. arXiv preprint arXiv: 2502.12530, 2025

    work page arXiv 2025

  37. [37]

    Mastering complex control in moba games with deep reinforcement learning

    Deheng Ye, Zhao Liu, Mingfei Sun, Bei Shi, Peilin Zhao, Hao Wu, Hongsheng Yu, Shaojie Yang, Xipeng Wu, Qingwei Guo, et al. Mastering complex control in moba games with deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 6672--6679, 2020

    work page 2020

  38. [38]

    C har P oet: A C hinese classical poetry generation system based on token-free LLM

    Chengyue Yu, Lei Zang, Jiaotuan Wang, Chenyi Zhuang, and Jinjie Gu. C har P oet: A C hinese classical poetry generation system based on token-free LLM . In Yixin Cao, Yang Feng, and Deyi Xiong, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 315--325, Bangkok, Thaila...

    work page doi:10.18653/v1/2024.acl-demos.30 2024

  39. [39]

    Training interactive agent in large fps game map with rule-enhanced reinforcement learning

    Chen Zhang, Huan Hu, Yuan Zhou, Qiyang Cao, Ruochen Liu, Wenya Wei, and Elvis S Liu. Training interactive agent in large fps game map with rule-enhanced reinforcement learning. In 2024 IEEE Conference on Games (CoG), pages 1--8. IEEE, 2024

    work page 2024

  40. [40]

    Ape210k: A large-scale and template-rich dataset of math word problems, 2020

    Wei Zhao, Mingyue Shang, Yang Liu, Liang Wang, and Jingming Liu. Ape210k: A large-scale and template-rich dataset of math word problems, 2020. URL https://arxiv.org/abs/2009.11506

    Pith/arXiv arXiv 2020

  41. [41]

    Instruction-following evaluation for large language models

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv: 2311.07911, 2023

    Pith/arXiv arXiv 2023

  42. [42]

    Pokerbench: Training large language models to become professional poker players

    Richard Zhuang, Akshat Gupta, Richard Yang, Aniket Rahane, Zhengyu Li, and Gopala Anumanchipalli. Pokerbench: Training large language models to become professional poker players. arXiv preprint arXiv:2501.08328, 2025

    Pith/arXiv arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.