Pith. sign in

REVIEW 4 major objections 6 minor 48 references

Multiple Weaks Win Single Strong: Large Language Models Ensemble Weak Reinforcement Learning Agents into a Supreme One

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM-Ens uses a language model to label game states as situations and switches to the agent with the best average reward in each, reporting gains over fixed RL ensembles of up to 20.9 percent.

desk verdict Original idea and clean writing, but the headline empirical claim is undercut by an unspecified reward-statistics protocol that may amount to selecting agents on the test episodes themselves. read the letter →

arxiv 2505.15306 v1 pith:ABBSKATC submitted 2025-05-21 cs.LG cs.AI

Yiwen Song , Qianyue Hao , Qingmin Liao , Jian Yuan , Yong Li This is my paper
classification cs.LGcs.AI
keywords reinforcementlearningmodelensemblelargelanguagemodelsstatecategorizationsituationgenerationdynamicagentselectionAtaribenchmarkpolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes LLM-Ens, a method for combining several weak reinforcement-learning agents into one stronger agent. A large language model defines the distinct 'situations' that occur in a task, such as exploration, combat, and evasion in a tank game, then the method measures the average reward each agent earns in each situation and, at run time, switches every $K$ steps to the agent with the best average reward for the current situation. The authors claim this situation-aware switching is more adaptive than fixed rules like majority voting or Boltzmann addition. On a 13-game subset of the Atari benchmark, LLM-Ens is reported to outperform those baselines by up to 20.9% and to beat the best individual agent.

What carries the argument

The machinery is the pairing of an LLM-generated situation taxonomy with a per-situation reward table. The Situation Generation LLM turns the game description into a handful of situations, the State Categorization LLM assigns the current frame to one of those situations every $K$ steps, and the selector applies the rule $m^* = \arg\max_{m\in M} R_{m,s}$, where $R_{m,s}$ is the average reward agent $m$ accumulated during past occurrences of situation $s$. This turns ensembling into a context-dependent switch whose context is expressed in natural-language terms the LLM understands.

What would settle it

Recompute the reward statistics $R_{m,s}$ on a separate set of rollouts that are disjoint from the episodes used to evaluate LLM-Ens, then check whether the reported edge over the best single agent and over Boltzmann addition survives; if the lead disappears, the improvement is an artifact of selecting agents based on the evaluation outcomes.

Watch

Extended reading notes

Core claim

The paper's central claim is that an RL model ensemble can be improved by replacing fixed combination rules with a semantic, situation-aware switching rule. The process begins when an LLM generates a small set of situations for the task; next, the average reward $R_{m,s}$ of each agent $m$ in each situation $s$ is recorded; and during inference the current state is labeled every $K$ steps and the agent $m^* = \arg\max_{m\in M} R_{m,s}$ is chosen to act. The paper reports that this dynamic selection outperforms majority voting, rank voting, aggregation, Boltzmann addition, and Boltzmann multiplication, and surpasses the best single agent, on the tested Atari games.

Load-bearing premise

The load-bearing assumption is that the per-situation reward averages, computed before the ensemble run, are honest forecasts of which agent will perform best the next time that situation appears, and are not derived from the same episodes used to score the final result; the authors also acknowledge the LLM may occasionally mislabel a situation.

Editorial extensions

If this is right

  • If the reported gains are correct, practitioners can combine several weak agents — trained with different seeds, learning rates, or architectures — into one strong agent without retraining or changing the underlying RL algorithm.
  • The method applies to a new task by re-running the LLM situation-generation prompt, so it avoids hand-crafting ensemble rules per game.
  • Because each switch is tied to a named situation, the ensemble's behavior is more interpretable than a vote: observers can see that it changes agents when the game enters 'combat' or 'evasion'.
  • The design incurs an LLM query every $K=30$ steps during inference, so its practical value depends on whether that latency and cost are acceptable for the deployment setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A revealing control experiment would replace the LLM situation labels with clusters of raw observations (for instance, DQN hidden activations) and apply the same $\arg\max_m R_{m,s}$ rule; matching performance would show the gain comes from per-situation reward maximization rather than semantic understanding.
  • The paper does not specify whether the reward table is static or updated online after inference begins; if it is static, the method assumes the situation–reward mapping is stationary, and if it is updated online, LLM-Ens is equivalent to a contextual bandit and could be compared against EXP4.
  • The authors note that LLMs may hallucinate situation labels; a cheap robustness test would be to query the categorizer several times per frame and only switch agents when the labels agree, a stability check the paper does not report.
  • If the approach transfers beyond Atari, it offers a general recipe for LLM-driven model selection in any domain where an LLM can interpret the state stream, such as robot navigation or traffic signal control.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes LLM-Ens, a three-stage RL model ensemble framework: a Situation Generation LLM partitions game states into named situations; a per-situation reward distribution R_{m,s} is computed for each agent; and at inference a State Categorization LLM labels the current state every K steps and selects the agent with the highest R_{m,s} in that situation. The authors evaluate on 13 Atari games using DQN agents with varying random seeds and hyperparameters, reporting mean episode rewards over 5 seeds and claiming improvements of up to 20.9% over static ensemble baselines. They also provide heatmap experiments across learning-rate and hidden-size configurations, and they include open-source code, implementation details, and a brief limitations section.

Significance. The core idea—using an LLM to produce semantically meaningful situation labels and then choosing among agents by per-situation empirical reward—is simple, testable, and potentially useful, and the paper is commendable for releasing code and showing example situations. However, the current evaluation does not establish the main empirical claim. The data protocol for computing R_{m,s} is ambiguous on the crucial train/evaluation split, the Atari task selection is partial and nonstandard, no statistical significance testing is provided, and there is no non-LLM dynamic selection baseline. If the authors can resolve these issues, the framework could be a meaningful contribution; as written, the evidence is insufficient to support the claimed superiority.

major comments (4)
  1. [§3.3–§3.4] The paper does not specify whether the interactions used to compute R_{m,s} are held out from the evaluation episodes whose rewards are reported in Table 1. Section 3.3 defines R_{m,s} as the average over 'all interactions associated with the identified situation' and says the distribution is 'continuously updating as new interactions occur,' while Section 3.4 refers to a 'precomputed' RewardDistribution and selects m* = argmax_m R_{m,s}. If the same episodes are used for both computing R_{m,s} and reporting final performance, the reported gains (e.g., BattleZone 10400 vs. 8600) are a direct consequence of selecting the agent that already scored best on those very episodes, which is an oracle artifact rather than evidence of generalization. The authors must state the exact data split, the timing of updates, and confirm that the Table 1 evaluation episodes did not contribute to R_{m,s}.
  2. [§4.1, Table 1] The experiments are restricted to 13 of the 26 Atari tasks, selected 'focusing on tasks where the original DQN achieves stable convergence,' and training length is set per task between 100k and 500k steps 'depending on the speed at which rewards increase.' This partial and outcome-dependent selection makes the broad Atari-benchmark claim unsupported. The authors should either evaluate on the full 26-game suite with a standard protocol or explicitly frame the results as a proof of concept on favorable tasks and argue that the selection does not bias the comparison.
  3. [Table 1] Table 1 reports only means and standard deviations over 5 seeds, and the standard deviations overlap substantially for most comparisons (e.g., BattleZone LLM-Ens 10400 ± 4159 vs. Aggregation 8600 ± 3507; ChopperCommand 1020 ± 217 vs. Aggregation 860 ± 230). No paired significance tests, confidence intervals, or effect sizes are provided, so the repeated use of 'significant' and 'consistently superior' is not justified by the reported statistics.
  4. [§4.2 and §3.4] The selection rule is simply the per-situation argmax over R_{m,s}, so the unique contribution of the LLM is the situation labeling. The paper does not compare against any non-LLM dynamic selection baseline that uses the same R_{m,s} table, such as selecting by state-feature histograms, visual similarities, or random situation assignments. Without such a baseline, the observed improvements cannot be attributed to LLM semantic understanding rather than to the per-situation reward statistics themselves.
minor comments (6)
  1. [Abstract and §4.2] The abstract reports 'up to 20.9%' improvement over baselines, while Section 4.2 later reports a 51.2% improvement over baselines for MsPacman; these numbers should be reconciled.
  2. [Table 2] The LLM temperature is set to 1.0 for both the Situation Generation and State Categorization calls, making the situation labels stochastic; the paper should report variance over repeated LLM calls or fix the sampling seed.
  3. [Figure 2] The heatmaps have axis labels that are hard to read, and the axes appear to repeat the same tick values without clear correspondence to the two hyperparameter dimensions; adding explicit axis labels, tick labels, and a colorbar would greatly improve interpretability.
  4. [§3.1] The phrase 'orignal weak agents' contains a typo and should read 'original weak agents.'
  5. [Appendix C.1] The limitation section says LLM errors may 'poison specific training processes,' but LLM-Ens is described as an inference-time selection method; it should clarify whether this refers to a possible training-time extension or is a wording error.
  6. [§4.1] The paper claims compatibility with 'various RL algorithms,' but all main experiments use DQN only; the claim should be softened or accompanied by at least one non-DQN experiment, and the definition of 'Best Single Agent' in Table 1 should be stated explicitly.

Circularity Check

1 steps flagged · score 6.0 of 10

Central reported gains may reduce to test-time oracle selection: R_{m,s} is computed from 'all interactions' during inference and then used in m* = argmax R_{m,s}, with no held-out split specified.

  1. fitted input called prediction [Section 3.3 (Agent Reward Distribution Analysis), Section 3.4 (Model Ensemble), Table 1]
    "After determining the situation ID, we compute the average reward achieved by each agent within that specific situation. This reward computation involves aggregating the rewards from all interactions associated with the identified situation... by continuously updating the reward distribution as new interactions occur... For each identified situation s, the framework refers to the precomputed {RewardDistribution}... m* = arg max_{m in M} R_{m,s}"

    R_{m,s} is defined as the mean of rewards r_{i,m} over all interactions in situation s, and Section 3.3 says this is done during inference and 'continuously updat[ed] ... as new interactions occur.' The paper never states that these interactions come from a held-out set separate from the episodes whose average rewards are reported in Table 1. Under the natural reading, the same evaluation episodes supply both the R_{m,s} statistics and the reported return, so the selection rule m* = argmax_m R_{m,s} is not predicting which agent is best; it is choosing the agent that has already accumulated the highest reward on the test data. The headline gains (e.g., BattleZone 10400, MsPacman 1116) are then a direct consequence of fitting the selector to the evaluation outcomes.

full rationale

The only load-bearing reduction I can exhibit is the reward-statistics protocol. Section 3.3 computes R_{m,s} by averaging rewards from 'all interactions associated with the identified situation' and says it updates online during inference; Section 3.4 then uses these values as a 'precomputed RewardDistribution' in m* = argmax_m R_{m,s}. If no split is made between episodes used to estimate R_{m,s} and episodes used for evaluation, the reported LLM-Ens return is a fitted quantity, not an independent prediction, which would make the main empirical claim circular. The paper does not specify such a split, and the 'continuously updating' language points the other way. I found no other circular steps: the LLM situation-generation and state-categorization are not defined in terms of the target result; the baseline comparisons are standard; and the self-citations in Related Work are not load-bearing. Because the central claim may reduce to test-time fitting under the paper's own description, the score is 6 rather than 0; the ambiguity prevents a higher score without a definitive internal proof that R_{m,s} is computed on held-out data.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on empirical assumptions about state partitioning and reward statistics rather than on a formal derivation. The method introduces no new physical entities. The free parameters are limited to the LLM interval K and temperature, but the validity of the approach depends on the unstated evaluation protocol for computing reward distributions.

free parameters (2)
  • State categorization interval K = 30
    Set in Section 4.1; controls how often the LLM reclassifies the state, trading off responsiveness and computational cost.
  • LLM temperature = 1.0
    Set in Appendix A for both Situation Generation and State Categorization LLMs; affects stochasticity of outputs and thus reproducibility.
assumptions (4)
  • domain assumption Atari game states can be partitioned into a small number of semantically meaningful situations.
    The Situation Generation LLM is asked to classify all possible states into a few categories (Section 3.2). The method relies on this simplification being accurate enough for agent selection.
  • domain assumption The average reward R_{m,s} estimated from historical interactions is a reliable predictor of future performance for agent m in situation s.
    Section 3.3 computes the mean reward for each agent per situation and Section 3.4 uses it to select the agent. If this statistic is not stable over time, the selection rule will be wrong.
  • domain assumption The State Categorization LLM correctly identifies the situation of the current state every K steps.
    Section 3.3 relies on the LLM's classification to map states to situations; misclassification would select the wrong agent.
  • domain assumption The situation of the state does not change significantly within the K-step interval.
    Section 3.4 switches agents only every K steps; if the situation changes within that window, the chosen agent may be suboptimal for part of the interval.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multiple Weaks Win Single Strong: Large Language Models Ensemble Weak Reinforcement Learning Agents into a Supreme One." pith.science (2026). https://pith.science/paper/ABBSKATC

@misc{pith2026250515306,
  author       = {Pith},
  title        = {Pith review of: Multiple Weaks Win Single Strong: Large Language Models Ensemble Weak Reinforcement Learning Agents into a Supreme One},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ABBSKATC}},
  note         = {Machine review of arXiv:2505.15306}
}
read the original abstract

Model ensemble is a useful approach in reinforcement learning (RL) for training effective agents. Despite wide success of RL, training effective agents remains difficult due to the multitude of factors requiring careful tuning, such as algorithm selection, hyperparameter settings, and even random seed choices, all of which can significantly influence an agent's performance. Model ensemble helps overcome this challenge by combining multiple weak agents into a single, more powerful one, enhancing overall performance. However, existing ensemble methods, such as majority voting and Boltzmann addition, are designed as fixed strategies and lack a semantic understanding of specific tasks, limiting their adaptability and effectiveness. To address this, we propose LLM-Ens, a novel approach that enhances RL model ensemble with task-specific semantic understandings driven by large language models (LLMs). Given a task, we first design an LLM to categorize states in this task into distinct 'situations', incorporating high-level descriptions of the task conditions. Then, we statistically analyze the strengths and weaknesses of each individual agent to be used in the ensemble in each situation. During the inference time, LLM-Ens dynamically identifies the changing task situation and switches to the agent that performs best in the current situation, ensuring dynamic model selection in the evolving task condition. Our approach is designed to be compatible with agents trained with different random seeds, hyperparameter settings, and various RL algorithms. Extensive experiments on the Atari benchmark show that LLM-Ens significantly improves the RL model ensemble, surpassing well-known baselines by up to 20.9%. For reproducibility, our code is open-source at https://anonymous.4open.science/r/LLM4RLensemble-F7EE.

Figures

Figures reproduced from arXiv: 2505.15306 by the authors.

Figure 1
Figure 1. Illustration of LLM-Ens, which leverages LLMs to dynamically categorize task-specific [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Performance improvement of LLM-Ens across different hyperparameter configurations on [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Examples of situations categorized by the Situation Generation LLM in different environ [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Representative frames for each situation categorized by the Situation Generation LLM in [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 25 canonical work pages

  1. [1]

    Reinforcement learning: An introduction.A Bradford Book, 2018

    Richard S Sutton. Reinforcement learning: An introduction.A Bradford Book, 2018. 9

    work page 2018

  2. [2]

    An introduction to deep reinforcement learning.Foundations and Trends® in Machine Learning, 11(3-4):219–354, 2018

    Vincent François-Lavet, Peter Henderson, Riashat Islam, Marc G Bellemare, Joelle Pineau, et al. An introduction to deep reinforcement learning.Foundations and Trends® in Machine Learning, 11(3-4):219–354, 2018

    work page 2018

  3. [3]

    Mastering the game of go without human knowledge.nature, 550(7676):354–359, 2017

    David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge.nature, 550(7676):354–359, 2017

    2017

  4. [4]

    Grandmaster level in starcraft ii using multi-agent reinforcement learning.nature, 575(7782):350–354, 2019

    Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Jun- young Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning.nature, 575(7782):350–354, 2019

    2019

  5. [5]

    Dota 2 with large scale deep reinforcement learning.arXiv preprint arXiv:1912.06680, 2019

    Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemysław D˛ ebiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al. Dota 2 with large scale deep reinforcement learning.arXiv preprint arXiv:1912.06680, 2019

    arXiv 1912

  6. [6]

    Mastering atari games with limited data.Advances in neural information processing systems, 34:25476–25488, 2021

    Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao. Mastering atari games with limited data.Advances in neural information processing systems, 34:25476–25488, 2021

    work page 2021

  7. [7]

    A graph placement methodology for fast chip design.Nature, 594(7862):207–212, 2021

    Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Wenjie Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Azade Nazi, et al. A graph placement methodology for fast chip design.Nature, 594(7862):207–212, 2021

    work page 2021

  8. [8]

    Hierarchical reinforcement learning for scarce medical resource allocation with imperfect information

    Qianyue Hao, Fengli Xu, Lin Chen, Pan Hui, and Yong Li. Hierarchical reinforcement learning for scarce medical resource allocation with imperfect information. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 2955–2963, 2021

    work page 2021

Show all 48 references
  1. [9]

    Reinforcement learning enhances the experts: Large-scale covid-19 vaccine allocation with multi-factor contact network

    Qianyue Hao, Wenzhen Huang, Fengli Xu, Kun Tang, and Yong Li. Reinforcement learning enhances the experts: Large-scale covid-19 vaccine allocation with multi-factor contact network. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4...

    2022

  2. [10]

    Gat-mf: Graph attention mean field for very large scale multi-agent reinforcement learning

    Qianyue Hao, Wenzhen Huang, Tao Feng, Jian Yuan, and Yong Li. Gat-mf: Graph attention mean field for very large scale multi-agent reinforcement learning. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 685–697, 2023

    2023

  3. [11]

    Spatial planning of urban communities via deep reinforcement learning.Nature Computational Science, 3(9):748– 762, 2023

    Yu Zheng, Yuming Lin, Liang Zhao, Tinghai Wu, Depeng Jin, and Yong Li. Spatial planning of urban communities via deep reinforcement learning.Nature Computational Science, 3(9):748– 762, 2023

    2023

  4. [12]

    A survey of machine learning for urban decision making: Applications in planning, transportation, and healthcare.ACM Computing Surveys, 2024

    Yu Zheng, Qianyue Hao, Jingwei Wang, Changzheng Gao, Jinwei Chen, Depeng Jin, and Yong Li. A survey of machine learning for urban decision making: Applications in planning, transportation, and healthcare.ACM Computing Surveys, 2024

    2024

  5. [13]

    Dyps: Dynamic parameter sharing in multi-agent reinforcement learning for spatio-temporal resource allocation

    Jingwei Wang, Qianyue Hao, Wenzhen Huang, Xiaochen Fan, Zhentao Tang, Bin Wang, Jianye Hao, and Yong Li. Dyps: Dynamic parameter sharing in multi-agent reinforcement learning for spatio-temporal resource allocation. InProceedings of the 30th ACM SIGKDD Conference on Knowledge ...

    2024

  6. [14]

    Coopride: Cooperate all grids in city-scale ride-hailing dispatching with multi-agent reinforcement learning

    Jingwei Wang, Qianyue Hao, Wenzhen Huang, Xiaochen Fan, Qin Zhang, Zhentao Tang, Bin Wang, Jianye Hao, and Yong Li. Coopride: Cooperate all grids in city-scale ride-hailing dispatching with multi-agent reinforcement learning. InProceedings of the 31st ACM SIGKDD Conference on ...

    2025

  7. [15]

    A method for evaluating hyperparameter sensitivity in reinforcement learning.arXiv preprint arXiv:2412.07165, 2024

    Jacob Adkins, Michael Bowling, and Adam White. A method for evaluating hyperparameter sensitivity in reinforcement learning.arXiv preprint arXiv:2412.07165, 2024

    2024 arXiv

  8. [16]

    How many random seeds? statistical power analysis in deep reinforcement learning experiments.arXiv preprint arXiv:1806.08295, 2018

    Cédric Colas, Olivier Sigaud, and Pierre-Yves Oudeyer. How many random seeds? statistical power analysis in deep reinforcement learning experiments.arXiv preprint arXiv:1806.08295, 2018. 10

    2018 arXiv

  9. [17]

    Ensemble deep learning: A review.Engineering Applications of Artificial Intelligence, 115:105151, 2022

    Mudasir A Ganaie, Minghui Hu, Ashwani Kumar Malik, Muhammad Tanveer, and Ponnuthu- rai N Suganthan. Ensemble deep learning: A review.Engineering Applications of Artificial Intelligence, 115:105151, 2022

    2022

  10. [18]

    A survey on ensemble learning.Frontiers of Computer Science, 14:241–258, 2020

    Xibin Dong, Zhiwen Yu, Wenming Cao, Yifan Shi, and Qianli Ma. A survey on ensemble learning.Frontiers of Computer Science, 14:241–258, 2020

    2020

  11. [19]

    Ensemble reinforcement learning: A survey.Applied Soft Computing, page 110975, 2023

    Yanjie Song, Ponnuthurai Nagaratnam Suganthan, Witold Pedrycz, Junwei Ou, Yongming He, Yingwu Chen, and Yutong Wu. Ensemble reinforcement learning: A survey.Applied Soft Computing, page 110975, 2023

    2023

  12. [20]

    Neural network ensembles in reinforcement learning

    Stefan Faußer and Friedhelm Schwenker. Neural network ensembles in reinforcement learning. Neural Processing Letters, 41:55–69, 2015

    2015

  13. [21]

    Reinforcement learning based dynamic weighing of ensemble models for time series forecasting.arXiv preprint arXiv:2008.08878, 2020

    Satheesh K Perepu, Bala Shyamala Balaji, Hemanth Kumar Tanneru, Sudhakar Kathari, and Vivek Shankar Pinnamaraju. Reinforcement learning based dynamic weighing of ensemble models for time series forecasting.arXiv preprint arXiv:2008.08878, 2020

    2008 arXiv

  14. [22]

    Ensemble algorithms in reinforcement learning.IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 38(4):930–936, 2008

    Marco A Wiering and Hado Van Hasselt. Ensemble algorithms in reinforcement learning.IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 38(4):930–936, 2008

    2008

  15. [23]

    The arcade learning environment: An evaluation platform for general agents.Journal of Artificial Intelligence Research, 47:253–279, 2013

    Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents.Journal of Artificial Intelligence Research, 47:253–279, 2013

    2013

  16. [24]

    Model- based reinforcement learning for atari.arXiv preprint arXiv:1903.00374, 2019

    Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, et al. Model- based reinforcement learning for atari.arXiv preprint arXiv:1903.00374, 2019

    1903 arXiv

  17. [25]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff...

    2020

  18. [26]

    A survey of gpt-3 family large language models including chatgpt and gpt-4.Natural Language Processing Journal, page 100048, 2023

    Katikapalli Subramanyam Kalyan. A survey of gpt-3 family large language models including chatgpt and gpt-4.Natural Language Processing Journal, page 100048, 2023

    2023

  19. [27]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  20. [28]

    Evaluation of openai o1: Opportunities and challenges of agi.arXiv preprint arXiv:2409.18486, 2024

    Tianyang Zhong, Zhengliang Liu, Yi Pan, Yutong Zhang, Yifan Zhou, Shizhe Liang, Zihao Wu, Yanjun Lyu, Peng Shu, Xiaowei Yu, et al. Evaluation of openai o1: Opportunities and challenges of agi.arXiv preprint arXiv:2409.18486, 2024

    2024

  21. [29]

    Early access for safety testing

    OpenAI. Early access for safety testing. 2024

    2024

  22. [30]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  23. [31]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  24. [32]

    Palm: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240):1– 113, 2023

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240):1– 113, 2023. 11

    2023

  25. [33]

    Hlm-cite: Hybrid language model workflow for text-based scientific citation prediction.arXiv preprint arXiv:2410.09112, 2024

    Qianyue Hao, Jingyang Fan, Fengli Xu, Jian Yuan, and Yong Li. Hlm-cite: Hybrid language model workflow for text-based scientific citation prediction.arXiv preprint arXiv:2410.09112, 2024

    2024 arXiv

  26. [34]

    Stance detection with collaborative role-infused llm-based agents

    Xiaochong Lan, Chen Gao, Depeng Jin, and Yong Li. Stance detection with collaborative role-infused llm-based agents. InProceedings of the international AAAI conference on web and social media, volume 18, pages 891–903, 2024

    2024

  27. [35]

    A survey of large language models.arXiv preprint arXiv:2303.18223, 2023

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models.arXiv preprint arXiv:2303.18223, 2023

    2023 arXiv

  28. [36]

    A survey on evaluation of large language models.ACM Transactions on Intelligent Systems and Technology, 15(3):1–45, 2024

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models.ACM Transactions on Intelligent Systems and Technology, 15(3):1–45, 2024

    2024

  29. [37]

    Towards large reasoning models: A survey of reinforced reasoning with large language models.arXiv preprint arXiv:2501.09686, 2025

    Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. Towards large reasoning models: A survey of reinforced reasoning with large language models.arXiv preprint arXiv:2501.09686, 2025

    2025 arXiv

  30. [38]

    Human-level control through deep reinforcement learning.Nature, 518(7540):529–533, 2015

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning.Nature, 518(7540):529–533, 2015

    2015

  31. [39]

    Deep reinforcement learning based ensemble model for rumor tracking.Information Systems, 103:101772, 2022

    Guohui Li, Ming Dong, Lingfeng Ming, Changyin Luo, Han Yu, Xiaofei Hu, and Bolong Zheng. Deep reinforcement learning based ensemble model for rumor tracking.Information Systems, 103:101772, 2022

    2022

  32. [40]

    An oppositional-cauchy based gsk evolutionary algorithm with a novel deep ensemble reinforcement learning strategy for covid-19 diagnosis.Applied Soft Computing, 111:107675, 2021

    Seyed Mohammad Jafar Jalali, Milad Ahmadian, Sajad Ahmadian, Abbas Khosravi, Mamoun Alazab, and Saeid Nahavandi. An oppositional-cauchy based gsk evolutionary algorithm with a novel deep ensemble reinforcement learning strategy for covid-19 diagnosis.Applied Soft Computing, 11...

    2021

  33. [41]

    Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods.arXiv preprint arXiv:2404.00282, 2024

    Yuji Cao, Huan Zhao, Yuheng Cheng, Ting Shu, Guolong Liu, Gaoqi Liang, Junhua Zhao, and Yun Li. Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods.arXiv preprint arXiv:2404.00282, 2024

    2024 arXiv

  34. [42]

    Augmenting autotelic agents with large language models

    Cédric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan, and Marc-Alexandre Côté. Augmenting autotelic agents with large language models. InConference on Lifelong Learning Agents, pages 205–226. PMLR, 2023

    2023

  35. [43]

    Read and reap the rewards: Learning to play atari with the help of instruction manuals.Advances in Neural Information Processing Systems, 36, 2024

    Yue Wu, Yewen Fan, Paul Pu Liang, Amos Azaria, Yuanzhi Li, and Tom M Mitchell. Read and reap the rewards: Learning to play atari with the help of instruction manuals.Advances in Neural Information Processing Systems, 36, 2024

    2024

  36. [44]

    Self-refined large language model as automated reward function designer for deep reinforcement learning in robotics.arXiv preprint arXiv:2309.06687, 2023

    Jiayang Song, Zhehua Zhou, Jiawei Liu, Chunrong Fang, Zhan Shu, and Lei Ma. Self-refined large language model as automated reward function designer for deep reinforcement learning in robotics.arXiv preprint arXiv:2309.06687, 2023

    2023 arXiv

  37. [45]

    Text2reward: Reward shaping with language models for reinforcement learning

    Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. Text2reward: Reward shaping with language models for reinforcement learning. In ICLR. OpenReview.net, 2024

    2024

  38. [46]

    Llm-empowered state representation for reinforcement learning.arXiv preprint arXiv:2407.13237, 2024

    Boyuan Wang, Yun Qu, Yuhang Jiang, Jianzhun Shao, Chang Liu, Wenming Yang, and Xi- angyang Ji. Llm-empowered state representation for reinforcement learning.arXiv preprint arXiv:2407.13237, 2024

    2024 arXiv

  39. [47]

    Large language model as a policy teacher for training reinforcement learning agents.arXiv preprint arXiv:2311.13373, 2023

    Zihao Zhou, Bin Hu, Chenyang Zhao, Pu Zhang, and Bin Liu. Large language model as a policy teacher for training reinforcement learning agents.arXiv preprint arXiv:2311.13373, 2023

    2023 arXiv

  40. [48]

    Exploration Situation

    Hengyuan Hu and Dorsa Sadigh. Language instructed reinforcement learning for human-ai coordination. InInternational Conference on Machine Learning, pages 13584–13598. PMLR, 2023. 12 A Implementation Details In this section, we provide the main implementation details for reprod...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.