Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

DVM: Towards Controllable LLM Agents in Social Deduction Games

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that an LLM agent can be made to play Werewolf at a requested win rate while still beating existing agents, by rewarding whole decision chains rather than single steps.

desk verdict A genuinely new control objective for LLM agents in social deduction games, but the win-rate evidence is thinner than the abstract suggests. read the letter →

arxiv 2501.06695 v1 pith:EBAKWYUN submitted 2025-01-12 cs.AI

Zheng Zhang , Yihuai Lan , Yangsen Chen , Lei Wang , Xiang Wang , Hao Wang This is my paper · ORCID
classification cs.AI
keywords largelanguagemodelsgameagentscontrollablereinforcementlearningsocialdeductiongamesWerewolfdecisionchainrewardwin-rateconstraint
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that controllability and competence are not in tension for LLM game agents: the same Werewolf agent can be made stronger or weaker on demand. The proposed system, DVM, splits the agent into a Predictor, a Decider, and a Discussor, and trains the Decider with a decision-chain reward that scores the whole sequence of a player's decisions, plus a win-rate-constrained term. The central claim is that DVM exceeds existing methods when unrestricted, and that its actual win rate moves in the intended direction when the win-rate constraint is changed, while prompting-only methods stay flat. That matters because game developers need NPCs with adjustable difficulty, and the same mechanism could be used to calibrate the strength of LLM agents for fair evaluation.

What carries the argument

The load-bearing object is the decision chain reward. A decision chain is the sequence of actions a player takes across a full Werewolf game; DVM builds a database of (decision chain, win rate) pairs from FanLang-9, a dataset of over 18,000 human-player Werewolf games, and after each game the agent's chain is looked up to give $cr(DC)=\alpha(WR-0.5)$. Controllability comes from $cr_{ctrl}$, which computes $d=(WR_{cons}-WR_{dc})^2$, applies a threshold $\epsilon$ and tanh scaling, and returns positive reward when $d$ is small and negative reward when $d$ is large, so the Decider is pushed to repeat decision chains whose database win rate is close to the requested target.

What would settle it

Retrain DVM with $cr_{ctrl}$ computed online from the agent's actual game outcomes against its real opponents, and compare the resulting dial curve to the one in the paper. If the monotone relationship between constraint and actual win rate disappears, or shifts meaningfully with more than 30 games per setting, the central claim is falsified. A second check is to take a DVM agent set to a low target (say 30%) and play it against human players from the same pool as FanLang-9: it should land near 30% if the database lookup is actually controlling performance.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is a training recipe for a "dialable" agent. The Decider is optimized by PPO with a reward $r_t = sr_t + cr$, where $cr(DC)=\alpha(WR-0.5)$ evaluates the whole decision chain using a precomputed win rate $WR$ from human games; a second term $cr_{ctrl}$ converts the squared deviation between the requested win rate and the chain's win rate into a bounded reward that is positive for small deviations and negative for large ones. In evaluation with 30 games per setting, DVM reached 66.6% win rate as werewolf, 63.3% as villager, and 53.3% for other roles, ahead of the Thinker baseline, and its actual win rate rose with the requested target when it controlled the village side, unlike ReAct, LtM, and Thinker given the same constraint in their prompts. The paper is explicit that a gap remains between target and achieved win rate, and treats the method as a way to tune an agent below its optimal ceiling.

Load-bearing premise

The training signal for controllability is not the win rate the agent actually achieves in games; it is the win rate of a decision chain found in a database built from human games, and if that database does not behave like the live environment where the agent plays other trained agents, hitting the database target does not mean hitting the requested win rate.

Editorial extensions

If this is right

  • If the claim holds, a single trained NPC policy can serve multiple difficulty levels: the same weights, with a different win-rate constraint at inference, produce a different skill tier.
  • The decision-chain reward should reduce the need for per-step reward engineering in conversation-driven games, because long-horizon quality is scored at the episode level.
  • For evaluation, a calibrated agent gives a controllable opponent for testing other agents; an evaluator can match strengths rather than relying on a fixed strong or weak baseline.
  • The method's limitation to targets below the agent's ceiling means the practical use is difficulty lowering, not superhuman boosting; the paper says so explicitly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because $cr_{ctrl}$ is optimized against the database win rate $WR_{dc}$ rather than outcomes in the live environment, the dial's accuracy is only as good as the match between the human-games database and the real opponent policy; an obvious extension is to rebuild the database online from self-play games and compare dial curves.
  • Nothing in the reward design is Werewolf-specific beyond the definition of a decision chain, so the same constrained chain reward should transfer to other social deduction games with a measurable win rate; a testable variant would apply DVM to a 5-player or 7-player setup and check whether the monotone dial survives.
  • The paper's controllability plots use 30 games per setting; with binomial noise, a more decisive demonstration would report confidence intervals or more games, and would test whether the achieved win rate tracks the constraint beyond the village side.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes DVM, a framework for controllable LLM agents in the social deduction game Werewolf. DVM consists of a Predictor, a Decider, and a Discussor, and is trained in two stages: supervised fine-tuning on the FanLang-9 human-game dataset, followed by PPO reinforcement learning. The central novelty is a decision-chain reward, where the win rate of a full decision chain is looked up from a precomputed (DC, WR) database, combined with a win-rate-constrained reward that is intended to steer the agent toward a specified target win rate. The paper reports that DVM outperforms prior methods in Werewolf and shows an upward trend in achieved win rate as the win-rate constraint is increased. The evaluation is conducted in a 9-player Werewolf setup with 30 games per condition, and the text acknowledges that a gap remains between achieved and target win rates.

Significance. If the central claims were fully supported, the paper would make a useful contribution to controllable LLM game agents, with potential applications to dynamic difficulty scaling and to the safety/fairness analysis of LLM agents. The idea of using decision-chain-level win rates from a human-game database as a reward signal is interesting and goes beyond single-step reward shaping. The authors also provide a component ablation and a prediction study, which are appropriate steps. However, the experimental evidence is currently too weak to support the abstract's claim that DVM 'meets predefined win rate targets,' and several load-bearing details of the reward mechanism and evaluation are missing. The contribution is promising but needs substantial additional evidence and clarification.

major comments (5)
  1. [III.A, Fig. 2] The controllability claim is not supported at the level stated in the abstract. Each condition uses only 30 games, no error bars or statistical tests are reported, and the text concedes 'there remains a gap between the achieved win rates and the target constraints.' A monotonic upward trend is consistent with a noisy proxy effect and does not establish that the agent 'meets predefined win rate targets.' Please report the exact achieved rates, confidence intervals, and a significance test for both the trend and the deviation from target.
  2. [II.B] The constrained reward cr_ctrl is computed from WR_dc, the win rate of a decision chain looked up in a database built from FanLang-9 human games. The deployment evaluation, however, pits DVM against Thinker-controlled roles, not against the human players who produced the database. The paper gives no coverage statistics for DVM's produced decision chains in this database and no calibration evidence that WR_dc tracks actual online win rates; without such evidence, optimizing this reward does not imply achieving the requested actual win rate. The hyperparameters α, ε, k, s and the step reward srt are also omitted, so the mechanism is not reproducible.
  3. [Tables II and III] The performance and ablation comparisons are based on 30 games per cell. A difference of one game corresponds to 3.3 percentage points; for example, DVM's werewolf win rate of 66.6% versus Thinker's 63.3% is a one-game difference, and the -w/o DCR ablation (63.6% versus 63.3%) is essentially unchanged. Without confidence intervals or significance tests, the claims that DVM 'outperforms existing methods' and that each ablated component is necessary are not established.
  4. [III.B] The prediction test set is described as 'selected 600 games from the FanLang-9 dataset and from games played by different agents.' Since the same FanLang-9 corpus is used for supervised training and for building the (DC, WR) database, the paper must specify how the 600-game test set is disjoint from these training and database sources. Without a strict split, the improvements in Table I may be inflated by overlap.
  5. [II.B, cr_ctrl definition] The equations defining cr_ctrl are dimensionally inconsistent. d is defined as (WR_cons − WR_dc)^2, but the second branch uses (d − ε)/(1 − ε) rather than (d − ε^2)/(1 − ε^2), and the first branch uses d/ε rather than d/ε^2. This is not merely a notational issue, because the sign and magnitude of the reward depend on this comparison; please clarify the intended formula and verify that the reward is continuous where intended.
minor comments (6)
  1. [II.B] The PPO objective shows A_π(s,a) = r_t + γV(s_{t+1}) − V(s_t), which is a TD residual rather than the standard advantage function used in PPO, and no clipped surrogate objective is presented; the training objective should be corrected or clarified.
  2. [Table I] The Random baseline probabilities do not match the hypergeometric distribution for selecting 3 werewolves out of 8; for example, ACC@3 should be 1/56 ≈ 1.8%, not 0.8%. Please verify the Random baseline computation.
  3. [Fig. 2] The actual achieved win-rate values are not listed in the text; please include the numerical values and error bars in the figure or caption.
  4. [General] The paper omits many implementation details needed for replication: the Werewolf environment, prompt templates, number of PPO iterations, self-play opponent details, and whether code or data will be released.
  5. [Tables II and III] The row labels 'Werewolf,' 'Villager,' and 'Other Roles' should be defined precisely, specifying which camp is controlled by the tested method and which roles are included in 'Other Roles.'
  6. [Notation] The notation 'W Rcons.' and 'W Rdc' appears with inconsistent spacing; please use consistent subscripts throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the controllability claim is evaluated on fresh actual games, and the offline FanLang-9 (DC, WR) database is an empirical transfer assumption, not a definitional reduction.

full rationale

The paper's derivation chain is: (1) build a (DC, WR) lookup table from FanLang-9 human games; (2) define the decision-chain reward cr = alpha * (WR - 0.5) and the constrained reward cr_ctrl from d = (WR_cons - WR_dc)^2; (3) train the Decider with PPO using these rewards; (4) evaluate actual win rates in fresh Werewolf games, with 30 games per setting in Fig. 2 and Table II. The central controllability claim is not derived from the reward formula alone; it is empirically tested by measuring actual win rates against Thinker-controlled opponents. No equation in the paper identifies the reported win rate with WR_dc by construction: the paper explicitly acknowledges a persistent gap between achieved and target win rates, which confirms that the offline database serves as a proxy and that actual outcomes are measured separately. The main in-scope concern is that the FanLang-9 (DC, WR) table may not cover the decision chains DVM produces during self-play, and the paper provides no coverage statistics or distributional analysis; that is a validity and transfer risk, not circularity. The self-citations to PPO variants ([22], [23]) are not load-bearing because the framework's contribution and evaluation do not rest on those citations. There is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation that reduces the result to its inputs.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the FanLang-9 decision chain win rate database, several unstated reward constants, and an assumption that the simulator and dataset are representative. These are empirical inputs, not derived quantities.

free parameters (6)
  • chain reward amplitude alpha
    Constant in cr = alpha * (WR - 0.5); value not given in paper, presumably tuned.
  • controllable reward threshold epsilon
    Threshold in d = (WR_cons - WR_dc)^2 and r formula; value not specified.
  • tanh smoothing factor k
    Smoothing factor in r = -tanh((d - epsilon^2) / k); not specified.
  • reward scaling factor s
    Scales cr_ctrl to [-s, s]; not specified.
  • decision chain win rate database = Win rates of chains computed from FanLang-9
    The (DC, WR) pairs are empirical estimates from human games and act as a lookup reward table during RL.
  • step reward srt
    Referenced in rt = srt + cr but never defined or specified.
assumptions (5)
  • domain assumption FanLang-9 dataset provides reliable decision chain win rate estimates
    The reward uses WR for a chain from this database; if the estimates are noisy or biased, the learned policy is misaligned.
  • domain assumption The Werewolf simulator faithfully implements game rules and conversation effects
    All training and evaluation depend on the environment, which is not described in detail or released.
  • domain assumption A decision chain is a sufficient statistic for win probability
    The chain reward assigns a single WR to a sequence of decisions, ignoring game-state context and opponent policies.
  • domain assumption PPO converges to a stable policy in this environment
    No convergence analysis is provided; standard RL assumptions are invoked.
  • domain assumption The prediction test set does not overlap with training set
    Test samples are drawn from FanLang-9 plus agent games, but no split is described; overlap with supervised training data is plausible.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DVM: Towards Controllable LLM Agents in Social Deduction Games." pith.science (2026). https://pith.science/paper/EBAKWYUN

@misc{pith2026250106695,
  author       = {Pith},
  title        = {Pith review of: DVM: Towards Controllable LLM Agents in Social Deduction Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EBAKWYUN}},
  note         = {Machine review of arXiv:2501.06695}
}
read the original abstract

Large Language Models (LLMs) have advanced the capability of game agents in social deduction games (SDGs). These games rely heavily on conversation-driven interactions and require agents to infer, make decisions, and express based on such information. While this progress leads to more sophisticated and strategic non-player characters (NPCs) in SDGs, there exists a need to control the proficiency of these agents. This control not only ensures that NPCs can adapt to varying difficulty levels during gameplay, but also provides insights into the safety and fairness of LLM agents. In this paper, we present DVM, a novel framework for developing controllable LLM agents for SDGs, and demonstrate its implementation on one of the most popular SDGs, Werewolf. DVM comprises three main components: Predictor, Decider, and Discussor. By integrating reinforcement learning with a win rate-constrained decision chain reward mechanism, we enable agents to dynamically adjust their gameplay proficiency to achieve specified win rates. Experiments show that DVM not only outperforms existing methods in the Werewolf game, but also successfully modulates its performance levels to meet predefined win rate targets. These results pave the way for LLM agents' adaptive and balanced gameplay in SDGs, opening new avenues for research in controllable game agents.

Figures

Figures reproduced from arXiv: 2501.06695 by the authors.

Figure 1
Figure 1. The framework of DVM. DVM consists of three parts: Predictor, Decider, and Discussor. The final reward is obtained by adding the step reward and the decision chain reward. II. DVM This section introduces the components of DVM, followed by a detailed description of the training methods for each module. The framework is shown in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Controllability performance of agents. For each method, we applied it to control the village side in the game and added different win rate constraints. The other roles was controlled by Thinker. We conducted 30 games under each setting and measured the actual win rate for the village side. TABLE I PREDICTION PERFORMANCE. WEREWOLF PREDICTION MEANS IDENTIFYING 3 WEREWOLVES OUT OF THE OTHER 8 PLAYERS. IDENTITY PREDICTI… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cracking Aegis: An Adversarial LLM-based Game for Raising Awareness of Vulnerabilities in Privacy Protection

    cs.HC 2025-05 conditional novelty 6.0 of 10

    Cracking Aegis, an adversarial LLM-driven dialogue game, led players to use manipulative language strategies and to self-report stronger awareness of privacy vulnerabilities after a single session.

Reference graph

Works this paper leans on

31 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  2. [2]

    Palm: Scal- ing language modeling with pathways,

    A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scal- ing language modeling with pathways,” Journal of Machine Learning Research, vol. 24, no. 240, pp. 1–113, 2023

    2023

  3. [3]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  4. [4]

    Determinants of llm-assisted decision- making,

    E. Eigner and T. H ¨andler, “Determinants of llm-assisted decision- making,” 2024. [Online]. Available: https://arxiv.org/abs/2402.17385

    arXiv 2024

  5. [5]

    A survey on large language model-based game agents,

    S. Hu, T. Huang, F. Ilhan, S. Tekin, G. Liu, R. Kompella, and L. Liu, “A survey on large language model-based game agents,” arXiv preprint arXiv:2404.02039, 2024

    arXiv 2024

  6. [6]

    Cradle: Empowering foundation agents towards general computer control,

    W. Tan, W. Zhang, X. Xu, H. Xia, Z. Ding, B. Li, B. Zhou, J. Yue, J. Jiang, Y . Li, R. An, M. Qin, C. Zong, L. Zheng, Y . Wu, X. Chai, Y . Bi, T. Xie, P. Gu, X. Li, C. Zhang, L. Tian, C. Wang, X. Wang, B. F. Karlsson, B. An, S. Yan, and Z. Lu, “Cradle: Empowering foundation agents towards general computer control,” 2024. [Online]. Available: https://arxiv...

    arXiv 2024

  7. [7]

    Mp5: A multi-modal open-ended embodied system in minecraft via active perception,

    Y . Qin, E. Zhou, Q. Liu, Z. Yin, L. Sheng, R. Zhang, Y . Qiao, and J. Shao, “Mp5: A multi-modal open-ended embodied system in minecraft via active perception,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 16 307–16 316

    work page 2024

  8. [8]

    Describe, explain, plan and select: Interactive planning with LLMs enables open-world multi-task agents,

    Z. Wang, S. Cai, G. Chen, A. Liu, X. Ma, and Y . Liang, “Describe, explain, plan and select: Interactive planning with LLMs enables open-world multi-task agents,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/forum?id=KtvPdGb31Z

    work page 2023

Show all 31 references
  1. [9]

    V oyager: An open-ended embodied agent with large language models,

    G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. Fan, and A. Anandkumar, “V oyager: An open-ended embodied agent with large language models,” in Intrinsically-Motivated and Open- Ended Learning Workshop @NeurIPS2023 , 2023. [Online]. Available: https://openrevie...

    2023

  2. [10]

    Baba is AI: Break the rules to beat the benchmark,

    N. Cloos, M. Jens, M. Naim, Y .-L. Kuo, I. Cases, A. Barbu, and C. J. Cueva, “Baba is AI: Break the rules to beat the benchmark,” in ICML 2024 Workshop on LLMs and Cognition , 2024. [Online]. Available: https://openreview.net/forum?id=jjN1A9CZn4

    2024

  3. [11]

    Avalonbench: Evaluating LLMs playing the game of avalon,

    J. Light, M. Cai, S. Shen, and Z. Hu, “Avalonbench: Evaluating LLMs playing the game of avalon,” in NeurIPS 2023 Foundation Models for Decision Making Workshop , 2023. [Online]. Available: https://openreview.net/forum?id=ltUrSryS0K

    2023

  4. [12]

    Playing repeated games with large language models,

    E. Akata, L. Schulz, J. Coda-Forno, S. J. Oh, M. Bethge, and E. Schulz, “Playing repeated games with large language models,” arXiv preprint arXiv:2305.16867, 2023

    2023 arXiv

  5. [13]

    Amongagents: Evaluating large language models in the interactive text-based social deduction game,

    Y . Chi, L. Mao, and Z. Tang, “Amongagents: Evaluating large language models in the interactive text-based social deduction game,” 2024. [Online]. Available: https://arxiv.org/abs/2407.16521

    2024 arXiv

  6. [14]

    Microscopic analysis on llm players via social deduction game,

    B. Kim, D. Seo, and B. Kim, “Microscopic analysis on llm players via social deduction game,” 2024. [Online]. Available: https://arxiv.org/abs/2408.09946

    2024

  7. [15]

    Language agents with reinforcement learning for strategic play in the werewolf game,

    Z. Xu, C. Yu, F. Fang, Y . Wang, and Y . Wu, “Language agents with reinforcement learning for strategic play in the werewolf game,” arXiv preprint arXiv:2310.18940, 2023

    2023 arXiv

  8. [16]

    Enhance reasoning for large language models in the game werewolf,

    S. Wu, L. Zhu, T. Yang, S. Xu, Q. Fu, Y . Wei, and H. Fu, “Enhance reasoning for large language models in the game werewolf,” arXiv preprint arXiv:2402.02330, 2024

    2024 arXiv

  9. [17]

    Emergent password signalling in the game of werewolf,

    O. Lipinski, A. Sobey, F. Cerutti, and T. Norman, “Emergent password signalling in the game of werewolf,” 2022

    2022

  10. [18]

    Exploring large language models for communication games: An empirical study on werewolf,

    Y . Xu, S. Wang, P. Li, F. Luo, X. Wang, W. Liu, and Y . Liu, “Exploring large language models for communication games: An empirical study on werewolf,” arXiv preprint arXiv:2309.04658 , 2023

    2023 arXiv

  11. [19]

    Werewolf among us: Multimodal resources for modeling persuasion behaviors in social deduction games,

    B. Lai, H. Zhang, M. Liu, A. Pariani, F. Ryan, W. Jia, S. A. Hayati, J. Rehg, and D. Yang, “Werewolf among us: Multimodal resources for modeling persuasion behaviors in social deduction games,” in Findings of the Association for Computational Linguistics: ACL 2023 , A. Rogers,...

    2023

  12. [20]

    Werewolf-xl: A database for identifying spontaneous affect in large competitive group interactions,

    K. Zhang, X. Wu, X. Xie, X. Zhang, H. Zhang, X. Chen, and L. Sun, “Werewolf-xl: A database for identifying spontaneous affect in large competitive group interactions,” IEEE Trans. Affect. Comput., vol. 14, no. 2, p. 1201–1214, apr 2023. [Online]. Available: https://doi.org/10....

    2023

  13. [21]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017. [Online]. Available: https://arxiv.org/abs/1707.06347

    2017 arXiv

  14. [22]

    Proximal policy optimization with mixed distributed training,

    Z. Zhang, X. Luo, T. Liu, S. Xie, J. Wang, W. Wang, Y . Li, and Y . Peng, “Proximal policy optimization with mixed distributed training,” in 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI) . Los Alamitos, CA, USA: IEEE Computer Society, no...

    2019

  15. [23]

    Proximal policy optimization via enhanced exploration efficiency,

    J. Zhang, Z. Zhang, S. Han, and S. L ¨u, “Proximal policy optimization via enhanced exploration efficiency,” Information Sciences , vol. 609, pp. 750–765, 2022. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0020025522008003

    2022

  16. [24]

    Secrets of rlhf in large language models part i: Ppo,

    R. Zheng, S. Dou, S. Gao, Y . Hua, W. Shen, B. Wang, Y . Liu, S. Jin, Q. Liu, Y . Zhou, L. Xiong, L. Chen, Z. Xi, N. Xu, W. Lai, M. Zhu, C. Chang, Z. Yin, R. Weng, W. Cheng, H. Huang, T. Sun, H. Yan, T. Gui, Q. Zhang, X. Qiu, and X. Huang, “Secrets of rlhf in large language mo...

    2023 arXiv

  17. [25]

    Is dpo superior to ppo for llm alignment? a comprehensive study,

    S. Xu, W. Fu, J. Gao, W. Ye, W. Liu, Z. Mei, G. Wang, C. Yu, and Y . Wu, “Is dpo superior to ppo for llm alignment? a comprehensive study,” 2024. [Online]. Available: https://arxiv.org/abs/2404.10719

    2024 arXiv

  18. [26]

    A theoretical analysis of optimistic proximal policy optimization in linear markov decision processes,

    H. Zhong and T. Zhang, “A theoretical analysis of optimistic proximal policy optimization in linear markov decision processes,” in Thirty- seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/forum?id=1bTG4sJ7tN

    2023

  19. [27]

    Finding friend and foe in multi-agent games,

    J. Serrino, M. Kleiman-Weiner, D. C. Parkes, and J. Tenenbaum, “Finding friend and foe in multi-agent games,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  20. [28]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools,

    T. GLM, A. Zeng, B. Xu, B. Wang, C. Zhang, D. Yin, D. Rojas, G. Feng, H. Zhao, H. Lai, H. Yu, H. Wang, J. Sun, J. Zhang, J. Cheng, J. Gui, J. Tang, J. Zhang, J. Li, L. Zhao, L. Wu, L. Zhong, M. Liu, M. Huang, P. Zhang, Q. Zheng, R. Lu, S. Duan, S. Zhang, S. Cao, S. Yang, W. L....

    2024

  21. [29]

    Direct preference optimization: Your language model is secretly a reward model,

    R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.ne...

    2023

  22. [30]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022

    2022 arXiv

  23. [31]

    Least-to-most prompting enables complex reasoning in large language models,

    D. Zhou, N. Sch ¨arli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. V . Le, and E. H. Chi, “Least-to-most prompting enables complex reasoning in large language models,” in The Eleventh International Conference on Learning Representations , 2023. [...

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.