Pith. sign in

REVIEW 4 major objections 6 minor 48 references

CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM role-player stays more faithful to a character when it explicitly reasons about the external scene and its own internal state before responding, with two self-derived rewards making that behavior stick through reinforcement learning.

desk verdict The SFT dual-cognition recipe is genuinely promising and survives alternate judges; the RL stage rests on an ICLG reward that looks hacksable, so referee it for the SFT claims and demand an analysis of the denominator-suppression path. read the letter →

arxiv 2507.17147 v1 pith:GBEXKU7Y submitted 2025-07-23 cs.CL

Cheng Liu , Yifei Lu , Fanghua Ye , Jian Li , Xingyu Chen , Feiliang Ren , Zhaopeng Tu , Xiaolong Li This is my paper
classification cs.CL
keywords role-playinglanguageagentsdualcognitionsituationalawarenessself-awarenessreinforcementlearningGRPOimplicitrule-basedrewardscharacterconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that role-playing language models respond inconsistently because they skip the cognitive step a human would take, and that making that step explicit fixes it. It claims that a cognize-then-respond pattern, in which the model first builds external situational awareness and then internal self-awareness before producing dialogue, improves character consistency and contextual alignment. To make the pattern stick, the paper first fine-tunes models on structured dual-cognition traces, then runs reinforcement learning with two rewards derived from the model itself: a likelihood-gain reward for the reasoning trace and a semantic-similarity reward against the reference response. On the CoSER benchmark and two generalization tasks, the resulting model is reported to beat vanilla, chain-of-thought, and supervised fine-tuning baselines, with the reinforcement stage adding a further gain. A sympathetic reader would take the central claim to be that explicit dual cognition, not merely extra reasoning tokens, carries the improvement.

What carries the argument

The load-bearing object is the dual cognition trajectory: a structured JSON-formatted thinking block containing situational awareness (environmental perception plus analysis of others' behavior, emotions, and intentions) and self-awareness (key memory activation, current emotions, perceived intentions, and internal thought), followed by the character's response in an answer block. What carries the argument is the combination of two reward functions computed entirely from the policy and a frozen reference. The Inference-Conditioned Likelihood Gain reward, R_ICLG = (likelihood of the gold response conditioned on prompt plus cognition divided by likelihood conditioned on prompt alone)^(1/|gold|), rewards reasoning traces that make the right response more probable. The Latent Semantic Alignment reward, R_LSA = cosine similarity of mean-pooled last hidden states of the reference model for the generated response versus the gold response, rewards semantic fidelity without requiring exact wording. These rewards are combined with fixed weights and optimized with GRPO, giving a training signal that is self-contained and does not require an external reward model.

What would settle it

Train the same two-stage pipeline with the dual-cognition traces replaced by a matched-length generic structured template and evaluate with a judge that was not used to generate or filter the training data; if the performance gap over baselines disappears, the claimed cognitive mechanism is not the active ingredient.

Watch

Extended reading notes

Core claim

The paper's central discovery is that an LLM role-player performs better when it is trained to think in two ordered layers before speaking: first about what the scene and other characters are doing, feeling, and intending, and then about its own memories, emotions, intentions, and strategy. The authors express this as situational awareness CA = environmental perception plus behavior, emotion, and intention analysis, followed by self-awareness = key memory activation, self-emotion, self-intention, and internal strategy, embedded in a cognize-then-respond generation format. They then show that this behavior can be induced by supervised fine-tuning on structured trajectories and further improved by GRPO reinforcement learning using two implicit rule-based rewards: ICLG, which measures how much the cognitive trace raises the per-token likelihood of the gold response, and LSA, which measures cosine similarity between the generated response and the gold response in the frozen SFT model's mean-pooled latent space. The claim is that, relative to vanilla, CoT, and SFT baselines, this combination yields higher storyline consistency, anthropomorphism, character fidelity, and storyline quality on CoSER, and higher accuracy on Cross-MR and LifeChoice.

Load-bearing premise

The method's training signal and its main evaluation both rely on GPT-4o's judgment of what counts as in-character cognition, so the reported gains could partly reflect imitation of that judge rather than a psychologically valid mechanism.

Editorial extensions

If this is right

  • Adding the cognitive-based chain-of-thought prompting version of dual cognition improves vanilla LLMs on CoSER without any training.
  • Two-stage CogDual (SFT then RL with ICLG and LSA) consistently beats CogDual-SFT and the listed baselines across CoSER metrics, with RL gains concentrated in storyline consistency and character fidelity.
  • The gains transfer to out-of-distribution multiple-choice tasks, with CogDual-RL outperforming SFT and baseline models on Cross-MR and LifeChoice.
  • The hybrid reward weighting matters: pure ICLG favors narrative causal structure, pure LSA favors persona-like language, and the 0.7/0.3 mix balances the two.
  • The full dual cognition structure outperforms variants missing situational or self-awareness on the main benchmark, showing that both components contribute to the result.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is right, dual cognition offers a cheap substitute for test-time reasoning in role-play, giving small open models a large part of the gain that a high-end reasoning model gets from long deliberation, without the style drift that long reasoning traces cause.
  • The ICLG and LSA rewards are model-intrinsic and could be dropped into other open-ended generation tasks with a reference response, such as story continuation or dialogue safety, without training a separate reward model.
  • A direct test of the paper's underlying thesis would be to vary only the content of the reasoning template: if a non-cognitive structured template of the same length produced the same gains, the improvement would be attributable to structured reasoning rather than dual cognition specifically.
  • The reliance on GPT-4o for both trace generation and filtering and benchmark scoring leaves open the possibility that the published gap over baselines partly reflects judge-model alignment; re-running with human-authored traces and a non-GPT-4o judge would separate the cognitive mechanism from evaluator bias.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CogDual, a role-playing language agent that interposes a structured dual-cognition trace (situational awareness followed by self-awareness) between context and response. The method is realized as a CB-CoT prompting baseline and a two-stage training pipeline: SFT on GPT-4o-generated dual-cognition trajectories, followed by GRPO-based RL with two rewards—an Inference-Conditioned Likelihood Gain (ICLG) reward that measures how much the model's own cognition trace increases the likelihood of the golden response, and a Latent Semantic Alignment (LSA) reward that computes cosine similarity between the generated response and the golden response in the frozen SFT model's embedding space. Experiments on CoSER, Cross-MR, and LifeChoice report consistent improvements over Vanilla, CoT, CoSER-SFT, and LongCoT baselines across LLaMA3.1-8B, Qwen2.5-7B, and LLaMA3.1-70B, with additional evaluator-robustness checks and a 50-sample human study.

Significance. If the results hold, the paper offers a practical recipe for applying RL to open-ended role-play without an external reward model, and it makes a psychologically motivated architectural claim that explicit dual cognition improves character consistency and context alignment. The strengths of the work are substantial: consistent wins across three open-source base models, verification with two alternative automatic judges (DeepSeek-v3 and Gemini-2.0-Flash), a human evaluation, detailed case studies, and component ablations. However, the ICLG reward's definition is self-referential and could be exploited in a way that undermines the claim that the RL stage improves causal reasoning, and the evaluation protocol has two places where training and evaluation overlap (GPT-4o as both data generator and judge; LSA optimizing semantic similarity while Cross-MR/LifeChoice accuracy is computed by GPT-4o semantic matching). These issues are load-bearing for the central claims and require additional analysis or protocol changes.

major comments (4)
  1. [3.4.2, Eq. (5)] The ICLG reward is defined using the current policy in both numerator and denominator. Because the denominator πθ(d_golden|x) is not a fixed baseline, the policy can increase the reward by suppressing the unconditioned likelihood of the golden response, which would make the likelihood gain appear large even if the cognition trace contributes no causal reasoning. The paper does not analyze this denominator-suppression path, and the KL penalty (β=0.001, Eq. 9) is likely too weak to prevent it over long sequences. The claim in §5.1 that 'the ICLG reward effectively guides the model to produce reasoning traces that advance the narrative in a causal, coherent manner' is therefore not established by the objective alone; please either use a frozen reference policy for the denominator, or report evidence (e.g., trajectories' denominator log-likelihoods over training) that this reward-hacking path is not exploited.
  2. [5.4, Table 3, and Appendix A] The reward weight ablation (Table 3) is evaluated on the same CoSER test set used for the final main results, and the optimal weights (λ_ICLG=0.7, λ_LSA=0.3) are then applied to produce the CogDual-RL rows in Table 1. This constitutes test-set selection: the reported numbers for the selected configuration are not independent of the model-choice procedure. Please select weights on a validation split (e.g., a held-out portion of D_cog or a separate set of novels) and report final performance on the test set only for the chosen configuration.
  3. [5.2, Appendix G] The Cross-MR and LifeChoice accuracy is computed by asking GPT-4o to choose the option most semantically similar to the model's generated response. Since the LSA reward (Eq. 6) explicitly optimizes cosine similarity to the gold response, the evaluation protocol is aligned with the training objective, which could inflate the apparent generalization gains of CogDual-RL relative to methods not trained with LSA. Please validate the generalization results with human accuracy judgments or with a metric that does not rely on semantic similarity (e.g., exact option selection in a forced-choice generation setting), and report whether the human evaluation in Appendix C covers these benchmarks.
  4. [Appendix C] The human evaluation is limited to 50 samples from five works, a single base model (LLaMA3.1-8B-Instruct), and five annotators. The average win rate for CogDual-RL is 31.1% versus 29.8% for CogDual-SFT, a small margin, and no inter-annotator agreement is reported. The conclusion that CogDual provides 'superior performance on subjective metrics' is thus only weakly supported; please add significance testing, agreement statistics, or a larger sample, and clarify whether the difference is statistically meaningful.
minor comments (6)
  1. [3.4.2] The paper refers to 'Grouped Reward Policy Optimization' for GRPO; the standard name is 'Group Relative Policy Optimization'.
  2. [Table 3 caption] The caption contains a doubled comma: 'λICLG=0.7, , λLSA=0.3' should read 'λ_ICLG=0.7, λ_LSA=0.3'.
  3. [5.4] The text states that the hybrid setting 'improves or maintains all in-domain metrics,' but Table 3 shows that the hybrid (0.7/0.3) has lower Anthropomorphism (45.89 vs. 46.92) and lower Storyline Quality (73.08 vs. 75.07) than CogDual-SFT, so the claim is inaccurate.
  4. [5.1] The 'average increase of 13.12%' does not match the two cited absolute gains of 11.65 and 21.84 points; please clarify whether the average is computed over different metrics/models or correct the number.
  5. [Table 10 prompt] The prompt contains the typo 'coginitive' in the output format; it should be 'cognitive'.
  6. [Title and Section 3.4.2] The term 'implicit rule-based rewards' is misleading because the ICLG reward depends on the policy being optimized, not on a fixed rule; consider renaming it to something like 'policy-derived rewards' or clarify the sense in which it is rule-based.

Circularity Check

3 steps flagged · score 4.0 of 10

ICLG reward is policy-dependent and can be inflated by denominator suppression; Cross-MR/LifeChoice evaluation overlaps with the LSA semantic-similarity objective; the paper itself concedes the GPT-4o train/judge overlap.

  1. self definitional [Section 3.4.2, Eq. (5)]
    "RICLG(x, dgolden, c) = ( πθ(dgolden | x ⊕ c) / πθ(dgolden | x) )^{1/|dgolden|} = ( ∏_{t=1}^{|dgolden|} πθ(dt | d<t, x⊕ c) / ∏_{t=1}^{|dgolden|} πθ(dt | d<t, x) )^{1/|dgolden|}"

    The 'Inference-Conditioned Likelihood Gain' is defined using the very policy θ being optimized. Maximizing R_ICLG can therefore be achieved not only by raising the cognition-conditioned likelihood of d_golden, but also by lowering the unconditioned likelihood πθ(d_golden | x). Section 5.1 then claims 'the ICLG reward effectively guides the model to produce reasoning traces that advance the narrative in a causal, coherent manner,' but the objective itself does not distinguish increased causal utility from denominator suppression. The KL penalty (β=0.001, Eq. 9) only bounds divergence from the frozen reference; it does not eliminate this spurious path. The RL 'gain' is thus partly a self-referential artifact of optimizing a reward written in terms of the optimized policy.

  2. fitted input called prediction [Section 5.2 and Section 3.4.2, Eq. (6)]
    "To align CogDual with this format, we use GPT-4o to choose the option that is most semantically similar to the response part generated by CogDual ... RLSA(x, dgolden, ˆd) = cos(fref(x, dgolden), fref(x, ˆd))"

    The LSA reward used in RL training optimizes cosine semantic similarity between the generated response and the gold reference in a frozen model's latent space. The Cross-MR and LifeChoice generalization evaluation then selects the option 'most semantically similar to the response part generated by CogDual' via GPT-4o. The reported generalization gains on these benchmarks are therefore measured by essentially the same semantic-similarity operation that the LSA reward directly optimizes. CogDual-RL's advantage over CogDual-SFT on these benchmarks is partly a re-measurement of the training objective rather than independent evidence of generalizable role-playing ability.

1 more flagged steps
  1. other [Appendix B]
    "CogDual-SFT is trained on reasoning chains partially generated by GPT-4o, potentially introducing evaluator bias and yielding evaluation results that disproportionately favor GPT-4o-aligned behavior."

    The primary CoSER evaluation in Section 4.3 also uses GPT-4o as the critic. The same model that wrote and filtered the cognition trajectories in Appendix A judges whether the trained model is consistent and faithful. This does not make the CoSER results vacuous: Appendix B re-runs with DeepSeek-v3 and Gemini-2.0-Flash, and Appendix C adds a human evaluation. However, the paper itself concedes the overlap, and the main SFT-vs-baseline comparison is partially a train/evaluator alignment effect rather than a purely task-general cognitive improvement.

full rationale

The central CoSER claim is not reduced to a fit: alternate judges and a human study provide independent support, and no load-bearing self-citation chain was found. However, the ICLG reward is policy-dependent and can be inflated by suppressing the unconditioned likelihood of the golden response, so the RL validation of 'reasoning utility' is partly self-referential. In addition, the Cross-MR/LifeChoice evaluation uses a semantic-matching procedure closely aligned with the LSA training objective, making the generalization claim partially a re-test of the reward. The paper's own Appendix B acknowledges the GPT-4o train/evaluate overlap. These are partial circularities in the RL and generalization claims, but not a full derivation-equals-input collapse, so a moderate score of 4 is appropriate.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

Central claim is empirical; the ledger captures the assumptions the experiments rely on. Free parameters include the reward weights chosen from an ablation on the same benchmark, the data subsampling choices, and the RL hyperparameter set. The axioms include the validity of GPT-4o-generated cognitive trajectories as gold supervision, the validity of ICLG and LSA as rewards, and the choice of mean-pooled frozen hidden states as a semantic space. No new physical or ontological entities are introduced; the dual cognition components are structured text formats, not invented entities with independent evidence.

free parameters (4)
  • Reward weights lambda_ICLG and lambda_LSA = 0.7 and 0.3
    Selected from the ablation in Table 3 on LLaMA3.1-8B using the CoSER evaluation itself; all RL results use this ratio.
  • SFT data subsample and filtering sizes = 400 characters; 38,724 cognitively filtered instances; 17,762 SFT examples
    Hand-chosen subsampling and GPT-4o filtering thresholds; no sensitivity analysis is reported.
  • RL hyperparameters (learning rate, steps, temperature, clip, KL coefficient) = LR 4e-7; 120 steps; temp 0.7; clip 0.2; KL 0.001
    Hand-set values from standard practice; no ablation is reported for these.
  • First-person versus third-person sampling probability in data generation = 50% / 50%
    Stochastic prompting choice in Appendix A; not varied or justified experimentally.
assumptions (6)
  • domain assumption Traces of situational awareness and self-awareness, in this order, are a valid and beneficial abstraction for role-play cognition.
    Section 3.3 and Figure 1; the whole method depends on this modeling choice, which is justified by cognitive psychology citations rather than by a derivation.
  • domain assumption GPT-4o-generated cognitive trajectories and GPT-4o filtering are high-quality gold supervision.
    Appendix A; if GPT-4o's cognition style is biased, the SFT and RL signal inherits that bias.
  • ad hoc to paper The ICLG likelihood-gain ratio in Eq. 5 is a valid reward for reasoning utility.
    Adapted from LATRO (Chen et al., 2024a); the formula assumes likelihood gain reflects causal coherence, but it can also be optimized by suppressing the denominator.
  • domain assumption Mean-pooled last hidden states of the frozen SFT reference model capture semantic similarity (Eq. 6).
    Based on Tao et al. (2024a); no role-play-specific validation of this embedding space is provided.
  • domain assumption GPT-4o as the CoSER critic produces scores that track the intended four quality dimensions.
    Section 4.3; alternative judges and a 50-sample human study mitigate this, but the automated metrics remain model-based.
  • standard math GRPO with the clipped surrogate loss and KL penalty is a valid optimizer for these rewards.
    Eq. 9 follows Shao et al. (2024); treated as background method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards." pith.science (2026). https://pith.science/paper/GBEXKU7Y

@misc{pith2026250717147,
  author       = {Pith},
  title        = {Pith review of: CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GBEXKU7Y}},
  note         = {Machine review of arXiv:2507.17147}
}
read the original abstract

Role-Playing Language Agents (RPLAs) have emerged as a significant application direction for Large Language Models (LLMs). Existing approaches typically rely on prompt engineering or supervised fine-tuning to enable models to imitate character behaviors in specific scenarios, but often neglect the underlying \emph{cognitive} mechanisms driving these behaviors. Inspired by cognitive psychology, we introduce \textbf{CogDual}, a novel RPLA adopting a \textit{cognize-then-respond } reasoning paradigm. By jointly modeling external situational awareness and internal self-awareness, CogDual generates responses with improved character consistency and contextual alignment. To further optimize the performance, we employ reinforcement learning with two general-purpose reward schemes designed for open-domain text generation. Extensive experiments on the CoSER benchmark, as well as Cross-MR and LifeChoice, demonstrate that CogDual consistently outperforms existing baselines and generalizes effectively across diverse role-playing tasks.

Figures

Figures reproduced from arXiv: 2507.17147 by the authors.

Figure 1
Figure 1. An example of CogDual. Before a character [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of CogDual training. (1) Stage 1: Supervised fine-tuning using the role-specific dual cognitive [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Ablation study on cognitive components. "SC", "AP", "CF", and "SQ" correspond to the four met￾rics: Storyline Consistency, Anthropomorphism, Char￾acter Fidelity, and Storyline Quality, respectively. "CM" denotes Cross-MR, and "LC" denotes LifeChoice. 5.3 Ablation Study I: Effect of Dual Cognition Components We first conduct an ablation study focusing on the effect of dual cognition structures during SFT stage [PITH… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 8 canonical work pages

  1. [1]

    let your characters tell their story

    Faeze Brahman, Meng Huang, Oyvind Tafjord, Chao Zhao, Mrinmaya Sachan, and Snigdha Chaturvedi. 2021. https://arxiv.org/abs/2109.05438 "let your characters tell their story": A dataset for character-centric narrative understanding . Preprint, arXiv:2109.05438

    arXiv 2021

  2. [2]

    Haolin Chen, Yihao Feng, Zuxin Liu, Weiran Yao, Akshara Prabhakar, Shelby Heinecke, Ricky Ho, Phil Mui, Silvio Savarese, Caiming Xiong, and Huan Wang. 2024 a . https://arxiv.org/abs/2411.04282 Language models are hidden reasoners: Unlocking latent reasoning capabilities via self-rewarding . Preprint, arXiv:2411.04282

    arXiv 2024

  3. [3]

    Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Gao Xing, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, and Fei Huang. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.125 S ocial B ench: Sociality evaluation of role-playing conversational agents . In Findings of the Association for Computational Linguistics: ACL 2024, pages 2108--2126, Bang...

    doi:10.18653/v1/2024.findings-acl.125 2024

  4. [4]

    Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao. 2024 c . https://openreview.net/forum?id=xrO70E8UIZ From persona to personalization: A survey on role-playing language agents . Transactions on Machi...

    work page 2024

  5. [5]

    Jiaqi Chen, Bang Zhang, Ruotian Ma, Peisong Wang, Xiaodan Liang, Zhaopeng Tu, Xiaolong Li, and Kwan-Yee K. Wong. 2025 a . https://arxiv.org/abs/2504.19162 Spc: Evolving self-play critic via adversarial games for llm reasoning . Preprint, arXiv:2504.19162

    arXiv 2025

  6. [6]

    Nuo Chen, Yan Wang, Yang Deng, and Jia Li. 2025 b . https://arxiv.org/abs/2407.11484 The oscars of ai theater: A survey on role-playing with language models . Preprint, arXiv:2407.11484

    arXiv 2025

  7. [7]

    Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, and Jia Li. 2023. https://arxiv.org/abs/2211.06869 Large language models meet harry potter: A bilingual dataset for aligning dialogue agents with characters . Preprint, arXiv:2211.06869

    arXiv 2023

  8. [8]

    Herbert H Clark and Susan E Brennan. 1991. Grounding in communication

    1991

Show all 48 references
  1. [9]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 9 others. 2025. https://arxiv.org/abs/2501.12948...

    2025 arXiv

  2. [10]

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025 a . https://arxiv.org/abs/2504.11536 Retool: Reinforcement learning for strategic tool use in llms . Preprint, arXiv:2504.11536

    2025 arXiv

  3. [11]

    Xiachong Feng, Longxu Dou, and Lingpeng Kong. 2025 b . https://arxiv.org/abs/2502.16940 Reasoning does not necessarily improve role-playing ability . Preprint, arXiv:2502.16940

    2025 arXiv

  4. [12]

    Herbert P Grice. 1975. Logic and conversation. In Speech acts, pages 41--58. Brill

    1975

  5. [13]

    Jiaming He, Cheng Liu, Guanyu Hou, Wenbo Jiang, and Jiachen Li. 2025. https://doi.org/10.1109/ICASSP49660.2025.10887843 Press: Defending privacy in retrieval-augmented generation via embedding space shifting . In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, S...

    2025

  6. [14]

    Ke Ji, Yixin Lian, Linxu Li, Jingsheng Gao, Weiyuan Li, and Bin Dai. 2025. https://arxiv.org/abs/2503.17662 Enhancing persona consistency for llms' role-playing using persona-aware contrastive learning . Preprint, arXiv:2503.17662

    2025 arXiv

  7. [15]

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. https://openreview.net/forum?id=3IyL2XWDkG CAMEL : Communicative agents for ''mind'' exploration of large language model society . In Thirty-seventh Conference on Neural Informati...

    2023

  8. [16]

    Yifei Lu, Fanghua Ye, Jian Li, Qiang Gao, Cheng Liu, Haibo Luo, Nan Du, Xiaolong Li, and Feiliang Ren. 2025. Codetool: Enhancing programmatic tool invocation of llms via process supervision. arXiv preprint arXiv:2503.20840

    2025 arXiv

  9. [17]

    Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, and Jia Li. 2025. https://arxiv.org/abs/2502.12853 S ^2 r: Teaching llms to self-verify and self-correct via reinforcement learning . Preprint, arXiv:2502.12853

    2025 arXiv

  10. [18]

    Youssef Mroueh. 2025. https://arxiv.org/abs/2503.06639 Reinforcement learning with verifiable rewards: Grpo's effective loss, dynamics, and success amplification . Preprint, arXiv:2503.06639

    2025

  11. [19]

    OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 45 othe...

    2024 arXiv

  12. [20]

    O'Brien, Carrie J

    Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. https://arxiv.org/abs/2304.03442 Generative agents: Interactive simulacra of human behavior . Preprint, arXiv:2304.03442

    2023 arXiv

  13. [21]

    Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. 2025. https://arxiv.org/abs/2504.13958 Toolrl: Reward is all tool learning needs . Preprint, arXiv:2504.13958

    2025 arXiv

  14. [22]

    Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. https://arxiv.org/abs/1511.06732 Sequence level training with recurrent neural networks . Preprint, arXiv:1511.06732

    2016 arXiv

  15. [23]

    Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. https://arxiv.org/abs/1811.00207 Towards empathetic open-domain conversation models: a new benchmark and dataset . Preprint, arXiv:1811.00207

    2019 arXiv

  16. [24]

    Soham Sane. 2025. https://arxiv.org/abs/2502.01652 Hybrid group relative policy optimization: A multi-sample approach to enhancing policy optimization . Preprint, arXiv:2502.01652

    2025 arXiv

  17. [25]

    Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. https://arxiv.org/abs/2310.10158 Character-llm: A trainable agent for role-playing . Preprint, arXiv:2310.10158

    2023 arXiv

  18. [26]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. https://arxiv.org/abs/2402.03300 Deepseekmath: Pushing the limits of mathematical reasoning in open language models . Preprint, arXiv:2402.03300

    2024 arXiv

  19. [27]

    Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. https://arxiv.org/abs/2303.11366 Reflexion: Language agents with verbal reinforcement learning . Preprint, arXiv:2303.11366

    2023 arXiv

  20. [28]

    Chongyang Tao, Tao Shen, Shen Gao, Junshuo Zhang, Zhen Li, Zhengwei Tao, and Shuai Ma. 2024 a . https://arxiv.org/abs/2412.12591 Llms are also effective embedding models: An in-depth overview . Preprint, arXiv:2412.12591

    2024 arXiv

  21. [29]

    Meiling Tao, Liang Xuechen, Tianyu Shi, Lei Yu, and Yiting Xie. 2024 b . https://aclanthology.org/2024.personalize-1.1/ R ole C raft- GLM : Advancing personalized role-playing in large language models . In Proceedings of the 1st Workshop on Personalization of Generative AI Sys...

    2024

  22. [30]

    Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Q...

    2022 arXiv

  23. [31]

    Michael Tomasello. 2010. Origins of human communication. MIT press

    2010

  24. [32]

    Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. 2024. https://doi.org/10.18653/v1/2024.acl-long.638 C haracter E val: A C hinese benchmark for role-playing conversational agent evaluation . In Proceedings of the 62nd Annual Meeting of the Ass...

    2024 doi

  25. [33]

    Qian Wang, Zhenheng Tang, and Bingsheng He. 2025 a . https://openreview.net/forum?id=dMrhmQdrdW Can LLM simulations truly reflect humanity? a deep dive . In The Fourth Blogpost Track at ICLR 2025

    2025

  26. [34]

    Xintao Wang, Heng Wang, Yifei Zhang, Xinfeng Yuan, Rui Xu, Jen tse Huang, Siyu Yuan, Haoran Guo, Jiangjie Chen, Wei Wang, Yanghua Xiao, and Shuchang Zhou. 2025 b . https://arxiv.org/abs/2502.09082 Coser: Coordinating llm-based persona simulation of established roles . Preprint...

    2025

  27. [35]

    Xintao Wang, Yunze Xiao, Jen tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao. 2024 a . https://arxiv.org/abs/2310.17976 Incharacter: Evaluating personality fidelity in role-playing agents through ps...

    2024 arXiv

  28. [36]

    Huang, Jie Fu, and Junran Peng

    Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Stephen W. Huang, Jie Fu, and Junran Peng. 2024 b . https://arxiv.org/abs/2310.00746 Rolellm...

    2024 arXiv

  29. [37]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. https://arxiv.org/abs/2201.11903 Chain-of-thought prompting elicits reasoning in large language models . Preprint, arXiv:2201.11903

    2023 arXiv

  30. [38]

    Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2025 a . https://openreview.net/forum?id=pZiyCaVuti Longmemeval: Benchmarking chat assistants on long-term interactive memory . In The Thirteenth International Conference on Learning Representations

    2025

  31. [39]

    Hongqiu Wu, Weiqi Wu, Tianyang Xu, Jiameng Zhang, and Hai Zhao. 2025 b . https://arxiv.org/abs/2502.17878 Towards enhanced immersion and agency for llm-based interactive drama . Preprint, arXiv:2502.17878

    2025 arXiv

  32. [40]

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. https://arxiv.org/abs/2502.14768 Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning . Preprint, arXiv:2502.14768

    2025 arXiv

  33. [41]

    Rui Xu, MingYu Wang, XinTao Wang, Dakuan Lu, Xiaoyu Tan, Wei Chu, and Yinghui Xu. 2025. https://arxiv.org/abs/2503.08193 Guess what i am thinking: A benchmark for inner thought reasoning of role-playing language agents . Preprint, arXiv:2503.08193

    2025 arXiv

  34. [42]

    Rui Xu, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, Xiaoqing Dong, and Yanghua Xiao. 2024. https://arxiv.org/abs/2404.12138 Character is destiny: Can role-playing language agents make persona-driven decisions? Preprint, arXiv:2404.12138

    2024 arXiv

  35. [43]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. 2024. https://arxiv.org/abs/2409.12122 Qwen2.5-math technical report: ...

    2024 arXiv

  36. [44]

    Mo Yu, Qiujing Wang, Shunchi Zhang, Yisi Sang, Kangsheng Pu, Zekai Wei, Han Wang, Liyan Xu, Jing Li, Yue Yu, and Jie Zhou. 2024. https://arxiv.org/abs/2211.04684 Few-shot character understanding in movies as an assessment to meta-learning of theory-of-mind . Preprint, arXiv:2211.04684

    2024 arXiv

  37. [45]

    Xinfeng Yuan, Siyu Yuan, Yuhan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, and Deqing Yang. 2024. https://arxiv.org/abs/2404.12726 Evaluating character understanding of large language models via character profiling from fictional works . Preprint, arXiv:2404.12726

    2024 arXiv

  38. [46]

    Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Libiao Peng, Jiaming Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Jie Tang, and Minlie Huang. 2023. https://arxiv.org/abs/2311.16832 Characterglm: Cust...

    2023 arXiv

  39. [47]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.