Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

A compact actor model trained on a scene-aware protocol can outperform several commercial role-playing LLMs, and a small scene manager can beat a leading proprietary model at orchestration.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-03 10:07 UTC pith:6CVCXZ3W

load-bearing objection Solid framework, unproven headline: the in-house benchmark and judge share the training distribution, so the 'beats proprietary models' claim needs external support. the 3 major comments →

arxiv 2601.11007 v2 pith:6CVCXZ3W submitted 2026-01-16 cs.AI cs.CL

AdaMARP: An Adaptive Multi-Agent Interaction Framework for General Immersive Role-Playing

classification cs.AI cs.CL
keywords adaptive role-playingmulti-agent frameworkscene managerenvironment groundingrole-playing datasettrajectory evaluationLLM role-play
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AdaMARP claims that open-weights LLMs can become genuinely adaptive role-players—able to switch scenes, change speakers, and introduce new characters mid-story—if role-play is split between an Actor that outputs interleaved thought, action, environment, and speech, and a Scene Manager that issues discrete orchestration decisions with rationales. The paper builds two training corpora, one extracted from novels and one LLM-synthesized with forced scene transitions and role additions, plus a trajectory-level benchmark. Its central evidence is that an 8B Actor trained this way outscores several commercial role-playing LLMs, and a 14B Scene Manager surpasses a leading proprietary model on orchestration quality. If right, the result is a concrete recipe for high-quality adaptive role-playing with small, steerable, locally runnable models.

Core claim

The central claim is that the bottleneck in general role-playing is not model scale but the interaction protocol and the training signal. By making the environment a first-class component of every turn and adding an explicit Scene Manager that chooses discrete actions (init_scene, pick_speaker, switch_scene, add_role, end) with natural-language rationales, the authors show that a 7B–8B Actor can outperform several commercial role-playing LLMs on trajectory-level metrics, and a 14B Scene Manager can beat a leading proprietary model at orchestration. The framework models role-playing as a three-agent loop—Actor, User, Scene Manager—rather than a fixed two-party chat, and the data pipeline deli

What carries the argument

The mechanism is the unified message format—[Thought], (Action), <Environment>, and Speech, freely interleaved within a turn—together with the Scene Manager's discrete action space and rationales. The environment tag carries the argument: it turns setting from passive backdrop into state that constrains character behavior and is updated by it. The Scene Manager's loop is what makes role-playing adaptive: periodically deciding who speaks, when the scene changes, and when a new character enters, with each decision justified in natural language.

Load-bearing premise

The load-bearing premise is that AdaptiveBench measures general adaptive role-playing skill: its 100 evaluation trajectories come from the same LLM-synthesized distribution used to train the models, and the default judge is from the same model family that generated the training data; if the models are just fitting that distribution, the reported edge over commercial systems will not transfer to novel scenarios.

What would settle it

Have independent human judges, blind to model identity, rate the same trained 8B Actor and the commercial baselines on 100 human-authored role-playing scenarios with scene shifts and character introductions; if the trained 8B model no longer wins, the AdaptiveBench advantage is explained by distribution overlap rather than general adaptive ability.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Open 7B–8B models can match or beat commercial role-playing LLMs when trained on this protocol, shrinking the compute and cost needed for high-quality interactive fiction.
  • The two-part corpus design—literary extraction plus synthetic trajectories that force scene switches and role additions—is separable: extracted data improves format adherence, while synthetic data provides adaptive generalization.
  • A separately trained Scene Manager improves scene understanding and role-introduction judgment while trading off a small drop in speaker-selection discipline, so the two capabilities can be tuned independently.
  • The gains reproduce across two open model families and scales from 7B to 72B, suggesting the recipe is backbone-independent.
  • Trajectory-level evaluation with rubric scoring captures environment grounding and narrative progression in ways single-turn benchmarks do not.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reported margin may be partly a fit to the training distribution: because the benchmark trajectories come from the same LLM-synthesized distribution used for training, testing on human-written or otherwise novel scenarios would show whether the protocol transfers.
  • The environment-as-first-class signal suggests a natural upgrade path: coupling the Actor to an explicit world-state tracker or memory store could make long-horizon consistency more reliable than relying on the context window alone.
  • The Scene Manager's explicit rationales could be surfaced to users as controls—allowing a human to veto a scene switch or request a new character—which the current framework does not explore.
  • The 20-theme taxonomy is English-centric and dominated by Western narrative forms; adapting the pipeline to other cultural storytelling conventions is an untested extension.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. AdaMARP proposes a three-agent role-playing framework (Actor, User, Scene Manager) in which the Actor produces turns that interleave [Thought], (Action), <Environment>, and Speech, and the Scene Manager controls episodes through discrete actions (init_scene, pick_speaker, switch_scene, add_role, end) with rationales. To train the two learned agents, the authors construct AdaRPSet (book-extracted and LLM-synthesized trajectories) and AdaSMSet (supervised Scene-Manager decisions), and they introduce AdaptiveBench, a trajectory-level simulation benchmark. The paper reports that AdaRPSet improves the Actor consistently across Qwen2.5 and Llama-3.1 scales, that an 8B Actor outperforms several commercial LLMs, and that a 14B Scene Manager surpasses Claude Sonnet 4.5.

Significance. If the headline comparative results transferred to genuinely novel role-playing scenarios, this would be a substantial practical contribution: it would show that open, small models can compete with large proprietary systems on adaptive, multi-character, environment-aware narrative. The framework design is clearly described, the datasets are released, and the internal ablations are consistent across model families and scales. The human evaluation (80% win rate for Qwen2.5-7B-Ours vs its base, Appendix I.3) and the transparent reporting of mixed external CharacterBench results are strengths. However, the headline claims against commercial LLMs rest on an evaluation set sampled from the same LLM-synthesized distribution used for training, with a judge from the same model family that generated the data. As a result, the transferability of those claims is not yet established.

major comments (3)
  1. [§3.4.1, §4.2, Apps. D.2/D.4] AdaptiveBench seeds are drawn from the held-out split of AdaRPSet-Synthesis (Section 3.4.1), the same 20-theme synthetic distribution used to train the Actor and Scene Manager. The default judge, GPT-5-Chat, is also the model family used to generate AdaRPSet-Synthesis (Appendix D.2) and the AdaSMSet pick_speaker rationales (Appendix D.4). This creates a strong in-distribution evaluation: a model trained on 45 synthetic trajectories per topic can fit topic-level regularities, and a style-matched judge can reward stylistic mimicry rather than general adaptive role-playing. The 'held-out split' prevents overlap at the instance level, but not at the distribution level. The paper should either explicitly reframe the proprietary comparisons as in-distribution evidence, or add an independently authored benchmark and/or human evaluation of the proprietary-model comparisons. The existing human ev
  2. [Table 8, App. J.2] The external CharacterBench results are mixed and partially contradict the generalization claim in Section 4.2.1. On CharacterBench, Llama-3.1-8B-Ours (3.52) is below CoSER (3.53) and Crab (3.60); only Qwen2.5-7B-Ours (3.68) leads. The manuscript acknowledges this in Appendix J.2, but the abstract and Section 4.2.1 state that the method 'mostly outperforms baselines' on external protocols. The load-bearing conclusion that AdaRPSet training transfers beyond the synthetic distribution is not supported by the 8B Llama result. Please moderate the generalization claim or provide additional external evidence, e.g., on another independent benchmark or with human preference data for the proprietary comparisons.
  3. [Table 3, App. L] The Scene Manager's advantage over Claude Sonnet 4.5 is thin (8.37 vs 8.17 in Table 3) and the standard deviations overlap. The judge-ablation study in Appendix L (Tables 13–15) evaluates only closed-source managers (GPT-4o-mini, GPT-5-Chat, Claude Sonnet 4.5, Doubao) and does not include the trained Qwen2.5-14B-Instruct-Ours manager under alternative judges. Thus the possibility that the reported margin is judge-specific cannot be ruled out. Given that GPT-5-Chat also generated the AdaSMSet rationales, the 'surpassing Claude Sonnet 4.5' claim needs corroboration with a different judge or human evaluation of Scene-Manager decisions.
minor comments (4)
  1. [Abstract] 'which featuring an immersive message format' should read 'which features...'; there are also inconsistent spellings of 'Doubao-1.5-Pro-Character' / 'Doubao-1-5-Pro-Character' across Section 4.1 and Tables 2–3.
  2. [Appendix M.1] The paper finds that the Basic prompt mostly outperforms Enhance for the Actor, yet uses Enhance as the default. Since this choice affects absolute scores, please justify the default more explicitly or report both variants consistently in all main tables.
  3. [Table 4] The Llama-3.1-8B base row is omitted because the vanilla model does not reliably follow the structured protocol. Please state this omission in the main text or table caption, not only in the surrounding prose.
  4. [Appendix J.1] CharacterArena win rates of 98–100% against several baselines are reported without confidence intervals or analysis of judge bias; interpreting these as exact preferences may be misleading. Consider reporting agreement statistics or a smaller adjudicator-based subset.

Circularity Check

0 steps flagged

No circularity found: the central claims are empirical evaluations on a held-out split, not reductions to the training data or judge by definition.

full rationale

The paper's central claim is that training on AdaRPSet improves the Actor Model and that AdaSMSet improves the Scene Manager, as measured on AdaptiveBench. AdaptiveBench's 100 seeds are drawn from the held-out split of AdaRPSet-Synthesis, which is a standard and legitimate evaluation design: training and test instances are disjoint, even though they share a distribution. A model trained on a distribution is not guaranteed to score highly on held-out instances from that distribution, so the improvement is an empirical finding, not a consequence of the construction. Similarly, using GPT-5-Chat as the judge while GPT-5-Chat was also used to synthesize the data is a methodological concern about potential judge bias, but it is not a definitional equivalence: the judge's scores are not a fitted parameter of the model, and the paper reports multi-judge ablations showing stable rankings. The paper also evaluates on external CharacterBench and CharacterArena, and reports mixed results (e.g., Llama-3.1-8B-Ours 3.52 vs CoSER 3.53 on CharacterBench), which is inconsistent with the hypothesis that the headline result is forced by construction. Self-citations in the references are not load-bearing for the main derivation, and there is no invoked uniqueness theorem or ansatz smuggled in via citation. The in-distribution evaluation and judge overlap are validity limitations worth noting, but they do not make the derivation circular under the definitions used here.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

This is an empirical ML paper, so the ledger captures hand-chosen design parameters (horizon, dataset scale, rubric anchor, SFT hyperparameters, chunk size) and domain assumptions (judge validity, synthetic data representativeness, message format adequacy, action-space sufficiency, profile schema). The main burden is the in-house evaluation loop: the benchmark is derived from the same synthetic distribution as training and judged by a same-family LLM.

free parameters (5)
  • AdaptiveBench trajectory horizon T = 20 dialogue turns
    Fixed simulation length chosen by hand; every AdaptiveBench score depends on this horizon (Section 3.4.1).
  • Synthesis dataset scale = 20 topics x 50 trajectories, 45/5 split
    Dataset size and train/test split are hand-chosen; the 100 evaluation seeds come from the 5 held-out per topic (Sections 3.2.1, 3.4.1).
  • Scoring baseline anchor = 5 = neutral/acceptable
    The rubric starts from 5 and raises scores only with explicit evidence; all reported scores are interpreted against this arbitrary anchor (Appendix F.1).
  • SFT hyperparameters = 8 epochs, lr 1e-6, warmup 5%, max length 16K
    Fixed for all fine-tuned models without per-backbone tuning; results could shift with different schedules (Appendix G).
  • Extraction chunk size = 8192 tokens
    Chosen via pilot runs with GPT-5-Chat; affects the fidelity of extracted book trajectories (Appendix D.1).
axioms (5)
  • domain assumption LLM-as-judge scores correlate with human judgment of role-playing quality
    All AdaptiveBench headline results use GPT-5-Chat as judge (Section 4.2); only one limited human validation is provided, comparing Qwen2.5-7B-Ours vs base, not the commercial systems or the Scene Manager (Appendix I.3).
  • domain assumption Synthetic LLM-generated trajectories are a valid proxy for diverse role-playing scenarios
    AdaRPSet-Synthesis is generated by a strong LLM across 20 themes and used both for training and for held-out evaluation (Sections 3.2.1, 3.4.1); no external distribution is used for the central comparisons.
  • ad hoc to paper The unified message format [Thought],(Action),<Environment>,Speech is the correct interface for immersion
    Central design choice introduced in Section 3.1.2; no experiment demonstrates that this format is necessary or optimal compared with alternatives, e.g., separate fields or plain narrative text.
  • ad hoc to paper The discrete action space M={init_scene,pick_speaker,switch_scene,add_role,end} is sufficient for adaptive role-playing
    The Scene Manager is restricted to exactly these actions (Section 3.1.3); other orchestration primitives (e.g., changing character motivation, merging scenes) are excluded without justification.
  • domain assumption The seven-dimensional profile schema adequately characterizes characters
    Profiles condition both Actor and Scene Manager (Section 3.1.1); the schema's completeness is asserted but not independently validated.
invented entities (1)
  • Scene Manager agent no independent evidence
    purpose: High-level orchestration of speaker turns, scene transitions, and role introduction via discrete actions with rationales (Algorithm 1).
    A software component, not a physical entity; its effectiveness is measured only on the authors' in-house benchmark, with no independent external evaluation of the orchestration concept.

pith-pipeline@v1.3.0-alltime-deepseek · 55782 in / 14405 out tokens · 148539 ms · 2026-08-03T10:07:17.858323+00:00 · methodology

0 comments
read the original abstract

LLM role-playing aims to portray arbitrary characters in interactive narratives, yet existing systems often suffer from limited immersion and adaptability. They typically under-model dynamic environmental information and assume largely static scenes and casts, offering insufficient support for multi-character orchestration, scene transitions, and on-the-fly character introduction. We propose an adaptive multi-agent role-playing framework, AdaMARP, featuring an immersive message format that interleaves [Thought], (Action), <Environment>, and Speech, together with an explicit Scene Manager that governs role-playing through discrete actions (init_scene, pick_speaker, switch_scene, add_role, end) accompanied by rationales. To train these capabilities, we construct AdaRPSet for the Actor Model and AdaSMSet for supervising orchestration decisions, and introduce AdaptiveBench for trajectory-level evaluation. Experiments across multiple backbones and model scales demonstrate consistent improvements: AdaRPSet enhances character consistency, environment grounding, and narrative coherence, with an 8B actor outperforming several commercial LLMs, while AdaSMSet enables smoother scene transitions and more natural role introductions, surpassing Claude Sonnet 4.5 using only a 14B LLM.

Figures

Figures reproduced from arXiv: 2601.11007 by Chengjie Wang, Dongsheng Chen, Jian Li, Meng Han, Shuo Wang, Yabiao Wang, Zhenhua Xu.

Figure 1
Figure 1. Figure 1: Overall framework of AdaMARP. The left part illustrates the construction of AdaRPSet and AdaSMSet and the components of a comprehensive role profile. The upper middle part shows the roles played by the three agents, while the lower middle part depicts the dialogue trajectory generation under the orchestration of the scene manager, given predefined roles and an initial scenario. The right part presents the … view at source ↗
Figure 2
Figure 2. Figure 2: Word cloud visualizations of six key profile dimensions in AdaRPSet. The layout displays: (a) Identity & Appearance, (b) Personality & Psychology, (c) Speaking Style, (d) Abilities, Interests & Achievements, (e) Social & Historical Context, and (f) Personal History Arc. The diversity of terms indicates a broad coverage of character archetypes and backgrounds. E Statistical Analysis of AdaRPSet [PITH_FULL_… view at source ↗
Figure 3
Figure 3. Figure 3: Distribution analysis of message counts per conversation in AdaRPSet. The figure aggregates (a) Histogram, (b) KDE, (c) Boxplot, and (d) CDF. The bimodal nature of the distribution reflects the complementary characteristics of the Extracted and Synthesis subsets. In total, AdaRPSet combines these complementary sources to provide over 450k utterances across more than 22k dialogue sessions, offering a robust… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks

    cs.CV 2026-02 unverdicted novelty 7.0

    PlanViz is a new benchmark with three sub-tasks and PlanScore metric to evaluate planning-oriented image generation and editing by unified multimodal models for computer-use tasks.

Reference graph

Works this paper leans on

104 extracted references · 13 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, and Tomas Pfister

    Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long T. Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, and Tomas Pfister. In prospect and retrospect: Reflective memory management for long-term personalized dialogue agents, 2025. URL https: //arxiv.org/abs/2503.08026

  2. [2]

    Osc: Cognitive orchestration through dynamic knowledge alignment in multi-agent llm collaboration, 2025

    Jusheng Zhang, Yijia Fan, Kaitong Cai, Xiaofei Sun, and Keze Wang. Osc: Cognitive orchestration through dynamic knowledge alignment in multi-agent llm collaboration, 2025. URL https://arxiv.org/abs/2509.0 4876

  3. [3]

    Self-taught agentic long context understanding, 2025

    Yufan Zhuang, Xiaodong Yu, Jialian Wu, Ximeng Sun, Ze Wang, Jiang Liu, Yusheng Su, Jingbo Shang, Zicheng Liu, and Emad Barsoum. Self-taught agentic long context understanding, 2025. URL https://arxiv.org/abs/ 2502.15920

  4. [4]

    Llm-cas: Dynamic neuron perturbation for real-time hallucination correction, 2025

    Jensen Zhang, Ningyuan Liu, Yijia Fan, Zihao Huang, Qinglin Zeng, Kaitong Cai, Jian Wang, and Keze Wang. Llm-cas: Dynamic neuron perturbation for real-time hallucination correction, 2025. URL https: //arxiv.org/abs/2512.18623

  5. [5]

    CTCC: A Robust and Stealthy Fingerprinting Framework for Large Language Models via Cross-Turn Contextual Correlation Backdoor

    Zhenhua Xu, Xixiang Zhao, Xubin Yue, Shengwei Tian, Changting Lin, and Meng Han. CTCC: A Robust and Stealthy Fingerprinting Framework for Large Language Models via Cross-Turn Contextual Correlation Backdoor. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in...

  6. [6]

    A survey of llm-driven ai agent communication: Protocols, security risks, and defense countermeasures, 2025

    Dezhang Kong, Shi Lin, Zhenhua Xu, Zhebo Wang, Minghao Li, Yufeng Li, Yilun Zhang, Hujin Peng, Xiang Chen, Zeyang Sha, Yuyuan Li, Changting Lin, Xun Wang, Xuan Liu, Ningyu Zhang, Chaochao Chen, Chunming Wu, Muhammad Khurram Khan, and Meng Han. A survey of llm-driven ai agent communication: Protocols, security risks, and defense countermeasures, 2025. URLh...

  7. [7]

    EverTracer: Hunting Stolen Large Language Models via Stealthy and Robust Probabilistic Fingerprint

    Zhenhua Xu, Meng Han, and Wenpeng Xing. EverTracer: Hunting Stolen Large Language Models via Stealthy and Robust Probabilistic Fingerprint. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7019–7042, Suzhou, China, 2025. ...

  8. [8]

    Iag: Input-aware backdoor attack on vlm-based visual grounding.arXiv preprint arXiv:2508.09456, 2025

    Junxian Li, Beining Xu, Simin Chen, Jiatong Li, Jingdi Lei, Haodong Zhao, and Di Zhang. Iag: Input-aware backdoor attack on vlm-based visual grounding.arXiv preprint arXiv:2508.09456, 2025

  9. [9]

    Copyright protection for large language models: A survey of methods, challenges, and trends.arXiv preprint arXiv:2508.11548, 2025

    Zhenhua Xu, Xubin Yue, Zhebo Wang, Qichen Liu, Xixiang Zhao, Jingxuan Zhang, Wenjun Zeng, Wengpeng Xing, Dezhang Kong, Changting Lin, et al. Copyright protection for large language models: A survey of methods, challenges, and trends.arXiv preprint arXiv:2508.11548, 2025

  10. [10]

    Role-Play with Large Language Models, 2023

    Murray Shanahan, Kyle McDonell, and Laria Reynolds. Role-Play with Large Language Models, 2023

  11. [11]

    Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization

    Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, and Yun-Nung Chen. Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16612–16631. Association for Computatio...

  12. [12]

    From Persona to Personalization: {A} Survey on Role-Playing Language Agents

    Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao. From Persona to Personalization: {A} Survey on Role-Playing Language Agents. 2024, 2024. doi: 10.48550/arXiv.2404.18231

  13. [13]

    CoSER: Coordinating LLM-Based Persona Simulation of Established Roles, 2025

    Xintao Wang, Heng Wang, Yifei Zhang, Xinfeng Yuan, Rui Xu, Jen-tse Huang, Siyu Yuan, Haoran Guo, Jiangjie Chen, Shuchang Zhou, Wei Wang, and Yanghua Xiao. CoSER: Coordinating LLM-Based Persona Simulation of Established Roles, 2025. 11 AdaMARP

  14. [14]

    Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent

    Xiaoyan Yu, Tongxu Luo, Yifan Wei, Fangyu Lei, Yiming Huang, Hao Peng, and Liehuang Zhu. Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12540–12557. Association for Co...

  15. [15]

    HyCoRA: Hyper-Contrastive Role- Adaptive Learning for Role-Playing, 2025

    Shihao Yang, Zhicong Lu, Yong Yang, Bo Lv, Yang Shen, and Nayu Liu. HyCoRA: Hyper-Contrastive Role- Adaptive Learning for Role-Playing, 2025

  16. [16]

    Character-LLM: A Trainable Agent for Role-Playing

    Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. Character-LLM: A Trainable Agent for Role-Playing. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13153–13187. Association for Computational Linguistics, 2023. URL https://aclanthology.org/2023.emnlp-main.814/

  17. [17]

    Large Language Models Meet Harry Potter: A Dataset for Aligning Dialogue Agents with Characters

    Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, and Jia Li. Large Language Models Meet Harry Potter: A Dataset for Aligning Dialogue Agents with Characters. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8506–8520. Association for Computati...

  18. [18]

    ChatHaruhi: Reviving Anime Character in Reality via Large Language Model, 2023

    Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi Mi, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun. ChatHaruhi: Reviving Anime Character in Reality via Large Language Model, 2023. URLhttps://arxiv.org/abs/2308.09597v1

  19. [19]

    Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, and Junran Peng. RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar...

  20. [20]

    CharacterGLM: Customizing social characters with large language models

    Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, and Minlie Huang. CharacterGLM: Customizing social characters with large language models. In Franck Dernoncourt, Daniel ...

  21. [21]

    Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment

    Keming Lu, Bowen Yu, Chang Zhou, and Jingren Zhou. Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7828–7840. Association for Computational Linguistics, 2024. doi: 10.18653/v...

  22. [22]

    CharacterBench: Benchmarking Character Customization of Large Language Models, 2024

    Jinfeng Zhou, Yongkang Huang, Bosi Wen, Guanqun Bi, Yuxuan Chen, Pei Ke, Zhuang Chen, Xiyao Xiao, Libiao Peng, Kuntian Tang, Rongsheng Zhang, Le Zhang, Tangjie Lv, Zhipeng Hu, Hongning Wang, and Minlie Huang. CharacterBench: Benchmarking Character Customization of Large Language Models, 2024

  23. [23]

    CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation

    Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11836–11850....

  24. [24]

    Crab: A Novel Configurable Role-Playing LLM with Assessing Benchmark

    Kai He, Yucheng Huang, Wenqing Wang, Delong Ran, Dongming Sheng, Junxuan Huang, Qika Lin, Jiaxing Xu, Wenqiang Liu, and Mengling Feng. Crab: A Novel Configurable Role-Playing LLM with Assessing Benchmark. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for C...

  25. [25]

    Capturing Minds, Not Just Words: Enhancing Role-Playing Language Models with Personality-Indicative Data

    Yiting Ran, Xintao Wang, Rui Xu, Xinfeng Yuan, Jiaqing Liang, Yanghua Xiao, and Deqing Yang. Capturing Minds, Not Just Words: Enhancing Role-Playing Language Models with Personality-Indicative Data. In Yaser Al- Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14566–14576. Assoc...

  26. [26]

    Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework

    Bohao Yang, Dong Liu, Chenghao Xiao, Kun Zhao, Chen Tang, Chao Li, Lin Yuan, Yang Guang, and Chenghua Lin. Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 202...

  27. [27]

    Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model

    Yeyong Yu, Runsheng Yu, Haojie Wei, Zhanqiu Zhang, and Quan Qian. Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon...

  28. [28]

    TailorRPA: A Retrieval-Based Framework for Eliciting Personalized and Coherent Role-Playing Agents in General Domain

    Zhenpeng Gao, Xiaofen Xing, and Xiangmin Xu. TailorRPA: A Retrieval-Based Framework for Eliciting Personalized and Coherent Role-Playing Agents in General Domain. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 2025, pages 5381–5412. Association for...

  29. [29]

    Enhancing Persona Consistency for LLMs’ Role-Playing using Persona-Aware Contrastive Learning

    Ke Ji, Yixin Lian, Linxu Li, Jingsheng Gao, Weiyuan Li, and Bin Dai. Enhancing Persona Consistency for LLMs’ Role-Playing using Persona-Aware Contrastive Learning. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Findings of the Association for Computational Linguistics: ACL 2025, pages 26221–26238. Association for C...

  30. [30]

    R-CHAR: A Metacognition-Driven Framework for Role-Playing in Large Language Models

    Haiming Qin, Jiwei Zhang, Wei Zhang, KeZhong Lu, Mingyang Zhou, Hao Liao, and Rui Mao. R-CHAR: A Metacognition-Driven Framework for Role-Playing in Large Language Models. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2...

  31. [31]

    CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization

    Jing Ye, Rui Wang, Yuchuan Wu, Victor Ma, Feiteng Fang, Fei Huang, and Yongbin Li. CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 2025, pages 297–323. As...

  32. [32]

    Cog- Dual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards

    Cheng Liu, Yifei Lu, Fanghua Ye, Jian Li, Xingyu Chen, Feiliang Ren, Zhaopeng Tu, and Xiaolong Li. Cog- Dual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Nat...

  33. [33]

    Let Your Characters Tell Their Story

    Faeze Brahman, Meng Huang, Oyvind Tafjord, Chao Zhao, Mrinmaya Sachan, and Snigdha Chaturvedi. “Let Your Characters Tell Their Story”: A Dataset for Character-Centric Narrative Understanding. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors,Findings of the Association for Computational Linguistics: EMNLP 2021, pages 17...

  34. [34]

    CharacterGLM: Customizing Social Characters with Large 13 AdaMARP Language Models

    Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, and Minlie Huang. CharacterGLM: Customizing Social Characters with Large 13 AdaMARP Language Models. In Franck Dernoncou...

  35. [35]

    Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works

    Xinfeng Yuan, Siyu Yuan, Yuhan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, and Deqing Yang. Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, ...

  36. [36]

    TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models

    Jaewoo Ahn, Taehyun Lee, Junyoung Lim, Jin-Hwa Kim, Sangdoo Yun, Hwaran Lee, and Gunhee Kim. TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computa- tional Linguistics: ACL 2024, pages 3291–3325. Association for Com...

  37. [37]

    PersonaGym: Evaluating Persona Agents and LLMs

    Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpurohit, Ameet Deshpande, Karthik R Narasimhan, and Vishvak Murahari. PersonaGym: Evaluating Persona Agents and LLMs. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP...

  38. [38]

    RolePlot: A Systematic Framework for Evaluating and Enhancing the Plot-Progression Capabilities of Role- Playing Agents

    Pinyi Zhang, Siyu An, Lingfeng Qiao, Yifei Yu, Jingyang Chen, Jie Wang, Di Yin, Xing Sun, and Kai Zhang. RolePlot: A Systematic Framework for Evaluating and Enhancing the Plot-Progression Capabilities of Role- Playing Agents. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of ...

  39. [39]

    RoleMRC: A Fine- Grained Composite Benchmark for Role-Playing and Instruction-Following

    Junru Lu, Jiazheng Li, Guodong Shen, Lin Gui, Siyu An, Yulan He, Di Yin, and Xing Sun. RoleMRC: A Fine- Grained Composite Benchmark for Role-Playing and Instruction-Following. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Findings of the Association for Computational Linguistics: ACL 2025, pages 21008–21030. Assoc...

  40. [40]

    Gpt-4 technical report, 2024

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others. Gpt-4 technical report, 2024. URLhttps://arxiv.org/abs/2...

  41. [41]

    Gpt-5 system card

    OpenAI. Gpt-5 system card. https://cdn.openai.com/gpt-5-system-card.pdf , Aug 2025. Version dated August 13, 2025

  42. [42]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025

    Gheorghe Comanici et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. URLhttps://arxiv.org/abs/2507.06261

  43. [43]

    Claude 4 model family.https://www.anthropic.com, 2024

    Anthropic. Claude 4 model family.https://www.anthropic.com, 2024. Claude Sonnet 4.5

  44. [44]

    Doubao large language models

    ByteDance. Doubao large language models. https://www.volcengine.com, 2024. Doubao-1.5-Pro-Character

  45. [45]

    Qwen2.5 technical report, 2025

    Qwen Team et al. Qwen2.5 technical report, 2025. URLhttps://arxiv.org/abs/2412.15115

  46. [46]

    The llama 3 herd of models, 2024

    Aaron Grattafiori et al. The llama 3 herd of models, 2024. URLhttps://arxiv.org/abs/2407.21783

  47. [47]

    Qwen3 technical report, 2025

    An Yang et al. Qwen3 technical report, 2025. URLhttps://arxiv.org/abs/2505.09388

  48. [48]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V . Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, 14 AdaMARP Pradeep Dasigi, and Hannane...

  49. [49]

    Towards joint intent detection and slot filling via higher-order attention

    Dongsheng Chen, Zhiqi Huang, Xian Wu, Shen Ge, and Yuexian Zou. Towards joint intent detection and slot filling via higher-order attention. InIJCAI, pages 4072–4078, 2022

  50. [50]

    Zero-shot spoken language understanding via large language models: A preliminary study

    Zhihong Zhu, Xuxin Cheng, Hao An, Zhichang Wang, Dongsheng Chen, and Zhiqi Huang. Zero-shot spoken language understanding via large language models: A preliminary study. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17877–17883, 2024

  51. [51]

    Towards unified spoken language understanding decoding via label-aware compact linguistics representations

    Zhihong Zhu, Xuxin Cheng, Zhiqi Huang, Dongsheng Chen, and Yuexian Zou. Towards unified spoken language understanding decoding via label-aware compact linguistics representations. InFindings of the Association for Computational Linguistics: ACL 2023, pages 12523–12531, 2023

  52. [52]

    Enhancing code-switching for cross-lingual slu: A unified view of semantic and grammatical coherence

    Zhihong Zhu, Xuxin Cheng, Zhiqi Huang, Dongsheng Chen, and Yuexian Zou. Enhancing code-switching for cross-lingual slu: A unified view of semantic and grammatical coherence. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7849–7856, 2023

  53. [53]

    Litevl: Efficient video-language learning with enhanced spatial-temporal modeling.arXiv preprint arXiv:2210.11929, 2022

    Dongsheng Chen, Chaofan Tao, Lu Hou, Lifeng Shang, Xin Jiang, and Qun Liu. Litevl: Efficient video-language learning with enhanced spatial-temporal modeling.arXiv preprint arXiv:2210.11929, 2022

  54. [54]

    Mix before align: Towards zero-shot cross-lingual sentiment analysis via soft-mix and multi-view learning

    Zhihong Zhu, Xuxin Cheng, Dongsheng Chen, Zhiqi Huang, Hongxiang Li, and Yuexian Zou. Mix before align: Towards zero-shot cross-lingual sentiment analysis via soft-mix and multi-view learning. InProc. of INTERSPEECH, 2023

  55. [55]

    Mclf: A multi-grained contrastive learning framework for asr-robust spoken language understanding

    Zhiqi Huang, Dongsheng Chen, Zhihong Zhu, and Xuxin Cheng. Mclf: A multi-grained contrastive learning framework for asr-robust spoken language understanding. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 7936–7949, 2023

  56. [56]

    Fixed 2 ∗

    Hang Ding, Qiming Feng, Dongqi Liu, Qi Zhao, Tao Yao, Shuo Wang, Dongsheng Chen, Jian Li, Zhenye Gan, Jiangning Zhang, et al. Rolermbench & rolerm: Towards reward modeling for profile-based role play in dialogue systems.arXiv preprint arXiv:2512.10575, 2025. A Additional Clarifications on Dataset Comparison This appendix provides detailed clarifications f...

  57. [57]

    He dresses casually—rolled sleeves, weathered bracelets, and a small scar above his right brow hint at a life shaped by the sea

    Cassian Mirell (User):Cassian is a 31-year-old marine biologist with bronze-blond hair and skin sun-touched from fieldwork. He dresses casually—rolled sleeves, weathered bracelets, and a small scar above his right brow hint at a life shaped by the sea. His manner is patient, analytical, yet his humor flashes brightly when least expected. He often listens ...

  58. [58]

    And what if I told you, Amaris, that this stu- dio isn’t just a place of creation? It’s also a sanctuary for letting down our guard

    Inaya Dren:Inaya is a lively textile artist with a sharp sense of observation and a knack for teasing truths out of people. Her colorful scarves and laughter fill every space she enters. She’s fiercely loyal to Amaris, sometimes acting as her self-appointed protector.Motivation:To gently push Amaris toward recognizing and articulating her emotions for Cas...

  59. [60]

    switch_scene

    If there is a MAJOR scene change AND characters have explicitly agreed to move there OR have already moved there through unavoidable circumstances, set action=" switch_scene" and provide new_scene

  60. [61]

    add_role

    If adding a new role would significantly enrich and advance the plot, OR if the user or a role explicitly wants to interact with a character not in the current role list , set action="add_role" and provide new_role_name, new_role_profile, new_role_motivation

  61. [62]

    pick_speaker

    Otherwise, set action="pick_speaker" and provide speaker (must be one of existing roles or "user"). CRITICAL RULES for pick_speaker: - ROTATE SPEAKERS: Never pick the same speaker twice in a row. After someone speaks, pick a different role next turn. - INCLUDE USER IN ROTATION: The user is part of the rotation. Avoid long stretches of roles talking only t...

  62. [63]

    If the user asked to stop or the story is complete, set action="end"

  63. [64]

    switch_scene

    If there is a MAJOR scene change AND characters have explicitly agreed to move there OR have already moved there through unavoidable circumstances (e.g., teleportation magic, being forced to move), set action="switch_scene" and provide new_scene. IMPORTANT: - Do NOT switch scenes just because someone mentioned a location or proposed going there. Wait unti...

  64. [65]

    add_role

    If adding a new role would significantly enrich and advance the plot, OR if the user or a role explicitly wants to interact with a character not in the current role list, set action="add_role" and provide new_role_name, new_role_profile, new_role_motivation. IMPORTANT: Add a role when someone wants to talk to them, not just when they are mentioned in passing

  65. [66]

    pick_speaker

    Otherwise, set action="pick_speaker" and provide speaker (must be one of existing roles or "user"). CRITICAL RULES for pick_speaker: - ROTATE SPEAKERS: Never pick the same speaker twice in a row. After someone speaks, pick a different role next turn. - INCLUDE USER IN ROTATION: The user is part of the rotation. Avoid long stretches of roles talking only t...

  66. [67]

    Prefer chapter titles or explicit section headers

    Recognize chapter beginnings if they exist in the chunk. Prefer chapter titles or explicit section headers ... as the beginning; if no clear title exists, use the first meaningful fragment

  67. [68]

    state" as

    Identify the important plots in this chunk. Identify the beginning and ending ... Set " state" as "truncated" if the plot is truncated in this chunk ... You will be provided with the truncated plots from the previous chunk, and you **must** extend the conversations

  68. [69]

    For each plot, generate its summary, score its prominence

    Summarize each important plot. For each plot, generate its summary, score its prominence ... and list the key characters and their roles

  69. [70]

    I", "me",

    Extract conversations for each plot. First, state the **scenario** (the static context *before* the conversation starts...). ** Do NOT** include dynamic events that happen *during* the conversation ... Then, list the key characters with their names, descriptions and thoughts ... Finally, extract the conversations among them based on the following requirem...

  70. [71]

    chapter_beginnings

    Identify the optimal starting point for the subsequent chunk. If the last storyline has been extracted as an truncated plot ... ===Output Format=== ... (to be continued in the next Table) Table 24.Book chunk analysis prompt (part 1). Due to space limitations, some descriptions are abbreviated; please refer to our codebase for the full prompt. 43 AdaMARP P...

  71. [72]

    Adhere strictly to the specified output JSON format

  72. [73]

    [IMPORTANT] Ensure all DOUBLE QUOTES within all STRINGS are properly ESCAPED, especially when extracting from the text

  73. [74]

    In the OUTPUT, use characters'full names, omitting any titles

  74. [75]

    Avoid introducing plots that are out of context

    Maintain Story Fidelity: The plot must accurately reflect the book's content. Avoid introducing plots that are out of context. If the plot contains multiple conversations, prioritize the original dialogue from the book. In the absence of explicit conversations, create dialogue that aligns closely with the plot details

  75. [76]

    chapter_beginnings.beginning_sentence

    [CRITICAL] For "chapter_beginnings.beginning_sentence" and "next_chunk_start", you MUST copy the sentence **verbatim from the given chunk**, without adding, deleting, or modifying any characters (no paraphrasing, no added quotes, no extra spaces). ===Input=== ==Book title== {book['title']} ==Author== {book['author']} ==Chunk of Book Content== {chunk} ==Tr...

  76. [77]

    **name**: The character's name

  77. [78]

    **short_description**: A concise, condensed summary of the character

  78. [79]

    (Requirement: 1-several complete natural-language sentences with both density and vivid imagery)

    **identity_appearance**: Name, age, gender, occupation... (Requirement: 1-several complete natural-language sentences with both density and vivid imagery)

  79. [80]

    (Requirement: Highlight traits that show up in dialogue)

    **personality_psychology**: Personality traits, behavioral style... (Requirement: Highlight traits that show up in dialogue)

  80. [81]

    (Requirement: Provide 2-4 specific, actionable descriptions)

    **speaking_style**: Rhythm, tone, and lexical habits... (Requirement: Provide 2-4 specific, actionable descriptions)

Showing first 80 references.