REVIEW 3 major objections 4 minor 1 cited by
A compact actor model trained on a scene-aware protocol can outperform several commercial role-playing LLMs, and a small scene manager can beat a leading proprietary model at orchestration.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 10:07 UTC pith:6CVCXZ3W
load-bearing objection Solid framework, unproven headline: the in-house benchmark and judge share the training distribution, so the 'beats proprietary models' claim needs external support. the 3 major comments →
AdaMARP: An Adaptive Multi-Agent Interaction Framework for General Immersive Role-Playing
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the bottleneck in general role-playing is not model scale but the interaction protocol and the training signal. By making the environment a first-class component of every turn and adding an explicit Scene Manager that chooses discrete actions (init_scene, pick_speaker, switch_scene, add_role, end) with natural-language rationales, the authors show that a 7B–8B Actor can outperform several commercial role-playing LLMs on trajectory-level metrics, and a 14B Scene Manager can beat a leading proprietary model at orchestration. The framework models role-playing as a three-agent loop—Actor, User, Scene Manager—rather than a fixed two-party chat, and the data pipeline deli
What carries the argument
The mechanism is the unified message format—[Thought], (Action), <Environment>, and Speech, freely interleaved within a turn—together with the Scene Manager's discrete action space and rationales. The environment tag carries the argument: it turns setting from passive backdrop into state that constrains character behavior and is updated by it. The Scene Manager's loop is what makes role-playing adaptive: periodically deciding who speaks, when the scene changes, and when a new character enters, with each decision justified in natural language.
Load-bearing premise
The load-bearing premise is that AdaptiveBench measures general adaptive role-playing skill: its 100 evaluation trajectories come from the same LLM-synthesized distribution used to train the models, and the default judge is from the same model family that generated the training data; if the models are just fitting that distribution, the reported edge over commercial systems will not transfer to novel scenarios.
What would settle it
Have independent human judges, blind to model identity, rate the same trained 8B Actor and the commercial baselines on 100 human-authored role-playing scenarios with scene shifts and character introductions; if the trained 8B model no longer wins, the AdaptiveBench advantage is explained by distribution overlap rather than general adaptive ability.
If this is right
- Open 7B–8B models can match or beat commercial role-playing LLMs when trained on this protocol, shrinking the compute and cost needed for high-quality interactive fiction.
- The two-part corpus design—literary extraction plus synthetic trajectories that force scene switches and role additions—is separable: extracted data improves format adherence, while synthetic data provides adaptive generalization.
- A separately trained Scene Manager improves scene understanding and role-introduction judgment while trading off a small drop in speaker-selection discipline, so the two capabilities can be tuned independently.
- The gains reproduce across two open model families and scales from 7B to 72B, suggesting the recipe is backbone-independent.
- Trajectory-level evaluation with rubric scoring captures environment grounding and narrative progression in ways single-turn benchmarks do not.
Where Pith is reading between the lines
- The reported margin may be partly a fit to the training distribution: because the benchmark trajectories come from the same LLM-synthesized distribution used for training, testing on human-written or otherwise novel scenarios would show whether the protocol transfers.
- The environment-as-first-class signal suggests a natural upgrade path: coupling the Actor to an explicit world-state tracker or memory store could make long-horizon consistency more reliable than relying on the context window alone.
- The Scene Manager's explicit rationales could be surfaced to users as controls—allowing a human to veto a scene switch or request a new character—which the current framework does not explore.
- The 20-theme taxonomy is English-centric and dominated by Western narrative forms; adapting the pipeline to other cultural storytelling conventions is an untested extension.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. AdaMARP proposes a three-agent role-playing framework (Actor, User, Scene Manager) in which the Actor produces turns that interleave [Thought], (Action), <Environment>, and Speech, and the Scene Manager controls episodes through discrete actions (init_scene, pick_speaker, switch_scene, add_role, end) with rationales. To train the two learned agents, the authors construct AdaRPSet (book-extracted and LLM-synthesized trajectories) and AdaSMSet (supervised Scene-Manager decisions), and they introduce AdaptiveBench, a trajectory-level simulation benchmark. The paper reports that AdaRPSet improves the Actor consistently across Qwen2.5 and Llama-3.1 scales, that an 8B Actor outperforms several commercial LLMs, and that a 14B Scene Manager surpasses Claude Sonnet 4.5.
Significance. If the headline comparative results transferred to genuinely novel role-playing scenarios, this would be a substantial practical contribution: it would show that open, small models can compete with large proprietary systems on adaptive, multi-character, environment-aware narrative. The framework design is clearly described, the datasets are released, and the internal ablations are consistent across model families and scales. The human evaluation (80% win rate for Qwen2.5-7B-Ours vs its base, Appendix I.3) and the transparent reporting of mixed external CharacterBench results are strengths. However, the headline claims against commercial LLMs rest on an evaluation set sampled from the same LLM-synthesized distribution used for training, with a judge from the same model family that generated the data. As a result, the transferability of those claims is not yet established.
major comments (3)
- [§3.4.1, §4.2, Apps. D.2/D.4] AdaptiveBench seeds are drawn from the held-out split of AdaRPSet-Synthesis (Section 3.4.1), the same 20-theme synthetic distribution used to train the Actor and Scene Manager. The default judge, GPT-5-Chat, is also the model family used to generate AdaRPSet-Synthesis (Appendix D.2) and the AdaSMSet pick_speaker rationales (Appendix D.4). This creates a strong in-distribution evaluation: a model trained on 45 synthetic trajectories per topic can fit topic-level regularities, and a style-matched judge can reward stylistic mimicry rather than general adaptive role-playing. The 'held-out split' prevents overlap at the instance level, but not at the distribution level. The paper should either explicitly reframe the proprietary comparisons as in-distribution evidence, or add an independently authored benchmark and/or human evaluation of the proprietary-model comparisons. The existing human ev
- [Table 8, App. J.2] The external CharacterBench results are mixed and partially contradict the generalization claim in Section 4.2.1. On CharacterBench, Llama-3.1-8B-Ours (3.52) is below CoSER (3.53) and Crab (3.60); only Qwen2.5-7B-Ours (3.68) leads. The manuscript acknowledges this in Appendix J.2, but the abstract and Section 4.2.1 state that the method 'mostly outperforms baselines' on external protocols. The load-bearing conclusion that AdaRPSet training transfers beyond the synthetic distribution is not supported by the 8B Llama result. Please moderate the generalization claim or provide additional external evidence, e.g., on another independent benchmark or with human preference data for the proprietary comparisons.
- [Table 3, App. L] The Scene Manager's advantage over Claude Sonnet 4.5 is thin (8.37 vs 8.17 in Table 3) and the standard deviations overlap. The judge-ablation study in Appendix L (Tables 13–15) evaluates only closed-source managers (GPT-4o-mini, GPT-5-Chat, Claude Sonnet 4.5, Doubao) and does not include the trained Qwen2.5-14B-Instruct-Ours manager under alternative judges. Thus the possibility that the reported margin is judge-specific cannot be ruled out. Given that GPT-5-Chat also generated the AdaSMSet rationales, the 'surpassing Claude Sonnet 4.5' claim needs corroboration with a different judge or human evaluation of Scene-Manager decisions.
minor comments (4)
- [Abstract] 'which featuring an immersive message format' should read 'which features...'; there are also inconsistent spellings of 'Doubao-1.5-Pro-Character' / 'Doubao-1-5-Pro-Character' across Section 4.1 and Tables 2–3.
- [Appendix M.1] The paper finds that the Basic prompt mostly outperforms Enhance for the Actor, yet uses Enhance as the default. Since this choice affects absolute scores, please justify the default more explicitly or report both variants consistently in all main tables.
- [Table 4] The Llama-3.1-8B base row is omitted because the vanilla model does not reliably follow the structured protocol. Please state this omission in the main text or table caption, not only in the surrounding prose.
- [Appendix J.1] CharacterArena win rates of 98–100% against several baselines are reported without confidence intervals or analysis of judge bias; interpreting these as exact preferences may be misleading. Consider reporting agreement statistics or a smaller adjudicator-based subset.
Circularity Check
No circularity found: the central claims are empirical evaluations on a held-out split, not reductions to the training data or judge by definition.
full rationale
The paper's central claim is that training on AdaRPSet improves the Actor Model and that AdaSMSet improves the Scene Manager, as measured on AdaptiveBench. AdaptiveBench's 100 seeds are drawn from the held-out split of AdaRPSet-Synthesis, which is a standard and legitimate evaluation design: training and test instances are disjoint, even though they share a distribution. A model trained on a distribution is not guaranteed to score highly on held-out instances from that distribution, so the improvement is an empirical finding, not a consequence of the construction. Similarly, using GPT-5-Chat as the judge while GPT-5-Chat was also used to synthesize the data is a methodological concern about potential judge bias, but it is not a definitional equivalence: the judge's scores are not a fitted parameter of the model, and the paper reports multi-judge ablations showing stable rankings. The paper also evaluates on external CharacterBench and CharacterArena, and reports mixed results (e.g., Llama-3.1-8B-Ours 3.52 vs CoSER 3.53 on CharacterBench), which is inconsistent with the hypothesis that the headline result is forced by construction. Self-citations in the references are not load-bearing for the main derivation, and there is no invoked uniqueness theorem or ansatz smuggled in via citation. The in-distribution evaluation and judge overlap are validity limitations worth noting, but they do not make the derivation circular under the definitions used here.
Axiom & Free-Parameter Ledger
free parameters (5)
- AdaptiveBench trajectory horizon T =
20 dialogue turns
- Synthesis dataset scale =
20 topics x 50 trajectories, 45/5 split
- Scoring baseline anchor =
5 = neutral/acceptable
- SFT hyperparameters =
8 epochs, lr 1e-6, warmup 5%, max length 16K
- Extraction chunk size =
8192 tokens
axioms (5)
- domain assumption LLM-as-judge scores correlate with human judgment of role-playing quality
- domain assumption Synthetic LLM-generated trajectories are a valid proxy for diverse role-playing scenarios
- ad hoc to paper The unified message format [Thought],(Action),<Environment>,Speech is the correct interface for immersion
- ad hoc to paper The discrete action space M={init_scene,pick_speaker,switch_scene,add_role,end} is sufficient for adaptive role-playing
- domain assumption The seven-dimensional profile schema adequately characterizes characters
invented entities (1)
-
Scene Manager agent
no independent evidence
read the original abstract
LLM role-playing aims to portray arbitrary characters in interactive narratives, yet existing systems often suffer from limited immersion and adaptability. They typically under-model dynamic environmental information and assume largely static scenes and casts, offering insufficient support for multi-character orchestration, scene transitions, and on-the-fly character introduction. We propose an adaptive multi-agent role-playing framework, AdaMARP, featuring an immersive message format that interleaves [Thought], (Action), <Environment>, and Speech, together with an explicit Scene Manager that governs role-playing through discrete actions (init_scene, pick_speaker, switch_scene, add_role, end) accompanied by rationales. To train these capabilities, we construct AdaRPSet for the Actor Model and AdaSMSet for supervising orchestration decisions, and introduce AdaptiveBench for trajectory-level evaluation. Experiments across multiple backbones and model scales demonstrate consistent improvements: AdaRPSet enhances character consistency, environment grounding, and narrative coherence, with an 8B actor outperforming several commercial LLMs, while AdaSMSet enables smoother scene transitions and more natural role introductions, surpassing Claude Sonnet 4.5 using only a 14B LLM.
Figures
Forward citations
Cited by 1 Pith paper
-
PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks
PlanViz is a new benchmark with three sub-tasks and PlanScore metric to evaluate planning-oriented image generation and editing by unified multimodal models for computer-use tasks.
Reference graph
Works this paper leans on
-
[1]
Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long T. Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, and Tomas Pfister. In prospect and retrospect: Reflective memory management for long-term personalized dialogue agents, 2025. URL https: //arxiv.org/abs/2503.08026
Pith/arXiv arXiv 2025
-
[2]
Osc: Cognitive orchestration through dynamic knowledge alignment in multi-agent llm collaboration, 2025
Jusheng Zhang, Yijia Fan, Kaitong Cai, Xiaofei Sun, and Keze Wang. Osc: Cognitive orchestration through dynamic knowledge alignment in multi-agent llm collaboration, 2025. URL https://arxiv.org/abs/2509.0 4876
2025
-
[3]
Self-taught agentic long context understanding, 2025
Yufan Zhuang, Xiaodong Yu, Jialian Wu, Ximeng Sun, Ze Wang, Jiang Liu, Yusheng Su, Jingbo Shang, Zicheng Liu, and Emad Barsoum. Self-taught agentic long context understanding, 2025. URL https://arxiv.org/abs/ 2502.15920
Pith/arXiv arXiv 2025
-
[4]
Llm-cas: Dynamic neuron perturbation for real-time hallucination correction, 2025
Jensen Zhang, Ningyuan Liu, Yijia Fan, Zihao Huang, Qinglin Zeng, Kaitong Cai, Jian Wang, and Keze Wang. Llm-cas: Dynamic neuron perturbation for real-time hallucination correction, 2025. URL https: //arxiv.org/abs/2512.18623
arXiv 2025
-
[5]
CTCC: A Robust and Stealthy Fingerprinting Framework for Large Language Models via Cross-Turn Contextual Correlation Backdoor
Zhenhua Xu, Xixiang Zhao, Xubin Yue, Shengwei Tian, Changting Lin, and Meng Han. CTCC: A Robust and Stealthy Fingerprinting Framework for Large Language Models via Cross-Turn Contextual Correlation Backdoor. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in...
2025
-
[6]
Dezhang Kong, Shi Lin, Zhenhua Xu, Zhebo Wang, Minghao Li, Yufeng Li, Yilun Zhang, Hujin Peng, Xiang Chen, Zeyang Sha, Yuyuan Li, Changting Lin, Xun Wang, Xuan Liu, Ningyu Zhang, Chaochao Chen, Chunming Wu, Muhammad Khurram Khan, and Meng Han. A survey of llm-driven ai agent communication: Protocols, security risks, and defense countermeasures, 2025. URLh...
arXiv 2025
-
[7]
EverTracer: Hunting Stolen Large Language Models via Stealthy and Robust Probabilistic Fingerprint
Zhenhua Xu, Meng Han, and Wenpeng Xing. EverTracer: Hunting Stolen Large Language Models via Stealthy and Robust Probabilistic Fingerprint. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7019–7042, Suzhou, China, 2025. ...
-
[8]
Iag: Input-aware backdoor attack on vlm-based visual grounding.arXiv preprint arXiv:2508.09456, 2025
Junxian Li, Beining Xu, Simin Chen, Jiatong Li, Jingdi Lei, Haodong Zhao, and Di Zhang. Iag: Input-aware backdoor attack on vlm-based visual grounding.arXiv preprint arXiv:2508.09456, 2025
Pith/arXiv arXiv 2025
-
[9]
Zhenhua Xu, Xubin Yue, Zhebo Wang, Qichen Liu, Xixiang Zhao, Jingxuan Zhang, Wenjun Zeng, Wengpeng Xing, Dezhang Kong, Changting Lin, et al. Copyright protection for large language models: A survey of methods, challenges, and trends.arXiv preprint arXiv:2508.11548, 2025
Pith/arXiv arXiv 2025
-
[10]
Role-Play with Large Language Models, 2023
Murray Shanahan, Kyle McDonell, and Laria Reynolds. Role-Play with Large Language Models, 2023
2023
-
[11]
Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, and Yun-Nung Chen. Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16612–16631. Association for Computatio...
-
[12]
From Persona to Personalization: {A} Survey on Role-Playing Language Agents
Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao. From Persona to Personalization: {A} Survey on Role-Playing Language Agents. 2024, 2024. doi: 10.48550/arXiv.2404.18231
-
[13]
CoSER: Coordinating LLM-Based Persona Simulation of Established Roles, 2025
Xintao Wang, Heng Wang, Yifei Zhang, Xinfeng Yuan, Rui Xu, Jen-tse Huang, Siyu Yuan, Haoran Guo, Jiangjie Chen, Shuchang Zhou, Wei Wang, and Yanghua Xiao. CoSER: Coordinating LLM-Based Persona Simulation of Established Roles, 2025. 11 AdaMARP
2025
-
[14]
Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent
Xiaoyan Yu, Tongxu Luo, Yifan Wei, Fangyu Lei, Yiming Huang, Hao Peng, and Liehuang Zhu. Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12540–12557. Association for Co...
doi:10.18653/v1/ 2024
-
[15]
HyCoRA: Hyper-Contrastive Role- Adaptive Learning for Role-Playing, 2025
Shihao Yang, Zhicong Lu, Yong Yang, Bo Lv, Yang Shen, and Nayu Liu. HyCoRA: Hyper-Contrastive Role- Adaptive Learning for Role-Playing, 2025
2025
-
[16]
Character-LLM: A Trainable Agent for Role-Playing
Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. Character-LLM: A Trainable Agent for Role-Playing. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13153–13187. Association for Computational Linguistics, 2023. URL https://aclanthology.org/2023.emnlp-main.814/
2023
-
[17]
Large Language Models Meet Harry Potter: A Dataset for Aligning Dialogue Agents with Characters
Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, and Jia Li. Large Language Models Meet Harry Potter: A Dataset for Aligning Dialogue Agents with Characters. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8506–8520. Association for Computati...
-
[18]
ChatHaruhi: Reviving Anime Character in Reality via Large Language Model, 2023
Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi Mi, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun. ChatHaruhi: Reviving Anime Character in Reality via Large Language Model, 2023. URLhttps://arxiv.org/abs/2308.09597v1
Pith/arXiv arXiv 2023
-
[19]
Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, and Junran Peng. RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar...
2024
-
[20]
CharacterGLM: Customizing social characters with large language models
Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, and Minlie Huang. CharacterGLM: Customizing social characters with large language models. In Franck Dernoncourt, Daniel ...
2024
-
[21]
Keming Lu, Bowen Yu, Chang Zhou, and Jingren Zhou. Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7828–7840. Association for Computational Linguistics, 2024. doi: 10.18653/v...
-
[22]
CharacterBench: Benchmarking Character Customization of Large Language Models, 2024
Jinfeng Zhou, Yongkang Huang, Bosi Wen, Guanqun Bi, Yuxuan Chen, Pei Ke, Zhuang Chen, Xiyao Xiao, Libiao Peng, Kuntian Tang, Rongsheng Zhang, Le Zhang, Tangjie Lv, Zhipeng Hu, Hongning Wang, and Minlie Huang. CharacterBench: Benchmarking Character Customization of Large Language Models, 2024
2024
-
[23]
CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation
Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11836–11850....
-
[24]
Crab: A Novel Configurable Role-Playing LLM with Assessing Benchmark
Kai He, Yucheng Huang, Wenqing Wang, Delong Ran, Dongming Sheng, Junxuan Huang, Qika Lin, Jiaxing Xu, Wenqiang Liu, and Mengling Feng. Crab: A Novel Configurable Role-Playing LLM with Assessing Benchmark. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for C...
2025
-
[25]
Yiting Ran, Xintao Wang, Rui Xu, Xinfeng Yuan, Jiaqing Liang, Yanghua Xiao, and Deqing Yang. Capturing Minds, Not Just Words: Enhancing Role-Playing Language Models with Personality-Indicative Data. In Yaser Al- Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14566–14576. Assoc...
-
[26]
Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework
Bohao Yang, Dong Liu, Chenghao Xiao, Kun Zhao, Chen Tang, Chao Li, Lin Yuan, Yang Guang, and Chenghua Lin. Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 202...
-
[27]
Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model
Yeyong Yu, Runsheng Yu, Haojie Wei, Zhanqiu Zhang, and Quan Qian. Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon...
-
[28]
Zhenpeng Gao, Xiaofen Xing, and Xiangmin Xu. TailorRPA: A Retrieval-Based Framework for Eliciting Personalized and Coherent Role-Playing Agents in General Domain. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 2025, pages 5381–5412. Association for...
-
[29]
Enhancing Persona Consistency for LLMs’ Role-Playing using Persona-Aware Contrastive Learning
Ke Ji, Yixin Lian, Linxu Li, Jingsheng Gao, Weiyuan Li, and Bin Dai. Enhancing Persona Consistency for LLMs’ Role-Playing using Persona-Aware Contrastive Learning. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Findings of the Association for Computational Linguistics: ACL 2025, pages 26221–26238. Association for C...
-
[30]
R-CHAR: A Metacognition-Driven Framework for Role-Playing in Large Language Models
Haiming Qin, Jiwei Zhang, Wei Zhang, KeZhong Lu, Mingyang Zhou, Hao Liao, and Rui Mao. R-CHAR: A Metacognition-Driven Framework for Role-Playing in Large Language Models. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2...
-
[31]
CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization
Jing Ye, Rui Wang, Yuchuan Wu, Victor Ma, Feiteng Fang, Fei Huang, and Yongbin Li. CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 2025, pages 297–323. As...
-
[32]
Cheng Liu, Yifei Lu, Fanghua Ye, Jian Li, Xingyu Chen, Feiliang Ren, Zhaopeng Tu, and Xiaolong Li. Cog- Dual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Nat...
-
[33]
Let Your Characters Tell Their Story
Faeze Brahman, Meng Huang, Oyvind Tafjord, Chao Zhao, Mrinmaya Sachan, and Snigdha Chaturvedi. “Let Your Characters Tell Their Story”: A Dataset for Character-Centric Narrative Understanding. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors,Findings of the Association for Computational Linguistics: EMNLP 2021, pages 17...
doi:10.18653/v 2021
-
[34]
CharacterGLM: Customizing Social Characters with Large 13 AdaMARP Language Models
Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, and Minlie Huang. CharacterGLM: Customizing Social Characters with Large 13 AdaMARP Language Models. In Franck Dernoncou...
2024
-
[35]
Xinfeng Yuan, Siyu Yuan, Yuhan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, and Deqing Yang. Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, ...
-
[36]
TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models
Jaewoo Ahn, Taehyun Lee, Junyoung Lim, Jin-Hwa Kim, Sangdoo Yun, Hwaran Lee, and Gunhee Kim. TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computa- tional Linguistics: ACL 2024, pages 3291–3325. Association for Com...
-
[37]
PersonaGym: Evaluating Persona Agents and LLMs
Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpurohit, Ameet Deshpande, Karthik R Narasimhan, and Vishvak Murahari. PersonaGym: Evaluating Persona Agents and LLMs. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP...
-
[38]
RolePlot: A Systematic Framework for Evaluating and Enhancing the Plot-Progression Capabilities of Role- Playing Agents
Pinyi Zhang, Siyu An, Lingfeng Qiao, Yifei Yu, Jingyang Chen, Jie Wang, Di Yin, Xing Sun, and Kai Zhang. RolePlot: A Systematic Framework for Evaluating and Enhancing the Plot-Progression Capabilities of Role- Playing Agents. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of ...
2025
-
[39]
RoleMRC: A Fine- Grained Composite Benchmark for Role-Playing and Instruction-Following
Junru Lu, Jiazheng Li, Guodong Shen, Lin Gui, Siyu An, Yulan He, Di Yin, and Xing Sun. RoleMRC: A Fine- Grained Composite Benchmark for Role-Playing and Instruction-Following. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Findings of the Association for Computational Linguistics: ACL 2025, pages 21008–21030. Assoc...
-
[40]
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others. Gpt-4 technical report, 2024. URLhttps://arxiv.org/abs/2...
Pith/arXiv arXiv 2024
-
[41]
Gpt-5 system card
OpenAI. Gpt-5 system card. https://cdn.openai.com/gpt-5-system-card.pdf , Aug 2025. Version dated August 13, 2025
2025
-
[42]
Gheorghe Comanici et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. URLhttps://arxiv.org/abs/2507.06261
Pith/arXiv arXiv 2025
-
[43]
Claude 4 model family.https://www.anthropic.com, 2024
Anthropic. Claude 4 model family.https://www.anthropic.com, 2024. Claude Sonnet 4.5
2024
-
[44]
Doubao large language models
ByteDance. Doubao large language models. https://www.volcengine.com, 2024. Doubao-1.5-Pro-Character
2024
-
[45]
Qwen2.5 technical report, 2025
Qwen Team et al. Qwen2.5 technical report, 2025. URLhttps://arxiv.org/abs/2412.15115
Pith/arXiv arXiv 2025
-
[46]
The llama 3 herd of models, 2024
Aaron Grattafiori et al. The llama 3 herd of models, 2024. URLhttps://arxiv.org/abs/2407.21783
Pith/arXiv arXiv 2024
-
[47]
An Yang et al. Qwen3 technical report, 2025. URLhttps://arxiv.org/abs/2505.09388
Pith/arXiv arXiv 2025
-
[48]
Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V . Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, 14 AdaMARP Pradeep Dasigi, and Hannane...
Pith/arXiv arXiv 2025
-
[49]
Towards joint intent detection and slot filling via higher-order attention
Dongsheng Chen, Zhiqi Huang, Xian Wu, Shen Ge, and Yuexian Zou. Towards joint intent detection and slot filling via higher-order attention. InIJCAI, pages 4072–4078, 2022
2022
-
[50]
Zero-shot spoken language understanding via large language models: A preliminary study
Zhihong Zhu, Xuxin Cheng, Hao An, Zhichang Wang, Dongsheng Chen, and Zhiqi Huang. Zero-shot spoken language understanding via large language models: A preliminary study. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17877–17883, 2024
2024
-
[51]
Towards unified spoken language understanding decoding via label-aware compact linguistics representations
Zhihong Zhu, Xuxin Cheng, Zhiqi Huang, Dongsheng Chen, and Yuexian Zou. Towards unified spoken language understanding decoding via label-aware compact linguistics representations. InFindings of the Association for Computational Linguistics: ACL 2023, pages 12523–12531, 2023
2023
-
[52]
Enhancing code-switching for cross-lingual slu: A unified view of semantic and grammatical coherence
Zhihong Zhu, Xuxin Cheng, Zhiqi Huang, Dongsheng Chen, and Yuexian Zou. Enhancing code-switching for cross-lingual slu: A unified view of semantic and grammatical coherence. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7849–7856, 2023
2023
-
[53]
Dongsheng Chen, Chaofan Tao, Lu Hou, Lifeng Shang, Xin Jiang, and Qun Liu. Litevl: Efficient video-language learning with enhanced spatial-temporal modeling.arXiv preprint arXiv:2210.11929, 2022
Pith/arXiv arXiv 2022
-
[54]
Mix before align: Towards zero-shot cross-lingual sentiment analysis via soft-mix and multi-view learning
Zhihong Zhu, Xuxin Cheng, Dongsheng Chen, Zhiqi Huang, Hongxiang Li, and Yuexian Zou. Mix before align: Towards zero-shot cross-lingual sentiment analysis via soft-mix and multi-view learning. InProc. of INTERSPEECH, 2023
2023
-
[55]
Mclf: A multi-grained contrastive learning framework for asr-robust spoken language understanding
Zhiqi Huang, Dongsheng Chen, Zhihong Zhu, and Xuxin Cheng. Mclf: A multi-grained contrastive learning framework for asr-robust spoken language understanding. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 7936–7949, 2023
2023
-
[56]
Hang Ding, Qiming Feng, Dongqi Liu, Qi Zhao, Tao Yao, Shuo Wang, Dongsheng Chen, Jian Li, Zhenye Gan, Jiangning Zhang, et al. Rolermbench & rolerm: Towards reward modeling for profile-based role play in dialogue systems.arXiv preprint arXiv:2512.10575, 2025. A Additional Clarifications on Dataset Comparison This appendix provides detailed clarifications f...
arXiv 2025
-
[57]
He dresses casually—rolled sleeves, weathered bracelets, and a small scar above his right brow hint at a life shaped by the sea
Cassian Mirell (User):Cassian is a 31-year-old marine biologist with bronze-blond hair and skin sun-touched from fieldwork. He dresses casually—rolled sleeves, weathered bracelets, and a small scar above his right brow hint at a life shaped by the sea. His manner is patient, analytical, yet his humor flashes brightly when least expected. He often listens ...
-
[58]
And what if I told you, Amaris, that this stu- dio isn’t just a place of creation? It’s also a sanctuary for letting down our guard
Inaya Dren:Inaya is a lively textile artist with a sharp sense of observation and a knack for teasing truths out of people. Her colorful scarves and laughter fill every space she enters. She’s fiercely loyal to Amaris, sometimes acting as her self-appointed protector.Motivation:To gently push Amaris toward recognizing and articulating her emotions for Cas...
-
[60]
switch_scene
If there is a MAJOR scene change AND characters have explicitly agreed to move there OR have already moved there through unavoidable circumstances, set action=" switch_scene" and provide new_scene
-
[61]
add_role
If adding a new role would significantly enrich and advance the plot, OR if the user or a role explicitly wants to interact with a character not in the current role list , set action="add_role" and provide new_role_name, new_role_profile, new_role_motivation
-
[62]
pick_speaker
Otherwise, set action="pick_speaker" and provide speaker (must be one of existing roles or "user"). CRITICAL RULES for pick_speaker: - ROTATE SPEAKERS: Never pick the same speaker twice in a row. After someone speaks, pick a different role next turn. - INCLUDE USER IN ROTATION: The user is part of the rotation. Avoid long stretches of roles talking only t...
-
[63]
If the user asked to stop or the story is complete, set action="end"
-
[64]
switch_scene
If there is a MAJOR scene change AND characters have explicitly agreed to move there OR have already moved there through unavoidable circumstances (e.g., teleportation magic, being forced to move), set action="switch_scene" and provide new_scene. IMPORTANT: - Do NOT switch scenes just because someone mentioned a location or proposed going there. Wait unti...
-
[65]
add_role
If adding a new role would significantly enrich and advance the plot, OR if the user or a role explicitly wants to interact with a character not in the current role list, set action="add_role" and provide new_role_name, new_role_profile, new_role_motivation. IMPORTANT: Add a role when someone wants to talk to them, not just when they are mentioned in passing
-
[66]
pick_speaker
Otherwise, set action="pick_speaker" and provide speaker (must be one of existing roles or "user"). CRITICAL RULES for pick_speaker: - ROTATE SPEAKERS: Never pick the same speaker twice in a row. After someone speaks, pick a different role next turn. - INCLUDE USER IN ROTATION: The user is part of the rotation. Avoid long stretches of roles talking only t...
-
[67]
Prefer chapter titles or explicit section headers
Recognize chapter beginnings if they exist in the chunk. Prefer chapter titles or explicit section headers ... as the beginning; if no clear title exists, use the first meaningful fragment
-
[68]
state" as
Identify the important plots in this chunk. Identify the beginning and ending ... Set " state" as "truncated" if the plot is truncated in this chunk ... You will be provided with the truncated plots from the previous chunk, and you **must** extend the conversations
-
[69]
For each plot, generate its summary, score its prominence
Summarize each important plot. For each plot, generate its summary, score its prominence ... and list the key characters and their roles
-
[70]
I", "me",
Extract conversations for each plot. First, state the **scenario** (the static context *before* the conversation starts...). ** Do NOT** include dynamic events that happen *during* the conversation ... Then, list the key characters with their names, descriptions and thoughts ... Finally, extract the conversations among them based on the following requirem...
-
[71]
chapter_beginnings
Identify the optimal starting point for the subsequent chunk. If the last storyline has been extracted as an truncated plot ... ===Output Format=== ... (to be continued in the next Table) Table 24.Book chunk analysis prompt (part 1). Due to space limitations, some descriptions are abbreviated; please refer to our codebase for the full prompt. 43 AdaMARP P...
-
[72]
Adhere strictly to the specified output JSON format
-
[73]
[IMPORTANT] Ensure all DOUBLE QUOTES within all STRINGS are properly ESCAPED, especially when extracting from the text
-
[74]
In the OUTPUT, use characters'full names, omitting any titles
-
[75]
Avoid introducing plots that are out of context
Maintain Story Fidelity: The plot must accurately reflect the book's content. Avoid introducing plots that are out of context. If the plot contains multiple conversations, prioritize the original dialogue from the book. In the absence of explicit conversations, create dialogue that aligns closely with the plot details
-
[76]
chapter_beginnings.beginning_sentence
[CRITICAL] For "chapter_beginnings.beginning_sentence" and "next_chunk_start", you MUST copy the sentence **verbatim from the given chunk**, without adding, deleting, or modifying any characters (no paraphrasing, no added quotes, no extra spaces). ===Input=== ==Book title== {book['title']} ==Author== {book['author']} ==Chunk of Book Content== {chunk} ==Tr...
-
[77]
**name**: The character's name
-
[78]
**short_description**: A concise, condensed summary of the character
-
[79]
(Requirement: 1-several complete natural-language sentences with both density and vivid imagery)
**identity_appearance**: Name, age, gender, occupation... (Requirement: 1-several complete natural-language sentences with both density and vivid imagery)
-
[80]
(Requirement: Highlight traits that show up in dialogue)
**personality_psychology**: Personality traits, behavioral style... (Requirement: Highlight traits that show up in dialogue)
-
[81]
(Requirement: Provide 2-4 specific, actionable descriptions)
**speaking_style**: Rhythm, tone, and lexical habits... (Requirement: Provide 2-4 specific, actionable descriptions)
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.