pith. sign in

arxiv: 2605.29256 · v1 · pith:NTSPMTLBnew · submitted 2026-05-28 · 💻 cs.CL · cs.AI

DynSess: Dynamic Session-Level Evaluation and Optimization Framework for Role-Playing Agents

Rongsheng Zhang , Jiji Tang , Junnan Ren , Zuyi Bao , Weijie Chen , Ruofan Hu , Zhou Zhao , Tangjie Lv
show 1 more author
Yan Zhang
This is my paper

Pith reviewed 2026-06-29 08:11 UTC · model grok-4.3

classification 💻 cs.CL cs.AI
keywords role-playing agentssession-level evaluationlong-horizon behaviorsmulti-turn dialogueagent optimizationhuman alignmentLLM evaluation
0
0 comments X

The pith

DynSess provides a session-level evaluation and optimization method for role-playing agents that aligns better with human judgments and enables smaller models to match larger ones.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Role-playing agents must maintain character and interaction quality over many turns, yet current methods evaluate and train only one turn at a time. DynSess introduces rubrics that score entire dialogue sessions for long-horizon behaviors. Using these session-level rewards, it generates training data via lookahead search and trains two model variants. The resulting models match top character systems despite using fewer parameters, and the evaluator itself agrees with humans more closely than earlier methods.

Core claim

DynSess-Eval scores complete dialogue sessions using rubrics for long-horizon behaviors, and this allows construction of high-quality training trajectories through multi-turn lookahead search to train DynSess-Character, which matches the strongest character model with substantially fewer parameters while maintaining role consistency.

What carries the argument

DynSess-Eval session-level rubrics that target long-horizon behaviors, combined with multi-turn lookahead search for training data and the DSPO and GSRPO optimization methods.

If this is right

  • DynSess-Eval provides more reliable automatic scoring of role-playing sessions than turn-level alternatives.
  • DynSess-Character achieves comparable performance to larger models in role consistency and interaction.
  • Session-level rewards enable better optimization for sustained agent behavior.
  • Releasing the dataset and code will support further development of session-aware role-playing systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar session-level approaches could improve evaluation in other multi-turn tasks like long conversations or task planning.
  • Reducing parameter count while preserving performance may lower computational costs for deploying role-playing agents.
  • Human alignment of the evaluator suggests it could replace some manual review in development pipelines.

Load-bearing premise

The session-level rubrics accurately and comprehensively capture the long-horizon behaviors that matter for role-playing quality.

What would settle it

A large-scale blind human study in which DynSess-Eval scores do not correlate with human preferences or DynSess-Character fails to match the strongest model in direct comparisons.

Figures

Figures reproduced from arXiv: 2605.29256 by Jiji Tang, Junnan Ren, Rongsheng Zhang, Ruofan Hu, Tangjie Lv, Weijie Chen, Yan Zhang, Zhou Zhao, Zuyi Bao.

Figure 1
Figure 1. Figure 1: Turn-level vs. Session-level evaluation. An [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of our framework. DynSess-Eval (left) simulates multi-turn user–agent sessions and scores [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A case study on a Batman role-playing session. Top: Coser-70B baseline vs. DynSess-Character-32B. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Impact of session length T on evaluation. Both LLM backbones show highly consistent trends. tematic preference of LLM judges toward outputs stylistically similar to their own. (iii) On-policy training overfits evaluator. While on-policy RL is conventionally regarded as the stronger paradigm, on-policy GSRPO surpasses off-policy DSPO on Auto (4.46 vs. 4.28) but underperforms on Human (3.35 vs. 3.37), indica… view at source ↗
Figure 5
Figure 5. Figure 5: Distribution of character categories in the test [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
read the original abstract

Role-playing with large language models is fundamentally a session-level task, requiring agents to sustain character identity and interaction quality across extended multi-turn conversations. Yet existing evaluation and optimization methods remain largely turn-level, failing to capture long-horizon quality. We propose DynSess, a unified session-level framework for role-playing agents. DynSess-Eval scores complete dialogue sessions via rubrics targeting long-horizon behaviors. Leveraging its session-level rewards, we construct high-quality training trajectories through multi-turn lookahead search and train DynSess-Character with two complementary variants: DSPO (off-policy) and GSRPO (on-policy). Experiments show that DynSess-Eval aligns with human judgments substantially better than prior evaluators, and blind human evaluation further shows that DynSess-Character matches the strongest character model despite using substantially fewer parameters, while maintaining strong role consistency and interactive ability. Our dataset and code will be released to facilitate future research.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper proposes DynSess, a unified session-level framework for role-playing agents. DynSess-Eval scores full dialogue sessions using rubrics that target long-horizon behaviors such as sustained character identity and interaction quality. These session-level rewards are then used to construct training trajectories via multi-turn lookahead search, training DynSess-Character with DSPO (off-policy) and GSRPO (on-policy) variants. The abstract claims that DynSess-Eval aligns substantially better with human judgments than prior evaluators, and that blind human evaluation shows DynSess-Character matches the strongest character model despite using substantially fewer parameters while preserving role consistency and interactive ability. Dataset and code will be released.

Significance. If the experimental claims hold with detailed validation, the work would meaningfully advance role-playing agent research by moving evaluation and optimization from turn-level to session-level, addressing a recognized gap in capturing long-horizon consistency. The planned public release of the dataset and code would further support reproducibility and follow-on work in the field.

major comments (2)
  1. [Abstract] Abstract: The central claim that DynSess-Eval 'aligns with human judgments substantially better than prior evaluators' is load-bearing for the entire contribution, yet the manuscript supplies no details on rubric construction, inter-rater agreement on the rubrics themselves, dataset sizes for the human correlation study, or statistical tests. Without these, it is impossible to assess whether the reported alignment reflects genuine improvement or stems from rubric design choices that overlap with the human evaluation criteria.
  2. [Abstract] Abstract: The claim that DynSess-Character 'matches the strongest character model despite using substantially fewer parameters' while maintaining 'strong role consistency and interactive ability' rests on blind human evaluation, but no information is given on the number of sessions evaluated, the exact baselines compared, the rubric dimensions used in the human study, or controls for session length. This information is required to evaluate whether the session-level optimization actually delivers the stated gains.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed feedback on the abstract and the importance of methodological transparency. We address each major comment below and will revise the manuscript to supply the requested details.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim that DynSess-Eval 'aligns with human judgments substantially better than prior evaluators' is load-bearing for the entire contribution, yet the manuscript supplies no details on rubric construction, inter-rater agreement on the rubrics themselves, dataset sizes for the human correlation study, or statistical tests. Without these, it is impossible to assess whether the reported alignment reflects genuine improvement or stems from rubric design choices that overlap with the human evaluation criteria.

    Authors: We agree that these details are essential and are currently absent from the manuscript. In the revised version we will add a dedicated subsection on rubric construction (including the iterative expert annotation process), report inter-rater agreement statistics, specify the exact dataset size used for the human correlation study, and include the statistical tests (with p-values) that support the alignment claims. The abstract will be updated to reference the new section. revision: yes

  2. Referee: [Abstract] Abstract: The claim that DynSess-Character 'matches the strongest character model despite using substantially fewer parameters' while maintaining 'strong role consistency and interactive ability' rests on blind human evaluation, but no information is given on the number of sessions evaluated, the exact baselines compared, the rubric dimensions used in the human study, or controls for session length. This information is required to evaluate whether the session-level optimization actually delivers the stated gains.

    Authors: We concur that this information is missing and required for proper evaluation. The revised manuscript will explicitly state the number of sessions in the blind human evaluation, list the precise baselines (including parameter counts), detail the rubric dimensions employed, and describe the session-length controls. These additions will appear in the experiments section, with the abstract revised to point readers to them. revision: yes

Circularity Check

0 steps flagged

No circularity: evaluation and optimization steps are independent of self-defined quantities

full rationale

The paper presents an empirical framework for session-level evaluation and optimization of role-playing agents. No equations, derivations, or parameter-fitting steps are described that reduce reported results (human alignment, model performance) to quantities defined by the same work's fitted parameters or self-citations. DynSess-Eval rubrics and the subsequent DSPO/GSRPO training are presented as separate from prior self-citations, with claims resting on external human judgments rather than internal redefinitions. This matches the default expectation of a non-circular empirical contribution.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no explicit free parameters, axioms, or invented entities are described. The framework implicitly depends on the assumption that rubric-based session scores constitute valid rewards, but this is not formalized.

pith-pipeline@v0.9.1-grok · 5710 in / 1214 out tokens · 34795 ms · 2026-06-29T08:11:02.830815+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 6 canonical work pages · 3 internal anchors

  1. [1]

    HER: Human-like Reasoning and Reinforcement Learning for LLM Role-playing

    Tract: Regression-aware fine-tuning meets chain-of-thought reasoning for llm-as-a-judge. In Proceedings of the 63rd Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 2934–2952. Xuan Long Do, Kenji Kawaguchi, Min-Yen Kan, and Nancy Chen. 2025. Aligning large language mod- els with human opinions through person...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  2. [2]

    InFindings of the Association for Computational Linguistics: ACL 2025, pages 10301– 10314, Vienna, Austria

    Reasoning does not necessarily improve role- playing ability. InFindings of the Association for Computational Linguistics: ACL 2025, pages 10301– 10314, Vienna, Austria. Association for Computa- tional Linguistics. Zhaolin Gao, Wenhao Zhan, Jonathan D Chang, Gokul Swamy, Kianté Brantley, Jason D Lee, and Wen Sun

    2025

  3. [3]

    Google DeepMind

    Regressing the relative future: Efficient pol- icy optimization for multi-turn rlhf.arXiv preprint arXiv:2410.04612. Google DeepMind. 2026. Gemini 3 pro. https: //deepmind.google/technologies/gemini/. Large language model. Accessed: 2026-03-23. Aobo Kong, Wentao Ma, Shiwan Zhao, Yongbin Li, Yuchuan Wu, Ke Wang, Xiaoqian Liu, Qicheng Li, Yong Qin, and Fei ...

    work page arXiv 2026

  4. [4]

    arXiv preprint arXiv:2308.09597 , year=

    Chatharuhi: Reviving anime character in reality via large language model.arXiv preprint arXiv:2308.09597. Yubo Li, Xiaobin Shen, Xinyu Yao, Xueying Ding, Yidi Miao, Ramayya Krishnan, and Rema Padman. 2025. Beyond single-turn: A survey on multi-turn inter- actions with large language models.arXiv preprint arXiv:2504.04717. Aixin Liu, Aoxue Mei, Bangcai Lin...

    work page arXiv 2025

  5. [5]

    5: Advancing superb reasoning models with reinforcement learning

    Large language models are superpositions of all characters: Attaining arbitrary role-play via self-alignment. InProceedings of the 62nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 7828–7840, Bangkok, Thailand. Association for Computational Linguistics. OpenAI. 2026. Gpt-5.4. https://openai.com/. Large l...

    work page arXiv 2026

  6. [6]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Multi-turn reinforcement learning with prefer- ence human feedback.Advances in Neural Informa- tion Processing Systems, 37:118953–118993. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language model...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 13555–13571, Suzhou, China

    RMTBench: Benchmarking LLMs through multi-turn user-centric role-playing. InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 13555–13571, Suzhou, China. Association for Computational Linguistics. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others

    2025

  8. [8]

    Qwen3 Technical Report

    Qwen3 technical report.arXiv preprint arXiv:2505.09388. Jing Ye, Rui Wang, Yuchuan Wu, Victor Ma, Feiteng Fang, Fei Huang, and Yongbin Li. 2025. CPO: Ad- dressing reward ambiguity in role-playing dialogue via comparative policy optimization. InFindings 10 of the Association for Computational Linguistics: EMNLP 2025, pages 297–323, Suzhou, China. Asso- cia...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    Read the multi-turn dialogue comprehensively

  10. [10]

    Check deduction criteria item by item

  11. [11]

    Look for bonus evidence of consistent human-like quality

  12. [12]

    human_likeness

    Write rationale, then provide score. Output Format {"human_likeness": {"reason": "<Around 150 words...>", "score": <Integer from 1-5>}} 18 DynSess-Eval Prompt: Role Consistency You are an expert in role-playing evaluation, proficient in various literary styles and historical/cultural backgrounds. You believe that "no obvious OOC (Out of Character)" is mer...

  13. [13]

    Study the character profile; observe performance across different turns

  14. [14]

    Character Drift

    Check deduction criteria, especially "Character Drift."

  15. [15]

    Look for bonus evidence of depth and consistency under pressure

  16. [16]

    role_consistency

    Write rationale, then provide score. Output Format {"role_consistency": {"reason": "<Around 150 words...>", "score": <Integer from 1-5>}} DynSess-Eval Prompt: Contextual Coherence You are a role-playing evaluation expert, highly sensitive to logical flaws and forgotten details. You believe that "not forgetting things in the short term" is merely a minimum...

  17. [17]

    Read the full dialogue, mapping timeline and key details

  18. [18]

    Check deduction criteria: amnesia, contradictions, dead loops

  19. [19]

    Look for bonus evidence of active callbacks and logical closure

  20. [20]

    context_consistency

    Write rationale, then provide score. Output Format {"context_consistency": {"reason": "<Around 150 words...>", "score": <Integer from 1-5>}} C.2 Prompt for User Persona Generation Prompt for User Persona Generation Case 1: Extracting the user persona from the role persona [System Prompt] You are a persona extraction assistant. Your task is to extract user...

  21. [21]

    reassuring

    Behave like a lazy user: only passively answer the other party’s questions, or give short comments on what they say.Do notproactively start new topics or ask questions frequently. 2.Colloquial style: speak naturally and casually, like a real person texting. 3.Keep replies short: each response should be strictly limited to about10–20 words, usually inone s...