{"id":"274765b4-638b-40a2-a8fc-b6ba218ae991","arxiv_id":"2608.06975","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"PHASE-Tree, a hierarchical character-state tree with gated persona evolution, and the LongEvoRoleBench benchmark improve evolved-state dialogue generation over static-profile baselines in the authors' evaluations.","lead":"This paper introduces PHASE-Tree, a layered character-state representation for long-horizon role-playing, plus a benchmark that tests whether models speak from a character's current evolved state. The system ranks first on most long-dialogue evaluation cells in its own experiments, though the character-score advantage weakens when the judge is given a different persona reference.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Char ranking inverts when judge reference changes to raw profile (Table 9), so the 19.7% character-score claim is reference-dependent rather than a property of PHASE-Tree.","rationale":"The most load-bearing condition for the central claim is the neutrality of the evaluation yardstick. For the Char metric, the paper uses the flattened PHASE-Tree profile as the judge's reference for every method, while PT's own prompt contains that same flattened tree. This creates a direct confound: the method being proposed defines the standard against which all methods are judged. Appendix L.1's own reference swap (Table 9) demonstrates the magnitude: RP's long-dialogue Char increases by +0.915 and surpasses PT when the judge is given the raw description, while PT decreases. The readability of the claim rests on which reference is chosen, and the paper's selection favors its own representation. The Sem and Emb findings are more robust (Sem persists across three judges and four backbones, and human correlation is r=0.65), which is why the paper should be conditionally accepted with the Char claim restated. The same model generating the trees and judging the responses compounds the risk, but the reference-sensitivity result alone is decisive: the claimed 19.7% Char improvement does not survive a neutral yardstick. Our proposed test—extending Table 9's reference swap to all judges, backbones, and external baselines—would settle the matter; if PT still ranks first across the board under the raw reference, the concern would land differently.","tokens_in":31293,"tokens_out":5743,"duration_ms":55680,"concrete_test":"Re-run the full long-dialogue external comparison (Table 2) with the Char judge's persona reference set to the raw character description instead of the flattened PHASE-Tree profile, using all three judge models (GPT-4.1, GLM-5.2, DeepSeek-V4-Flash). If Ours no longer ranks first on Char, or if the +19.7% margin over PAG reverses or drops below the Sem/Emb margins, the central evolved-state claim should be restated with the reference-sensitivity caveat and the benchmark should adopt a neutral reference (e.g., a human-authored gold state or transcript-only judging).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that textual PHASE-Tree 'speaks from a currently evolved state' rests on the long-dialogue Char metric, where Ours exceeds the best textual baseline by +19.7% (Table 2). The load-bearing assumption is that judging every method against the flattened PHASE-Tree profile is a neutral measurement of evolved-state fidelity. Appendix L.1 (Table 9) shows this assumption fails: when the judge's persona reference is switched to the raw character description, RP's long-dialogue Char rises from 2.454 to 3.369 while PT falls from 3.004 to 2.935, reversing the order and erasing the claimed margin. The source is the judge input itself: PT's conditioning prompt contains the flattened tree verbatim (1358 profile tokens on long dialogue, Table 14), which is also the Char judge's reference, giving PT a lexical-consistency advantage that RP/RAG/PAG, which condition on different text, do not share. The same GPT-4.1 is used for tree extraction and judging, adding a second channel of self-consistency bias. Although Sem and Emb are less reference-sensitive, the headline 19.7% Char improvement is an artifact of the chosen yardstick, not an established property of the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PHASE-Tree, a four-stratum character-state representation (immutable identity root; mutable persona, session, and moment layers) with resistance-gated cross-episode evolution, and introduces LongEvoRoleBench, a next-utterance benchmark built from four long-dialogue and four short-dialogue corpora. The central claim is that conditioning generation on the flattened PHASE-Tree state improves evolved-state fidelity, reported as rank-first in 11 of 12 long-dialogue cells against internal variants and all 12 cells against external textual baselines, with long-dialogue character, semantic, and embedding gains of 19.7%, 12.4%, and 15.1%. The paper validates the representation under both explicit textual provision and implicit parametric (profile-to-LoRA) adaptation, with additional human-rating, cross-judge, and cross-backbone analyses.","tokens_in":31566,"tokens_out":3034,"duration_ms":33076,"significance":"If its findings hold, the paper would make a useful contribution: the multi-timescale, field-addressable tree with a deterministic gating rule is a concrete proposal for localized character-state updates, and the benchmark unifies eight existing corpora under a common protocol. The paper ships code, data, and model releases, and it includes a human audit of accepted field updates, a blinded 200-response human study, and a multi-judge/multi-backbone robustness analysis. The semantic-score advantage is reasonably supported: PT vs NR long-dialogue Sem Cohen's d = 0.25 and PT vs ST d = 0.40, with the Sem ranking stable across three judges and four backbones. The character-score superiority, however, is not supported by the evidence: the internal ablation effect is negligible (PT vs NR Char d = 0.01), and the headline Char gain reverses when the judge reference is changed to the raw profile (Appendix L.1, Table 9). The paper's significance therefore rests mainly on the semantic and embedding findings and on the benchmark infrastructure, not on the character-score claim as stated.","major_comments":[{"comment":"The long-dialogue character-score ranking inverts under a neutral judge reference. When the judge's persona reference is switched from the flattened PHASE-Tree profile to the raw character description, RP rises from 2.454 to 3.369 while PT falls from 3.004 to 2.935, reversing the order and erasing the claimed advantage. This directly contradicts the Abstract's headline 'improving character-level ... by 19.7%' and the corresponding claim in Section 5.2. The reference-sensitivity analysis in the appendix is candid, but the main text continues to state the Char gain as a primary result; a revision must either withdraw the Char superiority claim or re-derive it from a reference-neutral evaluation.","section":"Appendix L.1, Table 9"},{"comment":"The character-score evaluation is partially circular. The judge is given the flattened PHASE-Tree profile as the persona reference, and this is exactly the conditioning text PT receives (1358 profile tokens on long dialogue, Table 14), while RP, RAG, and PAG condition on different text. PT therefore enjoys a lexical-consistency advantage that is an artifact of the yardstick rather than a measured property of evolved-state fidelity. The same GPT-4.1 model is used for tree extraction and for judging, adding a second channel of self-consistency bias. The Sem and Emb metrics are less sensitive to this issue, but the Char score as implemented does not support the paper's central claim.","section":"Section 4.3, Appendix G, Table 14"},{"comment":"Even under the reference condition most favorable to PT, the internal ablation shows no practical character-score effect: PT vs NR long-dialogue Char Cohen's d = 0.01 with per-cell range [-0.13, 0.08], and PT vs ST d = 0.10. The 19.7% improvement in the Abstract is computed against the external PAG baseline, whose lower score is partly explained by the judge-reference mismatch identified in Appendix L.1. The Sem and Emb gains (d = 0.25-0.40) are credible, but the character-score claim should be either removed from the headline results or replaced with an analysis that uses a reference-neutral profile for all methods.","section":"Section 5.2, Table 8"},{"comment":"The gating mechanism in Eq. (2) depends on manually chosen thresholds (tau_ep, tau_high, tau_cd) and the paper states that learning them automatically is future work. While the thresholds are held fixed across corpora and the human audit (Appendix D, Table 5) shows that accepted updates are evidence-supported, the experiments do not test sensitivity to these threshold values. A brief sensitivity analysis or an explicit argument that the headline results are robust to threshold choice would strengthen the claim that cross-episode evolution, rather than the particular thresholds, drives the Sem gains.","section":"Section 3.4, Eq. (2)"}],"minor_comments":[{"comment":"The phrase 'on descriptive n=10 PT and NR prompt subsets' reports a very small, descriptive human comparison; the +0.20 Overall difference is not a statistical claim. Please either move this detail to Appendix L.3 or state clearly that it is an anecdotal pilot.","section":"Abstract"},{"comment":"The sentence 'they were chosen so that core-tier updates require evidence spanning roughly two-thirds of a typical television season of a typical television corpus' is redundant; 'of a typical television corpus' can be deleted.","section":"Appendix C"},{"comment":"The affiliation 'MemT ensor (Shanghai) Technology' contains an odd space; please fix to 'MemTensor (Shanghai) Technology'.","section":"Author affiliations"},{"comment":"The caption contains the fragment 'abl main' in the y-axis labels; this appears to be a LaTeX truncation of 'ablation' and should be corrected.","section":"Figure 4 caption"},{"comment":"The notation 'S persona t' and similar inline subscripts are visually confusing; consider using S^persona_t or a consistently typeset subscript throughout the paper.","section":"Section 3.1, Eq. (1)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a CS.CL venue and the releases are valuable. The main concern is the character-score claim: the internal effect is negligible and the headline gain is reference-dependent. This is fixable by reframing the contributions around the semantic-score findings, the benchmark, and the representation, and by reporting both judge-reference conditions in the main tables. I would not reject, but the revision needs to be substantive rather than cosmetic."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my read.\n\nThe genuinely new thing is the packaged system: the four-stratum PHASE-Tree state with an immutable identity root and persona/session/moment layers gated by resistance–evidence–cooldown rules, plus a unified next-utterance benchmark over eight existing corpora. That combination did not exist, and releasing code, data, and model weights makes it directly usable. The semantic-score advantage is real evidence: PT vs NR long-dialogue Sem Cohen's d = 0.25, PT vs ST d = 0.40, and it holds across three LLM judges and four generation backbones. The human correlation (r = 0.65 on Sem/Overall) and the human audit of narrative updates are also good practice. The paper deserves credit for reporting effect sizes and a reference-sensitivity analysis instead of burying them.\n\nThe soft spot is the character-score claim, and it is load-bearing. The paper's own Appendix L.1, Table 9 shows that when the judge's persona reference is switched from the flattened PHASE-Tree profile to the raw character description, RP's long-dialogue Char jumps from 2.454 to 3.369 while PT drops from 3.004 to 2.935, inverting the leaderboard. PT conditions on the same PHASE-Tree text the judge uses as the Char reference, so the +19.7% headline gain partly measures lexical self-consistency, not evolved-state fidelity. The effect-size table gives this away: PT vs NR Char d = 0.01. I also note that the same GPT-4.1 is used for tree extraction and judging, adding a second channel of potential bias. These caveats do not kill the Sem or Emb results, but the abstract overstates things by leading with the Char number without that caveat. A smaller practical point: textual PHASE-Tree on long dialogue carries 1358 profile tokens out of a 1736-token prompt, which is expensive at scale, and the parametric route shows the profile-to-LoRA bottleneck erases the tree's distinctions.\n\nWho this is for: people working on long-horizon role-playing, AI companions, or dynamic persona representation. The benchmark and pipeline are worth using even if the Char metric needs to be reworked with a neutral judge reference. I would send it to serious referees, asking them to scrutinize the evaluation protocol; the paper is internally coherent and the main claims are checkable. My recommendation: engage, but require the Char claims to be re-stated with the reference-sensitivity caveat, or re-evaluated with a judge reference that does not mirror one method's prompt.","headline":"The benchmark and the semantic-score finding are solid, but the headline character-score gain does not survive the paper's own reference-sensitivity check (Table 9 inverts the ranking).","tokens_in":32121,"tokens_out":3215,"would_cite":true,"duration_ms":33043,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PHASE-Tree claims that role-playing models can generate from a character's currently evolved state by storing that state in a multi-timescale tree, not a static profile.","keywords":["role-playing dialogue","character-state evolution","long-horizon dialogue","persona modeling","LLM-as-judge evaluation","multi-timescale representation","stale-state failure","next-utterance benchmark"],"falsifier":"Take the released LongEvoRoleBench responses and score them under the raw character description as the judge reference; if, as in Appendix L.1, PHASE-Tree's long-dialogue character score drops to 2.935 while the raw-profile baseline rises to 3.369, the claimed character-level superiority is an artifact of the chosen judge reference rather than of evolved-state fidelity.","tokens_in":31095,"feed_emoji":"🎭","tokens_out":6108,"duration_ms":55496,"temperature":0.7,"pith_summary":"PHASE-Tree is a representation for long-horizon role-playing that stores a character's state as a four-layer tree: an immutable identity root plus persona, session, and moment strata that update on different timescales. The paper's central claim is that this lets a model generate from the character's currently evolved state, avoiding the \"stale-state failure\" where a character's voice persists but their narrative situation is outdated. To support that, the paper introduces a benchmark that tests whether responses match the state at the current narrative time, and reports that textual provision of the tree outperforms raw profiles, rewritten profiles, static trees, retrieval, and profile-prompt baselines across long-dialogue corpora. The load-bearing mechanism is the update gate: a persona field changes only when evidence spans enough episodes, includes enough high-significance events, and respects a cooldown. If the claim holds, role-playing agents can stay recognizable over long arcs while tracking relationships, mood, and commitments.","feed_headline":"PHASE-Tree lets characters evolve without breaking who they are","feed_subtitle":"A four-layer character-state tree beats static profiles on long-horizon role-play, with semantic gains holding across models.","key_machinery":"The central object is the PHASE-Tree itself: a four-stratum hierarchy with an immutable identity root (name, gender, backstory) and three mutable strata—persona (slow-moving traits with resistance tiers), session (within-scene learned information, commitments, and attitudes), and moment (transient emotion and intensity). The update rule in Eq. (2) is the gating mechanism: a persona field is updated only if the number of contributing episodes $n_{\\mathrm{ep}}(f)$, the number of high-significance evidence entries $n_{\\mathrm{high}}(f)$, and the number of episodes since the last update $\\Delta_{\\mathrm{ep}}(f)$ all clear resistance-dependent thresholds. That gate is what lets core traits like personality change only under season-long evidence while relationships can shift from a single decisive scene. The benchmark LongEvoRoleBench supplies the measuring instrument: eight corpora unified under a next-utterance protocol, with long-dialogue corpora testing cross-episode evolution and short-dialogue corpora serving as within-scene state-tracking checks.","core_discovery":"The paper discovers that decomposing a character's state by timescale—fixed identity, slow-moving persona, session-level adaptations, and moment-level affect—and gating persona updates by episode count, significance, and cooldown produces generations that align better with the character's current narrative state than either a raw profile or an unstructured rewrite. PHASE-Tree's extracted profiles are used as prompt text, and on the long-dialogue benchmark the full pipeline ranks first in all external textual comparisons, improving character, semantic, and embedding scores over the strongest textual baseline by 19.7%, 12.4%, and 15.1%. The paper also claims that the semantic-score advantage survives swapping judge models and generation backbones, while its own appendix shows the character-score advantage is more sensitive to the judge's reference text.","pith_inferences":["Beyond the paper: the reference-sensitivity result in Appendix L.1 implies that the headline character-score gain is not robust to the judge's profile text; a fair comparison would score all methods against a neutral, method-independent reference.","Beyond the paper: because the same model used to build the trees also serves as the primary judge, an independent judge that has never read PHASE-Tree profiles would test whether the semantic-score advantage reflects the state representation or the judge's familiarity with its wording.","Beyond the paper: the gating thresholds could be learned from data rather than hand-set; a useful extension would correlate learned thresholds with actual narrative-pacing statistics, such as the number of episodes before relationship changes occur.","Beyond the paper: the parametric bottleneck suggests that richer state-conditioned adapters with more capacity might close the gap between the textual and implicit routes, making evolved-state conditioning feasible at fixed prompt cost."],"forward_implications":["Role-playing systems can maintain character recognizability across long arcs without freezing a persona, because each editable field is updated locally rather than rewriting the whole profile.","Benchmarks for persona fidelity should score against the character state at the current narrative time, not against a static profile, if evolved-state generation is the target.","Textual provision of an evolving tree is a practical prompting strategy: it beats retrieval and profile-prompt baselines on long-dialogue semantic and embedding scores.","Encoding the same tree into LoRA adapters is more token-efficient but loses the distinctions among tree variants, indicating that the profile-to-LoRA mapping is the bottleneck rather than the tree representation itself.","Hand-set resistance thresholds transfer across four different long-dialogue corpora without per-corpus tuning, suggesting that one narrative-pacing prior can govern multiple story domains."],"supporting_citations":[{"why":"Supplies the trait-versus-characteristic-adaptation psychological split that motivates separating the persona and session strata.","marker":"[24]"},{"why":"Supplies the profile-to-LoRA hypernetwork used for the implicit parametric conditioning variant and its comparison.","marker":"[36]"},{"why":"Supplies retrieval-augmented generation, the strongest textual baseline on long-dialogue semantic and embedding scores.","marker":"[11]"},{"why":"Supplies ChatHaruhi as both a profile-prompt role-playing baseline and a short-dialogue corpus in the benchmark.","marker":"[12]"},{"why":"Supplies the Friends transcript corpus used as a long-dialogue evolution testbed.","marker":"[1]"},{"why":"Supplies the Harry Potter dialogue corpus used as a long-dialogue evolution testbed.","marker":"[3]"},{"why":"Supplies the RAIDEN benchmark converted into a short-dialogue state-tracking set.","marker":"[44]"}],"fun_headline_variants":["Character tree keeps evolved roles intact across long dialogues","PHASE-Tree: local updates, stable identity in role-play","Role-play model preserves character while letting them grow","Tree-structured state beats static profiles for evolving roles","Long-horizon role-play gets a state tree that updates locally"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that grading every method against PHASE-Tree's own flattened profile is a fair yardstick for character fidelity; the paper's own reference-swap shows the headline character-score lead flips when the raw character description is used instead.","fun_headline_variants_meta":{"raw":{"variants":["Character tree keeps evolved roles intact across long dialogues","PHASE-Tree: local updates, stable identity in role-play","Role-play model preserves character while letting them grow","Tree-structured state beats static profiles for evolving roles","Long-horizon role-play gets a state tree that updates locally"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00012,"raw_usage":{"total_tokens":1115,"prompt_tokens":999,"completion_tokens":116,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":37}},"tokens_in":615,"tokens_out":116,"duration_ms":2250,"temperature":1.0,"reasoning_tokens":37,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:15:49.522347+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the released LongEvoRoleBench responses and score them under the raw character description as the judge reference; if, as in Appendix L.1, PHASE-Tree's long-dialogue character score drops to 2.935 while the raw-profile baseline rises to 3.369, the claimed character-level superiority is an artifact of the chosen judge reference rather than of evolved-state fidelity.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the trait-versus-characteristic-adaptation psychological split that motivates separating the persona and session strata."},{"cited_title":"RAIDEN benchmark: Evaluating role-playing conver- sational agents with measurement-driven custom dialogues","cited_arxiv_id":null,"evidence_quote":"Supplies the RAIDEN benchmark converted into a short-dialogue state-tracking set."}],"review_version":1}