Pith. sign in

REVIEW 5 major objections 5 minor 35 references

Codifying Character Logic in Role-Playing

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Precompiling character profiles into executable if-then logic outperforms free-text prompts for role-playing consistency, updatability, and controlled randomness.

desk verdict Codified Profiles is a genuinely new idea for role-play—profiles as executable parse_by_scene functions—with plausible but not yet airtight evidence; the main open question is whether the Fandom benchmark's guiding questions leak the answer. read the letter →

arxiv 2505.07705 v2 pith:SUUSQ6QI submitted 2025-05-12 cs.CL

Letian Peng , Jingbo Shang This is my paper · ORCID
classification cs.CL
keywords codifiedprofilesrole-playingcharacterlogicexecutablefunctionsconditioncheckingnaturallanguageinferencecontrollablerandomnesssmallmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a character's behavioral logic can be written down as executable functions—parse_by_scene(scene) returning triggered_statements, with semantic checks like check_condition(scene, question)—and that this codified form makes role-playing more consistent, more updateable, and more controllable than appending free-text profiles to prompts. On a new 83-character, 5,141-scene benchmark built from fan wiki narrative summaries, the authors report average NLI scores of 68.38 for codified profiles versus 65.98 for original textual profiles on main characters, and larger gaps on minor characters. They also report that evolving the codified profile episode-by-episode yields bigger gains than evolving a textual profile, and that explicit random.choice logic produces precise stochastic behavior that temperature sampling cannot. The practical upshot, if the claim holds, is that a 1B-parameter model with a codified profile and a distilled condition checker approaches the role-playing quality of an 8B model prompted with text, which matters for local and low-resource deployment.

What carries the argument

The load-bearing object is the codified profile: a set of executable functions parse_by_scene(scene) that return triggered_statements, a list of character-consistent assertions, by running explicit if-then-else control flow over semantic condition checks. The helper check_condition(scene, question) queries the role-playing LLM with a natural-language question and reads its logits over "yes", "no", and "unknown" to return True, False, or None, so the code can test scene properties that symbolic matching cannot. This machinery offloads most of the character-reasoning burden from open-ended generation to localized classification and control flow, which is what lets smaller models perform competitively and what makes behavioral rules traceable and patchable.

What would settle it

Have human annotators audit a random sample of benchmark scenes to check whether the guiding question can be answered from the ground-truth sentence alone, or whether scene truncation removes context the character logic depends on; if a substantial share leak the reference or lose decisive context, the NLI gap between codified and textual profiles is an artifact of benchmark construction.

Watch

Extended reading notes

Core claim

The central discovery is that role-playing consistency can be manufactured rather than inferred: convert each paragraph of a character profile into a Python function parse_by_scene(scene) whose control flow decides which triggered_statements apply to the current scene. Semantic calls to check_condition(scene, question) let the function ask an LLM yes/no/unknown questions such as "Is the character in danger?" and branch on the answer. Because the logic is explicit, it can be inspected, revised along the storyline, and given precise randomness with constructs such as random.choice([...]) and random.random() < p. The paper's evidence is a set of comparisons on its Fandom Benchmark, where codified profiles outperform original textual profiles and a retrieval-style Codified RAG baseline in NLI scores and in human/LLM preference, and where a 1B model using a codified profile plus a distilled 0.1B condition checker reaches 60.21 average NLI on main characters versus 65.98 for an 8B model with a textual profile.

Load-bearing premise

The Fandom Benchmark must faithfully measure character logic: the LLM-extracted scenes, ground-truth actions, and guiding questions have to be accurate and free of leakage, because every comparison is scored against that reference.

Editorial extensions

If this is right

  • Character profiles become inspectable artifacts: a behavioral contradiction in a role-play output can be traced to a specific code block and fixed by editing that block, rather than by rewriting prose and hoping the model complies.
  • Precompiling logic into conditions cuts inference cost, because the role-playing LLM no longer has to reason over the full profile text at every response; the paper measures this efficiency gain in reduced forward passes.
  • Small models become viable role-play engines: a 1B model with a codified profile and distilled condition checker approaches an 8B text-prompted model, and beats it on some minor-character subsets.
  • Storyline evolution can be handled as a code update loop, and the paper shows codified evolving profiles (average 69.28 NLI) outperform evolving textual profiles (65.60) across eight Game-of-Thrones protagonists.
  • Precise stochastic personality traits, such as equal-probability emotion, low-probability humor, and fair rock-paper-scissors choices, can be encoded with explicit probability statements rather than left to temperature, and Best@K coverage improves as a result.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If codification works by decomposing long profile text into localized yes/no decisions, the same pattern might improve other instruction-following tasks that require persistent rules, such as tool-use policies or safety guardrails, by making each decision auditable.
  • The character-level win-rate pattern suggests a testable routing rule: codify profiles for logic-heavy, strategic characters and keep textual prompting for emotionally expressive characters; an automatic classifier that picks the format per character might beat either method alone.
  • The distilled condition checker result implies that much of role-playing can be reduced to binary scene classification; a natural extension would be to train the condition checker on a larger corpus and measure whether small models then close the residual gap to frontier models entirely.
  • Nothing in the mechanism is fiction-specific; one could codify a customer-service persona or an AI agent's decision policy the same way, with the same promises of persistence and updatability.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Codified Profiles, a representation of role-play character logic as executable parse_by_scene functions that emit triggered statements conditioned on scene checks, and claims this improves persistence, updatability, and controllable randomness relative to textual profiles. The authors build an 83-character, 5,141-scene Fandom benchmark with NLI-based scoring, compare codified profiles against textual profiles and Codified RAG under llama-3.1-8b, test profile evolution along storylines, and show that smaller models benefit from codification. They also analyze segmentation and codification fidelity, and include a distilled condition checker to lower deployment cost.

Significance. If the benchmark is valid, the paper offers a practical and falsifiable mechanism for making role-play behavior more deterministic and updatable, and it demonstrates that reasoning can be partially offloaded to smaller models. The authors provide a public code/dataset link, a human validation of the NLI metric on 415 cases, and a manual audit of codification quality, which are genuine strengths. The leakage and statistical-significance concerns below are the main obstacles to accepting the headline claims.

major comments (5)
  1. [§4, Fandom Benchmark; Fig. 17] The benchmark construction in §4 generates each guiding question with access to the reference action, and the paper provides no audit for leakage. The text states that questions are 'designed to constrain the response without providing clues toward the correct answer', but the caption of Figure 3 says the questions are 'generated to steer predictions to be relevant to the reference', and the Tyrion example in Figure 17 asks 'How does Tyrion respond to Cersei's confrontation about his decision to arrest Janos Slynt and appoint Bronn as Commander of the City Watch?', which restates the core of the reference. If even a fraction of the 5,141 questions behave this way, the NLI scores in Table 3 can be inflated by answer-copying rather than by character logic, and the comparison between codified and textual profiles becomes uninterpretable. Please release the questions, have humans rate leakage on a random sample, and add a control condition with generic or entity-removed questions.
  2. [§5.3, Table 3] Table 3 reports only artifact-level means, without confidence intervals, per-character standard deviations, or significance tests. The headline difference on main characters is 68.38 vs 65.98 (2.40 points) and on minor characters 69.87 vs 65.70 (4.17 points); given that scene counts per character range from about 48 to 214 and that Table 6 already shows large per-character variation (e.g., Sansa 81.25% vs Bran 38.89% win rates), these averages may not be statistically robust. Report bootstrap CIs or per-character paired tests, and show the full per-character distribution for the central codified-vs-textual comparison.
  3. [§5.4, Table 4] The claim that '1B + Code + Distill' approaches the 8B + Text baseline is not supported by the reported numbers. The distilled condition checker achieves only 70.53% agreement with gpt-4.1 on the held-out 10% of 415 scenes (about 42 examples), and the main-character NLI improves from 58.43 (1B+Code) to 60.21 (1B+Code+Distill), still 5.77 points below 8B+Text (65.98). The margin is within the likely noise given the small held-out set and the absence of error bars in Table 4. Please report the checker's per-class agreement, confidence intervals for Table 4, and either a larger distillation set or a threshold analysis for when the distilled checker becomes beneficial.
  4. [§6.2, Table 5] The manual codification audit in Table 5 shows that recall falls to 81.3% for if-depth 4 and 'Both' falls to 75.0%, based on only 16 cases. Because the paper's persistence claim depends on codified profiles executing 'complete' character logic, a one-in-five miss rate on the most complex segments is not a minor caveat; it may directly cap the measured advantage on logic-heavy characters. Please report error categories, evaluate Table 3 with imperfectly codified segments removed or flagged, and state whether the main results are robust when the 16 depth-4 cases are excluded.
  5. [§5.1, §5.3, Appendix C] The same model (gpt-4.1) both performs codification and serves as the NLI judge, and the human NLI validation in Appendix C is a reliability check of the judge in general, not a check against judge bias between the codified and textual conditions. Since the codified responses are generated under conditions that gpt-4.1 itself wrote, the 2.4-point gap could partly reflect stylistic compatibility with the judge. A judge-swap experiment with a different LLM scorer, or a blinded human re-scoring on a random subsample of Table 3, would establish that the advantage is not an artifact of judge preference.
minor comments (5)
  1. [§3.3, Figure 12 caption] Please correct the typos 'Codifed Profiles' and 'scholastic response' to 'Codified Profiles' and 'stochastic response'.
  2. [Figure 5 and surrounding text] Report the number of human judges, their inter-annotator agreement, and the exact tie-handling protocol; the current aggregate bars lack error bars and the reader cannot tell whether the 26–38 win/tie/lose counts are per-character or per-scene.
  3. [Figure 8 and Figure 4] Specify the exact model variants (llama-3.2-1b, llama-3.2-3b, llama-3.1-8b) in the axis labels and define the 'forward pass count' used in Figure 4, since the efficiency comparison depends on this unit.
  4. [§6.3] The Jotaro/Hol Horse case study is a single hand-built scene; state explicitly that it is illustrative and should not be interpreted as quantitative evidence, and consider moving it to the appendix.
  5. [Abstract and Contributions] The contribution list says the dataset 'will be open-sourced' while the abstract describes it as introduced; clarify the current release status and provide the exact license and access instructions.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: randomness demo is definitional and distilled checker is fitted to benchmark scenes; core benchmark comparison remains external.

  1. self definitional [Section 3.4 and Figure 7 / Appendix E]
    "Codified profile addresses this limitation by externalizing randomness into explicit control logic. Rather than relying on the LLM to probabilistically choose responses at generation time, we prompt it to generate executable Python code that encodes randomness via constructs such as random.choice([...]) and random.random() < p. ... In contrast, codifying probability can precisely control the randomness for the character to perform different responses with explicit probabilistic control flow."

    The target behaviors in Figure 7 (uniform rock/paper/scissors, uniform positive/negative emotion, 10% humor) are written into the codified profile as literal Python random calls; Appendix E shows random.choice(['positive','negative']) and if random.random() < 0.1. Therefore the measured P(Positive)=0.5 and P(Humor)=0.1 for the codified condition are semantic consequences of executing the supplied code, not empirical predictions about role-playing. The paper presents this as evidence that codified profiles 'can precisely control randomness,' which is true by definition of the code. The textual-profile failure is an independent empirical result, but the codified-side 'prediction' is its own construction.

  2. fitted input called prediction [Section 5.4, Table 4]
    "we distill from gpt-4.1's condition-checking outputs using 415 scenes (5 per character, 8% of all scenes) and obtain 20,759 labeled discrimination cases. A 3-class deberta-v3-base model (0.1B) (He et al., 2021) is trained on 90% of the data for 5 epochs and achieves 70.53% consistency with gpt-4.1 on the held-out 10%. As shown in Table 4, integrating this distilled discriminator into a 1B model with codified profiles leads to stronger role-playing performance than relying solely on the 1B LLM for both generation and condition checking."

    The distilled deberta model is trained on gpt-4.1's condition annotations for 415 scenes sampled from the same Fandom Benchmark used for Table 4. The paper reports consistency on the held-out 10% of condition labels, but never states that those 415 scenes are excluded from the end-to-end role-playing evaluation. Since check_condition outputs determine triggered_statements and hence the final response, a checker that memorizes scene-specific condition values for those 415 scenes makes the reported 1B+Code+Distill scores on those scenes partly fitted rather than predicted. The claim that the distilled system 'approaches the performance of 8B + Text' is thus partially supported by data the discriminator has already seen.

full rationale

The core Table 3 comparison is not circular: both codified and textual conditions are scored by the same gpt-4.1 NLI judge against Fandom ground-truth actions that are external to the method, and the Codified RAG baseline provides an additional control. The persistence and updating results likewise evaluate on future scenes after updates, so they do not reduce to their inputs. However, two partial circularities lower the score. First, the controllable-randomness demonstration encodes the target distribution directly in Python random calls; showing that the output frequencies match the code is executing the code, not testing a prediction. Second, the distilled condition checker is fitted to condition labels from 415 Fandom Benchmark scenes, and Table 4 reports end-to-end performance on the benchmark without a stated held-out split, so part of the reported 1B + Code + Distill gain can reflect memorized scene-specific conditions. The guiding-question and judge-style concerns (Section 4, Figure 3) are evaluation-validity risks rather than construction-level circularity, and are therefore not counted as circular steps here.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces a new representation and benchmark, not new physical entities. The central quantitative claims depend on a small number of hand-chosen evaluation weights, on the fidelity of LLM extraction and codification, and on a trained condition checker. None of these are justified from first principles, and the sensitivity of the rankings to them is not reported.

free parameters (3)
  • NLI score mapping = entailed=100, neutral=50, contradicted=0
    The benchmark's core metric is defined by this hand-chosen mapping. It affects every average score, and the paper does not test sensitivity to these weights.
  • Target probabilities in randomness case studies = p=0.1 for humor; 1/3 for paper/scissors/stone; 0.5 for positive/negative emotion
    These probabilities are authored into the codified profiles by the experimenters to match the expected character logic. The toy demonstrations therefore confirm the code sampler rather than an externally derived prediction.
  • Distilled condition checker = DeBERTa-v3-base trained on 20,759 labels; 70.53% consistency with gpt-4.1
    The 1B + Code + Distill result depends on this trained checker. Its weights are fitted to gpt-4.1 condition outputs, and the consistency on held-out labels is 70.53%, which is a parameter of the pipeline rather than a closed-form constant.
assumptions (4)
  • domain assumption LLM-based NLI scoring is a valid proxy for whether a generated action follows the character's ground-truth logic.
    All numeric comparisons in Tables 3, 4, and 6 rely on gpt-4.1 NLI scores. The paper manually checks 5 cases per character and reports roughly 89 to 92 percent alignment with human judgments, but the NLI model itself is treated as a trusted measuring instrument.
  • domain assumption Fandom profiles and storyline summaries accurately represent character logic, and spoiler filtering removes information that would leak future behavior.
    The benchmark is built entirely from Fandom. Section 4 states that profiles are filtered to remove spoilers, but no audit of filtering errors or of the extraction of scenes and reference actions is provided.
  • domain assumption gpt-4.1 can faithfully translate natural language profile segments into executable functions.
    The codified profiles are produced by gpt-4.1 using the Appendix B prompts. Manual evaluation in Section 6.2 reports high precision but recall drops to 81.3% for if-depth 4, so the method inherits any codification losses.
  • domain assumption Next-action prediction conditioned on a scene, a profile, and a guiding question is a sufficient probe of role-playing quality.
    The benchmark evaluates a single next action per scene. This operationalizes character consistency but ignores dialogue style, emotional nuance, and multi-turn behavior, which the preference evaluation only partially recovers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Codifying Character Logic in Role-Playing." pith.science (2026). https://pith.science/paper/SUUSQ6QI

@misc{pith2026250507705,
  author       = {Pith},
  title        = {Pith review of: Codifying Character Logic in Role-Playing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SUUSQ6QI}},
  note         = {Machine review of arXiv:2505.07705}
}
read the original abstract

This paper introduces Codified Profiles for role-playing, a novel approach that represents character logic as structured, executable functions for behavioral decision-making. Each profile defines a set of functions parse_by_scene(scene) that outputs a list of logic-grounded assertions triggered_statements, using both explicit control structures (e.g., if-then-else) and condition checks like check_condition(scene, question), where each question is a semantically meaningful prompt about the scene (e.g., "Is the character in danger?") discriminated by the role-playing LLM as true, false, or unknown. This explicit representation offers three key advantages over traditional prompt-based profiles, which append character descriptions directly into text prompts: (1) Persistence, by enforcing complete and consistent execution of character logic, rather than relying on the model's implicit reasoning; (2) Updatability, through systematic inspection and revision of behavioral logic, which is difficult to track or debug in prompt-only approaches; (3) Controllable Randomness, by supporting stochastic behavior directly within the logic, enabling fine-grained variability that prompting alone struggles to achieve. To validate these advantages, we introduce a new benchmark constructed from 83 characters and 5,141 scenes curated from Fandom, using NLI-based scoring to compare character responses against ground-truth actions. Our experiments demonstrate the significant benefits of codified profiles in improving persistence, updatability, and behavioral diversity. Notably, by offloading a significant portion of reasoning to preprocessing, codified profiles enable even 1B-parameter models to perform high-quality role-playing, providing a scalable and efficient foundation for local deployment of role-play agents.

Figures

Figures reproduced from arXiv: 2505.07705 by the authors.

Figure 1
Figure 1. A presentation of the workflow and advantages of Codified Profiles. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The evolving mechanism of codified profile to synchronize character with the storyline. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Curation and evaluation scenarios of our Fandom Benchmark. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: The role-playing performance with rea￾soning mechanism. In [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The Human/LLM-based preference evaluation results. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Role-playing performance with profiles evolving with the storyline. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Left: Case experiments on controllable randomness simulation by textual and codified profiles. Right: Scenario coverage comparison, evaluated by Best@K performance under stochastic response setting. Controllable Randomness [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: The role-playing performance with smaller role-playing LLMs. We compare vanilla prompting, textual pro￾files with chain-of-thought, and codified profiles across LLaMA-3 models of 1B (3.2), 3B (3.2), and 8B (3.1) parameters in [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: The role-playing perfor￾mance with different segmentation strategies [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Case study of multi-turn role-playing in a given scene: [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: The preprocessing prompts used in our experiments. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: The shared prompts used for evolving profile and scholastic response. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: The shared prompts used in our experiments. [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Codification cases for personality, relation, and working mechanism of superpower. [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: A case for the evolving mechanism for codified profiles. [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: Codification cases with randomness simulation involved. [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]
Figure 17
Figure 17. Figure 17: An example of a profile and testing cases used in our experiments. (Tyrion Lannister) [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 11 canonical work pages

  1. [1]

    R., Angeli, G., Potts, C., and Manning, C

    Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326, 2015

    arXiv 2015

  2. [2]

    let your characters tell their story

    Brahman, F., Huang, M., Tafjord, O., Zhao, C., Sachan, M., and Chaturvedi, S. "let your characters tell their story": A dataset for character-centric narrative understanding. In Moens, M., Huang, X., Specia, L., and Yih, S. W. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-...

    doi:10.18653/v1/2021.findings-emnlp.150 2021

  3. [3]

    From persona to personalization: A survey on role-playing language agents, 2024 a

    Chen, J., Wang, X., Xu, R., Yuan, S., Zhang, Y., Shi, W., Xie, J., Li, S., Yang, R., Zhu, T., Chen, A., Li, N., Chen, L., Hu, C., Wu, S., Ren, S., Fu, Z., and Xiao, Y. From persona to personalization: A survey on role-playing language agents, 2024 a . URL https://arxiv.org/abs/2404.18231

    arXiv 2024

  4. [4]

    Large language models meet harry potter: A dataset for aligning dialogue agents with characters

    Chen, N., Wang, Y., Jiang, H., Cai, D., Li, Y., Chen, Z., Wang, L., and Li, J. Large language models meet harry potter: A dataset for aligning dialogue agents with characters. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023 , pp.\ 8506--8520. Association f...

    doi:10.18653/v1/2023.findings-emnlp.570 2023

  5. [5]

    The oscars of AI theater: A survey on role-playing with language models

    Chen, N., Deng, Y., and Li, J. The oscars of AI theater: A survey on role-playing with language models. CoRR, abs/2407.11484, 2024 b . doi:10.48550/ARXIV.2407.11484. URL https://doi.org/10.48550/arXiv.2407.11484

    arXiv doi:10.48550/arxiv.2407.11484 2024

  6. [6]

    Compress to impress: Unleashing the potential of compressive memory in real-world long-term conversations

    Chen, N., Li, H., Chang, J., Huang, J., Wang, B., and Li, J. Compress to impress: Unleashing the potential of compressive memory in real-world long-term conversations. In Rambow, O., Wanner, L., Apidianaki, M., Al - Khalifa, H., Eugenio, B. D., and Schockaert, S. (eds.), Proceedings of the 31st International Conference on Computational Linguistics, COLING...

    work page 2025

  7. [7]

    Gallotta, R., Todd, G., Zammit, M., Earle, S., Liapis, A., Togelius, J., and Yannakakis, G. N. Large language models and games: A survey and roadmap. CoRR, abs/2402.18659, 2024. doi:10.48550/ARXIV.2402.18659. URL https://doi.org/10.48550/arXiv.2402.18659

    arXiv doi:10.48550/arxiv.2402.18659 2024

  8. [8]

    Do llms play dice? exploring probability distribution sampling in large language models for behavioral simulation

    Gu, J., Pang, L., Shen, H., and Cheng, X. Do llms play dice? exploring probability distribution sampling in large language models for behavioral simulation. In Rambow, O., Wanner, L., Apidianaki, M., Al - Khalifa, H., Eugenio, B. D., and Schockaert, S. (eds.), Proceedings of the 31st International Conference on Computational Linguistics, COLING 2025, Abu ...

    work page 2025

Show all 35 references
  1. [9]

    Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., and Liu, S. S. Bias in large language models: Origin, evaluation, and mitigation. CoRR, abs/2411.10915, 2024. doi:10.48550/ARXIV.2411.10915. URL https://doi.org/10.48550/arXiv.2411.10915

    2024 doi arXiv

  2. [10]

    Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing

    He, P., Gao, J., and Chen, W. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543, 2021

    2021 arXiv

  3. [11]

    Karpukhin, V., Oguz, B., Min, S., Lewis, P. S. H., Wu, L., Edunov, S., Chen, D., and Yih, W. Dense passage retrieval for open-domain question answering. In Webber, B., Cohn, T., He, Y., and Liu, Y. (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Lang...

    2020 doi

  4. [12]

    u ttler, H., Lewis, M., Yih, W., Rockt \

    Lewis, P. S. H., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K \" u ttler, H., Lewis, M., Yih, W., Rockt \" a schel, T., Riedel, S., and Kiela, D. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Larochelle, H., Ranzato, M., Hadsell, R., B...

    2020

  5. [13]

    Chatharuhi: Reviving anime character in reality via large language model

    Li, C., Leng, Z., Yan, C., Shen, J., Wang, H., MI, W., Fei, Y., Feng, X., Yan, S., Wang, H., Zhan, L., Jia, Y., Wu, P., and Sun, H. Chatharuhi: Reviving anime character in reality via large language model. CoRR, abs/2308.09597, 2023. doi:10.48550/ARXIV.2308.09597. URL https://...

    2023 doi arXiv

  6. [14]

    Artificial intelligence-enabled metaverse for sustainable smart cities: Technologies, applications, challenges, and future directions

    Lifelo, Z., Ding, J., Ning, H., Qurat-Ul-Ain, and Dhelim, S. Artificial intelligence-enabled metaverse for sustainable smart cities: Technologies, applications, challenges, and future directions. Electronics, 13 0 (24), 2024. ISSN 2079-9292. doi:10.3390/electronics13244874. UR...

    2024 doi

  7. [15]

    RECAP: retrieval-enhanced context-aware prefix encoder for personalized dialogue response generation

    Liu, S., Cho, H., Freedman, M., Ma, X., and May, J. RECAP: retrieval-enhanced context-aware prefix encoder for personalized dialogue response generation. In Rogers, A., Boyd - Graber, J. L., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for ...

    2023 doi

  8. [16]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. CoRR, abs/2303.08774, 2023. doi:10.48550/arXiv.2303.08774. URL https://doi.org/10.48550/arXiv.2303.08774

    2023 doi arXiv

  9. [17]

    and Shang, J

    Peng, L. and Shang, J. Quantifying and optimizing global faithfulness in persona-driven role-playing. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference...

    2024

  10. [18]

    X., Jin, S., Gao, Z., Fan, M., and Hui, P

    Qin, H. X., Jin, S., Gao, Z., Fan, M., and Hui, P. Charactermeet: Supporting creative writers' entire story character construction processes through conversation with llm-powered chatbot avatars. In Mueller, F. F., Kyburz, P., Williamson, J. R., Sas, C., Wilson, M. L., Dugas, ...

    2024

  11. [19]

    Sadeq, N., Xie, Z., Kang, B., Lamba, P., Gao, X., and McAuley, J. J. Mitigating hallucination in fictional character role-play. In Al - Onaizan, Y., Bansal, M., and Chen, Y. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, Nov...

    2024

  12. [20]

    Lamp: When large language models meet personalization

    Salemi, A., Mysore, S., Bendersky, M., and Zamani, H. Lamp: When large language models meet personalization. In Ku, L., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 20...

    2024 doi

  13. [21]

    Character-llm: A trainable agent for role-playing

    Shao, Y., Li, L., Dai, J., and Qiu, X. Character-llm: A trainable agent for role-playing. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023 , pp.\ 13...

    2023 doi

  14. [22]

    Reflexion: language agents with verbal reinforcement learning

    Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. Reflexion: language agents with verbal reinforcement learning. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Co...

    2023

  15. [23]

    O., Karpov, I., Miikkulainen, R., and Gold, A

    Stanley, K. O., Karpov, I., Miikkulainen, R., and Gold, A. Real-time interactive learning in the NERO video game. In Proceedings, The Twenty-First National Conference on Artificial Intelligence and the Eighteenth Innovative Applications of Artificial Intelligence Conference, J...

    2006

  16. [24]

    D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H., Jin, A., Bos, T., Baker, L., Du, Y., Li, Y., Lee, H., Zheng, H

    Thoppilan, R., Freitas, D. D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H., Jin, A., Bos, T., Baker, L., Du, Y., Li, Y., Lee, H., Zheng, H. S., Ghafouri, A., Menegali, M., Huang, Y., Krikun, M., Lepikhin, D., Qin, J., Chen, D., Xu, Y., Chen, Z., Roberts, A., Bosma, M., ...

    2022 arXiv

  17. [25]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M., Lacroix, T., Rozi \` e re, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. Llama: Open and efficient foundation language models. CoRR, abs/2302.13971, 2023. doi:10.48550...

    2023 doi arXiv

  18. [26]

    Does role-playing chatbots capture the character personalities? assessing personality traits for role-playing chatbots

    Wang, X., Fei, Y., Leng, Z., and Li, C. Does role-playing chatbots capture the character personalities? assessing personality traits for role-playing chatbots. arXiv preprint arXiv:2310.17976, 2023 a

    2023 arXiv

  19. [27]

    M., Peng, Z., Que, H., Liu, J., Zhou, W., Wu, Y., Guo, H., Gan, R., Ni, Z., Zhang, M., et al

    Wang, Z. M., Peng, Z., Que, H., Liu, J., Zhou, W., Wu, Y., Guo, H., Gan, R., Ni, Z., Zhang, M., et al. Rolellm: Benchmarking, eliciting, and enhancing role-playing abilities of large language models. arXiv preprint arXiv:2310.00746, 2023 b

    arXiv

  20. [28]

    H., Le, Q

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022. URL http://papers.nips.cc/paper\_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abc...

    2022

  21. [29]

    End-to-end spoken conversational question answering: Task, dataset and model

    You, C., Chen, N., Liu, F., Ge, S., Wu, X., and Zou, Y. End-to-end spoken conversational question answering: Task, dataset and model. In Carpuat, M., de Marneffe, M., and Ru \' z, I. V. M. (eds.), Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, ...

    2022 doi

  22. [30]

    Y., Xu, Z., Zhu, Y., Shi, X., Li, M., and Smola, A

    Yu, P., Shen, D., Meng, S., Lee, J., Yin, W., Cui, A. Y., Xu, Z., Zhu, Y., Shi, X., Li, M., and Smola, A. RPGBENCH: evaluating large language models as role-playing game engines. CoRR, abs/2502.00595, 2025. doi:10.48550/ARXIV.2502.00595. URL https://doi.org/10.48550/arXiv.2502.00595

    2025 doi arXiv

  23. [31]

    Evaluating character understanding of large language models via character profiling from fictional works

    Yuan, X., Yuan, S., Cui, Y., Lin, T., Wang, X., Xu, R., Chen, J., and Yang, D. Evaluating character understanding of large language models via character profiling from fictional works. In Al - Onaizan, Y., Bansal, M., and Chen, Y. (eds.), Proceedings of the 2024 Conference on ...

    2024

  24. [32]

    Optimizing generative ai by backpropagating language model feedback

    Yuksekgonul, M., Bianchi, F., Boen, J., Liu, S., Lu, P., Huang, Z., Guestrin, C., and Zou, J. Optimizing generative ai by backpropagating language model feedback. Nature, 639: 0 609--616, 2025

    2025

  25. [33]

    Unleasing chatgpt on the metaverse: Savior or destroyer? arXiv preprint arXiv:2303.13856, 2023

    Zhou, P. Unleasing chatgpt on the metaverse: Savior or destroyer? arXiv preprint arXiv:2303.13856, 2023

    2023 arXiv

  26. [34]

    Cognitive personalized search integrating large language models with an efficient memory mechanism

    Zhou, Y., Zhu, Q., Jin, J., and Dou, Z. Cognitive personalized search integrating large language models with an efficient memory mechanism. In Chua, T., Ngo, C., Kumar, R., Lauw, H. W., and Lee, R. K. (eds.), Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, ...

    2024

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.