Pith. sign in

REVIEW 3 major objections 4 minor 45 references

When Harry Meets Superman: The Role of The Interlocutor in Persona-Based Dialogue Generation

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that in persona-based dialogue, knowing the interlocutor's biography, more than their turns, determines how identifiable the target speaker is.

desk verdict A useful and reusable evaluation paradigm for interlocutor effects in persona dialogue, but the central 'adaptation' claim is confounded with evaluator cue availability and needs a control condition before it is asserted. read the letter →

arxiv 2505.24613 v1 pith:I3XHWB6I submitted 2025-05-30 cs.CL

Daniela Occhipinti , Marco Guerini , Malvina Nissim This is my paper
classification cs.CL
keywords persona-baseddialoguegenerationinterlocutoradaptationauthoridentificationLLM-as-a-judgerole-playagentsevaluationzero-shotfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies whether dialogue agents adapt not only to the persona they are asked to play but also to the persona of the person they are talking to. The authors test this by generating dialogues between movie characters and others, then asking an LLM-as-a-judge and human raters to identify the target speaker's biography from the dialogue under four conditions: with the interlocutor's biography and turns both visible, only one visible, or neither. Their central claim is that access to the interlocutor's persona improves recognition of the target speaker, while masking it hurts, and that the interlocutor's biography contributes more than their conversational turns. They also find that zero-shot models often copy rare biographical words into their lines, which makes identification easier but the dialogue more superficial; fine-tuning reduces this copying and produces dialogues closer to human-written ones. If right, the result would change how persona-based dialogue systems are evaluated and designed: who the agent is talking to is a primary axis of persona fidelity.

What carries the argument

The key machinery is an author-identification evaluation design with four interlocutor-disclosure conditions: BothDisc, BioDisc, TurnsDisc, and BothMask. In each condition, an evaluator, either a fine-tuned Llama 3.1 LLM-as-a-judge or a human rater, must pick the target speaker's biography from three semantically similar options. The underlying logic is that if hiding the interlocutor's biography or turns makes the target speaker harder to recognise, then the generated dialogue must have been adapted to that interlocutor information. The design is paired with rare-word overlap and standard text-overlap metrics between biographies and turns, which expose superficial copying.

What would settle it

Hold the generated dialogues fixed and vary only what the evaluator sees: if giving the evaluator the true interlocutor biography raises identification accuracy by the same margin of about 0.08 for fine-tuned generations, even for dialogues that were generated without access to that biography, then the disclosure effect is an evaluator-side cue and does not demonstrate adaptation.

Watch

Extended reading notes

Core claim

The paper's central claim is that the interlocutor's identity information is a first-order determinant of speaker identifiability in persona-based dialogue, and that the effect is driven by biography rather than conversation turns. Across PRODIGy gold dialogues and fine-tuned Llama 3.1 8B Instruct generations, an LLM-as-a-judge identifies the target speaker's biography at much higher accuracy when the interlocutor's biography is disclosed (gold: 0.820 BothDisc, 0.805 BioDisc; generated: 0.594 for both) than when it is masked and only turns are visible (gold: 0.588; generated: 0.517) or everything is masked (gold: 0.577; generated: 0.515). The same pattern appears in zero-shot generations but at higher overall accuracy, because the zero-shot model copies rare biographical terms into dialogue (about 40% of dialogues carry at least one rare biography word, versus about 5% for fine-tuned and gold), making the speaker trivially easy to identify. The paper also reports that models generalise across topics but degrade for unfamiliar speaker pairings, and that human raters show the same direction of effect, though for humans the interlocutor's turns helped slightly more than biography.

Load-bearing premise

The conclusion that the model adapts to its interlocutor assumes that higher evaluator accuracy when the interlocutor's biography is disclosed reflects information actually present in the generated dialogue, rather than the evaluator using the disclosed biography as an external clue to guess what the target speaker would say.

Editorial extensions

If this is right

  • Evaluation reports for persona-based dialogue should state whether the interlocutor's biography and turns were visible, because hiding that information changes measured speaker identifiability by roughly eight accuracy points for fine-tuned models.
  • High identification accuracy in zero-shot role-play can be a sign of verbatim biography reuse, not genuine persona adaptation, so evaluations should control for prompt-to-output lexical overlap.
  • Fine-tuning on persona-based dialogues is the more reliable path toward persona consistency because it reduces surface copying and produces generations closer to gold dialogues in overlap metrics.
  • Robustness engineering for role-play agents should prioritise unfamiliar interlocutor pairings, where accuracy drops sharply, before worrying about unfamiliar topics, which show little effect.
  • Because LLM judges and human judges disagree on which interlocutor cue matters most, conclusions about the mechanism of adaptation should rely on multiple judge types.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The disclosure-condition asymmetry may be partly an evaluator-side inference effect: an evaluator who knows the interlocutor is Superman can predict what Harry would say to Superman, even if the generated dialogue was not actually adapted. The paper's own human results, where turns mattered more than biography, are consistent with this alternative reading, so the 'adaptation' conclusion is stronge
  • A direct test of adaptation would hold the generated dialogue fixed and vary only what the evaluator sees; if accuracy still jumps when the interlocutor's biography is disclosed, the effect is cue-based rather than generative.
  • If biography is the dominant identification signal, persona-evaluation benchmarks could add an 'interlocutor masking' dimension as a standard control, and systems that avoid biography copying would likely show smaller disclosure gaps and better generalisation to novel interlocutors.
  • The copy-paste finding suggests that zero-shot role-play success in other benchmarks may be artificially high; measuring rare-word overlap between prompt biography and generation would be a cheap diagnostic for that artefact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies whether persona-based dialogue agents adapt their responses to the interlocutor's profile. The authors generate dialogues with Llama 3.1 8B Instruct in zero-shot and fine-tuned settings, using PRODIGy and newly created non-PRODIGy characters, and evaluate them via an author identification task. Evaluators (an LLM judge and human annotators) are given one of four disclosure conditions: both interlocutor biography and turns, biography only, turns only, or neither. The paper reports that access to the interlocutor's biography improves target-speaker identification, that biography matters more than dialogue turns, that models generalize across topics but struggle with unfamiliar interlocutors, and that zero-shot models achieve high identifiability by copy-pasting biographical details. The copy-paste analysis is supported by rare-word overlap and BLEU/ROUGE/METEOR comparisons.

Significance. If the central adaptation claim were established, the proposed masking/disclosure framework would be a useful evaluation paradigm for persona-based dialogue. The copy-paste finding is a solid, concrete contribution: it gives direct evidence that zero-shot models reproduce biographical surface cues, and the fine-tuned models' lower overlap with gold dialogues is an interesting contrast. However, the central claim that differences across disclosure conditions reflect the model's adaptation to the interlocutor is not supported by the experimental design, because those conditions vary what the evaluator sees rather than what the generator saw. The aggregate numeric differences are also not tested for significance, and the per-experiment and human results contain contradictions with the headline claim. The significance is therefore conditional on additional experimental control.

major comments (3)
  1. [Section 1, Section 3.3, Table 2] The central inference stated in Section 1 — "if masking interlocutor information makes it harder to identify the target speaker's biography, it is likely that the model has appropriately adapted its responses" — is confounded. The four disclosure conditions change what the evaluator sees, not what the generator saw. Under BioDisc, an evaluator who knows the interlocutor is Superman can reason about what Harry would plausibly say to Superman even if the generated dialogue is not actually adapted to the interlocutor; the same external-cue account predicts BioDisc > TurnsDisc because a biography is a richer cue than turns. Table 2's aggregate difference (0.594 vs 0.517) is exactly what this cue-based account predicts. The paper needs a matched/mismatched control, e.g., taking a fixed generated dialogue and providing the correct interlocutor biography versus a mismatched biography; if the model truly adapts its content to the interlocutor, the mismatched biography should reduce evaluator accuracy relative to the correct one. Without such a control, the adaptation claim is not established.
  2. [Sections 4.1, 4.2, 5; Tables 2 and 8] The paper repeatedly uses the word "significantly" (e.g., "significantly enhances the judge's accuracy" in Section 4.1, "performance drops significantly" in Section 4.2, "significantly improves biography recognition" in Section 5) but reports no statistical significance tests, confidence intervals, or inter-annotator agreement. In Table 2, the difference between BioDisc (0.594) and TurnsDisc (0.517) is a single aggregate without variance; in Table 8, the human-evaluation difference between BioDisc (0.449) and TurnsDisc (0.474) is in the opposite direction and also lacks error bars. The claims of significance are therefore unsupported. The authors should report bootstrap confidence intervals, a proper significance test (e.g., McNemar's test for paired accuracy), or clearly label the results as descriptive.
  3. [Section 4.2, Appendix E.1, Appendix E.2, Table 8] The headline claim that "the interlocutor's biography plays a more significant role than their conversational turns" (Section 4.2) is contradicted by several per-experiment results. In the fine-tuned model's Exp3 and Exp4 (Table 17), TurnsDisc outperforms BioDisc (0.578 vs 0.553 and 0.572 vs 0.558). In zero-shot results (Table 19), Exp3–Exp6 consistently show that disclosing the biography is not helpful (e.g., Exp5: BothDisc 0.657, BioDisc 0.654, TurnsDisc 0.723, BothMask 0.730). The human evaluation (Table 8) also shows TurnsDisc (0.474) above BioDisc (0.449). The aggregate ordering in Table 2 therefore does not reflect a stable effect across the design. The paper should present the per-experiment pattern, explain when biography disclosure helps and when it does not, and restrict the claim accordingly.
minor comments (4)
  1. [Section 3.3 and Figure 2] The label "BothM ask" appears with an internal space in Section 3.3 and Figure 2; it should be "BothMask" for consistency with the other condition names.
  2. [Section 4.4 and Appendix E.3] The claim that rare words appear "up to eight times more frequently" is based on a single aggregated percentage (40.69% vs 5.18%); the per-experiment values in Table 20 show a range, and the abstract should avoid implying a uniform eightfold increase across all conditions.
  3. [Table 8] The F1 values in Table 8 do not match the standard F1 formula applied to the reported precision and recall (e.g., BothDisc: 2*0.478*0.375/(0.478+0.375) = 0.420, not 0.395). The paper should specify whether F1 is macro-averaged, micro-averaged, or computed differently.
  4. [Limitations] The Limitations section acknowledges data contamination and human evaluation variability but does not mention the evaluator-cue confound described above or the absence of significance testing; adding these would accurately represent the scope of the conclusions.

Circularity Check

2 steps flagged · score 4.0 of 10

The 'interlocutor adaptation' result is an evaluator cue effect renamed as generator behavior; no matched control separates the two.

  1. renaming known result [Abstract; Section 1 (Introduction)]
    "By systematically masking or disclosing information about the interlocutor, we assess its impact on dialogue generation. Results show that access to the interlocutor's persona improves the recognition of the target speaker, while masking it does the opposite. The key idea is that if masking interlocutor information makes it harder to identify the target speaker's biography, it is likely that the model has appropriately adapted its responses to the interlocutor's information, while still staying coherent to the target persona."

    The masking/disclosure is applied only to the evaluator's input (Figure 3); the generated dialogues are fixed before evaluation and are identical across disclosure conditions. The observed regularity is the well-known one that giving a reader more cue information improves identification accuracy. The paper renames this evaluator-side cue effect as 'the model has appropriately adapted its responses to the interlocutor.' The conclusion would hold even if the generator ignored the interlocutor entirely, as long as the evaluator can use the disclosed biography to reason about what the target would say.

  2. self definitional [Section 4.2 (Fine-Tuned Model's Generations)]
    "As shown in Table 2, the interlocutor's biography plays a more significant role than their conversational turns. This indicates that the model efficiently tailors its responses to the interlocutor's biography."

    Table 2 compares BioDisc (evaluator is given the interlocutor's biography, turns masked) with TurnsDisc (evaluator is given interlocutor's turns, biography masked). The higher accuracy in BioDisc shows that the biography is a more useful cue for the evaluator; it does not show that the generator tailored its responses to the biography. The paper's own human evaluation (Table 8) reverses this ordering (TurnsDisc 0.474 > BioDisc 0.449), confirming that the automatic result reflects judge cue preference rather than a stable generation property. The claim 'the model efficiently tailors its responses to the interlocutor's biography' is thus defined by the evaluator input manipulation, making the conclusion equivalent to the Section 1 inference rule rather than derived from dialogue content.

full rationale

The paper does not fit parameters and does not rest on a uniqueness theorem or an ansatz smuggled in via self-citation; the authors' prior PRODIGy dataset is used as a data resource, not as evidence for the adaptation claim. The central problem is that the four disclosure conditions alter the evaluator's information, while the conclusions are about the generator's behavior. The abstract's 'access to the interlocutor's persona improves recognition' is true by construction of the evaluation protocol, and Section 1's key idea defines 'adapted responses' as 'harder to identify when masked,' so the subsequent finding is that definition relabeled. This is partial circularity/renaming rather than a formal self-citation loop, and the copy-paste analysis in Section 4.4 is an independent contribution; hence a moderate score of 4.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its main assumptions are about the validity of the evaluation framework and the representativeness of the dataset. Free parameters are design choices (thresholds, limits) that affect quantitative claims.

free parameters (3)
  • rare_word_zipf_threshold = 4.0
    Used in Appendix E.3 to define rare words for the copy-paste overlap analysis. The quantitative comparison (e.g., 41.84% vs 9.68% in Exp1) depends on this threshold.
  • biography_sentence_limit = 5
    Each character's biography is truncated to the first five sentences to match the minimum in PRODIGy (Section 3.2). This affects how much persona information is available to both generator and evaluator.
  • dialogue_turns = 8
    Dialogue length is fixed at 8 turns based on prior findings on persona alignment decay (Section 3.2). This choice affects the difficulty of speaker identification.
assumptions (4)
  • domain assumption Author identification task is a valid proxy for how well a dialogue model aligns with target and interlocutor personas.
    The paper relies on this to interpret accuracy differences in speaker identification as evidence about generation quality (Section 1).
  • domain assumption The two most semantically similar biographies (by SBERT cosine similarity) provide a sufficiently challenging distractor set such that identification reflects persona-relevant content rather than superficial overlap.
    The evaluation design uses this to prevent trivial identification (Section 3.3).
  • domain assumption The fine-tuned Llama 3.1 8B judge is a reliable evaluator of persona identification, comparable to human judgment.
    The paper uses the LLM judge as the primary evaluation tool but does not report correlation or agreement with human judgments; human results show different relative ordering for some conditions.
  • domain assumption PRODIGy dialogues and character biographies are representative of persona-based dialogue tasks.
    The study is based on this dataset; results may not generalize beyond fictional movie characters (stated in Limitations).

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Harry Meets Superman: The Role of The Interlocutor in Persona-Based Dialogue Generation." pith.science (2026). https://pith.science/paper/I3XHWB6I

@misc{pith2026250524613,
  author       = {Pith},
  title        = {Pith review of: When Harry Meets Superman: The Role of The Interlocutor in Persona-Based Dialogue Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I3XHWB6I}},
  note         = {Machine review of arXiv:2505.24613}
}
read the original abstract

Endowing dialogue agents with persona information has proven to significantly improve the consistency and diversity of their generations. While much focus has been placed on aligning dialogues with provided personas, the adaptation to the interlocutor's profile remains largely underexplored. In this work, we investigate three key aspects: (1) a model's ability to align responses with both the provided persona and the interlocutor's; (2) its robustness when dealing with familiar versus unfamiliar interlocutors and topics, and (3) the impact of additional fine-tuning on specific persona-based dialogues. We evaluate dialogues generated with diverse speaker pairings and topics, framing the evaluation as an author identification task and employing both LLM-as-a-judge and human evaluations. By systematically masking or disclosing information about the interlocutor, we assess its impact on dialogue generation. Results show that access to the interlocutor's persona improves the recognition of the target speaker, while masking it does the opposite. Although models generalise well across topics, they struggle with unfamiliar interlocutors. Finally, we found that in zero-shot settings, LLMs often copy biographical details, facilitating identification but trivialising the task.

Figures

Figures reproduced from arXiv: 2505.24613 by the authors.

Figure 1
Figure 1. Example of a generated dialogue between Harry and Superman. The upper section displays the prompt, while the lower the beginning of the generated conversation. One aspect that has received much less atten￾tion instead, is the ability of such persona-based agents to adapt to the other speaker they are engag￾ing with (i.e., the interlocutor). Indeed, the impact on the model’s responses of the interlocutor’s per￾sona, … view at source ↗
Figure 2
Figure 2. Pipeline: (i) Fine-tune Llama 3.1 8B Instruct on PRODIGy dialogues and speaker pairings. (ii) Create [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Information disclosure configurations used in the evaluation: (a) Both [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of dialogues generated in a zero [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 14 canonical work pages

  1. [1]

    Jaewoo Ahn, Taehyun Lee, Junyoung Lim, Jin-Hwa Kim, Sangdoo Yun, Hwaran Lee, and Gunhee Kim. 2024. https://doi.org/10.18653/v1/2024.findings-acl.197 T ime C hara: Evaluating point-in-time character hallucination of role-playing large language models . In Findings of the Association for Computational Linguistics: ACL 2024, pages 3291--3325, Bangkok, Thaila...

    doi:10.18653/v1/2024.findings-acl.197 2024

  2. [2]

    Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909/ METEOR : An automatic metric for MT evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , pages 65--72, Ann Arbor, Michigan. Association for Compu...

    2005

  3. [3]

    Hezekiah J Branch, Jonathan Rodriguez Cefalu, Jeremy McHugh, Leyla Hujer, Aditya Bahl, Daniel del Castillo Iglesias, Ron Heichman, and Ramesh Darwishi. 2022. Evaluating the susceptibility of pre-trained language models via handcrafted adversarial examples. arXiv preprint arXiv:2209.02128

    arXiv 2022

  4. [4]

    Camilla Casula, Sebastiano Vecellio Salto, Alan Ramponi, and Sara Tonelli. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1099 Delving into qualitative implications of synthetic data for hate speech detection . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19709--19726, Miami, Florida, USA. Association...

    work page doi:10.18653/v1/2024.emnlp-main.1099 2024

  5. [5]

    Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao. 2024. https://arxiv.org/abs/2404.18231 From persona to personalization: A survey on role-playing language agents . Preprint, arXiv:2404.18231

    arXiv 2024

  6. [6]

    Hope B Corrigan, Georgiana Craciun, and Allison M Powell. 2014. How does target know so much about its customers? utilizing customer analytics to make marketing decisions. Marketing Education Review, 24(2):159--166

    work page 2014

  7. [7]

    Lorenzo De Mattei, Michele Cafagna, Felice Dell ' Orletta, and Malvina Nissim. 2020. https://aclanthology.org/2020.lrec-1.828/ Invisible to people but not to machines: Evaluation of style-aware H eadline G eneration in absence of reliable human judgment . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6709--6717, Marseil...

    work page 2020

  8. [8]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314

    arXiv 2023

Show all 45 references
  1. [9]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  2. [10]

    Lucie Flekova, Jordan Carpenter, Salvatore Giorgi, Lyle Ungar, and Daniel Preo t iuc-Pietro. 2016. https://doi.org/10.18653/v1/P16-1080 Analyzing biases in human perception of user age and gender from text . In Proceedings of the 54th Annual Meeting of the Association for Comp...

    2016 doi

  3. [11]

    Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. 2024. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint arXiv:2406.20094

    2024 arXiv

  4. [12]

    Jia-Chen Gu, Hui Liu, Zhen-Hua Ling, Quan Liu, Zhigang Chen, and Xiaodan Zhu. 2021. Partner matters! an empirical study on fusing personas for personalized response selection in retrieval-based chatbots. In Proceedings of the 44th International ACM SIGIR Conference on Research...

    2021

  5. [13]

    Balachander Krishnamurthy, Konstantin Naryshkin, and Craig Wills. 2011. Privacy leakage vs. protection measures: the growing disconnect. In Proceedings of the Web, volume 2, pages 1--10

    2011

  6. [14]

    Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun. 2023. https://arxiv.org/abs/2308.09597 Chatharuhi: Reviving anime character in reality via large languag...

    2023 arXiv

  7. [15]

    Kenneth Li, Tianle Liu, Naomi Bashkansky, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024. https://arxiv.org/abs/2402.10962 Measuring and controlling instruction (in)stability in language model dialogs . Preprint, arXiv:2402.10962

    2024 arXiv

  8. [16]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  9. [17]

    Qian Liu, Yihong Chen, Bei Chen, Jian-Guang Lou, Zixuan Chen, Bin Zhou, and Dongmei Zhang. 2020. https://doi.org/10.18653/v1/2020.acl-main.131 You impress me: Dialogue generation via mutual persona perception . In Proceedings of the 58th Annual Meeting of the Association for C...

    2020 doi

  10. [18]

    Hongyuan Lu, Wai Lam, Hong Cheng, and Helen Meng. 2022. https://doi.org/10.18653/v1/2022.naacl-main.382 Partner personas generation for dialogue response generation . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Lingu...

    2022 doi

  11. [19]

    Keming Lu, Bowen Yu, Chang Zhou, and Jingren Zhou. 2024. https://arxiv.org/abs/2401.12474 Large language models are superpositions of all characters: Attaining arbitrary role-play via self-alignment . Preprint, arXiv:2401.12474

    2024 arXiv

  12. [20]

    Daniela Occhipinti, Serra Sinem Tekiro g lu, and Marco Guerini. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.222 PRODIG y: a PRO file-based DI alogue generation dataset . In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3500--3514, Mexic...

    2024 doi

  13. [21]

    OpenAI. 2023. https://doi.org/10.48550/arXiv.2303.08774 Gpt-4 technical report . arXiv

    2023 doi arXiv

  14. [22]

    OpenAI. 2024. https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/ Gpt-4o mini: Advancing cost-efficient intelligence

    2024

  15. [23]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  16. [24]

    Bernstein

    Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. https://doi.org/10.1145/3586183.3606763 Generative agents: Interactive simulacra of human behavior . In Proceedings of the 36th Annual ACM Symposium on User Int...

    2023

  17. [25]

    F \'a bio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527

    2022 arXiv

  18. [26]

    Daniel Russo, Shane Kaszefski-Yaschuk, Jacopo Staiano, and Marco Guerini. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.703 Countering misinformation via emotional response generation . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processi...

    2023 doi

  19. [27]

    Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models. Nature, 623(7987):493--498

    2023

  20. [28]

    Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. https://aclanthology.org/2023.emnlp-main.814 Character- LLM : A trainable agent for role-playing . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13153--13187, Singapore...

    2023

  21. [29]

    Yifan Song, Guoyin Wang, Sujian Li, and Bill Yuchen Lin. 2024. The good, the bad, and the greedy: Evaluation of llms should not ignore non-determinism. arXiv preprint arXiv:2407.10457

    2024 arXiv

  22. [30]

    Efstathios Stamatatos, Walter Daelemans, Ben Verhoeven, Martin Potthast, Benno Stein, Patrick Juola, Miguel A Sanchez-Perez, Alberto Barr \'o n-Cede \ n o, et al. 2014. Overview of the author identification task at pan 2014. In CEUR Workshop Proceedings, volume 1180, pages 877...

    2014

  23. [31]

    Nandan Thakur, Nils Reimers, Johannes Daxenberger, and Iryna Gurevych. 2021. https://www.aclweb.org/anthology/2021.naacl-main.28 Augmented SBERT : Data augmentation method for improving bi-encoders for pairwise sentence scoring tasks . In Proceedings of the 2021 Conference of ...

    2021

  24. [32]

    Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, and Yun-Nung Chen. 2024. https://arxiv.org/abs/2406.01171 Two tales of persona in llms: A survey of role-playing and personalization . Preprint, arXiv:2406.01171

    2024 arXiv

  25. [33]

    Walter JB Van Heuven, Pawel Mandera, Emmanuel Keuleers, and Marc Brysbaert. 2014. Subtlex-uk: A new and improved word frequency database for british english. Quarterly journal of experimental psychology, 67(6):1176--1190

    2014

  26. [34]

    Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, and Junran Peng. 2024 a . https://doi.org/10.18653/v1/2024.findings-acl.878 R ole ...

    2024 doi

  27. [35]

    Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.102 I n C haracter: Evaluating personality fidelity in role-playing a...

    2024 doi

  28. [36]

    Jiaxin Wen, Pei Ke, Hao Sun, Zhexin Zhang, Chengfei Li, Jinfeng Bai, and Minlie Huang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.84 Unveiling the implicit toxicity in large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language...

    2023 doi

  29. [37]

    Weiqi Wu, Hongqiu Wu, Lai Jiang, Xingyuan Liu, Hai Zhao, and Min Zhang. 2024. https://doi.org/10.18653/v1/2024.findings-acl.196 From role-play to drama-interaction: An LLM solution . In Findings of the Association for Computational Linguistics: ACL 2024, pages 3271--3290, Bang...

    2024 doi

  30. [38]

    Chen Xu, Piji Li, Wei Wang, Haoran Yang, Siyun Wang, and Chuangbai Xiao. 2022. Cosplay: Concept set guided personalized dialogue generation across both party personas. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retr...

    2022

  31. [39]

    Jintang Xue, Yun-Cheng Wang, Chengwei Wei, Xiaofeng Liu, Jonghye Woo, and C-C Jay Kuo. 2023. Bias and fairness in chatbots: An overview. arXiv preprint arXiv:2309.08836

    2023 arXiv

  32. [40]

    Wu Youyou, Michal Kosinski, and David Stillwell. 2015. Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences, 112(4):1036--1040

    2015

  33. [41]

    Xiaoyan Yu, Tongxu Luo, Yifan Wei, Fangyu Lei, Yiming Huang, Hao Peng, and Liehuang Zhu. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.697 Neeko: Leveraging dynamic L o RA for efficient multi-character role-playing agent . In Proceedings of the 2024 Conference on Empirical...

    2024 doi

  34. [42]

    Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. https://doi.org/10.18653/v1/P18-1205 Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational ...

    2018 doi

  35. [43]

    Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, and Minlie Huang. 2024. https://doi.org/...

    2024 doi

  36. [44]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.