{"id":"a9a4610b-5827-4367-9f8e-787f4b9449ae","arxiv_id":"2509.04343","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"MBTI personality prompts measurably change how LLM agents write stories and play strategic games, with self-reflection before communication supporting cooperative behavior.","lead":"This paper introduces MBTI-in-Thoughts, a prompt-only framework that gives LLM agents fictional MBTI personality types and tests how those types change behavior. It finds that personality priming shifts emotional expression in stories and strategy in games, and that self-reflection before group discussion helps cooperation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Narrative 'excel' claim rests on an unvalidated LLM judge that likely measures surface style, not reader-valued quality; human validation is needed.","rationale":"The reader's weakest assumption is the same as mine: the narrative-effectiveness claim depends on LLM-as-a-judge scores that are never validated against human judgments. I consider this the most load-bearing concern because the paper's value proposition is 'enhancing the effectiveness' of LLM agents; the only direct evidence for that enhancement is the narrative evaluation. The game-theory section provides objective behavioral differences, but those show personality-conditioned behavior, not improved task effectiveness. I also considered two other weaknesses: the 16Personalities verification is weakened by exemplar-laden prompts, and the paper explicitly says it omits uninformative results. Both are real, but they are secondary: the game metrics are independent of the personality test, and the omission issue affects the breadth of the 'consistent' claim rather than the specific narrative-effectiveness claim. The proposed check—masking personality references and obtaining human ratings—would settle whether the observed T/F gap reflects style or quality. Because the paper could potentially pass this check, the appropriate outcome is to keep the reader's CONDITIONAL verdict rather than reject.","tokens_in":40770,"tokens_out":8718,"duration_ms":86841,"concrete_test":"Re-score a stratified sample of 100 stories (e.g., 20 each from Feeling types, Thinking types, EXPERT, NONE) under three variants: (a) full JSON output, (b) story field only, (c) story field with all first-person personality references masked. Then collect 3–5 human ratings on the same masked stories for believability, emotional engagement, and overall quality. Compute the Feeling-vs-Thinking gap and judge–human agreement (ICC or Spearman). If the gap disappears under masking, or if humans do not reproduce it, the 'excel' claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2's flagship claim ('emotionally expressive agents excel in narrative generation') is supported only by LLM-as-a-judge scores from PersonaLLM metrics. No human validation, inter-annotator agreement, judge model, or confidence intervals are reported. The headline attributes (Emotionally Chargedness, Happy Ending, Personalness) are surface stylistic features that an LLM judge can infer from emotion-word density, sentiment polarity, and first-person pronouns—exactly the cues that the Feeling priming prompts explicitly instruct. The problem is compounded by the generation template in Appendix C.3: each agent must output a 'relation to personality' field beginning with 'I am a(n) [personality type]' and a 'reasoning related to personality' field that references personality traits. If the judge consumes the full JSON output, the condition label is directly visible; if it sees only the story field, no masking is reported. Either way, the T/F gap in 'emotionally charged' scores does not establish that Feeling-primed agents produce better narratives for human readers. The objective game metrics in Section 4.3 are a useful independent support for behavioral divergence, but they do not validate the narrative-effectiveness claim. The paper's central 'enhanced effectiveness' result therefore is conditional on an unvalidated judge.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MBTI-in-Thoughts (MiT), a framework that conditions LLM agents on MBTI personality profiles via prompt engineering. The authors claim that such priming induces consistent behavioral biases: Feeling-primed agents produce more emotionally expressive narratives, Thinking-primed agents adopt more stable strategies in game-theoretic settings, Introverts and Judging types are more honest, and self-reflection before communication improves cooperative outcomes and reasoning. They integrate the 16Personalities test for automated verification of trait persistence and propose a formal mapping to other psychological frameworks (Big Five, HEXACO, Enneagram, DISC) to support generalization. The evaluation uses multiple LLMs and compares against NONE and EXPERT baselines across narrative generation, multi-agent communication protocols, and several two-player games.","tokens_in":41045,"tokens_out":4168,"duration_ms":39960,"significance":"If the central claims hold, the paper would make a useful practical contribution: lightweight, fine-tuning-free control of LLM behavior along affective and cognitive axes, with potential applications in narrative generation, negotiation, and multi-agent coordination. The release of code is a concrete strength, as is the use of objective behavioral metrics (defection rates, strategy-switch rates, honesty rates) in Section 4.3. However, the effectiveness claims rest on two fragile pillars: a verification procedure that leaks the intended answer through exemplars, and an unvalidated LLM-as-a-judge evaluation for the narrative results. The generalization-to-other-frameworks claim is only supported by a formal restatement, not by experiments. These issues are substantial enough that the paper's headline conclusions are not yet established.","major_comments":[{"comment":"The 16Personalities verification is circular. The verification prompt explicitly includes four few-shot exemplars aligned with the target type's stance on each axis (e.g., 'As an introverted personality...' for E/I, 'As a feeling personality...' for T/F). This tells the model exactly how to answer, so the separability in Figure 2 merely shows the model can follow the exemplars, not that priming induces persistent traits. The 'robustness' claim in Section 4.1 is therefore not supported by an independent test. Please redesign the verification without answer-revealing exemplars, or clearly label the current result as a prompt-following check rather than a personality verification.","section":"Section 3.1, Appendix C.1"},{"comment":"The narrative-generation claim that 'Feeling types excel' rests entirely on LLM-as-a-judge scores from PersonaLLM metrics. No human validation, inter-annotator agreement, judge model, or confidence intervals are reported. Moreover, the generation template in Appendix C.3 forces each output to include a 'relation to personality' field beginning with 'I am a(n) [personality type]' and a 'reasoning related to personality' field. If the judge sees these fields, the type label is directly visible; if not, no masking procedure is described. Thus the reported T/F differences may reflect surface style or even the condition label, not reader-valued narrative quality. Human evaluation of a masked subset is necessary to support the 'excel' claim.","section":"Section 4.2, Appendix C.3"},{"comment":"The key behavioral comparisons are presented without any statistical support. The text states that Thinking types defect in 'roughly 90%' of rounds versus '≈50%' for Feeling types, and that Introverts have 'significantly higher' honesty rates, but no test statistic, p-value, standard error, number of games, or number of runs is given. Figure 4 shows only point estimates without error bars. Since these objective metrics are the strongest evidence for behavioral divergence, the authors should report variance across repeated runs and perform appropriate significance tests (e.g., bootstrap or mixed-effects models).","section":"Section 4.3, Figure 4"},{"comment":"The paper states: 'we present representative results and omit data that does not yield relevant insights.' This selective reporting makes it impossible to assess whether the omitted data contradict the central claims. The criteria for 'relevant insights' are undefined, and no appendix or supplementary provides the full results. To allow independent verification, all experimental outcomes should be released (at least in a supplementary file), and the selection process should be described transparently.","section":"Section 4, 'Evaluation & Use Cases'"},{"comment":"The claim that the framework 'generalizes seamlessly' to Big Five, HEXACO, Enneagram, and DISC is unsupported by any experiments. The formal mapping in Appendix B simply defines each framework as a function F: Agent -> R^n and re-describes existing trait models; it does not demonstrate that the prompt-conditioning method works for those frameworks. No Big Five, HEXACO, Enneagram, or DISC agents were instantiated or evaluated. This is a contribution listed in the abstract and introduction, so it needs either empirical validation or a clear rephrasing as a theoretical framing rather than a demonstrated capability.","section":"Section 3.3, Appendix B"}],"minor_comments":[{"comment":"The sentence '...providing a detailed plan...' ends with an incomplete fragment: 'Neuroticism, providing.' Should be 'providing...' or restructured.","section":"Section 2.2"},{"comment":"The dataset name 'W RITING PROMPTS' contains a stray space; also 'PersonaLLM' is cited but the exact metric names and versions are not defined in the text.","section":"Section 4.2"},{"comment":"'more performance on various cognitive and affective tasks' should be 'better performance' or 'higher performance'.","section":"Section 4.1"},{"comment":"'which is a statistically significant' is grammatically incomplete; should be 'which is statistically significant' and be backed by the missing test details noted in the major comments.","section":"Section 4.3"},{"comment":"In the ESTP example, the phrase 'you are an known for' appears to be a typo; should read 'you are known for'.","section":"Appendix C.2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong practical premise and the code release is commendable, but the verification circularity and unvalidated judge are serious. The authors should be asked to either provide human evaluation for the narrative tasks or temper the 'excel' language, to fix the 16Personalities verification, and to report proper statistical details. The generalization claim should be demoted to a conceptual contribution unless experiments are added. These are addressable within a revision, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate extension of persona prompting, not a breakthrough, and the core behavioral-divergence finding is probably real, but the two load-bearing claims—that personality primes persist and that Feeling agents 'excel' at narrative—are each undercut by fixable but real evaluation problems.\n\nWhat's new: existing work shapes personas or measures LLM personality; this applies MBTI priming to task effectiveness in narrative and game-theoretic settings, and compares multi-agent protocols with and without self-reflection. That comparison is a genuinely useful contribution, and releasing code helps. The game metrics (defection, strategy switch, honesty) are objective and show consistent E/I and T/F effects across several games; those results are the paper's strongest evidence and I think they hold up.\n\nThe soft spots are the ones you'd expect. First, trait verification via 16Personalities is circular as executed: Appendix C.1 gives the model four few-shot exemplars aligned with each target axis, so the test is effectively teaching the answer. The authors acknowledge the S/N axis is weak, but the circularity applies to all axes. Second, the narrative 'excel' claim depends entirely on an LLM-as-a-judge scoring emotionally chargedness, happy endings, etc. No human validation, no inter-annotator agreement, no judge model reported. The stress-test note is right that these attributes are surface stylistic features the judge can infer from word counts and sentiment, and the generation template explicitly asks the agent to state its personality type, so the judge may see conditioning labels. That means the T/F gap in narrative scores could reflect prompt-consistent style, not reader-valued quality. The paper needs human ratings or at least a masked judge with calibration. Third, Section 4 explicitly says it omits data not yielding insights. That's cherry-picking by their own admission; it weakens confidence in the effect size.\n\nMinor points: the 'generalization' to Big Five/HEXACO is just a vector-space reformulation; they test only MBTI. And the 'effectiveness' framing oversells: Thinking types defecting more isn't obviously better, just different. The paper's own caveats on MBTI validity are honest, which I credit.\n\nBottom line: the empirical mapping of MBTI dimensions to game behavior is credible and worth building on; the narrative-effectiveness claim is not yet supported. This deserves a serious referee—major revision, with human narrative evaluation and a non-circular verification prompt. I'd bring it to a reading group as a case study in evaluation design, but I wouldn't cite it yet.","headline":"A plausible prompt-engineering study whose narrative-effectiveness claim rests on an unvalidated LLM judge and whose trait persistence check is circular; the game-theoretic results are more credible.","tokens_in":41580,"tokens_out":3131,"would_cite":false,"duration_ms":27931,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Prompt-based MBTI priming steers LLM agents along affective and cognitive axes, and self-reflection before communication improves cooperation and reasoning quality.","keywords":["MBTI","personality priming","large language models","prompt engineering","affective-cognitive axes","multi-agent communication","LLM-as-a-judge","game theory"],"falsifier":"Have independent human raters score the 100 generated stories for believability and emotional tone without knowing which personality prompt produced them, and check whether Feeling-primed stories still outscore Thinking-primed and unprimed ones; if the gap vanishes, the affective advantage is an artifact of the judging model.","tokens_in":40689,"feed_emoji":"🎭","tokens_out":8921,"duration_ms":81939,"temperature":0.7,"pith_summary":"The paper sets out to show that a language model's behavior can be steered along two psychological axes—affect and cognition—simply by priming it with a personality profile in the prompt, with no fine-tuning. It introduces MBTI-in-Thoughts, which assigns each agent one of the 16 Myers–Briggs types, verifies the assignment with the 16Personalities test, and then measures behavior on narrative writing and strategic games. The reported effects are consistent and interpretable: Feeling types produce more emotionally expressive stories, Thinking types defect more often and switch strategies less, and Introverted or Judging types are more honest in communication. A separate result shows that letting agents reflect privately before exchanging messages improves cooperation and reasoning in multi-agent tasks. If these findings hold, personality prompts become a cheap, reversible dial for matching agent behavior to task demands.","feed_headline":"MBTI prompts alone shift LLM behavior along emotion and logic axes","feed_subtitle":"Feeling-primed agents write more emotional stories; thinking-primed agents defect more; reflection boosts cooperation.","key_machinery":"The load-bearing object is the personality prompt: a role-setting context that tells the agent it has a given MBTI type, either naming the type explicitly or describing its traits without the label. The paper couples this with the official 16Personalities questionnaire as an external verification instrument, treating each MBTI dichotomy as a continuous [0,1] pair rather than a binary switch. For multi-agent settings, the framework defines three communication protocols: independent majority voting, interactive blackboard dialogue, and interactive dialogue preceded by a private self-reflection scratchpad. The protocols are what let the paper isolate the effect of personality priming on group r","core_discovery":"The central claim is that a psychological profile injected through prompt text is enough to give an LLM agent a stable, task-relevant behavioral orientation. Across 16 MBTI profiles the paper shows reproducible separability on three of the four MBTI axes when agents take the official 16Personalities test, with Sensing/Intuition weaker. In narrative generation, Feeling-primed agents score higher on emotionally charged, personal, optimistic attributes than Thinking-primed agents or unprimed controls. In repeated Prisoner's Dilemma and related games, Thinking-primed agents defect in roughly 90% of rounds versus about 50% for Feeling-primed agents, and Thinking types switch strategies half as of","pith_inferences":["I would expect the Sensing/Intuition axis to become more visible in tasks with explicit information-gathering choices, such as deciding whether to seek concrete evidence versus explore abstract hypotheses, rather than in single-turn text scoring.","The reported honesty rates are about matching an announced action; varying the payoff for lying in the same games would reveal whether introverted/judging agents are truly more honest or simply more instruction-following.","Because priming lives entirely in the prompt, the same agent could switch personality profiles between rounds or tasks, giving a cheap route to adaptive multi-agent behavior without retraining."],"forward_implications":["MBTI type becomes a zero-retraining design knob: choose a Feeling profile for emotional-support or narrative applications and a Thinking profile for competitive, payoff-maximizing ones.","Private self-reflection before agent communication can be adopted as a protocol default, since it improves cooperation and reasoning quality while preserving epistemic independence.","Introverted and Judging profiles can be selected when truthful, trust-preserving communication is required, such as negotiation or safety-critical coordination.","The same prompt infrastructure can expose Big Five, HEXACO, Enneagram, or DISC dimensions, so users are not locked into MBTI."],"supporting_citations":[{"why":"Defines the 16 MBTI types and four dichotomous dimensions that the priming prompts are built on.","marker":"Myers and Briggs 1944"},{"why":"Source text the authors summarize to construct general MBTI-oriented priming contexts in one prompt variant.","marker":"Myers and Myers 1980"},{"why":"Supplies the 100 WritingPrompts story prompts and human reference stories used in the affective evaluation.","marker":"Fan, Lewis, and Dauphin 2018"},{"why":"Provides the PersonaLLM evaluation metrics that score generated stories on believability, emotional tone, readability, and related attributes.","marker":"Jiang et al. 2024"},{"why":"Supplies the benchmark tasks used to compare communication protocols and measure self-reflection effects.","marker":"Srivastava et al. 2023"},{"why":"Defines the Big Five/OCEAN model that the framework claims to generalize to.","marker":"Barrick and Mount 1991"},{"why":"Defines the HEXACO model used to show the priming approach extends beyond MBTI.","marker":"Lee and Ashton 2004"}],"fun_headline_variants":["MBTI prompt primes shift LLM behavior without fine-tuning","Feeling-primed LLMs excel at emotional stories; thinking-primed defect more","Self-reflection before LLM interaction improves cooperation and reasoning","Psychological profiles via prompts give LLM agents stable behavioral traits","Personality-conditioned prompts steer LLM agents across cognition and affect"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The central results depend on the LLM-as-judge scores for narrative attributes and the game-play metrics (defection, switching, honesty) being valid measures of what they claim to measure, rather than reflections of the prompt's surface style.","fun_headline_variants_meta":{"raw":{"variants":["MBTI prompt primes shift LLM behavior without fine-tuning","Feeling-primed LLMs excel at emotional stories; thinking-primed defect more","Self-reflection before LLM interaction improves cooperation and reasoning","Psychological profiles via prompts give LLM agents stable behavioral traits","Personality-conditioned prompts steer LLM agents across cognition and affect"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1168,"prompt_tokens":701,"completion_tokens":467,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":445,"completion_tokens_details":{"reasoning_tokens":380}},"tokens_in":445,"tokens_out":467,"duration_ms":4663,"temperature":1.0,"reasoning_tokens":380,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:12:51.710469+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have independent human raters score the 100 generated stories for believability and emotional tone without knowing which personality prompt produced them, and check whether Feeling-primed stories still outscore Thinking-primed and unprimed ones; if the gap vanishes, the affective advantage is an artifact of the judging model.","supporting_citations":[{"cited_title":"Myers, I","cited_arxiv_id":null,"evidence_quote":"Source text the authors summarize to construct general MBTI-oriented priming contexts in one prompt variant."}],"review_version":1}