Pith. sign in

REVIEW 4 major objections 6 minor 18 references

Assessing Social Alignment: Do Personality-Prompted Large Language Models Behave Like Humans?

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Personality-prompted LLMs do not behave like humans with the same personality in social decision-making tasks.

desk verdict Cross-model negative result on personality prompting is worth taking seriously, but the human-benchmark grounding is thinner than the headline claims. read the letter →

arxiv 2412.16772 v3 pith:PJEKI6VC submitted 2024-12-21 cs.CY cs.AIcs.LG

Ivan Zakazov , Mikolaj Boronski , Lorenzo Drudi , Robert West This is my paper · ORCID
classification cs.CYcs.AIcs.LG
keywords personalitypromptinglargelanguagemodelsUltimatumGameMilgramexperimentBigFivetraitssocialalignmentLLM-as-a-judgebehavioralbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Personality prompting is the cheapest and most common way to give a chatbot a character, yet this paper asks whether a prompted trait actually changes what the model does when a social decision must be made. It tests models from four vendors in two classic behavioral experiments, the Ultimatum Game and the Milgram obedience paradigm, with Big Five traits prompted at nine intensity levels. The answer to both research questions is no: in two of four trait–task combinations behavior moved in the opposite direction from the human trend, and in no case did stronger trait prompts produce a monotonic increase in the corresponding behavior. The authors conclude that a model can pass a personality questionnaire while failing to act in a human-aligned way in a consequential social interaction, so personality benchmarks should evaluate behavior in realistic tasks rather than self-report style answers.

What carries the argument

The machinery is a pair of standardized social-interaction testbeds borrowed from behavioral economics and social psychology, re-run as iterative prompt loops. In the Ultimatum Game, a responder prompted with a personality description sees offers from $0 to $10 and must accept or reject; acceptance is recorded over 50 runs per offer and trait level. In the Milgram paradigm, a prompted teacher is progressively ordered to increase shock voltage while an LLM-as-a-judge classifies whether the teacher stopped, hesitated, or obeyed. The quantitative core is a linear regression of acceptance on one-hot trait levels plus normalized offer, $y(trait, o) = \sum_{i=1}^{9} \Theta_i x_i + \Theta_o o + c$, whose $\Theta_i$ coefficients encode whether trait intensity moves behavior and whether that movement is monotonic. Personality induction uses a nine-point adjective-plus-qualifier prompting scale that previous work showed to shift questionnaire scores, and the authors verify that prompted traits register on the IPIP questionnaire in a supplementary check.

What would settle it

A replication in which at least one LLM shows acceptance rates for unfair Ultimatum offers increasing monotonically with prompted Openness from 1 to 9, or a Milgram run in which the high-Agreeableness condition reaches a significantly higher mean final shock level than the low-Agreeableness condition under this paper's prompting scheme, would contradict the reported opposite-direction findings.

Watch

Extended reading notes

Core claim

The paper's central claim is that prompt-based personality induction does not reliably transfer to social decision behavior. In the Ultimatum Game, where human studies show Openness and Agreeableness positively correlated with accepting unfair offers, all seven tested models showed increasing Agreeableness increasing acceptance but increasing Openness decreasing acceptance, reversing the human correlation. In the Milgram setup, where human data show Agreeableness and Conscientiousness positively correlated with administering higher shocks, high-Agreeableness models withdrew earlier than low-Agreeableness models in every model that could complete the task, while Conscientiousness produced no significant difference from baseline. Across the four case studies, two results went against the human trend, one was null, and the one aligned trend was not monotonic across the nine trait levels. The paper presents this as evidence that personality-prompted LLMs cannot be expected to exhibit human-aligned behavior by default, even when the model correctly reports the prompted trait on psychological questionnaires.

Load-bearing premise

The conclusions treat the two human-study correlations used as ground truth, Openness and Agreeableness predicting acceptance of unfair offers, and Agreeableness and Conscientiousness predicting higher shocks, as correct and transferable to a text-based LLM testbed; if either correlation is weak, non-replicable, or does not transfer, the observed reversals can no longer be read as a failure of personality prompting.

Editorial extensions

If this is right

  • If a model answers a personality questionnaire in line with its prompt, that tells us little about whether it will act accordingly when bargaining, obeying orders, or otherwise making a social decision.
  • Fine control of personality by prompt intensity is not achievable with current adjective-based prompting: levels 1 through 9 do not produce a monotonic behavioral gradient.
  • The negative results hold across open- and closed-source models from four vendors and persist under several prompt phrasings, so the failure is not isolated to one model or one wording.
  • Behavioral benchmarks grounded in classic experiments, rather than self-report or text-style evaluation, are needed to validate personality-prompted agents before they are deployed as conversational companions or advisors.
  • A model that behaves in a human-like way in the unprompted baseline can become less human-like when a personality is injected, so prompting can actively degrade social alignment rather than only fail to improve it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, a plausible mechanism is that adjective prompts evoke text about a trait rather than a stable decision policy; comparing adjective prompts with prompts containing concrete trait-consistent examples would test this.
  • The null Conscientiousness result hints that safety and refusal behavior can override the trait signal; one could test whether the opposite-direction effects shrink when the experimental frame is explicitly separated from real harm.
  • The paper's limitation about brief, consequence-free agents points to a testable extension: run the same protocols with memory and long-term stakes to see whether persistence restores human-like trait effects.
  • If the Openness reversal is robust, it implies openness-prompted agents may react more strongly to perceived unfairness in other allocation tasks, a prediction that can be checked against existing negotiation benchmarks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper evaluates whether personality-prompted large language models behave consistently with the personality they are prompted to embody, using two classic social decision-making settings: the Ultimatum Game (UG) and the Milgram Experiment (ME). The authors prompt seven models from four vendors with nine levels of trait intensity for Agreeableness, Openness, and Conscientiousness, then measure acceptance rates in UG and withdrawal levels and disobediences in ME. They report two main findings: (i) in two of four case studies (Openness in UG, Agreeableness in ME) the direction of behavior change is opposite to the direction reported in human studies, and in a third case (Conscientiousness in ME) no significant change is found; (ii) the effect of trait intensity on behavior is generally non-monotonic, so behavior cannot be finely steered by prompt intensity. The authors conclude that personality-prompted LLMs should not be assumed to exhibit human-aligned behavior even when questionnaire-based assessments suggest the personality has been induced.

Significance. If the results hold, the paper makes a useful contribution by moving personality evaluation away from self-report questionnaires and toward behavioral benchmarks, and by providing cross-vendor evidence of shared failure modes. The robustness checks (prompt perturbations, model-update comparison, filtering of invalid responses) are valuable, and the negative answer to RQ2 regarding monotonic steering is plausible and interesting. However, the central negative answers to RQ1 rest on the direction of two specific human correlations, neither of which is established with effect sizes or replication in the manuscript, and the main regression trends are reported without uncertainty quantification. These are load-bearing gaps that currently make the headline claim stronger than the evidence supports.

major comments (4)
  1. [§Introduction; §Results and Discussion (Ultimatum Game)] The claim that the Openness–UG trend is 'opposite to human data' rests entirely on Mehta (2007), a single doctoral dissertation, and the paper reports no effect size, confidence interval, or independent replication for the Openness–acceptance correlation. The secondary citation Zhao and Smillie (2015) supports a general link between prosocial traits and bargaining behavior, not specifically the Openness–unfair-offer correlation that the paper needs. As written, the 'opposite direction' finding for Openness is uninterpretable if the human benchmark does not replicate. Please add meta-analytic or multiple-study evidence for the sign of the human correlation, or alternatively reframe the conclusion as 'opposite to the direction reported in Mehta (2007)' and weaken RQ1 accordingly.
  2. [§Results and Discussion, Eq. (1) and Fig. 6] The regression coefficients Θi in Eq. (1) are the quantitative basis for both RQ1 and RQ2, but Fig. 6 plots point estimates with no confidence intervals, standard errors, or significance tests. Since each Θi is estimated from many trials (e.g., 50 runs per offer level), standard errors are readily available. Without them, the paper cannot support claims that the Openness trend is significantly downward, that the Agreeableness trend is significantly upward, or that non-monotonicity is a robust phenomenon rather than sampling noise. Please report uncertainty for all Θi and provide inferential tests for trend direction and monotonicity.
  3. [§Methodology (Milgram Experiment); Fig. 7, Fig. 8, Table 3] The ME measurement of withdrawal levels and disobediences depends on LLM-as-a-judge classifications in the Stop? and Obey? steps, which the paper itself acknowledges as imperfect (citing Zheng et al. 2023). Because the Agreeableness-opposite result is one of the two headline 'opposite direction' findings, a judge that misclassifies hesitation or stopping in a personality-dependent way could produce the observed pattern without reflecting genuine behavioral differences. Please validate the judge against a labeled sample of completions or against the log-probability-based method of Aher et al. (2023), and report agreement rates broken down by personality condition.
  4. [§Results and Discussion (Milgram Experiment); Table 3] The paper reports that low- and high-Conscientiousness withdrawal levels are 'not significantly different' from baseline based on Welch's t-test, but no test statistics, p-values, effect sizes, or multiple-comparison corrections are provided. Given that the null Conscientiousness result is one of the four case-study outcomes used to answer RQ1, the absence of these statistics makes the null claim unverifiable. Please report the full test results or replace the claim with descriptive evidence.
minor comments (6)
  1. [Conclusion] The Conclusion refers to varying 'Agreeableness and Consciousness'; this should be 'Conscientiousness'.
  2. [Fig. 7 caption] The caption contains a duplicated word: 'results results'.
  3. [Fig. 8 caption] The caption contains a typo: 'Least Agreeablee' should be 'Least Agreeable'.
  4. [Table 3 caption] The caption contains a typo: 'firs-order response' should be 'first-order response'.
  5. [§Results and Discussion (Ultimatum Game)] The non-monotonicity claim is currently supported by visual inspection of Fig. 6; a formal test, such as comparing adjacent trait levels or testing a quadratic term, would make the RQ2 conclusion more rigorous.
  6. [References] The citation to Mehta (2007) gives a page number for the UG correlation, but the paper does not state whether the reported correlation is partial, zero-order, or controlled for other Big Five traits; please clarify the statistical basis for the human benchmark.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's behavioral comparisons are anchored in external human studies and independent manipulation checks, not in quantities derived from its own models or self-citations.

full rationale

The paper's central claims—that personality-prompted LLMs fail to match human behavioral trends (RQ1) and that trait-intensity steering is not monotonic (RQ2)—are derived from direct measurements of LLM behavior in two externally established experimental paradigms (Ultimatum Game and Milgram Experiment). The human reference points are external studies (Mehta 2007; Bègue et al. 2015), not quantities computed from the paper's own fitted parameters. The logistic regression in Eq. (1) is a descriptive summary of the observed acceptance data, not a model whose assumptions encode the conclusion; the Θ_i coefficients are estimated from, not defined by, the outcome they are used to describe. The paper's adoption of the Serapio-García et al. (2023) personality-prompting method is not a self-citation, and the paper independently verifies the manipulation via IPIP questionnaires (Fig. 12), so the behavioral outcome is not assumed by the prompting scheme. The LLM-as-a-judge step in the Milgram adaptation is an acknowledged measurement limitation, but it is not circular: the judge classifications are inputs to the measured withdrawal levels, whereas the human-alignment conclusion is made by comparing those levels against externally reported human correlations. Concerns about the robustness or replicability of the two human benchmark studies, or about LLM-as-a-judge accuracy, are correctness and external-validity risks, not circularity. No load-bearing argument reduces to a self-citation, a fitted parameter renamed as a prediction, or a definitional identity. Therefore the paper's derivation chain is self-contained with respect to circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the human benchmark correlations, the validity of the LLM-as-a-judge in ME, and the regression coefficients used to summarize UG behavior. No new entities are postulated. The regression coefficients are descriptive fits, not physical constants, but the paper's RQ1/RQ2 conclusions are read off their sign and shape.

free parameters (2)
  • Trait intensity coefficients Θi (i=1..9) = estimated per model per trait (Fig. 6)
    The paper's RQ1/RQ2 conclusions are based on the sign and monotonicity of these fitted coefficients in Eq. 1; no confidence intervals are reported.
  • Offer coefficient Θo and bias c = estimated per model (Fig. 6 table)
    Regression controls in Eq. 1; not the object of the claim but part of the same fit.
assumptions (3)
  • domain assumption Mehta (2007) and Bègue et al. (2015) provide correct, generalizable human directional correlations between Big Five traits and UG/ME behavior.
    The 'opposite direction' findings are defined relative to these external benchmarks; if the benchmarks are wrong, the conclusion is unsupported.
  • domain assumption The personality prompting scheme (Serapio-García et al. 2023) induces the target trait in the model's behavior-relevant state, not just its verbal self-report.
    The paper verifies via IPIP questionnaires (Fig. 12), but the central claim is that questionnaire alignment does not guarantee behavioral alignment; this axiom is the premise that the prompt changes something meaningful about the model.
  • domain assumption LLM-as-a-judge classification of Teacher actions (Stop/Obey/Hesitate) in the Milgram setup is accurate enough to measure withdrawal behavior.
    The ME results rely on automated judge decisions at blocks (5) and (6); the paper acknowledges imperfect judge behavior (Zheng et al. 2023) and uses retries to filter failures.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing Social Alignment: Do Personality-Prompted Large Language Models Behave Like Humans?." pith.science (2026). https://pith.science/paper/PJEKI6VC

@misc{pith2026241216772,
  author       = {Pith},
  title        = {Pith review of: Assessing Social Alignment: Do Personality-Prompted Large Language Models Behave Like Humans?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PJEKI6VC}},
  note         = {Machine review of arXiv:2412.16772}
}
read the original abstract

The ongoing revolution in language modeling has led to various novel applications, some of which rely on the emerging social abilities of large language models (LLMs). Already, many turn to the new cyber friends for advice during the pivotal moments of their lives and trust them with the deepest secrets, implying that accurate shaping of the LLM's personality is paramount. To this end, state-of-the-art approaches exploit a vast variety of training data, and prompt the model to adopt a particular personality. We ask (i) if personality-prompted models behave (i.e., make decisions when presented with a social situation) in line with the ascribed personality (ii) if their behavior can be finely controlled. We use classic psychological experiments, the Milgram experiment and the Ultimatum Game, as social interaction testbeds and apply personality prompting to open- and closed-source LLMs from 4 different vendors. Our experiments reveal failure modes of the prompt-based modulation of the models' behavior that are shared across all models tested and persist under prompt perturbations. These findings challenge the optimistic sentiment toward personality prompting generally held in the community.

Figures

Figures reproduced from arXiv: 2412.16772 by the authors.

Figure 2
Figure 2. Milgram Experiment – a flowchart of the setup [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Milgram Experiment – percentage of subjects re [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 3
Figure 3. Ultimatum Game – comparison of acceptance rates [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: Ultimatum Game – heatmap showing the probability of offer acceptance for Openness, Agreeableness, and Baseline [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Ultimatum Game – regression computed as shown in Eq. 1. The general trend in the [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Milgram Experiment – percentage of subjects remaining at each step of the experiment with personality shaped and [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Milgram Experiment – cumulative sum of disobediences per subject for ”Least Agreeablee” and ”Most Agreeable” [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Ultimatum Game – comparison of acceptance rates [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Ultimatum Game – heatmap showing the probability of offer acceptance for Openness, Agreeableness, and Baseline [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Milgram Experiment – example of the prompt [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: Alignment between prompted and measured trait values (Agreeableness, Conscientiousness, Openness) across [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: Milgram Experiment – trajectories and disobediences for different GPT-4o checkpoints. The newer model (2024- [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 13 canonical work pages

  1. [2]

    During each run, we construct a story-like prompt that is being iteratively ex- panded and passed to the model until the experiment is fin- ished

    illustrates the whole experiment, while Fig.11 presents composition of the experiment prompt in greater detail. During each run, we construct a story-like prompt that is being iteratively ex- panded and passed to the model until the experiment is fin- ished. • Intro Text introduces the experiment and sets the scene. It is added only once at the very begin...

    work page 2024

  2. [4]

    Accessed: 2025- 08-01

    https://character.ai/. Accessed: 2025- 08-01. G¨uth, W.; Schmittberger, R.; and Schwarze, B

    work page 2025

  3. [5]

    As shown in the baseline plots (Fig

    – four employing first-person prompts and one utilizing a third-person form. As shown in the baseline plots (Fig. 9), the prompted behavior is highly similar between the runs. Examining the acceptance rate heatmap (Fig. 10), we also observe that the rate patterns are largely consistent across different prompts. Notably, Openness results remain the op- pos...

    work page 2023

  4. [6]

    Accessed: 2025-08-

    https://www.liwc.app/. Accessed: 2025-08-

    work page 2025

  5. [7]

    arXiv:2310.02168

    Editing Personality for Large Language Models. arXiv:2310.02168. Mehta, P

    arXiv

  6. [8]

    Accessed: 2025-08-01

    https://ai.meta.com/ai-studio/. Accessed: 2025-08-01. Milgram, S

    work page 2025

  7. [9]

    LLMs with Personalities in Multi-issue Negotiation Games

    LLMs with Person- alities in Multi-issue Negotiation Games. arXiv preprint arXiv:2405.05248. OpenAI

    work page Pith review arXiv

  8. [10]

    https://openai.com/index/expanding-on-sycophan cy/

    Expanding on what we missed with syco- phancy. https://openai.com/index/expanding-on-sycophan cy/. Accessed: 2025-08-01. Raad, B

    work page 2025

Show all 18 references
  1. [11]

    Accessed: 2025-08-01

    https://replika.com/. Accessed: 2025-08-01. Serapio-Garc´ıa, G.; Safdari, M.; Crepy, C.; Sun, L.; Fitz, S.; Romero, P.; Abdulhai, M.; Faust, A.; and Matari ´c, M

    2025

  2. [12]

    In line with the prior works, we observe each model consis- tently following induced personality. 0 2 4 6 8 10 Offer 0.0 0.2 0.4 0.6 0.8 1.0Acceptance Rate Ultimatum Game: Baseline Acceptance Rate (varying prompts) Houser, McCabe 2014 Krawczyk 2018 GPT-4o-mini-description-1 GP...

    2014

  3. [13]

    Accessed: 2025-08-01

    What is My AI on Snapchat and how do I use it? https://help.snapchat.com/hc/en-us/articles/1326 6788358932-What-is-My-AI-on-Snapchat-and-how-do-I- use-it. Accessed: 2025-08-01. Sorokovikova, A.; Fedorova, N.; Rezagholi, S.; and Yamshchikov, I. P

    2025

  4. [14]

    arXiv:2402.01765

    LLMs Simulate Big Five Person- ality Traits: Further Evidence. arXiv:2402.01765. Zhao, K.; and Smillie, L. D

    arXiv

  5. [15]

    arXiv:2306.05685

    Judg- ing LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685. Measuring personality with questionnaires The personality measured with psychological questionnaires (we use 300-item version from IPIP) is depicted in Fig

    arXiv

  6. [306]

    Bianchi, F.; Chia, P

    Accessed: 2025-08-01. Bianchi, F.; Chia, P. J.; Yuksekgonul, M.; Tagliabue, J.; Ju- rafsky, D.; and Zou, J

    2025

  7. [1982]

    Journal of Economic Behavior & Organization, 3(4): 367–388

    An experimental analysis of ultimatum bargaining. Journal of Economic Behavior & Organization, 3(4): 367–388. Jiang, G.; Xu, M.; Zhu, S.-C.; Han, W.; Zhang, C.; and Zhu, Y . 2023a. Evaluating and Inducing Personality in Pre-trained Language Models. In NeurIPS. Jiang, H.; Zhang...

    arXiv

  8. [2023]

    arXiv:2307.00184

    Personality Traits in Large Language Models. arXiv:2307.00184. Snapchat

    arXiv

  9. [2024]

    In NeurIPS 2024 Workshop on Behavioral Machine Learning

    As- sessing Behavioral Alignment of Personality-Driven Gener- ative Agents in Social Dilemma Games. In NeurIPS 2024 Workshop on Behavioral Machine Learning. Character.ai

    2024

  10. [2025]

    Accessed: 2025-08-01

    https://www.applymagicsauce.co m/. Accessed: 2025-08-01. B`egue, L.; Beauvois, J. L.; Courbet, D.; Oberl´e, D.; Lepage, J. L.; and Duke, A. A

    2025

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.