Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Political Ideology Shifts in Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Larger models display broader, more polarized, and more steerable political ideology under persona prompting.

desk verdict Useful large-scale measurement of persona-driven political shifts; the scale-dependence headline rests on two confounded pairs. read the letter →

arxiv 2508.16013 v1 pith:YYUPBL66 submitted 2025-08-22 cs.CL

Pietro Bernardelle , Stefano Civelli , Leon Fröhling , Riccardo Lunardi , Kevin Roitero , Gianluca Demartini This is my paper
classification cs.CL
keywords largelanguagemodelspoliticalideologysyntheticpersonaspersona-basedpromptingideologicalmalleabilitymodelscaleCompassTestbias
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to prove that the political ideology an LLM expresses is not a fixed property of its weights: when a model is asked to answer as a persona, its political outputs shift, and the shifts are systematic enough to be predicted and steered. Using the 62-statement Political Compass Test as a probe and 200,000 synthetic persona descriptions as conditions, the authors report four scale-linked patterns: larger models spread their outputs across a wider and more polarized ideological range; they respond more strongly to explicit political labels in prompts; right-authoritarian priming moves them several times further than left-libertarian priming; and persona themes such as history, business, and politics pull outputs in stable, stereotyped directions that sharpen as models grow. If these patterns hold, the persona prompt is a political steering wheel: whoever writes the persona—user, developer, or adversary—partly chooses the model's stance, and the steering power grows with model size. The authors care because the same mechanism that lets personas generate diverse annotations or simulate survey respondents is also a channel through which deployed systems could be quietly biased.

What carries the argument

The machinery is persona-conditioned prompting, probed by the Political Compass Test (PCT), a 62-statement questionnaire that scores respondents on an economic left–right axis and a social libertarian–authoritarian axis. Each of 200,000 synthetic persona descriptions is fed to each model with every PCT statement, and the model must answer on a fixed four-point scale, producing a political map of 12.4 million stances per model. Study 2 adds explicit ideological labels to the persona text; Study 3 embeds the persona descriptions, clusters them into 15 themes, and compares each theme's output distribution to the model's overall baseline using bin-wise Z-score deviation maps. The PCT supplies th

What would settle it

Run the same three studies across many sizes of one model family whose checkpoints share training data and alignment (for example 1B, 3B, 8B, 14B, 70B, and 405B siblings). If coverage, explicit-cue shift, and thematic deviation do not increase with parameter count, or if a same-size model with different alignment matches the larger sibling's shifts, the scale claim is an artifact of training choices rather than size. A cheaper check: compare base and instruction-tuned variants of the same 70B model—if instruction tuning alone reproduces the effect, scale is not the driver.

Watch

Extended reading notes

Core claim

The paper's central claim is that ideological malleability in LLMs is real, measurable, and grows with scale, on three separate axes. Implicit malleability: with no political wording added, impersonating different personas already spreads outputs across the political compass, and the larger sibling in each model family covers more territory—in the Llama family coverage rises from 35% to 49% of the compass, in the Qwen family from 14% to 37%—with polarized extremes appearing mainly in the bigger models. Explicit malleability: prepending 'right-authoritarian' or 'left-libertarian' to a persona shifts the average position significantly in the intended quadrant in every model, but asymmetrically

Load-bearing premise

The scale-dependence findings assume the two small/large model pairs (8B vs 70B in one family, 7B vs 72B in the other) differ essentially in parameter count, with no evidence that training data, alignment pipeline, or safety filtering were matched between siblings.

Editorial extensions

If this is right

  • Within the same model family, the larger sibling shows broader ideological coverage and larger shifts under identical prompts, so model scale itself—not just training corpus—is a driver of political malleability.
  • Explicit ideological labels in prompts function as a reliable steering mechanism: all seven models shift significantly toward the labeled quadrant, so persona wording is a practical tool for orienting LLM output, for good or ill.
  • Steering is not symmetric: right-authoritarian priming produces shifts several times larger than left-libertarian priming, so claims of 'neutral' behavior must be stated per prompt direction, not averaged.
  • Persona themes are a covert ideological channel: descriptions mentioning history, business, or politics pull outputs in different stable directions even when no political word appears, so theme alone biases answers.
  • Neutrality evaluations that skip persona conditioning understate risk; a model that looks balanced unprimed can still slant strongly when asked to play a role, so audits should include counterfactual and role-play prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Untested extrapolation: if the scale trend continues past 70B, the largest deployed models are the most persona-steerable, making them both the most useful simulators and the most attractive targets for hidden ideological injection; the paper's largest models stop at 72B.
  • The left-libertarian ceiling explanation implies a flippable asymmetry: a model whose default leans right-authoritarian should show large shifts under left-libertarian priming and small shifts rightward—an experiment the paper does not run but its own logic predicts.
  • Because same-size models from different families differ enormously (one 8B model covers 35% of the compass, another 7B model only 14%), training-environment factors can swamp scale at small sizes; the scale trend should therefore be read as a tendency that other design choices can override.
  • Testable diagnostic: the same three studies repeated in open-ended conversational form, which the paper flags as a limitation of its multiple-choice format, would show whether thematic and scale effects persist, grow, or invert when models write full sentences instead of picking a stance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates whether LLMs' political expression shifts when they adopt synthetic personas. Using 200,000 personas from PersonaHub and the 62-item Political Compass Test, the authors run three studies on seven instruction-tuned models (7B–70B+): Study 1 measures baseline dispersion and coverage of persona-prompted responses; Study 2 prepends explicit "left-libertarian" or "right-authoritarian" descriptors and measures mean shifts, Wilcoxon tests, and Cohen's d; Study 3 embeds and k-means clusters personas into 15 themes and uses bin-wise Z-score deviation maps. The abstract claims four scale-dependent patterns: larger models show broader, more polarized implicit coverage; explicit ideological cues have stronger effects with scale; right-authoritarian priming is more effective than left-libertarian priming; thematic content induces systematic ideological shifts that amplify with size.

Significance. If the scale-dependent claims hold, the paper would provide a scalable, interpretable methodology for auditing ideological malleability and would support an important policy-relevant conclusion: persona-based prompting can steer LLM outputs in predictable ideological directions, and this steerability grows with model scale. Strengths include the unusually large experimental sweep (200,000 personas × 62 statements per condition per model), the use of standardized prompts and structured output decoding, and the public release of data and code on Zenodo/GitHub. These assets make the descriptive measurements reproducible. The main risk is inferential: the central scale attributions rest on only two within-family pairs, and the paper itself acknowledges non-scale confounds for one of those pairs. The thematic and asymmetry findings are also subject to specific methodological concerns.

major comments (3)
  1. [§3.1.2, §3.2.1, §3.3, Table 1, Table 2] The repeated claim that "larger models" show broader coverage, stronger explicit shifts, and amplified thematic deviations rests on exactly two within-family comparisons: Llama-3.1-8B vs 70B and Qwen2.5-7B vs 72B. Section 5.2 says same-family models were chosen to "minimize confounding variables," but larger checkpoints are separately trained systems with different pretraining data, alignment pipelines, safety filtering, and release-time choices. The paper itself makes this point in §3.1.1, where Qwen2.5-7B's narrow distribution is attributed to "architectural or training-related design choices" and a "distinct sociopolitical and regulatory environment." If that explanation is accepted for the small model, the same unmeasured variation can explain the 7B→72B differences. With two binary comparisons there is no evidence for monotonicity or for a class-level "grows with scale" conclusion.
  2. [§3.2.2 and Table 2] The right-vs-left asymmetry is confounded with a floor/ceiling effect. All models have left-libertarian baseline positions, so a left-libertarian injection can only reinforce the existing tendency, while a right-authoritarian injection moves responses across a larger portion of the compass. The paper acknowledges this "representational ceiling" in §3.2.2 but still reports the asymmetry as a substantive finding. To support "models respond more strongly to right-authoritarian than to left-libertarian priming," the analysis should account for the different initial distances to the two target quadrants, for example by normalizing shifts by the distance available, or by comparing injections placed symmetrically around each model's baseline. Without such an adjustment, the result may simply reflect available response room rather than asymmetric susceptibility.
  3. [§5.4.5, Eq. (Z-score)] The background distribution used to compute expected counts for each thematic cluster is the full set of 200,000 personas, which includes the foreground cluster itself. For a cluster with N_F personas, the bin-wise expectation E_i = N_F p_i is pulled toward the foreground's own counts, attenuating all Z-scores and, more importantly, biasing comparisons across clusters of different sizes. The thematic deviation maps in §3.3 and Figure 4 are the primary evidence for Study 3's claims, so this is not a cosmetic issue. The authors should use a leave-one-cluster-out background, or a held-out set of personas that excludes each foreground cluster, and re-run the deviation analysis.
minor comments (5)
  1. [§5.5] The computational-resources section states "Across the eight models and three experimental configurations," but only seven models are evaluated. Please correct this inconsistency.
  2. [§5.4.1 / §5.4.5] The scoring procedure that converts 62 four-point PCT responses into x/y coordinates is referenced but not specified. The paper says responses are aggregated using a "weighted scoring system" but does not give the weights or the mapping from stances to numeric values. The released code may resolve this, but the Methods section should be self-contained to the extent possible.
  3. [Figure 5 caption] The cluster label “Envionment” is a typo for “Environment.” Please fix.
  4. [§3.3 / Appendix D] The number of thematic clusters (k=15) is selected "heuristically by inspecting the top keywords." No stability analysis or alternative k values are reported; a brief sensitivity check would strengthen the claim that the 15-cluster solution is not an artifact of the k-means initialization.
  5. [§4 Discussion] The sentence "the authors expect the directional patterns observed to hold in other settings" uses third-person phrasing inconsistent with the rest of the paper; consider rewriting in first person or passive voice.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the central claims are direct measurements; there are only minor self-citations and a conservative self-referential baseline in Study 3.

full rationale

The paper is an observational measurement study, not a derivation with fitted parameters. The central quantities—dispersion, coverage, explicit-shift magnitude, and thematic Z-scores—are all computed directly from elicited Political Compass Test responses; no parameter is fitted to a subset and then re-predicted on a closely related quantity, and no uniqueness theorem or ansatz is imported from prior work. The scale comparisons are direct between-model measurements, so the claims about larger models are empirical comparisons rather than consequences of the definitions. The one self-referential aspect is in Section 5.4.5: the background distribution used for the Z-score test is built from all 200,000 personas, which includes each foreground thematic cluster. This makes the expected count E_i partially dependent on the foreground count F_i and therefore biases Z-scores conservatively toward zero; it cannot manufacture the reported thematic deviations, so it is not a circular derivation of the paper's conclusions. There are several self-citations ([6], [12], [20], [32]), but they are peripheral—e.g., supporting the left-libertarian baseline, PersonaHub adoption, and PCT wording—and the main analysis relies on external standardized instruments (PCT, PersonaHub) and external references. No load-bearing circularity is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the PCT coordinate mapping, the within-family scale comparison premise, and PersonaHub as a neutral persona source. No new theoretical entities are introduced; the paper is an empirical measurement study. Three hand-chosen analysis settings (k=15, 35x35 grid, Z>2) affect the reported thematic and coverage results.

free parameters (3)
  • number of thematic clusters k = 15
    Chosen heuristically by inspecting top keywords for candidate clusterings (Section 5.3.3); the number of themes and their boundaries determine the deviation maps in Study 3.
  • grid resolution (bins per axis) = 35x35
    Selected as a balance between resolution and clarity (Section 5.4.1); coverage percentages and Z-score maps depend directly on this binning.
  • Z-score significance threshold = |Z| > 2
    Used for red/blue over-/under-representation in deviation maps (Section 5.4.5); with 200,000 samples this threshold marks trivial deviations as significant.
assumptions (4)
  • domain assumption The Political Compass Test's weighted scoring maps four categorical stances to a two-dimensional plane in which Euclidean distance is a meaningful measure of ideological difference.
    Dispersion, coverage, shift magnitudes, and deviation maps are all computed using the PCT x/y coordinates (Section 5.4). The specific weighted scoring function is not disclosed, only referenced to [32].
  • domain assumption Within-family model pairs (Llama-3.1-8B/70B, Qwen2.5-7B/72B) differ primarily in parameter count rather than in training data or alignment choices.
    The paper's scale claims rely on comparing these pairs and states in Section 5.2 that same-family models were chosen 'to minimize confounding variables from differing architectures or training data', but provides no matching evidence.
  • domain assumption PersonaHub's 200,000 persona descriptions are a sufficiently diverse and unbiased set of identity descriptions to serve as a neutral baseline for 'implicit' ideological variation.
    Study 1 maps the distribution of persona-conditioned responses over this set and uses it as the baseline for Studies 2 and 3 (Section 5.3.1). If PersonaHub is skewed by its synthetic generation process, the 'baseline' distribution inherits that skew.
  • domain assumption The fixed four-way forced-choice prompt template and structured decoding do not materially distort the models' ideological expression relative to other prompt wordings.
    All measurements use a single template from prior work [41] (Appendix B) and constrain outputs to one of four stances via vLLM structured decoding. Prior work cited in the paper (e.g., [41]) shows political positions can be sensitive to prompt format.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Political Ideology Shifts in Large Language Models." pith.science (2026). https://pith.science/paper/YYUPBL66

@misc{pith2026250816013,
  author       = {Pith},
  title        = {Pith review of: Political Ideology Shifts in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YYUPBL66}},
  note         = {Machine review of arXiv:2508.16013}
}
read the original abstract

Large language models (LLMs) are increasingly deployed in politically sensitive settings, raising concerns about their potential to encode, amplify, or be steered toward specific ideologies. We investigate how adopting synthetic personas influences ideological expression in LLMs across seven models (7B-70B+ parameters) from multiple families, using the Political Compass Test as a standardized probe. Our analysis reveals four consistent patterns: (i) larger models display broader and more polarized implicit ideological coverage; (ii) susceptibility to explicit ideological cues grows with scale; (iii) models respond more strongly to right-authoritarian than to left-libertarian priming; and (iv) thematic content in persona descriptions induces systematic and predictable ideological shifts, which amplify with size. These findings indicate that both scale and persona content shape LLM political behavior. As such systems enter decision-making, educational, and policy contexts, their latent ideological malleability demands attention to safeguard fairness, transparency, and safety.

Figures

Figures reproduced from arXiv: 2508.16013 by the authors.

Figure 1
Figure 1. Experimental framework for probing ideological malleability in large language models. We evaluate how seven LLMs respond to political statements when adopting synthetic personas across three studies. In study 1, 200,000 personas are prompted to answer 62 Political Compass Test (PCT) statements, establishing a baseline map of ideological variation (implicit malleability). In study 2, we follow the same procedure as i… view at source ↗
Figure 2
Figure 2. Political compass distribution of 200,000 personas when impersonated by different 7-8B parameters LLMs under different conditions. Darker regions indicate higher density of personas on a logarithmic scale. The bar charts show the marginal distributions along each axis. White dots represents the leaning of the original LLM (without any form of persona prompting). White triangles (left and right columns) shows the ave… view at source ↗
Figure 3
Figure 3. Political compass distribution of 200,000 personas when impersonated by different 70B+ parameters LLMs under different conditions. Darker regions indicate higher density of personas on a logarithmic scale. The bar charts show the marginal distributions along each axis. White dots represents the leaning of the original LLM (without any form of persona prompting). White triangles (left and right columns) shows the ave… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Thematic deviations in ideological output distributions across Llama-3.1 and Qwen2.5 models. Statistical deviation maps show how personas associated with three thematic clusters—History, Political, and Business—shift the political orientation of model outputs relative …
Figure 5
Figure 5. Figure 5: Thematic deviations in ideological output distributions across Llama-3.1 and Qwen2.5 models. Statistical deviation maps show how personas associated with twelve of the fifteen thematic clusters shift the political orientation of model outputs relative to each model’s b…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Auditing Alignment Controllability in LLMs via Political Axes

    cs.CY 2026-07 conditional novelty 6.0 of 10

    On a 63,700-response Political Compass stress test of seven frontier LLMs, system-prompt framing dominates model identity, and steerability needs dispersion, symmetry, saturation, and refusal-floor metrics.

Reference graph

Works this paper leans on

52 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Argyle, Ethan C

    Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023. Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis 31, 3 (Feb. 2023), 337–351

    work page 2023

  2. [2]

    Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics . PMLR, 4447–4455

    work page 2024

  3. [3]

    Leif Azzopardi and Yashar Moshfeghi. 2024. PRISM: a methodology for auditing biases in large language models. arXiv preprint arXiv:2410.18906 (2024)

    arXiv 2024

  4. [4]

    Yejin Bang, Delong Chen, Nayeon Lee, and Pascale Fung. 2024. Measuring Political Bias in Large Language Models: What Is Said and How It Is Said. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 11142–11159

    work page 2024

  5. [5]

    Pietro Bernardelle and Gianluca Demartini. 2024. Optimizing LLMs with direct preferences: A data efficiency perspective. In Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region . 236–240

    work page 2024

  6. [6]

    Pietro Bernardelle, Leon Fröhling, Stefano Civelli, Riccardo Lunardi, Kevin Roitero, and Gianluca Demartini. 2025. Mapping and influencing the political ideology of large language models using synthetic personas. In Companion Proceedings of the ACM on Web Conference 2025 . 864–867

    work page 2025

  7. [7]

    Alexander Bick, Adam Blandin, and David J Deming. 2024. The rapid adoption of generative AI . Technical Report. National Bureau of Economic Research

    work page 2024

  8. [8]

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

    arXiv 2021

Show all 52 references
  1. [9]

    Kai Chen, Zihao He, Jun Yan, Taiwei Shi, and Kristina Lerman. 2024. How Susceptible are Large Language Models to Ideological Manipulation?. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . 17140–17161

    2024

  2. [10]

    Sahil Chinoy and Martin Koenen. 2024. Political Sorting in the US Labor Market: Evidence and Explanations. Unpub- lished Manuscript 6 (2024)

    2024

  3. [11]

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems 30 (2017)

    2017

  4. [12]

    Stefano Civelli, Pietro Bernardelle, and Gianluca Demartini. 2025. The Impact of Persona-based Political Perspectives on Hateful Content Detection. In Companion Proceedings of the ACM on Web Conference 2025 . 1963–1968

    2025

  5. [13]

    Andrew M Dai and Quoc V Le. 2015. Semi-supervised sequence learning. Advances in neural information processing systems 28 (2015)

    2015

  6. [14]

    Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity in chatgpt: Analyzing persona-assigned language models. In Findings of the Association for Computational Linguistics: EMNLP 2023. 1236–1270

    2023

  7. [15]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  8. [16]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models.arXiv e-prints (2024), arXiv–2407

    2024

  9. [17]

    Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models, arXiv. arXiv prepri...

    2023 arXiv

  10. [18]

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306 (2024)

    2024 arXiv

  11. [19]

    Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models. In Proceedings of the 61st Annual Meeting of the Association for Computation...

    2023

  12. [20]

    Leon Fröhling, Gianluca Demartini, and Dennis Assenmacher. 2024. Personas with Attitudes: Controlling LLMs for Diverse Data Annotation. arXiv preprint arXiv:2410.11745 (2024)

    2024 arXiv

  13. [21]

    Iason Gabriel, Geoff Keeling, Arianna Manzini, and James Evans. 2025. We need a new ethics for a world of AI agents. Nature 644, 8075 (2025), 38–40

    2025

  14. [22]

    Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. 2024. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint arXiv:2406.20094 (2024). , Vol. 1, No. 1, Article . Publication date: September 2025. 18 Bernardelle et al

    2024 arXiv

  15. [23]

    Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. 2023. The political ideology of conversational AI: Converging evidence on ChatGPT’s pro-environmental, left-libertarian orientation. arXiv preprint arXiv:2301.01768 (2023)

    2023 arXiv

  16. [24]

    Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch Critch, Jerry Li Li, Dawn Song, and Jacob Steinhardt

  17. [25]

    Tiancheng Hu, Yara Kyrychenko, Steve Rathje, Nigel Collier, Sander van der Linden, and Jon Roozenbeek. 2025. Generative language models exhibit social identity biases. Nature Computational Science 5, 1 (2025), 65–75

    2025

  18. [26]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  19. [27]

    Hang Jiang, Doug Beeferman, Brandon Roy, and Deb Roy. 2022. CommunityLM: Probing Partisan Worldviews from Language Models. In Proceedings of the 29th International Conference on Computational Linguistics . 6818–6826

    2022

  20. [28]

    Weixin Liang, Yaohui Zhang, Mihai Codreanu, Jiayu Wang, Hancheng Cao, and James Zou. 2025. The widespread adoption of large language model-assisted writing across society. arXiv preprint arXiv:2502.09747 (2025)

    2025 arXiv

  21. [29]

    Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, and Xuan-Jing Huang. 2024. Aligning Large Language Models with Human Preferences through Representation Engineering. In Proceedings of the 62nd Annual Meeting...

    2024

  22. [30]

    Turning right

    Yifei Liu, Yuang Panwang, and Chao Gu. 2025. “Turning right”? An experimental study on the political value shift in large language models. Humanities and Social Sciences Communications 12, 1 (2025), 1–10

    2025

  23. [31]

    Charles G Lord, Lee Ross, and Mark R Lepper. 1979. Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of personality and social psychology 37, 11 (1979), 2098

    1979

  24. [32]

    Riccardo Lunardi, David La Barbera, and Kevin Roitero. 2024. The Elusiveness of Detecting Political Bias in Language Models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (CIKM ’24). Association for Computing Machinery, New Yor...

    2024

  25. [33]

    Marilù Miotto, Nicola Rossberg, and Bennett Kleinberg. 2022. Who is GPT-3? An exploration of personality, values and demographics. In Proceedings of the Fifth Workshop on Natural Language Processing and Computational Social Science (NLP+ CSS). 218–227

    2022

  26. [34]

    Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. 2024. More human than human: measuring ChatGPT political bias. Public Choice 198, 1 (2024), 3–23

    2024

  27. [35]

    Raymond S Nickerson. 1998. Confirmation bias: A ubiquitous phenomenon in many guises.Review of general psychology 2, 2 (1998), 175–220

    1998

  28. [36]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35...

    2022

  29. [37]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  30. [38]

    Alec Radford. 2018. Improving language understanding with unsupervised learning. OpenAI Res (2018)

    2018

  31. [39]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems 36 (2023), 53728–53741

    2023

  32. [40]

    Martina Raue and Sabine G Scholl. 2018. The use of heuristics in decision making under risk and uncertainty. In Psychological perspectives on risk and risk analysis: Theory, models, and applications . Springer, 153–179

    2018

  33. [41]

    Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy

  34. [42]

    David Rozado. 2024. The political preferences of LLMs. PloS one 19, 7 (2024), e0306621

    2024

  35. [43]

    Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect?. In International Conference on Machine Learning . PMLR, 29971–30004

    2023

  36. [44]

    Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems 36 (2023), 51778–51809. , Vol. 1, No. 1, Article . Publication date: September 2025. Political Ideology Shifts in Large...

    2023

  37. [45]

    Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models. Nature 623, 7987 (2023), 493–498

    2023

  38. [46]

    Herbert Alexander Simon. 1957. Models of man: social and rational; mathematical essays on rational human behavior in society setting. Wiley

    1957

  39. [47]

    Lewis Tunstall, Edward Emanuel Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro Von Werra, Clémentine Fourrier, Nathan Habib, et al . [n. d.]. Zephyr: Direct Distillation of LM Alignment. In First Conference on Language Modeling

  40. [48]

    Amos Tversky and Daniel Kahneman. 1974. Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty. science 185, 4157 (1974), 1124–1131

    1974

  41. [49]

    Gerard Yeo, Fiona Tan An Ting, Kokil Jaidka, Shaz Furniturewala, Wu Fanyou, Weijie Xu, Vinija Jain, Aman Chadha, Yang Liu, and See Kiong Ng. 2025. PHAnToM: Persona-Based Prompting Has an Effect on Theory-of-Mind Reasoning in Large Language Models. In Proceedings of the Interna...

    2025

  42. [50]

    strongly agree

    Di Zhou and Yinxian Zhang. 2024. Political biases and inconsistencies in bilingual GPT models—the cases of the US and China. Scientific Reports 14, 1 (2024), 25048. A Political Compass Test statements The PCT comprises 62 one-sentence statements spanning six thematic areas: na...

    2024

  43. [2021]

    In International Conference on Learning Representations

    Aligning AI With Shared Human Values. In International Conference on Learning Representations

  44. [2024]

    In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguist...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.