Pith. sign in

REVIEW 4 major objections 5 minor 57 references

Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM agents do not emulate the social media users they are asked to simulate: given a user's real tweets as context, they grow more ideologically consistent but systematically exaggerate partisan hashtags, emojis, and toxicity beyond the…

desk verdict Paired human-baseline study shows LLM agents exaggerate emoji, hashtags, and toxicity; the ideological-consistency metric is entangled with its own classifier, but the core phenomenon holds. read the letter →

arxiv 2507.00657 v1 pith:K5ERPJLN submitted 2025-07-01 cs.HC cs.AIcs.SI

Jacopo Nudo , Mario Edoardo Pandolfo , Edoardo Loru , Mattia Samory , Matteo Cinelli , Walter Quattrociocchi This is my paper
classification cs.HCcs.AIcs.SI
keywords LLMagentsgenerativeexaggerationsocialsimulationpoliticaldiscourseideologicalconsistencytoxicity2024USpresidentialelectionpersonaprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Using 21 million X interactions around the 2024 U.S. presidential election, this paper builds LLM agents from 1,186 real users and has each agent reply to the same politically salient tweets the human user replied to, under two conditions: Zero Shot (only the user's inferred political leaning) and Few Shot (username, bio, and 30 recent tweets). The central claim is that LLMs do not emulate users but reconstruct them: richer context makes agent replies more ideologically consistent with the target user while systematically amplifying salient traits, including partisan hashtags, emojis, and toxic language, beyond human baselines. The authors name this distortion 'generative exaggeration' and show it is asymmetric, caricaturing right-leaning users more than left-leaning ones across three model families (Gemini, Mistral, and DeepSeek). They argue the distortion is a byproduct of models optimizing for salience over subtlety, and that it compromises the reliability of LLM agents as proxies for human behavior in content moderation, deliberative simulations, and policy modeling.

What carries the argument

The argument is carried by a paired controlled comparison: for each of 1,186 real X users, an agent replies to the same tweet thread the user replied to, under two initialization conditions — Zero Shot, which feeds only the user's inferred leaning score, and Few Shot, which feeds the username, bio, and 30 prior tweets. Political leaning is estimated by an automated stance classifier applied to at least 50 tweets per user, each labeled $+1$ (pro-Republican), $-1$ (pro-Democrat), or $0$ (neutral) and averaged, and the same classifier labels every generated reply, feeding the ideological consistency metric $C = 1 - L$, where $L$ averages the normalized ideological distance $|C_i - s_{ik}|/2$ between binned user leaning and reply label. Toxicity is scored with the Perspective API, and each generated reply's toxicity is ranked as a percentile within the user's 30 prompt tweets to show that agents overshoot the user's own distribution. Emoji and hashtag amplification is quantified as the ratio of LLM to human relative frequencies, with a ratio above 1 marking overrepresentation.

What would settle it

Re-run the measurement pipeline with a second, independently trained political stance classifier, or with human annotation of a random subsample of replies, and compare how well the two labeling methods agree on human versus LLM-generated tweets. If agreement is markedly higher for human tweets, then part of the improvement in consistency and part of the exaggeration ratios is an artifact of the classifier rewarding stereotyped language rather than a sign of genuine ideological fidelity.

Watch

Extended reading notes

Core claim

The paper's central discovery is a distortion it names 'generative exaggeration': when an LLM agent receives more behavioral context about a real user, its replies become more consistent with that user's political stance and simultaneously more extreme in the cues that mark the stance. In the Few Shot condition, agent-generated tweets contain the rainbow emoji nearly 20 times more often and partisan hashtags such as #MAGA and #kamalaharris2024 up to 10–15 times more often than human tweets do, and several model configurations exceed the human toxicity baseline of roughly 5% of replies above a 0.6 toxicity score. The effect is systematic rather than random: models overshoot the toxicity distribution of the very tweets they are prompted with, and the amplification appears across all six tested models, with right-leaning users caricatured more heavily than left-leaning ones. The paper reads this as evidence that the models capture superficial identity markers while misrepresenting the underlying behavioral profile, so fidelity to ideology collapses into caricature.

Load-bearing premise

Everything rests on the automated stance classifier being equally accurate for human tweets and for LLM-generated replies: the same classifier assigns each user a political leaning and labels every reply, so a classifier bias toward stereotyped or polarized language would inflate both the measured consistency gains and the apparent caricature.

Editorial extensions

If this is right

  • LLM agents cannot be treated as neutral proxies in political simulation: adding context improves ideological alignment while amplifying polarization, so improving simulation fidelity can make output less realistic.
  • Few Shot prompting can weaken safety constraints: models that produce few toxic replies with minimal context generate toxic content once user tweets are added, in some cases exceeding human baselines.
  • Deploying agents in content moderation, deliberation, or policy modeling risks importing a systematic skew that amplifies loud partisan cues and mutes low-frequency nuance.
  • The observed link between ideology and toxicity suggests that safety-oriented alignment protocols may carry an implicit political valence.
  • Because the distortion is stronger for right-leaning users across model families, simulations of political discourse are not symmetrically distorted; the bias is structural rather than prompt-specific.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Generative exaggeration may not be political at root: if next-token optimization over-weights salient markers generally, then persona-steered models should also amplify professional jargon, dialectal phrases, or subcultural symbols, making the phenomenon a general property of persona simulation rather than an election-specific quirk.
  • A direct mitigation follows from the paper's own percentile analysis: constrain generation to the user's empirical base rates by matching the user's actual frequency of emoji, hashtag, and toxic-language use, then test whether ideological consistency survives; if it does, exaggeration is a calibration problem, and if not, it marks a fidelity ceiling.
  • Re-running the pipeline with human-annotated political labels on a subsample of replies would separate true ideological fidelity from classifier agreement and sharpen the paper's core claim about reconstruction versus emulation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies whether LLM agents initialized with real user data from X (formerly Twitter) reproduce the linguistic, ideological, and toxicity profiles of the users they simulate, using 21 million interactions from the 2024 U.S. presidential election and 1,186 users. Agents are prompted in Zero Shot (political leaning only) or Few Shot (bio plus 30 past tweets) conditions, and compared against the original human replies. The central claim is that LLM agents exhibit "generative exaggeration": richer context improves apparent ideological consistency but also amplifies salient traits such as partisan hashtags, emojis, and toxicity beyond empirical human baselines. The paper reports higher LogTTR and TTR for small samples but lower diversity at scale, stronger ideological consistency under Few Shot prompting, increased toxicity in several model configurations, and large overrepresentation of stereotypical emojis and hashtags (e.g., rainbow emoji, #MAGA). The authors conclude that LLMs reconstruct rather than emulate users, introducing structural biases relevant to content moderation and deliberative simulations.

Significance. If the exaggeration effect is real, this is a timely and important result for computational social science and for the deployment of LLM-based social agents. The paper has notable strengths: it uses a large public dataset, compares six models across three families, provides one-to-one human-vs-agent comparisons, and supplements model-dependent classifiers with direct, interpretable measurements of emoji and hashtag frequencies. The toxicity analysis using Perspective API is an external benchmark and therefore not circular. However, the evidence is uneven across the three pillars of the central claim. The emoji/hashtag ratios are the most robust pillar but lack statistical grounding given very low human base rates; the toxicity-amplification claim is partially compromised by the prompt-window baseline; and the ideological-consistency and caricature claims rest on a stance classifier used on both sides of the comparison, with no distribution-shift validation. These issues are fixable with additional robustness analyses, and the paper is well positioned to make a strong contribution after revision.

major comments (4)
  1. [Methods, Eqs. (2)-(8); Results, Ideological Bias and Consistency] The ideological consistency metric uses the same automated stance classifier (Burnham et al., [49]) both to estimate each user's leaning L_i (Eq. 2) and to label the agent-generated replies s_ik that enter Eqs. (5)-(8). This creates a risk of measurement entanglement: if the classifier keys on surface markers such as partisan hashtags and slogans, then Few Shot replies that mimic those markers will be scored as more consistent even if they do not reflect deeper ideological fidelity. The paper itself acknowledges in the Limitations that standard classifiers may carry biases. A concrete robustness test is needed: validate the classifier on a human-labeled sample of LLM-generated replies, or recompute the consistency metric with an independent stance classifier and with manual annotations, and report whether the consistency gains and the caricature interpretation survive.
  2. [Results, Toxicity; Eq. (1)] Figure 4B computes the percentile of each generated reply against the 30 prompt tweets only (M = 30 in Eq. (1)), not against the full distribution of the user's past replies or against the specific human reply being emulated. Because the model is conditioned on exactly those 30 tweets, a high percentile can reflect simple copying or amplification of the most toxic item in its own context rather than exaggeration beyond the empirical human baseline. To support the claim of generative exaggeration in toxicity, the percentile should be computed against a larger random sample of the user's history, and the toxicity of the generated reply should be compared with the toxicity of the human reply it is meant to emulate. Reporting absolute counts and confidence intervals for the 100th-percentile peak would also strengthen the result.
  3. [Results, The Style of Generative Exaggeration; Fig. 5] The emoji and hashtag ratios in Fig. 5 are the most direct and convincing evidence for exaggeration, but the human base rates are extremely low: 87.47% of human tweets omit emojis and 96.29% omit hashtags. A ratio of 10-20x can therefore arise from a small absolute number of model-generated tweets containing a given token, and the paper reports no absolute counts, statistical tests, or confidence intervals for these ratios. Since the text states that the over-production is 'systematic, not random', the authors should provide per-emoji and per-hashtag frequencies with counts, binomial or permutation-based significance tests, and effect sizes, especially for the headline examples such as rainbow emoji and #MAGA.
  4. [Results and Discussion; Conclusions] The paper asserts that the observed effects 'are not driven by prompt design but are a byproduct of model training and optimization' and offers a mechanistic explanation in terms of next-token prediction weighting high-salience ideological tokens. However, the experimental design only varies Zero Shot versus Few Shot initialization; it does not ablate prompt wording, example selection, instruction length, or sampling temperature. The mechanistic claim is therefore not directly tested. A minimal ablation varying the number of few-shot examples and the prompt phrasing would help determine whether the exaggeration is a model-internal bias or an artifact of the specific prompting strategy.
minor comments (5)
  1. [Methods, Political Leaning Estimation] Eq. (2) sets M_i = 50, but the text says the leaning is estimated from 'at least 50 comments previously posted by the user.' Please clarify whether users with more than 50 tweets contribute all tweets or exactly 50, and how users with fewer than 50 are handled.
  2. [Results, Ideological Bias and Consistency] The text says DeepSeek in the Zero Shot condition generates ideologically aligned responses in approximately 45% of cases for Republican-leaning users, but the corresponding row in Fig. 2 appears to show a different value; please ensure the prose numbers match the figure cells and state whether these are conditional probabilities.
  3. [Supplementary Table S2] Table S2 has malformed column headers such as 'Emojis, Hashtags # Mentions @ Tweets No, Tweets No # Tweets No @'. The headers should be clearly separated and labeled so that the counts for emojis, hashtags, and mentions are unambiguous.
  4. [Prompts in Figs. 6 and 7] Both prompts instruct the model to write a tweet 'with not more than 100 characters,' but the paper elsewhere discusses Twitter's 280-character limit and analyzes tweets up to 280 characters. Please reconcile these thresholds and specify whether the 100-character cap was applied consistently across all generations.
  5. [Acknowledgements] The acknowledgements thank 'Geronimo Stilton and the Hypnotoad.' If these are not real contributors, they should be removed from the published version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: exaggeration claims are benchmarked against human ground truth and external classifiers, with no load-bearing self-citation or fitted-parameter-as-prediction step.

full rationale

The paper's central empirical claims are comparisons between LLM outputs and human-authored tweets, using external measurement instruments. The emoji/hashtag exaggeration ratios in Fig. 5 are relative frequencies computed from human and generated tweets, so they are not defined in terms of the model outputs. Toxicity is scored by Google's Perspective API for both humans and agents against a human baseline, again an external benchmark. The ideological consistency metric (Eqs. 2-8) does use the same Burnham et al. stance classifier to estimate user leaning (from past tweets) and to label generated replies; this shared-instrument design can inflate measured consistency if the classifier keys on surface cues, and the paper itself acknowledges that 'standard classifiers used for toxicity and ideological labeling may themselves carry biases.' However, this is a construct-validity caveat, not circularity: no parameter is fitted within the paper to the target outcome, the classifier is an external pre-trained tool, and the metric is an observational statistic rather than a prediction derived from an input. The Fig. 4B percentile baseline is the 30-tweet prompt set ('we set M = 30, which corresponds to the number of tweets provided to each agent for the Few Shot initialization'), so the claim of toxicity overshoot is explicitly relative to the conditioning context; while methodologically limited, it is not an algebraic consequence of the definition. No load-bearing self-citations appear: the dataset [37], stance classifier [49], and toxicity API [50] are all external to the author group. No uniqueness theorem or ansatz is imported from the authors' prior work. The study is self-contained against empirical baselines, so no circular step can be exhibited.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces a conceptual label, "generative exaggeration", but no new physical or mechanistic entity. The main ledger items are methodological thresholds and classifier validity assumptions. The most important assumption is that the same stance classifier works equally well on human and LLM text, because that assumption underwrites the ideological consistency results. The mechanistic explanation about token salience is an ad hoc postulate without independent evidence.

free parameters (5)
  • Political leaning bin threshold = 0.25
    Eq. (4) bins continuous L_i into Democrat, Neutral, and Republican classes using +/-0.25 thresholds; no sensitivity analysis is reported.
  • Minimum tweets per user = 50
    Selection criterion used to estimate user political leaning and style; changes which users enter the analysis.
  • Few Shot prompt tweet count = 30
    Chosen to keep the prompt under length limits; the toxicity percentile reference set also uses M=30.
  • Toxicity threshold = 0.6
    Tweets are labeled toxic if Perspective API score exceeds 0.6; this threshold is borrowed from prior work but is arbitrary.
  • LogTTR smoothing constant = 1
    Added to numerator and denominator in Eq. (9) to avoid undefined values for short tweets.
assumptions (5)
  • domain assumption Stance classifier [49] correctly classifies political leaning of both human and LLM-generated tweets
    Used to compute user leaning L_i (Eq. 2) and to label agent replies in the consistency metric (Eqs. 5-8); no validation on LLM text is reported.
  • domain assumption Perspective API toxicity scores are a valid measure of harmful language for both human and synthetic tweets
    The entire toxicity analysis depends on an external classifier that may behave differently on model-generated text.
  • domain assumption Users with at least 50 politically relevant tweets are representative of the population of interest
    The sample is restricted to highly active political users, which may not generalize to casual users.
  • domain assumption LLM outputs are stable enough that one generated reply per human reply supports the comparisons
    No sampling temperature, number of runs per prompt, or variance analysis is reported for the paired reply generation.
  • ad hoc to paper Training assigns extra weight to high-salience ideological tokens, causing the observed exaggeration
    Stated as the cause of generative exaggeration in the 'Style of Generative Exaggeration' section but no training analysis or evidence is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity." pith.science (2026). https://pith.science/paper/K5ERPJLN

@misc{pith2026250700657,
  author       = {Pith},
  title        = {Pith review of: Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K5ERPJLN}},
  note         = {Machine review of arXiv:2507.00657}
}
read the original abstract

We investigate how Large Language Models (LLMs) behave when simulating political discourse on social media. Leveraging 21 million interactions on X during the 2024 U.S. presidential election, we construct LLM agents based on 1,186 real users, prompting them to reply to politically salient tweets under controlled conditions. Agents are initialized either with minimal ideological cues (Zero Shot) or recent tweet history (Few Shot), allowing one-to-one comparisons with human replies. We evaluate three model families (Gemini, Mistral, and DeepSeek) across linguistic style, ideological consistency, and toxicity. We find that richer contextualization improves internal consistency but also amplifies polarization, stylized signals, and harmful language. We observe an emergent distortion that we call "generation exaggeration": a systematic amplification of salient traits beyond empirical baselines. Our analysis shows that LLMs do not emulate users, they reconstruct them. Their outputs, indeed, reflect internal optimization dynamics more than observed behavior, introducing structural biases that compromise their reliability as social proxies. This challenges their use in content moderation, deliberative simulations, and policy modeling.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 38 canonical work pages

  1. [49]

    , Kahn , K

    botherref Burnham , M. , Kahn , K. , Wang , R.Y. , Peng , R.X. : Political debate: Efficient zero-shot and few-shot classifiers for political text. arXiv preprint arXiv:2409.02078 (2024) botherref

    arXiv 2024

  2. [1]

    , Zhang , J

    bchapter Debenedetti , E. , Zhang , J. , Balunovic , M. , Beurer-Kellner , L. , Fischer , M. , Tram \`e r , F. : Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents . In: The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track ( 2024 ) bchapter

    work page 2024

  3. [2]

    , Cao , Y

    barticle Boskabadi , M.R. , Cao , Y. , Khadem , B. , Clements , W. , Gerek , Z.N. , Reuthe , E. , Sivaram , A. , Savoie , C.J. , Mansouri , S.S. : Industrial agentic ai and generative modeling in complex systems . Current Opinion in Chemical Engineering 48 , 101150 ( 2025 ) barticle

    work page 2025

  4. [3]

    , Song , K

    barticle Shen , Y. , Song , K. , Tan , X. , Li , D. , Lu , W. , Zhuang , Y. : Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face . Advances in Neural Information Processing Systems 36 , 38154 -- 38180 ( 2023 ) barticle

    work page 2023

  5. [4]

    , Chu , S.N

    barticle Goodell , A.J. , Chu , S.N. , Rouholiman , D. , Chu , L.F. : Large language model agents can use tools to perform clinical calculations . npj Digital Medicine 8 ( 1 ), 163 ( 2025 ) barticle

    work page 2025

  6. [5]

    Bran , A

    barticle M. Bran , A. , Cox , S. , Schilter , O. , Baldassari , C. , White , A.D. , Schwaller , P. : Augmenting large language models with chemistry tools . Nature Machine Intelligence 6 ( 5 ), 525 -- 535 ( 2024 ) barticle

    work page 2024

  7. [6]

    , Yu , Y

    barticle Kim , S. , Yu , Y. , Seo , H. : Artificial intelligence orchestration for text-based ultrasonic simulation via self-review by multi-large language model agents . Scientific Reports 15 ( 1 ), 12474 ( 2025 ) barticle

    work page 2025

  8. [7]

    , Cinus , F

    botherref Coppolillo , E. , Cinus , F. , Minici , M. , Bonchi , F. , Manco , G. : Engagement-driven content generation with large language models. arXiv preprint arXiv:2411.13187 (2024) botherref

    arXiv 2024

Show all 57 references
  1. [8]

    , O'Brien , J

    bchapter Park , J.S. , O'Brien , J. , Cai , C.J. , Morris , M.R. , Liang , P. , Bernstein , M.S. : Generative agents: Interactive simulacra of human behavior . In: Proceedings of the 36th Annual Acm Symposium on User Interface Software and Technology , pp. 1 -- 22 ( 2023 ) bchapter

    2023

  2. [9]

    , Axtell , R

    bbook Epstein , J.M. , Axtell , R. : Growing Artificial Societies: Social Science from the Bottom Up . Brookings Institution Press , ??? ( 1996 ) bbook

    1996

  3. [10]

    , Edmonds , B

    barticle Conte , R. , Edmonds , B. , Moss , S. , Sawyer , R.K. : Sociology and social theory in agent based social simulation: A symposium . Computational & Mathematical Organization Theory 7 ( 3 ), 183 -- 205 ( 2001 ) barticle

    2001

  4. [11]

    , Romero , D.M

    botherref M ller , A.G. , Romero , D.M. , Jurgens , D. , Aiello , L.M. : The impact of generative ai on social media: An experimental study. arXiv preprint arXiv:2506.14295 (2025) botherref

    2025 arXiv

  5. [12]

    , De Francisci Morales , G

    barticle Cinelli , M. , De Francisci Morales , G. , Galeazzi , A. , Quattrociocchi , W. , Starnini , M. : The echo chamber effect on social media . Proceedings of the national academy of sciences 118 ( 9 ), 2023301118 ( 2021 ) barticle

    2021

  6. [13]

    , Galeazzi , A

    botherref Di Martino , E. , Galeazzi , A. , Starnini , M. , Quattrociocchi , W. , Cinelli , M. : Characterizing the fragmentation of the social media ecosystem. arXiv preprint arXiv:2411.16826 (2024) botherref

    2024 arXiv

  7. [14]

    , Bessi , A

    barticle Del Vicario , M. , Bessi , A. , Zollo , F. , Petroni , F. , Scala , A. , Caldarelli , G. , Stanley , H.E. , Quattrociocchi , W. : The spreading of misinformation online . Proceedings of the national academy of Sciences 113 ( 3 ), 554 -- 559 ( 2016 ) barticle

    2016

  8. [15]

    , Ziegler , J

    bchapter Donkers , T. , Ziegler , J. : Understanding online polarization through human-agent interaction in a synthetic llm-based social network . In: Proceedings of the International AAAI Conference on Web and Social Media , vol. 19 , pp. 457 -- 478 ( 2025 ) bchapter

    2025

  9. [16]

    : Salm: A multi-agent framework for language model-driven social network simulation

    botherref Koley , G. : Salm: A multi-agent framework for language model-driven social network simulation. arXiv preprint arXiv:2505.09081 (2025) botherref

    2025

  10. [17]

    , Stella , M

    botherref Rossetti , G. , Stella , M. , Cazabet , R. , Abramski , K. , Cau , E. , Citraro , S. , Failla , A. , Improta , R. , Morini , V. , Pansanella , V. : Y social: an llm-powered social media digital twin. arXiv preprint arXiv:2408.00818 (2024) botherref

    2024 arXiv

  11. [18]

    , Failla , A

    bchapter Cau , E. , Failla , A. , Rossetti , G. : Bots of a feather: Mixing biases in llms’ opinion dynamics . In: International Conference on Complex Networks and Their Applications , pp. 166 -- 176 ( 2024 ). Springer bchapter

    2024

  12. [19]

    , Assenmacher , D

    botherref Holtdirk , T. , Assenmacher , D. , Bleier , A. , Wagner , C. : Fine-tuning large language models to simulate german voting behaviour. Technical report, Center for Open Science (2024) botherref

    2024

  13. [20]

    , Popowski , L

    bchapter Park , J.S. , Popowski , L. , Cai , C. , Morris , M.R. , Liang , P. , Bernstein , M.S. : Social simulacra: Creating populated prototypes for social computing systems . In: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology , pp. 1 -...

    2022

  14. [21]

    , Wurth , E

    botherref Ahnert , G. , Wurth , E. , Strohmaier , M. , Mata , J. : Simulating persuasive dialogues on meat reduction with generative agents. arXiv preprint arXiv:2504.04872 (2025) botherref

    2025

  15. [22]

    , Manco , G

    botherref Coppolillo , E. , Manco , G. , Aiello , L.M. : Unmasking conversational bias in ai multiagent systems. arXiv preprint arXiv:2501.14844 (2025) botherref

    2025 arXiv

  16. [23]

    , Dover , Y

    botherref Taubenfeld , A. , Dover , Y. , Reichart , R. , Goldstein , A. : Systematic biases in llm simulations of debates. arXiv preprint arXiv:2402.04049 (2024) botherref

    2024 arXiv

  17. [24]

    , Piccardi , T

    botherref Cheng , M. , Piccardi , T. , Yang , D. : Compost: Characterizing and evaluating caricature in llm simulations. arXiv preprint arXiv:2310.11501 (2023) botherref

    2023 arXiv

  18. [25]

    , Diab , M

    bchapter Liu , A. , Diab , M. , Fried , D. : Evaluating large language model biases in persona-steered generation . In: Findings of the Association for Computational Linguistics ACL 2024 , pp. 9832 -- 9850 . Association for Computational Linguistics , ??? ( 2024 ). 10.18653/v1...

    2024 doi

  19. [26]

    , Chen , H

    botherref Li , A. , Chen , H. , Namkoong , H. , Peng , T. : LLM generated persona is a promise with a catch (2025) https://arxiv.org/abs/2503.16527 2503.16527 [cs.CL] botherref

    2025 arXiv

  20. [27]

    , Sen , I

    botherref Alipour , S. , Sen , I. , Samory , M. , Mitra , T. : Robustness and confounders in the demographic alignment of llms with human perceptions of offensiveness. arXiv preprint arXiv:2411.08977 (2024) botherref

    2024 arXiv

  21. [28]

    , Kyrychenko , Y

    barticle Hu , T. , Kyrychenko , Y. , Rathje , S. , Collier , N. , Linden , S. , Roozenbeek , J. : Generative language models exhibit social identity biases . Nature Computational Science 5 ( 1 ), 65 -- 75 ( 2025 ) barticle

    2025

  22. [29]

    , Nudo , J

    botherref Loru , E. , Nudo , J. , Di Marco , N. , Cinelli , M. , Quattrociocchi , W. : Decoding ai judgment: How llms assess news credibility and bias. arXiv preprint arXiv:2502.04426 (2025) botherref

    2025

  23. [30]

    , Rogiers , A

    botherref Buyl , M. , Rogiers , A. , Noels , S. , Bied , G. , Dominguez-Catena , I. , Heiter , E. , Johary , I. , Mara , A.-C. , Romero , R. , Lijffijt , J. , et al.: Large language models reflect the ideology of their creators. arXiv preprint arXiv:2410.18417 (2024) botherref

    2024 arXiv

  24. [31]

    , Bied , G

    botherref Noels , S. , Bied , G. , Buyl , M. , Rogiers , A. , Fettach , Y. , Lijffijt , J. , De Bie , T. : What large language models do not talk about: An empirical study of moderation and censorship practices. arXiv preprint arXiv:2504.03803 (2025) botherref

    2025 arXiv

  25. [32]

    , He , Z

    botherref Chen , K. , He , Z. , Yan , J. , Shi , T. , Lerman , K. : How susceptible are large language models to ideological manipulation? arXiv preprint arXiv:2402.11725 (2024) botherref

    2024 arXiv

  26. [33]

    , Bernardelle , P

    bchapter Civelli , S. , Bernardelle , P. , Demartini , G. : The impact of persona-based political perspectives on hateful content detection . In: Companion Proceedings of the ACM on Web Conference 2025 , pp. 1963 -- 1968 ( 2025 ) bchapter

    2025

  27. [34]

    , Lu , Z

    botherref Piao , J. , Lu , Z. , Gao , C. , Xu , F. , Santos , F.P. , Li , Y. , Evans , J. : Emergence of human-like polarization among large language model agents. arXiv preprint arXiv:2501.05171 (2025) botherref

    2025 arXiv

  28. [35]

    , Liao , Q.V

    bchapter Sharma , N. , Liao , Q.V. , Xiao , Z. : Generative echo chamber? effect of llm-powered search systems on diverse information seeking . In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems , pp. 1 -- 17 ( 2024 ) bchapter

    2024

  29. [36]

    : Filter bubbles and affective polarization in user-personalized large language model outputs

    bchapter Lazovich , T. : Filter bubbles and affective polarization in user-personalized large language model outputs . In: Proceedings On , pp. 29 -- 37 ( 2023 ). PMLR bchapter

    2023

  30. [37]

    , Zou , V

    botherref Balasubramanian , A. , Zou , V. , Narayana , H. , You , C. , Luceri , L. , Ferrara , E. : A public dataset tracking social media discourse about the 2024 us presidential election on twitter/x. arXiv preprint arXiv:2411.00376 (2024) botherref

    2024 arXiv

  31. [38]

    , Loru , E

    barticle Di Marco , N. , Loru , E. , Bonetti , A. , Serra , A.O.G. , Cinelli , M. , Quattrociocchi , W. : Patterns of linguistic simplification on social media platforms over time . Proceedings of the National Academy of Sciences 121 ( 50 ), 2412105121 ( 2024 ) barticle

    2024

  32. [39]

    , Baayen , R.H

    barticle Tweedie , F.J. , Baayen , R.H. : How variable may a constant be? measures of lexical richness in perspective . Computers and the Humanities 32 , 323 -- 352 ( 1998 ) barticle

    1998

  33. [40]

    , Jarvis , S

    barticle McCarthy , P.M. , Jarvis , S. : Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment . Behavior research methods 42 ( 2 ), 381 -- 392 ( 2010 ) barticle

    2010

  34. [41]

    , Miguel , M.S

    botherref Rosillo-Rodes , P. , Miguel , M.S. , Sanchez , D. : Entropy and type-token ratio in gigaword corpora (2025). https://arxiv.org/abs/2411.10227 botherref

    2025 arXiv

  35. [42]

    : Type/token ratios: What do they really tell us? Journal of child language 14 ( 2 ), 201 -- 209 ( 1987 ) barticle

    barticle Richards , B. : Type/token ratios: What do they really tell us? Journal of child language 14 ( 2 ), 201 -- 209 ( 1987 ) barticle

    1987

  36. [43]

    : Can type-token ratio be used to show morphological complexity of languages? Journal of Quantitative Linguistics 21 ( 3 ), 223 -- 245 ( 2014 ) barticle

    barticle Kettunen , K. : Can type-token ratio be used to show morphological complexity of languages? Journal of Quantitative Linguistics 21 ( 3 ), 223 -- 245 ( 2014 ) barticle

    2014

  37. [44]

    : Type-token mathematics: A textbook of mathematical linguistics

    botherref Herdan , G. : Type-token mathematics: A textbook of mathematical linguistics. (No Title) (1960) botherref

    1960

  38. [45]

    barticle Chotlos , J.W. : Iv. a statistical and comparative analysis of individual written language samples. Psychological Monographs 56 ( 2 ), 75 ( 1944 ) barticle

    1944

  39. [46]

    : How useful is the logarithmic type/token ratio? Journal of Linguistics 7 ( 2 ), 237 -- 243 ( 1971 ) 10.1017/S0022226700002930 barticle

    barticle Weitzman , M. : How useful is the logarithmic type/token ratio? Journal of Linguistics 7 ( 2 ), 237 -- 243 ( 1971 ) 10.1017/S0022226700002930 barticle

    1971 doi

  40. [47]

    : The political preferences of llms

    barticle Rozado , D. : The political preferences of llms . PloS one 19 ( 7 ), 0306621 ( 2024 ) barticle

    2024

  41. [48]

    o ttger , P. , Hofmann , V. , Pyatkin , V. , Hinck , M. , Kirk , H.R. , Sch \

    botherref R \"o ttger , P. , Hofmann , V. , Pyatkin , V. , Hinck , M. , Kirk , H.R. , Sch \"u tze , H. , Hovy , D. : Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models. arXiv preprint arXiv:2402.16786 (2024...

    2024 arXiv

  42. [50]

    , Tran , V.Q

    bchapter Lees , A. , Tran , V.Q. , Tay , Y. , Sorensen , J. , Gupta , J. , Metzler , D. , Vasserman , L. : A new generation of perspective api: Efficient multilingual character-level transformers . In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Da...

    2022

  43. [51]

    , Di Marco , N

    barticle Avalle , M. , Di Marco , N. , Etta , G. , Sangiorgio , E. , Alipour , S. , Bonetti , A. , Alvisi , L. , Scala , A. , Baronchelli , A. , Cinelli , M. , : Persistent interaction patterns across social media platforms and over time . Nature 628 ( 8008 ), 582 -- 589 ( 202...

    2024

  44. [52]

    , Hu , Y

    barticle Zhang , L. , Hu , Y. , Li , W. , Bai , Q. , Nand , P. : Llm-aidsim: Llm-enhanced agent-based influence diffusion simulation in social networks . Systems 13 ( 1 ), 29 ( 2025 ) barticle

    2025

  45. [53]

    , Yan , Y

    botherref Piao , J. , Yan , Y. , Zhang , J. , Li , N. , Yan , J. , Lan , X. , Lu , Z. , Zheng , Z. , Wang , J.Y. , Zhou , D. , et al.: Agentsociety: Large-scale simulation of llm-driven generative agents advances understanding of human behaviors and society. arXiv preprint arX...

    2025 arXiv

  46. [54]

    , Montani , I

    botherref Honnibal , M. , Montani , I. : spaCy 2 : Natural language understanding with B loom embeddings, convolutional neural networks and incremental parsing. To appear (2017) botherref

    2017

  47. [55]

    , : Exploratory Data Analysis vol

    bbook Tukey , J.W. , : Exploratory Data Analysis vol. 2 . Springer , ??? ( 1977 ) bbook

    1977

  48. [56]

    Exploratory data analysis, volume 2

    John Wilder Tukey et al. Exploratory data analysis, volume 2. Springer, 1977

    1977

  49. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.