Pith. sign in

REVIEW 3 major objections 6 minor 35 references

Directional Hallucinations: Ideological Drift in News-Grounded LLM Question Answering

T0 review · 3 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM hallucinations skew left even when the source article is right-leaning; the paper reports 67.5% of fabricated sentences classified as left-leaning overall, and 63.2% from right-leaning sources.

desk verdict Solid, important empirical claim, but the headline drift figure hinges on a stance classifier that is never validated on the short hallucinated sentences it is applied to; a human-annotation spot-check should be a required revision. read the letter →

arxiv 2607.20487 v2 pith:FYZIKTKY submitted 2026-06-02 cs.AI cs.CL

Chendi Wang , Liam Cunningham , Tom Yishay , Jieying Chen This is my paper
classification cs.AIcs.CL
keywords hallucinationideologicaldriftpoliticalbiasquestionansweringLLMstanceclassificationuncertaintynewsgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that LLM hallucinations are not ideologically neutral. In a document-grounded question-answering task over 21,727 U.S. political news articles, it finds that hallucinated sentences are classified as left-leaning 67.5% of the time, and that this leftward drift persists even when the model is grounded on right-leaning articles (63.2% left). The pattern holds across four independently developed models, with only modest differences in hallucination frequency by source ideology. The paper also shows hallucinations concentrate in high-entropy generation contexts, and that in some models uncertainty predicts leftward drift, suggesting an 'uncertainty to guessing' mechanism. The broader point: when models lose grounding, they fall back on a latent left-leaning prior, making errors directional rather than random.

What carries the argument

The central machinery is a directional-drift measurement pipeline: a reference-based hallucination detector (ANAH-v2) labels each answer sentence as supported, unverifiable, or contradictory; a fine-tuned DeBERTa-v3 binary left/right stance classifier (macro-F1=0.74) labels the ideology of hallucinated sentences; and one-sided binomial tests compare the left-share against a symmetric null. The argument turns on the dissociation between hallucination rate (which varies modestly by source ideology) and hallucination content ideology (consistently left), plus a logit-based entropy analysis linking hallucination to uncertainty.

What would settle it

Compute the stance classifier's left-prediction rate on hallucinated sentences that are near-verbatim paraphrases of source text, or on correct (supported) sentences of the same length; if those are also classified as left-leaning a majority of the time, the drift claim collapses. Alternatively, have human annotators label a random sample of the hallucinated sentences and check whether the 67.5% left-share survives human judgment.

Watch

Extended reading notes

Core claim

The central claim is that hallucination content exhibits robust leftward drift: a majority of hallucinated sentences are classified as left-leaning, including among hallucinations generated from right-leaning sources. Quantitatively, the paper reports an overall left-share of 67.5% (95% CI [66.7, 68.3], p<0.001, Cohen's h=0.36), and 63.2% for right-source-only hallucinations (95% CI [61.9, 64.5], p<0.001). This asymmetry is independent of source ideology, and it is consistent across all four models tested. The paper further finds that hallucinations arise in high-uncertainty generation contexts, and that for some models uncertainty also predicts leftward drift, supporting an uncertainty-gate

Load-bearing premise

The drift metrics all depend on the fine-tuned DeBERTa-v3 binary stance classifier applied to short, decontextualized hallucinated sentences—a distribution different from the full-article text it was trained on—so if the classifier's accuracy or its mild left base rate shifts on that distribution, the observed leftward drift could be inflated or even an artifact.

Editorial extensions

If this is right

  • Even if overall hallucination rates are reduced, the errors that remain may still be directionally biased, so reliability improvements alone do not remove ideological skew.
  • Because leftward drift persists when the source is right-leaning, grounding a model on a source of one ideology does not prevent it from injecting content of another ideology.
  • Hallucinations concentrate in contested, election-relevant topics, meaning the directional bias is strongest exactly where misinformation is most consequential.
  • Uncertainty signals predict hallucination, suggesting abstention or warning policies could reduce ungrounded content; the heterogeneity across models implies interventions may need to be model-specific.
  • The framework provides an auditable, model-agnostic workflow for assessing both hallucination prevalence and ideological directionality of AI-mediated political information.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The consistency of leftward drift across models with different training data suggests the prior may originate in the composition of English-language pretraining corpora; this is an inference beyond the paper's explicit claims.
  • A testable extension would apply the same pipeline to non-U.S. or non-English political corpora: if the drift direction follows the corpus distribution, that would strengthen the latent-prior explanation; if it stays left in all languages, alignment or safety tuning would be implicated.
  • The finding implies that users interpreting AI answers as neutral may systematically absorb a left-leaning framing of contested facts, which could deepen perceived platform bias and polarization even if no single answer is overtly partisan.
  • The mild left base rate of the classifier (53.4%) is far below the observed drift, but a direct calibration check on hallucinated-sentence-length inputs would clarify how much of the drift is classifier distribution shift rather than model behavior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a reproducible, end-to-end framework for measuring whether hallucinated content in news-grounded LLM question answering is ideologically directional. Using 21,727 QBias articles labeled left/center/right, the authors generate one neutral question per article, collect answers from four LLMs, detect sentence-level hallucinations with ANAH-v2, and label the ideology of hallucinated sentences with a fine-tuned DeBERTa-v3 classifier. The main empirical claim is that hallucinated content exhibits a robust leftward drift: 67.5% of hallucinated sentences are classified as left-leaning (95% CI [66.7, 68.3]), including 63.2% when the source article is right-leaning. The paper also reports that hallucination frequency varies by model and topic, that mean token entropy predicts hallucination, and that uncertainty predicts leftward drift for some models but not others. The authors interpret the results as evidence that hallucinations are not symmetric reliability failures but can expose latent ideological priors.

Significance. If the central claim holds, the paper makes a valuable contribution to AI auditing in civic-information settings: it separates hallucination prevalence from the ideological direction of hallucinated content, uses a large expert-labeled corpus, reports confidence intervals and effect sizes, and includes a falsification check against the question-generation confound. The dissociation between source-article ideology and hallucination-content ideology is socially important, and the logit-level uncertainty analysis adds mechanistic evidence. The paper's strengths include the scale of the evaluation, the consistency of the leftward drift across four independently developed models, and the transparency of the pipeline description in the appendix. The main weakness is that the headline result rests entirely on a stance classifier that is trained on full articles and applied to short, decontextualized hallucinated sentences without dedicated validation; this is the load-bearing point that needs to be addressed before the empirical claim can be accepted.

major comments (3)
  1. [§5.3, Appendix 12.4, Table 2] The central drift metrics are all produced by a fine-tuned DeBERTa-v3 classifier trained on QBias full articles (headline + body, truncated to 256 tokens) and applied to hallucinated sentences, which are typically one short sentence without headline context. The paper reports a test-set left-prediction base rate of 53.4% (Appendix 12.3), but this is measured on article-level inputs and does not control for distribution shift. If the classifier over-predicts 'left' on short political sentences—for example, because topic words such as 'Trump' or 'immigration' co-occur with left-leaning phrasings in the training articles—the observed 67.5% overall left-share and 63.2% right-source left-share could be an artifact. The Limitations section (§8) acknowledges that differential accuracy across ideologies cannot be fully ruled out, but this is not a substitute for validation on the target distribu
  2. [§5.2 and §8 (Detection validity)] ANAH-v2 was not validated on U.S. partisan news sources, and the paper's limitation states this. Since hallucination labels define the population of sentences that enter the ideology analysis, any systematic error in the hallucination detector that correlates with source ideology or sentence topic could propagate into the drift estimates. The paper reports that ANAH-v2 achieves approximately 90% accuracy on its original validation data, but a small human-validated subset for the current article-grounded QA setting would strengthen the claim. This is particularly relevant because the detector is used at the sentence level on a genre (partisan news) that differs from the general validation corpus.
  3. [§6.2, Figure 5] The confusion matrices show that left-source articles contribute substantially more hallucinated sentences than right-source articles (e.g., Deepseek: 3,546 vs. 2,288; GPT-4o-Mini: 908 vs. 643), reflecting both the corpus imbalance and higher left-source hallucination rates. The overall left-share of 67.5% is therefore partly driven by the source-article composition. The paper does report the right-source-only left-share of 63.2%, which is the more direct evidence for source-independent drift, and I recommend foregrounding this conditional estimate in the abstract and discussion. The current framing of the headline as a global 'majority of hallucinated sentences' could be read as overstating the independence from source composition.
minor comments (6)
  1. [§5.3 / Table 1 / Figure 1] The stance classifier is called 'ownBERT' in Table 1 and Figure 1, but the text and Appendix 12 describe it as a fine-tuned DeBERTa-v3 model. Please unify the terminology.
  2. [Figure 5] The confusion matrices use 'Actual' and 'Predicted' labels. The rows are source-article ideology, not ground-truth hallucination ideology. This may mislead readers into thinking the rows are gold labels. Please rename the axes, e.g., 'Source article orientation' and 'Predicted hallucination orientation'.
  3. [Abstract and §6.2] The phrase 'a majority of hallucinated sentences are classified as left-leaning' should explicitly state that the classification is produced by the article-trained stance classifier, and that the estimate is subject to the distribution-shift caveat in Major Comment 1. As written, it could be read as direct human-labeled ideology.
  4. [Appendix / Reproducibility] The paper does not provide a data or code availability statement. Given the reproducible-framework claim, please include a link to the pipeline code and, if possible, the model outputs or human-annotation samples.
  5. [References] The reference for [Cinelli et al., 2025] lists 'Cinelli, Gianmarco De Francisci Morales, Alessandro Galeazzi, ...' without the first author's full name. Please fix the bibliographic entry.
  6. [§6.3] For Mistral, the OR = 0.71 with 95% CI [0.50, 1.02] and p = 0.065 is not statistically significant. The sentence 'Mistral shows no significant uncertainty-ideology relationship' is correct, but the subsequent interpretive sentence could note that the confidence interval includes substantial negative and positive effects, so the conclusion of 'no relationship' should be expressed as 'not detected' rather than 'absent.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central drift finding is a measured property of classifier outputs on generated sentences, not constructed from the classifier's training labels or from self-citation.

full rationale

The paper's load-bearing claim is empirical: hallucinated sentences are classified as left-leaning at 67.5% (Table 2). The DeBERTa-v3 stance classifier is fine-tuned on QBias article text and applied to hallucinated sentences; this is a measurement pipeline, not a derivation. The drift metric Pr(ŷ=L | hallucination) is not defined in terms of the classifier's training objective or base rate: Appendix 12.3 reports a 53.4% left-prediction rate on the balanced test set, so the 67.5% result is not forced by the classifier's own tendency. The classifier is trained on external article labels and applied to unseen generated sentences, so no fitted parameter is renamed as a prediction. The acknowledged risk that the classifier may behave differently on short, decontextualized sentences (Section 8: "We cannot fully rule out differential accuracy across ideologies") is a validity/distribution-shift limitation, not circularity. The two self-citations ([Chen et al., 2026], [de Swart et al., 2025]) appear only as corroborating literature in Related Work and Discussion and are not load-bearing; no uniqueness theorem or ansatz is imported from prior work by the same authors. The RQ3 logistic regressions are standard statistical fits, not equivalences to inputs. Overall, no step in the claimed derivation chain reduces to its own input.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim is an empirical measurement, so the ledger records the instrument-validity assumptions (stance classifier, hallucination detector) and the QG-neutrality assumption on which the drift claim rests. No numbers are fitted to produce the headline statistic.

free parameters (1)
  • Stance classifier decision threshold = 0.5 (default)
    The left-share metric is the proportion of hallucinated sentences with p(left)>0.5; this threshold is not separately fitted or validated for the hallucinated-sentence distribution, and moving it would change the headline 67.5%.
assumptions (5)
  • domain assumption QBias expert-annotated source ideology is treated as ground truth for left/right/center.
    The alignment and drift analyses compare classifier outputs against these labels (Section 4, §6.2).
  • domain assumption ANAH-v2 hallucination labels are valid on U.S. partisan news.
    The H indicator is defined from ANAH-v2 output; the paper notes it was not validated on this genre (Section 8).
  • domain assumption DeBERTa-v3 stance classifier remains accurate on hallucinated sentences under distribution shift.
    All drift metrics are computed from these labels; only article-level validation is provided (Appendix 12), and the 53.4% base-rate check is not a distribution-shift test.
  • domain assumption Llama-generated questions do not systematically bias content direction.
    The paper argues via Llama's weakest drift, but questions may still affect other models (Section 5.1, §6.2).
  • standard math Standard statistical assumptions for binomial tests and logistic regression.
    Used in §6.2 and §6.3 for CIs, p-values, ORs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Directional Hallucinations: Ideological Drift in News-Grounded LLM Question Answering." pith.science (2026). https://pith.science/paper/FYZIKTKY

@misc{pith2026260720487,
  author       = {Pith},
  title        = {Pith review of: Directional Hallucinations: Ideological Drift in News-Grounded LLM Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FYZIKTKY}},
  note         = {Machine review of arXiv:2607.20487}
}
read the original abstract

Large language models (LLMs) are increasingly used to answer questions about political information, including in election-adjacent information settings where factual errors and ideological distortions are high-stakes. We present a reproducible measurement framework that treats hallucinations, unsupported statements in document-grounded QA, as diagnostic signals of ideological drift. Using 21,727 expert-labeled U.S. political news articles from QBias spanning left, center, and right sources, we (i) generate an article-specific question, (ii) elicit document-grounded answers from three open-weight LLMs and one proprietary model, (iii) detect sentence-level hallucinations via reference-based comparison, (iv) classify the ideological valence of hallucinated sentences with a fine-tuned stance classifier, and (v) probe output logits to relate token-level uncertainty to hallucination and drift. Hallucination rates vary substantially across models and concentrate in contentious topics, while source-ideology differences in hallucination frequency are modest. In contrast, hallucination content exhibits robust leftward drift: a majority of hallucinated sentences are classified as left-leaning, including among hallucinations generated from right-leaning sources. Logit-level analysis shows hallucinations arise in high-entropy generation contexts, and in some models uncertainty also predicts leftward drift, consistent with an "uncertainty to guessing" mechanism. We discuss implications for auditing AI-mediated political information and for designing safeguards in election-relevant deployments.

Figures

Figures reproduced from arXiv: 2607.20487 by the authors.

Figure 1
Figure 1. Pipeline overview: (1) question generation, (2) news [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Normalized hallucination proportions by source leaning. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Ideological alignment of hallucinations: proportion [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Hallucinated topics by model (normalized by baseline [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 5
Figure 5. Figure 5: Confusion matrices: source article orientation (rows) vs. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Confusion matrix for the binary stance classifier on the [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 3 linked inside Pith

  1. [1]

    Social media and fake news in the 2016 elec- tion.Journal of economic perspectives, 31(2):211–236,

    [Allcott and Gentzkow, 2017] Hunt Allcott and Matthew Gentzkow. Social media and fake news in the 2016 elec- tion.Journal of economic perspectives, 31(2):211–236,

    2017

  2. [7]

    Detecting linguistic bias in govern- ment documents using large language models

    [de Swartet al., 2025 ] Milena de Swart, Floris den Hengst, and Jieying Chen. Detecting linguistic bias in govern- ment documents using large language models. In Guodong Long, Michale Blumestein, Yi Chang, Liane Lewin-Eytan, Zi Helen Huang, and Elad Yom-Tov, editors,Proceedings of the ACM on Web Conference 2025, WWW 2025, Syd- ney, NSW, Australia, 28 Apri...

    2025

  3. [8]

    Bias and fairness in large language models: A survey

    [Gallegoset al., 2024 ] Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Der- noncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey. Computational Linguistics, 50(3):1097–1179,

    2024

  4. [9]

    Selective classification for deep neural networks

    [Geifman and El-Yaniv, 2017] Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. Advances in neural information processing systems, 30,

    2017

  5. [10]

    Anah-v2: Scaling analytical hallucination annotation of large language mod- els.Advances in Neural Information Processing Systems, 37:60012–60039,

    [Guet al., 2024 ] Yuzhe Gu, Ziwei Ji, Wenwei Zhang, Chengqi Lyu, Dahua Lin, and Kai Chen. Anah-v2: Scaling analytical hallucination annotation of large language mod- els.Advances in Neural Information Processing Systems, 37:60012–60039,

    2024

  6. [11]

    On calibration of modern neural net- works

    [Guoet al., 2017 ] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural net- works. InInternational conference on machine learning, pages 1321–1330. PMLR,

    2017

  7. [12]

    Qbias - A Dataset on Media Bias in Search Queries and Query Suggestions

    [Haak and Schaer, 2023] Fabian Haak and Philipp Schaer. Qbias - A Dataset on Media Bias in Search Queries and Query Suggestions. InProceedings of the 15th ACM Web Science Conference 2023, pages 239–244, April

    2023

  8. [14]

    The levers of political persua- sion with conversational artificial intelligence.Science, 390(6777):eaea3884,

    [Hackenburget al., 2025 ] Kobi Hackenburg, Ben M Tap- pin, Luke Hewitt, Ed Saunders, Sid Black, Hause Lin, Catherine Fist, Helen Margetts, David G Rand, and Christopher Summerfield. The levers of political persua- sion with conversational artificial intelligence.Science, 390(6777):eaea3884,

    2025

Show all 35 references
  1. [15]

    Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding shar- ing,

    [Heet al., 2021 ] Pengcheng He, Jianfeng Gao, and Weizhu Chen. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding shar- ing,

    2021

  2. [16]

    A survey on hallucination in large language models: Prin- ciples, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1–55,

    [Huanget al., 2025 ] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qiang- long Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Prin- ciples, taxonomy, challenges, and open questions.ACM Tr...

    2025

  3. [17]

    Anah: Analyti- cal annotation of hallucinations in large language models

    [Jiet al., 2024 ] Ziwei Ji, Yuzhe Gu, Wenwei Zhang, Chengqi Lyu, Dahua Lin, and Kai Chen. Anah: Analyti- cal annotation of hallucinations in large language models. arXiv preprint arXiv:2405.20315,

    2024 arXiv

  4. [18]

    How can we know when language models know? on the calibration of language models for question answering.Transactions of the Association for Computational Linguistics, 9:962–977,

    [Jianget al., 2021 ] Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering.Transactions of the Association for Computational Linguistics, 9:962–977,

    2021

  5. [19]

    Evaluating large language models for accuracy incentivizes hallucinations

    [Kalaiet al., 2026 ] Adam Tauman Kalai, Ofir Nachum, San- tosh S Vempala, and Edwin Zhang. Evaluating large language models for accuracy incentivizes hallucinations. Nature, pages 1–3,

    2026

  6. [20]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural infor- mation processing systems, 33:9459–9474,

    [Lewiset al., 2020 ] Patrick Lewis, Ethan Perez, Aleksan- dra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K ¨uttler, Mike Lewis, Wen-tau Yih, Tim Rockt¨aschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural infor...

    2020

  7. [22]

    Persuad- ing voters using human–artificial intelligence dialogues

    [Linet al., 2025 ] Hause Lin, Gabriela Czarnek, Benjamin Lewis, Joshua P White, Adam J Berinsky, Thomas Costello, Gordon Pennycook, and David G Rand. Persuad- ing voters using human–artificial intelligence dialogues. Nature, pages 1–8,

    2025

  8. [23]

    Singh, Rylan Schaeffer, Andrew Poulton, Sanmi Koyejo, Pontus Stenetorp, Sharan Narang, and Dieuwke Hupkes

    [Madaanet al., 2024 ] Lovish Madaan, Aaditya K. Singh, Rylan Schaeffer, Andrew Poulton, Sanmi Koyejo, Pontus Stenetorp, Sharan Narang, and Dieuwke Hupkes. Quanti- fying Variance in Evaluation Benchmarks, June

    2024

  9. [24]

    [Motokiet al., 2025 ] Fabio Y . S. Motoki, Valdemar Pinho Neto, and Victor Rangel. Assessing political bias and value misalignment in generative artificial intelli- gence.Journal of Economic Behavior & Organization, page 106904, February

    2025

  10. [25]

    Biases in large language models: origins, in- ventory, and discussion.ACM Journal of Data and Infor- mation Quality, 15(2):1–21,

    [Navigliet al., 2023 ] Roberto Navigli, Simone Conia, and Bj¨orn Ross. Biases in large language models: origins, in- ventory, and discussion.ACM Journal of Data and Infor- mation Quality, 15(2):1–21,

    2023

  11. [26]

    RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models, May

    [Niuet al., 2024 ] Cheng Niu, Yuanhao Wu, Juno Zhu, Sil- iang Xu, Kashun Shum, Randy Zhong, Juntong Song, and Tong Zhang. RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models, May

    2024

  12. [27]

    What large language models do not talk about: An empirical study of moderation and censorship practices,

    [Noelset al., 2025 ] Sander Noels, Guillaume Bied, Maarten Buyl, Alexander Rogiers, Yousra Fettach, Jefrey Lijffijt, and Tijl De Bie. What large language models do not talk about: An empirical study of moderation and censorship practices,

    2025

  13. [28]

    https://huggingface.co/open-llm-leaderboard, March

    [OpenLLM, 2025] Open LLM Leaderboard. https://huggingface.co/open-llm-leaderboard, March

    2025

  14. [29]

    Assessing political bias in large language models.Journal of Computational Social Sci- ence, 8(2):42, May

    [Rettenbergeret al., 2025 ] Luca Rettenberger, Markus Reis- chl, and Mark Schutera. Assessing political bias in large language models.Journal of Computational Social Sci- ence, 8(2):42, May

    2025

  15. [30]

    Measuring Political Prefer- ences in AI Systems: An Integrative Approach, March

    [Rozado, 2025] David Rozado. Measuring Political Prefer- ences in AI Systems: An Integrative Approach, March

    2025

  16. [31]

    Whose Opinions Do Language Models Re- flect?, March

    [Santurkaret al., 2023 ] Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose Opinions Do Language Models Re- flect?, March

    2023

  17. [32]

    The spread of true and false news online.sci- ence, 359(6380):1146–1151,

    [V osoughiet al., 2018] Soroush V osoughi, Deb Roy, and Sinan Aral. The spread of true and false news online.sci- ence, 359(6380):1146–1151,

    2018

  18. [34]

    Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models, September

    [Zhanget al., 2023 ] Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Mod...

    2023

  19. [35]

    Detained in Russia since Feb

    Appendix 10 Data example Table 3: Example Row From the QBias Dataset id 21629 title How the White House is Handling Brittney Griner’s Russia Detention tags [World, White House, Russia, Brittney Griner] heading Biden under fire for handling of Brittney Griner’s detention in Rus...

    2024

  20. [2017]

    Llm-generated messages can persuade humans on policy issues.Nature Communications, 16(1):6037,

    [Baiet al., 2025 ] Hui Bai, Jan G V oelkel, Shane Muldowney, Johannes C Eichstaedt, and Robb Willer. Llm-generated messages can persuade humans on policy issues.Nature Communications, 16(1):6037,

    2025

  21. [2018]

    Ethical and social risks of harm from language models.arXiv preprint arXiv:2112.04359,

    [Weidingeret al., 2021 ] Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. Ethical and social risks of harm from language models.arXiv preprint arXiv:2112.04359,

    2021 arXiv

  22. [2020]

    Towards trustworthy LLMs: A review on debiasing and dehallucinating in large language models.Artificial Intel- ligence Review, 57(9):243, August

    [Linet al., 2024 ] Zichao Lin, Shuyan Guan, Wending Zhang, Huiyan Zhang, Yugang Li, and Huaping Zhang. Towards trustworthy LLMs: A review on debiasing and dehallucinating in large language models.Artificial Intel- ligence Review, 57(9):243, August

    2024

  23. [2021]

    Uncovering political bias in large lan- guage models using parliamentary voting records.arXiv preprint arXiv:2601.08785,

    [Chenet al., 2026 ] Jieying Chen, Karen de Jong, Andreas Poole, Jan Burakowski, Elena Elderson Nosti, Joep Windt, and Chendi Wang. Uncovering political bias in large lan- guage models using parliamentary voting records.arXiv preprint arXiv:2601.08785,

    2026

  24. [2023]

    Evaluating the persuasive influence of political microtargeting with large language mod- els.Proceedings of the National Academy of Sciences, 121(24):e2403116121,

    [Hackenburg and Margetts, 2024] Kobi Hackenburg and He- len Margetts. Evaluating the persuasive influence of political microtargeting with large language mod- els.Proceedings of the National Academy of Sciences, 121(24):e2403116121,

    2024

  25. [2024]

    On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610– 623,

    [Benderet al., 2021 ] Emily M Bender, Timnit Gebru, An- gelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610– 623,

    2021

  26. [2025]

    Measuring political bias in large lan- guage models: What is said and how it is said.arXiv preprint arXiv:2403.18932,

    [Banget al., 2024 ] Yejin Bang, Delong Chen, Nayeon Lee, and Pascale Fung. Measuring political bias in large lan- guage models: What is said and how it is said.arXiv preprint arXiv:2403.18932,

    2024 arXiv

  27. [2026]

    The echo chamber effect on social media.ResearchGate, February

    [Cinelliet al., 2025 ] Cinelli, Gianmarco De Francisci Morales, Alessandro Galeazzi, Walter Quattrociocchi, and Michele Starnini. The echo chamber effect on social media.ResearchGate, February

    2025

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.