REVIEW 5 cited by
Measuring Political Bias in Large Language Models: What Is Said and How It Is Said
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We propose to measure political bias in LLMs by analyzing both the content and style of their generated content regarding political issues. Existing benchmarks and measures focus on gender and racial biases. However, political bias exists in LLMs and can lead to polarization and other harms in downstream applications. In order to provide transparency to users, we advocate that there should be fine-grained and explainable measures of political biases generated by LLMs. Our proposed measure looks at different political issues such as reproductive rights and climate change, at both the content (the substance of the generation) and the style (the lexical polarity) of such bias. We measured the political bias in eleven open-sourced LLMs and showed that our proposed framework is easily scalable to other topics and is explainable.
Forward citations
Cited by 5 Pith papers
-
Directional Hallucinations: Ideological Drift in News-Grounded LLM Question Answering
LLM hallucinations in news-grounded QA skew leftward: 64–70% of hallucinated sentences are classified as left-leaning, including 63% from right-leaning sources.
-
Auditing LLM Editorial Bias in News Media Exposure
Compared with Google News, GPT-4o-Mini, Claude-3.7-Sonnet, and Gemini-2.0-Flash surface fewer unique news outlets, distribute attention more unevenly, and lean ideologically in system-specific ways.
-
Designing Effective AI Explanations for Misinformation Detection: A Comparative Study of Content, Social, and Combined Explanations
Aligned content and social explanations improve misinformation detection accuracy, while misaligned explanations offer no accuracy benefit over no explanation.
-
LLMs on Trial: Evaluating Judicial Fairness for Large Language Models
A new 177,100-case benchmark shows that 16 LLMs systematically vary criminal sentences based on extra-legal demographic and procedural details, revealing pervasive judicial unfairness.
-
Obscured but Not Erased: Evaluating Nationality Bias in LLMs via Name-Based Bias Benchmarks
A name-substituted variant of the BBQ benchmark shows that LLMs retain nationality stereotypes even when explicit labels are removed, with smaller models showing more bias and lower accuracy.
Discussion (0). Sign in to comment.