Pith. sign in

REVIEW 2 major objections 4 minor 24 references

Leveraging In-Context Learning for Political Bias Testing of LLMs

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that feeding LLMs prior survey answers as in-context examples—Questionnaire Modeling—stabilizes political bias measurement enough to compare instruction-tuned models with their base versions.

desk verdict A genuinely new probing task with honest reporting, but the main empirical findings are conditional on one in-context ordering that the paper's own ablations show can flip the results. read the letter →

arxiv 2506.22232 v1 pith:SXU6A6LO submitted 2025-06-27 cs.CL

Patrick Haller , Jannis Vamvas , Rico Sennrich , Lena A. Jäger This is my paper · ORCID
classification cs.CL
keywords politicalbiasin-contextlearningquestionnairemodelingstabilityinstructiontuningLLMevaluationsurveydatapersonalizationaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Political bias testing by directly asking a language model a question and reading off 'yes' or 'no' is unstable: paraphrase or reword the question and the answer flips. This paper proposes Questionnaire Modeling (QM), which instead shows the model a partially filled human questionnaire—the answers of one real survey respondent to sixty earlier questions—and asks it to predict that respondent's answer to the target question. Repeating this over many respondents and averaging estimates the model's bias relative to the human population. The authors report that QM reduces the variability of bias scores across fifty paraphrases of each target question, and that it exposes cases where instruction tuning flips the direction of bias, such as overestimating agreement to stay-at-home parenting before tuning and underestimating it after. They also find that larger instruction-tuned models predict respondents' answers more accurately and show smaller biases.

What carries the argument

The central object is the Questionnaire Modeling task: a conversation in which a user asks a sequence of questions from a real voting advice questionnaire and the assistant answers, with the model asked to predict the final answer of the same human respondent. The bias score is the mean predicted probability of 'yes' across respondents minus the mean observed human probability of 'yes'. The personalization accuracy measures how often the model predicts the respondent's actual answer. Monte Carlo sampling over the 192 respondents turns a single brittle prompt into an expectation over many prompts, which is what carries the stability improvement.

What would settle it

Recompute the reported bias scores with, say, twenty random permutations of the sixty in-context examples for each model and target question; if the polarity of the bias flips across permutations for a substantial share of model–question pairs—or the spread of permuted scores is comparable to the difference between base and instruct models—then the stability claim collapses.

Watch

Extended reading notes

Core claim

The central claim is that bias evaluation becomes more reliable when the probing question is embedded in a questionnaire context drawn from real human responses. Formally, the paper defines the bias score as the difference between the model's average predicted probability of the answer 'yes' to a target question and the observed mean human answer; the prediction is made per respondent, conditioned on that respondent's answers to the other sixty questions. Averaging over the 192 elected Swiss parliamentarian respondents marginalizes over the influence of any single in-context example. The paper demonstrates that this procedure yields lower standard deviations of bias across paraphrases than zero-shot prompting, and that comparing base and instruction-tuned versions of the same model becomes meaningful because the method does not require the model to follow an instruction. The empirical finding that instruction tuning can change the direction of bias—for example, on state security, stay-at-home parenting, and digitalization—is presented as evidence that the method can detect real behavioral differences between model versions.

Load-bearing premise

The main load-bearing assumption is that one fixed ordering of the sixty in-context examples gives a representative bias estimate; the paper's own ablations show that permuting the order, and especially changing the final example, shifts scores and can flip their polarity.

Editorial extensions

If this is right

  • QM reduces the standard deviation of bias scores across fifty paraphrases compared with zero-shot prompting for the tested models.
  • Instruction tuning can flip the polarity of bias scores—e.g., Llama 3.1 70B switches from over- to under-estimating agreement with stay-at-home parenting.
  • Larger instruction-tuned Llama models show higher personalization accuracy and lower absolute bias scores, with 405B near zero bias on five of seven questions.
  • Randomly assigned in-context answers also reduce paraphrase variability but produce different bias scores, indicating that much of the stability comes from learning the label space rather than personalized prediction.
  • QM separates instructability from bias, allowing base models to be compared with their instruction-tuned counterparts even when the base model cannot follow the 'answer yes or no' instruction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the stability holds across a broader set of questions and languages, QM could serve as a standardized audit for political bias per model release, using the same survey population as a fixed reference.
  • The ablation's sensitivity to the identity of the last in-context example suggests a recency effect; sampling the last example randomly alongside respondent sampling could turn this sensitivity into a measured uncertainty rather than an uncontrolled confound.
  • The bias scores are defined relative to a specific human population (elected Swiss parliamentarians), so the same model could appear differently biased against another population; using QM as a benchmark would require fixing the reference population.
  • Extending QM beyond binary answers to the original Likert scale or to multi-token free responses would test whether the stability benefit persists when the discrete yes/no restriction is removed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes Questionnaire Modeling (QM), a probing task in which an LLM is given 60 human survey question-answer pairs as in-context examples and must predict a target respondent's answer to a seventh attitude statement. The authors define a personalization accuracy score and a bias score equal to the difference between the model's average predicted 'yes' probability and the human mean, then report these scores for Llama 3.1 8B/70B/405B, OLMo 7B, and GPT-3.5 across seven target questions. They claim that QM improves the stability of question-based bias evaluation relative to zero-shot prompting, that instruction tuning can flip the direction of bias on several statements, and that larger models exhibit higher personalization accuracy and smaller bias scores. Ablations vary the order and length of the in-context examples and compare true human responses with randomized answers.

Significance. If validated, QM is a useful contribution: it grounds LLM bias measurement in a human response distribution, it is transparently defined, and the authors release code and data. The paper's honesty is a strength: it reports a randomized-context baseline, full per-question variability tables, and an order-sensitivity ablation rather than hiding negative results. The empirical pattern that instruction tuning can flip the polarity of bias on specific attitude statements is intriguing, and the trend that larger models show lower bias in this setup is worth investigating. However, the central reliability claim is currently weakened by the fact that the main point estimates are computed under a single fixed ordering of the in-context examples while the paper's own ablation shows that ordering can change and even flip bias scores substantially.

major comments (2)
  1. [Appendix F.1 / Tables 2-3] The headline bias scores in Tables 2 and 3 are computed with one fixed ordering of the 60 in-context examples, and the reported standard errors reflect respondent sampling only (as defined in Section 3.3). The order-permutation ablation in Figures 7 and 8 shows that bias scores can shift by tens of percentage points and can flip polarity for several model-question pairs when the last in-context example is changed. Because the paper's central comparisons are differences between these point estimates (e.g., the instruction-tuning flips on Q13.4 and Q13.5, and the size trend in Table 3), the authors should either average bias scores over multiple random order permutations, or explicitly quantify and propagate order-induced uncertainty into the reported estimates and confidence intervals. As written, the reader cannot tell whether the headline flips are robust properties of the models or artifacts of one arbitrarily chosen prompt arrangement.
  2. [Section 5.1 / Table 4] The randomized in-context baseline achieves bias variability almost identical to true QM for many model-question pairs (for example, Llama 3.1 8B Instruct: 16.5 vs 16.2; OLMo Instruct: 11.6 vs 11.7; GPT-3.5: 21.5 vs 21.7). The text acknowledges this, but the abstract and introduction still attribute the stability improvement to QM's use of human survey data. The paper should restate the contribution more precisely: providing in-context examples yields the stability gain, while the human survey responses mainly shift the bias scores and anchor them to a population. Otherwise the reader may infer a causal role for the survey content that the randomized baseline does not support.
minor comments (4)
  1. [Throughout] There are several typographical inconsistencies, including 'LLama' in Table 2 and 'counter-parts' in Section 5; these should be corrected.
  2. [Appendix F.1, Figures 7-8] Only the original configuration has a confidence interval; the permuted estimates are shown without uncertainty. Adding error bars or intervals to the permuted conditions would help the reader judge which polarity flips are statistically meaningful.
  3. [Section 4 / Limitations] The Limitations paragraph says the paper does not investigate varying 'other aspects of the prompt,' but the ablation in Appendix F.1 does vary the order and length of in-context examples; please cross-reference that section to make the scope clear.
  4. [Figure 3 and Appendix G] Some token labels in the probability distribution figures appear truncated (e.g., a bare 'n'), and the captions refer to color coding that may not be accessible in grayscale; please ensure the final PDF renders these correctly.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the bias score is a direct comparison of model predictions to human averages, with no parameter fitted to the target data and no load-bearing self-citation chain.

full rationale

The paper's central bias score is defined as Bias(Q_tgt) = p_hat_yes,tgt - (1/n) sum A_i,tgt, where p_hat_yes,tgt is the model's mean predicted probability of 'yes' and the second term is the observed human mean. The target answer A_i,tgt is used only as an evaluation gold label, never as an in-context example: the conditioning set in the prediction p_theta(u | {Q_j, A_i,j}_{j != tgt}; Q_tgt) explicitly excludes the target question's answer. No parameter is fitted to the target data, and the reported stability claim is an empirical comparison against a zero-shot baseline rather than a consequence of the definition. The only self-citation, Haller et al. (2024), appears in Related Work as background evidence of response instability and does not carry the paper's central argument. The paper's own ablations (Figures 7 and 8) show sensitivity to in-context example order and especially to the identity of the last example, and the authors acknowledge this in the Limitations section; this is a genuine validity concern about the stability claim, but it is not circular reasoning, because the order-conditioned estimates are not baked into the definition of the bias score. Overall, the derivation is self-contained and externally falsifiable against human survey data, so no circularity step can be exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no fitted parameters or new theoretical entities. Its empirical claims rest on several domain assumptions about the reference population, prompt ordering, and the validity of token probabilities as opinions.

free parameters (3)
  • Likert-to-binary mapping threshold = 1-3 to 'no', 5-7 to 'yes', 4 discarded
    Human 7-point answers are collapsed to binary; dropping neutral answers changes the denominator and can shift both the human mean and the bias score.
  • Top-k token probability window = k = 10
    Only the 10 most likely tokens are used to estimate p(yes) and p(no); this truncation can bias estimates for models that spread probability mass over many tokens.
  • Sampling temperature = 1.0
    Used for all model queries; the paper notes bias scores are not invariant to temperature scaling.
assumptions (4)
  • domain assumption The 192 elected Swiss parliamentary candidates are representative of the population P to which bias is compared.
    Stated in Appendix A; if false, the bias scores are relative to a skewed reference population.
  • domain assumption The set of 60 in-context questions, presented in Smartvote's original order, is a sufficient and unbiased context for predicting the target answer.
    The paper's ablations show order and length changes shift bias substantially, so this assumption is empirically tenuous.
  • domain assumption Token probabilities in the top-10 window provide a sound estimator of the model's binary response distribution.
    The method relies on this to compute p(yes) and p(no); the paper acknowledges sensitivity to truncation and temperature.
  • domain assumption The LLM's next-token prediction is a valid proxy for what the model believes when asked a survey question.
    The entire bias measure is built on this proxy, a known simplification discussed in Limitations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging In-Context Learning for Political Bias Testing of LLMs." pith.science (2026). https://pith.science/paper/SXU6A6LO

@misc{pith2026250622232,
  author       = {Pith},
  title        = {Pith review of: Leveraging In-Context Learning for Political Bias Testing of LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SXU6A6LO}},
  note         = {Machine review of arXiv:2506.22232}
}
read the original abstract

A growing body of work has been querying LLMs with political questions to evaluate their potential biases. However, this probing method has limited stability, making comparisons between models unreliable. In this paper, we argue that LLMs need more context. We propose a new probing task, Questionnaire Modeling (QM), that uses human survey data as in-context examples. We show that QM improves the stability of question-based bias evaluation, and demonstrate that it may be used to compare instruction-tuned models to their base versions. Experiments with LLMs of various sizes indicate that instruction tuning can indeed change the direction of bias. Furthermore, we observe a trend that larger models are able to leverage in-context examples more effectively, and generally exhibit smaller bias scores in QM. Data and code are publicly available.

Figures

Figures reproduced from arXiv: 2506.22232 by the authors.

Figure 1
Figure 1. We provide the LLM with a political ques [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Prompt used for the Questionnaire Modeling [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Token probabilities predicted by Llama 3.1 8B models given an attitude question. Paraphrase 1 and 2 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Relationship between absolute bias and personalization accuracy for selected questions. Significant [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Visualization of political bias scores across [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Distribution of human answers to the attitude statements, given as percentages. The answers are based [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Effect of permuted in-context example order on model bias across target questions. In three of these [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Effect of reducing ICL context length on model bias across target questions. We created six reduced [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Relationship between absolute bias and personalization accuracy for remaining questions not in the main [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 7 canonical work pages

  1. [1]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  2. [2]

    Marcel Binz and Eric Schulz. 2023. https://doi.org/10.1073/pnas.2218523120 Using cognitive psychology to understand GPT-3 . Proceedings of the National Academy of Sciences, 120(6):e2218523120

    doi:10.1073/pnas.2218523120 2023

  3. [3]

    Su Lin Blodgett, Solon Barocas, Hal Daum \'e III, and Hanna Wallach. 2020. https://doi.org/10.18653/v1/2020.acl-main.485 Language (technology) is power: A critical survey of `` bias '' in NLP . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454--5476, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.acl-main.485 2020

  4. [4]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  5. [5]

    Tanise Ceron, Neele Falk, Ana Bari \'c , Dmitry Nikolaev, and Sebastian Pad \'o . 2024. https://doi.org/10.1162/tacl_a_00710 Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLM s . Transactions of the Association for Computational Linguistics, 12:1378--1400

    doi:10.1162/tacl_a_00710 2024

  6. [6]

    Wenchao Dong, Assem Zhunis, Hyojin Chin, Jiyoung Han, and Meeyoung Cha. 2024. https://doi.org/10.48550/arXiv.2402.10436 I am not them: Fluid identities and persistent out-group bias in large language models . arXiv preprint arXiv:2402.10436

    work page Pith review arXiv doi:10.48550/arxiv.2402.10436 2024

  7. [7]

    Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023. https://doi.org/10.18653/v1/2023.acl-long.783 Mitigating label biases for in-context learning . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14014--14031, Toronto, Canada. Association for Computational Linguistics

    work page doi:10.18653/v1/2023.acl-long.783 2023

  8. [8]

    Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. https://doi.org/10.18653/v1/2023.acl-long.656 From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa...

    doi:10.18653/v1/2023.acl-long.656 2023

Show all 24 references
  1. [9]

    Dirk Groeneveld, Iz Beltagy, Evan Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tusha...

    2024

  2. [10]

    Patrick Haller, Jannis Vamvas, and Lena Ann J \"a ger. 2024. https://openreview.net/forum?id=7xUtka9ck9 Yes, no, maybe? revisiting language models' response stability under paraphrasing for the assessment of political leaning . In First Conference on Language Modeling, Philade...

    2024

  3. [11]

    Jen-tse Huang, Wenxiang Jiao, Man Ho Lam, Eric John Li, Wenxuan Wang, and Michael Lyu. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.354 On the reliability of psychological scales on large language models . In Proceedings of the 2024 Conference on Empirical Methods in Natu...

    2024 doi

  4. [12]

    Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/21f7b745f73ce0d1f9bcea7f40b1388e-Paper-Conference.pdf Evaluating and inducing personality in pre-trained language models . In Adva...

    2023

  5. [13]

    Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. https://doi.org/10.18653/v1/2022.acl-long.556 Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity . In Proceedings of the 60th Annual Meeting of th...

    2022 doi

  6. [14]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.759 Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on ...

    2022 doi

  7. [15]

    Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. 2024. https://doi.org/10.1007/s11127-023-01097-2 More human than human: measuring chatgpt political bias . Public Choice, 198(1):3--23

    2024 doi

  8. [16]

    OpenAI. 2023. GPT 3.5 . https://platform.openai.com/docs/models/gpt-3-5-turbo. [Online; accessed 23-March-2024]

    2023

  9. [17]

    Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. https://doi.org/10.18653/v1/D18-1302 Reducing gender bias in abusive language detection . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799--2804, Brussels, Belgium. Association...

    2018 doi

  10. [18]

    Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. https://doi.org/10.18653/v1/2020.acl-main.647 Null it out: Guarding protected attributes by iterative nullspace projection . In Proceedings of the 58th Annual Meeting of the Association for Com...

    2020 doi

  11. [19]

    Yuval Reif and Roy Schwartz. 2024. https://doi.org/10.18653/v1/2024.naacl-long.378 Beyond performance: Quantifying and mitigating label bias in LLM s . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human L...

    2024 doi

  12. [20]

    Paul R \"o ttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. 2024. https://doi.org/10.18653/v1/2024.acl-long.816 Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large lan...

    2024 doi

  13. [21]

    David Rozado. 2023. The political biases of chatgpt. Social Sciences, 12(3):148

    2023

  14. [22]

    Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/a2cf225ba392627529efef14dc857e22-Paper-Conference.pdf Evaluating the moral beliefs encoded in LLMs . In Advances in Neural Information Processing Systems, v...

    2023

  15. [23]

    Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. 2024. https://doi.org/10.18653/v1/2024.naacl-long.295 You don ' t need a personality test to know these models are unreliable: Assessing the reliability ...

    2024 doi

  16. [24]

    Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. https://doi.org/10.18653/v1/D18-1334 Getting gender right in neural machine translation . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3003--3008, Brussels, Belgium....

    2018 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.