REVIEW 3 major objections 5 minor 34 references
No for Some, Yes for Others: Persona Prompts and Other Sources of False Refusal in Language Models
T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read Persona prompts change false refusal rates less than model and task choice do.
desk verdict Solid multi-factor empirical ranking of false-refusal sources (model > task > persona > prompt), with an untested refusal detector and missing artifacts keeping it short of a reference result. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a nested Monte Carlo sampling scheme that randomly pairs each input text with a persona and a prompt template, so that the combinatorial space of persona×prompt×model×task is explored in a balanced, sample-efficient way. Quantification then rests on a squared Wasserstein-2 global sensitivity index, computed via optimal transport, which measures how much knowing each factor shifts the refusal distribution; a regularized logistic regression on refusal outcome confirms the ranking and gives per-level coefficients.
What would settle it
Hand-label a random sample of model responses across all personas, tasks, and models, and compare human refusal judgments with the string-match detector; if disagreements are concentrated in particular personas or models, the reported refusal rates and the model-vs-persona ranking would shift.
Extended reading notes
Core claim
The paper claims that false refusals in LLM classification tasks are driven primarily by model choice, secondarily by task, third by sociodemographic persona, and least by prompt paraphrase. Using a Monte Carlo sampling design and a Wasserstein-distance sensitivity analysis, the authors rank these four sources of refusal variation. They also find a generational pattern: newer Llama and Qwen models refuse far less, while the newest Gemma2 models refuse more, and offensiveness classification triggers far more refusals than NLI. Certain personas—Black, White, transgender, and Muslim—elicit disproportionately more refusals in some models, which the paper reads as evidence of bias in safety align
Load-bearing premise
The automatic refusal detector only counts responses that begin with a fixed list of refusal phrases, so refusals phrased differently would be missed; the paper acknowledges this and assumes the misses are spread evenly enough not to change the factor ranking.
Editorial extensions
If this is right
- Future studies of persona effects must control for model, task, and prompt design, or they risk attributing to personas what is actually caused by other experimental choices.
- Refusal rates in classification tasks decrease substantially in newer model generations within some families, but not all, so progress on false refusal is not uniform.
- Tasks involving sensitive content, especially offensiveness classification, will keep eliciting far more false refusals than objective tasks like NLI.
- Open-ended or unforced prompts increase refusals; forcing a label output suppresses most refusals, which is an experimental-design choice that can change reported results dramatically.
- Some sociodemographic groups experience systematically higher false refusals in particular models, suggesting that safety tuning has fairness implications that should be audited per model rather than assumed from persona labels.
Reading between the lines
- An interaction between persona and model generation may be the real story: the same persona label moves refusal rates strongly in older models and barely at all in newer ones, so a single 'persona effect' number will be misleading without specifying the model generation.
- The string-match refusal detector could be tested directly against human annotation; if modern models refuse in longer, hedged forms that do not start with the listed phrases, the reported rates and the ranking of factors would need revision, although the paper argues any such miss is likely evenly distributed.
- The same Monte Carlo plus Wasserstein sensitivity pipeline could be transferred to other outcome variables, such as answer accuracy, sycophancy, or formatting compliance, to separate which experimental controls actually drive behavior.
- The logistic regression's persona coefficients are main effects only; whether Black, Muslim, and transgender personas remain the top-refusal groups in multimodal, larger, or proprietary models is an open empirical question.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a large-scale controlled study of false refusals in open-weight LLMs. The authors prompt 16 models with 15 sociodemographic personas across three classification tasks (NLI, politeness, offensiveness) and nine prompt paraphrases, using a Monte Carlo sampling scheme to keep the evaluation feasible. They quantify the contribution of model, task, persona, and prompt to false refusal rates using a Wasserstein-distance sensitivity index and a logistic regression. The central finding is that model choice is the most influential factor, followed by task, then persona, then prompt; the authors conclude that persona effects on false refusal have been overestimated in prior work, while model safety behavior and sensitive-task content dominate. They also report that newer Llama/Qwen models refuse less, while Gemma2 models show increased refusal rates, and that certain personas (Black, White, transgender, Muslim) are associated with more refusals in some models.
Significance. If the results hold, the paper provides a valuable corrective to the existing persona-refusal literature: it controls for several confounds simultaneously and quantifies their relative importance rather than reporting pairwise persona differences. The Monte Carlo sampling design and the use of a global sensitivity measure are appropriate for the combinatorial search space, and the inclusion of 16 models and three tasks is a strength. The main quantitative conclusion—model and task dominate persona—is plausible and would be practically useful for researchers designing fairness or safety evaluations. However, the paper's central claim is only as strong as its refusal detector, and the detector is neither validated nor released; this, together with the lack of code/data and uncertainty quantification, currently limits the confidence that can be placed in the ranking. The paper also makes a testable claim about the distribution of detector misses, but does not test it.
major comments (3)
- [§3.3 and Appendix C] The refusal detector is load-bearing for the paper's central claim, and it has both false-negative and false-positive risks. It flags only responses beginning with a fixed list (e.g., 'sorry', 'i cannot', 'as an ai') as refusals; any refusal phrased differently (e.g., 'I'd rather not', 'This cannot be classified') is missed. Conversely, the list includes 'no,' and 'no.' as refusal markers, so a legitimate negative answer such as 'No, a man would not label this as offensive' would be counted as a refusal. The Limitations section concedes that refusal rates 'might thus be marginally higher than reported, but likely to be evenly enough distributed to not change results,' but this is asserted, not demonstrated. Since output format and phrasing interact with prompt strictness (unforced vs. semi-forced/forced) and with model generation style, differential misclassification across models or per
- [§3.1–3.2] The Monte Carlo sampling procedure is described only at a high level. The authors say they 'sample at random' tuples of (prompt template, persona) and 'ensure a balanced distribution,' but they do not provide the exact algorithm, the number of samples drawn per combination, the random seed, or any pseudocode. Without this information, and without code or data release, the reported refusal rates and sensitivity indices cannot be reproduced. The paper's reproducibility claim would be substantially strengthened by releasing the sampling code, the generated prompts, and the model outputs (or an anonymized subset), or at least by specifying the full procedure in an appendix.
- [§4.1, Table 1 and §4.6.1, Figure 5] The refusal rates and the Wasserstein importance indices are reported as point estimates without confidence intervals. The paper's ranking (model > task > persona > prompt) is a comparative claim, and it is not shown that the differences between adjacent factors, especially persona vs. prompt, are statistically distinguishable. The authors should provide bootstrap or exact binomial confidence intervals for the refusal rates in Table 1 and for the importance indices in Figure 5. This is particularly relevant for near-zero rates (e.g., many models in the NLI condition), where small absolute differences can be driven by a handful of responses.
minor comments (5)
- [§4.4] The text states that false refusal rates are 'much higher in the persona setup (14.68%),' but the persona-free average is not reported in the text. Please give both numbers and the delta explicitly, or refer to the figure with the actual values.
- [Table 7] The prompt coefficient 'pp5' appears to be a typo for 'p5'; the prompt identifiers are otherwise p1–p9. Please align the labels between the table and the appendix prompt templates.
- [Appendix D.2] The caption says 'Figures 7, 8 show the variation...' but it is referring to Figures 10 and 11. The cross-reference is wrong.
- [Author affiliations] The affiliation for Emanuele Borgonovo is spelled 'Boconni University' in the author block; it should be 'Bocconi University'.
- [Figure 6 / §4.6.2] The text says 'Llama3.8B' when referring to Llama3-8B. Please fix the model name for consistency with Table 1.
Circularity Check
No significant circularity: the refusal-rate measurements and importance rankings are empirical outputs of a stated detector and statistical tools, not re-statements of their inputs.
full rationale
The paper is an empirical measurement study. Refusal is operationalized by the explicit prefix-string detector in Appendix C; the Wasserstein and logistic-regression analyses in §4.6 are applied to the resulting binary labels. There is no equation in which the reported importance ranking is defined in terms of itself: the detector's prefix list is fixed before the experiments and does not encode model/task/persona/prompt as factors. The sensitivity measure from Borgonovo et al. is cited from prior work by co-authors, but it is a generic statistical tool with stated formulas, not a uniqueness theorem or an ansatz that pre-selects the conclusion. The prompt typology from Röttger et al. (2024a) is an experimental design choice, and the finding that unforced prompts elicit more refusals is an empirical output, not an assumption. The Limitations section's caveat that the detector 'might not have identified the full range of refusals' is a measurement-validity concern; it does not make the ranking circular, because the analysis would be equally well-defined for any detector and the conclusion is not logically entailed by the detector's construction. No fitted parameter is relabeled as a prediction. Overall, the central claims are self-contained relative to their operational definitions.
Assumptions & free parameters
free parameters (3)
- Number of persona/prompt variations per input sample =
10
- Number of samples per class in each task =
1000
- Logistic regression regularization strength
assumptions (4)
- domain assumption Refusals are detectable by a fixed list of starting phrases such as "I'm sorry" or "I cannot".
- domain assumption The three tasks (NLI, politeness, offensiveness) are representative of safe classification tasks where false refusals are expected to be low.
- domain assumption Temperature-0 greedy decoding is deterministic and adequate for measuring refusal behavior.
- domain assumption The Monte Carlo sampling scheme yields an unbiased, balanced estimate of factor importance.
Cite this review
Pith. "Pith review of No for Some, Yes for Others: Persona Prompts and Other Sources of False Refusal in Language Models." pith.science (2026). https://pith.science/paper/ZHE74RHF
@misc{pith2026250908075,
author = {Pith},
title = {Pith review of: No for Some, Yes for Others: Persona Prompts and Other Sources of False Refusal in Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZHE74RHF}},
note = {Machine review of arXiv:2509.08075}
}
read the original abstract
Large language models (LLMs) are increasingly integrated into our daily lives and personalized. However, LLM personalization might also increase unintended side effects. Recent work suggests that persona prompting can lead models to falsely refuse user requests. However, no work has fully quantified the extent of this issue. To address this gap, we measure the impact of 15 sociodemographic personas (based on gender, race, religion, and disability) on false refusal. To control for other factors, we also test 16 different models, 3 tasks (Natural Language Inference, politeness, and offensiveness classification), and nine prompt paraphrases. We propose a Monte Carlo-based method to quantify this issue in a sample-efficient manner. Our results show that as models become more capable, personas impact the refusal rate less and less. Certain sociodemographic personas increase false refusal in some models, which suggests underlying biases in the alignment strategies or safety mechanisms. However, we find that the model choice and task significantly influence false refusals, especially in sensitive content tasks. Our findings suggest that persona effects have been overestimated, and might be due to other factors.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card
2024
-
[2]
Bang An, Sicheng Zhu, Ruiyi Zhang, Michael-Andrei Panaitescu-Liess, Yuancheng Xu, and Furong Huang. 2024. Automatic pseudo-harmful prompt generation for evaluating false refusals in large language models. In ICML 2024 Next Generation of AI Safety Workshop
work page 2024
-
[3]
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609
arXiv 2023
-
[4]
Emanuele Borgonovo, Alessio Figalli, Elmar Plischke, and Giuseppe Savar \'e . 2024. Global sensitivity analysis via optimal transport. Management Science. Online First
work page 2024
-
[5]
Emanuele Borgonovo, Valentina Ghidini, Roman Hahn, and Elmar Plischke. 2023. Classifier explainability with measures of statistical association. Computational Statistics and Data Analysis, 182:197701/1--16
work page 2023
-
[6]
Emanuele Borgonovo, Gordon B. Hazen, and Elmar Plischke. 2016. A common rationale for global sensitivity measures and their estimation. Risk Analysis, 36(10):1871--1895
work page 2016
-
[7]
Khaoula Chehbouni, Megha Roshan, Emmanuel Ma, Futian Wei, Afaf Taik, Jackie Cheung, and Golnoosh Farnadi. 2024. https://doi.org/10.18653/v1/2024.findings-acl.927 From representational harms to quality-of-service harms: A case study on llama 2 safety safeguards . In Findings of the Association for Computational Linguistics: ACL 2024, pages 15694--15710, Ba...
-
[8]
Yongxin Chen, Tryphon T. Georgiou, and Michele Pavon. 2021. Stochastic control liaisons: R ichard S inkhorn meets G aspard M onge on a S chr \"o dinger bridge. SIAM Review, 63(2):249--313
work page 2021
Show all 34 references
-
[9]
Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. https://doi.org/10.18653/v1/D18-1269 XNLI : Evaluating cross-lingual sentence representations . In Proceedings of the 2018 Conference on Empirical Methods ...
2018 doi
-
[10]
Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. 2024. Or-bench: An over-refusal benchmark for large language models. arXiv preprint arXiv:2405.20947
2024 arXiv
-
[11]
Pedro Henrique Luz de Araujo and Benjamin Roth. 2024. Helpful assistant or fruitful facilitator? investigating how personas affect language model behavior. arXiv preprint arXiv:2407.02099
2024 arXiv
-
[12]
Prannaya Gupta, Le Qi Yau, Hao Han Low, I-Shiang Lee, Hugo Maximus Lim, Yu Xin Teoh, Koh Jia Hng, Dar Win Liew, Rishabh Bhardwaj, Rajat Bhardwaj, and Soujanya Poria. 2024 a . https://doi.org/10.18653/v1/2024.emnlp-demo.42 W alled E val: A comprehensive safety evaluation toolki...
2024 doi
-
[13]
Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, and Tushar Khot. 2024 b . Bias R uns D eep: Implicit reasoning biases in persona-assigned LLM s. In The Twelfth International Conference on Learning Representations
2024
-
[14]
Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2023. Evaluating and inducing personality in pre-trained language models. Advances in Neural Information Processing Systems, 36:10622--10643
2023
-
[15]
Meta. 2024. https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/ Llama 3.2: Revolutionizing edge ai and vision with open, customizable models
2024
-
[16]
Jiaxin Pei and David Jurgens. 2023. https://doi.org/10.18653/v1/2023.law-1.25 When do annotator demographics matter? measuring the influence of annotator demographics with the POPQUORN dataset . In Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII), pages 252--2...
2023 doi
-
[17]
Flor Miriam Plaza-del-Arco , Amanda Cercas Curry, Susanna Paoli, Alba Cercas Curry, and Dirk Hovy. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.251 Divine LL a MA s: Bias, stereotypes, stigmatization, and emotion representation of religion in large language models . I...
2024 doi
-
[18]
Omid Rafieian and Hema Yoganarasimhan. 2023. Ai and personalization. Artificial Intelligence in Marketing, pages 77--102
2023
-
[19]
Paul R \"o ttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.816 Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large ...
2024 doi
-
[20]
Paul R \"o ttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024 b . https://doi.org/10.18653/v1/2024.naacl-long.301 XST est: A test suite for identifying exaggerated safety behaviours in large language models . In Proceedings of the 2024...
2024 doi
-
[21]
Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024. https://doi.org/10.18653/v1/2024.acl-long.399 L a MP : When large language models meet personalization . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu...
2024 doi
-
[22]
Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems, 36:51778--51809
2023
-
[23]
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324
2023 arXiv
-
[24]
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi \`e re, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024 a . Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295
2024 arXiv
-
[25]
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , and 1 others. 2024 b . Gemma 2: Improving open language models at a practical size. arXiv preprint arX...
2024 arXiv
-
[26]
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288
2023 arXiv
-
[27]
C \'e dric Villani. 2009. Optimal Transport: Old and New. Springer Verlag, Berlin
2009
-
[28]
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024 a . Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191
2024 arXiv
-
[29]
Xinpeng Wang, Chengzhi Hu, Paul R \"o ttger, and Barbara Plank. 2024 b . Surgical, cheap, and flexible: Mitigating false refusal in language models via single vector ablation. arXiv preprint arXiv:2410.03415
2024 arXiv
-
[30]
Johannes C. W. Wiesel. 2022. Measuring association with W asserstein distances. Bernoulli, 28(4):2816--2832
2022
-
[31]
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computation...
2018 doi
-
[32]
Zhehao Zhang, Ryan A Rossi, Branislav Kveton, Yijia Shao, Diyi Yang, Hamed Zamani, Franck Dernoncourt, Joe Barrow, Tong Yu, Sungchul Kim, and 1 others. 2024. Personalization of large language models: A survey. arXiv preprint arXiv:2411.00027
2024 arXiv
-
[33]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[34]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.