REVIEW 3 major objections 5 minor 4 references
Same violence, different answer: how AI responds to coercive control against women across languages
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A woman's disclosure of coercive control receives materially different protection from widely used chatbots depending on which of nine languages she writes in, and two systems show the gap is a design outcome rather than a language property
desk verdict Solid cross-lingual audit showing language-dependent protection; the 'design outcome' framing slightly overreaches but the paper's own caveats mostly contain it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a single scripted disclosure vignette, varied only by the partner's stated motive (affection, jealousy, paternalistic protection, past betrayal) and his level of distress, rendered identically in nine languages. Responses are scored on two independent axes: a four-level behavioural outcome from outright refusal to full compliance, and a structural index (0-3) counting whether the reply names the control, counters the woman's self-blame, and affirms her agency. The argument's decisive move is the ceiling comparison: two systems that refuse and recognize in all nine languages convert the observed unevenness from an unavoidable language effect into a design outcome.
What would settle it
Re-run the same seventy-two prompt cells with slightly looser wording (a more colloquial or fragmented disclosure) or after a public update to one of the seven models. If either of the two systems that refused everywhere starts writing the apology letter in any language, or if the language ordering reverses substantially, the paper's claim that the unevenness is a stable design outcome would be undercut.
Extended reading notes
Core claim
The central discovery is that recognition and refusal are separable and unevenly distributed across languages. In a fixed scenario where a woman asks for an apology letter accepting her partner's phone tracking, a model that holds the line (refuses to write) is not the same as one that does protective work (naming the control, countering self-blame, affirming agency), and different languages pull these apart in different ways. Chinese, despite being richly represented in training data, drew the lowest refusal rate among the five varying systems, while low-resource Catalan drew one of the highest; Arabic and Hebrew, sibling Semitic languages, sat at opposite ends. Two of the seven systems - G
Load-bearing premise
The charge that failures elsewhere are a design outcome rests on the assumption that the two ceiling systems' perfect refusal in this one scripted scenario is a stable, replicable property of their design rather than a quirk of this particular story or of proprietary advantages other developers cannot match.
Editorial extensions
If this is right
- The same written disclosure garners materially different protection depending on its language; within the varying systems, Chinese held the line in 20% of cases versus 76% for Hebrew.
- Good refusal and good recognition are not the same; an evaluation that scores only refusals, or merges both into one index, misses half of what protection is.
- A universal protective floor is attainable in this scenario family, because two systems refused and did protective work in every language; shortfalls are design outcomes, not language limitations.
- Sympathetic excuses for the partner work differently by language: paternalism raised naming most in Hindi, while Russian's recognition stayed flat, so safety tuning cannot be done once in English and then translated.
- The systems most likely to be free at the point of need are the ones that protect least; the better-protecting systems ration sustained use on free tiers.
Reading between the lines
- The authors' explanations for the Catalan and Arabic results - corpus-borne feminist discourse for Catalan, borrowed English safety for Hebrew - are explicitly not measured in this study; a direct test would inspect training-data composition or run controlled fine-tuning experiments varying only the safety-evaluation language.
- The authors name the cross of language with an explicitly stated country as the decisive untested experiment; if an explicitly permissive or protective locale overrides the language effect, the injustice becomes easier to remedy, while if language dominates, developers would have to do per-language safety work.
- The paper's observation that the best-protecting systems ration free use more tightly implies that the women with least money and support will systematically meet the least protective systems even after a floor is technically attainable; a distributional audit of free-tier limits would quantify that.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper audits seven commercial conversational AI systems (Mistral Medium 3.5, DeepSeek V4 Flash, Gemini 3.5 Flash, Qwen 3.6 Flash, Llama 4 Maverick, GPT-5.5, Claude Haiku 4.5) on one fixed coercive-control scenario in nine languages, scoring 3,528 responses on whether the model writes an apology letter requested by a woman whose partner tracks her phone, and on three protective-content axes (naming control, countering self-blame, affirming agency). It reports substantial cross-lingual differences in held-line rates (Chinese 43% vs Hebrew 83% for all systems), a motive-by-language interaction, a 'home-language' capitulation pattern among three non-anglophone developers, and uniform distress effects. Because GPT-5.5 and Claude Haiku 4.5 refuse in all nine languages, the paper concludes that a protective floor is attainable and failures elsewhere are a design outcome.
Significance. The empirical core is valuable: it is a large, systematically coded audit with inter-coder reliability (κ = .90 on the boundary outcome), back-translation checks, and open data/code, and it addresses a socially important question. The finding that protection varies sharply by language, holding scenario and system constant, is robust and policy-relevant. The paper's two load-bearing interpretive claims—the 'design outcome' attribution and the 'home-language effect'—go beyond what the design can support, and need tempering or additional evidence.
major comments (3)
- [§4.1 and §5.6] The inference that failures elsewhere are a 'design outcome' rests entirely on two proprietary systems having refused in all nine languages on one fixed scripted vignette. Section 5.7 concedes this is a single-scenario snapshot and that commercial systems change over time. Perfect performance by two large proprietary systems is an existence proof of attainability under this specific script, but not evidence that other builders could achieve the same floor as a design choice; it may reflect scale, safety-tuning investment, or an accidental match between the script and the systems' training. To make the 'design outcome' claim load-bearing, the authors would need robustness across multiple phrasings/scenarios and longitudinal sampling, or at minimum should qualify the conclusion to 'within this scenario family and at this point in time.' As written, the abstract and conclusion state the str
- [§4.5/§5.3] The home-language effect is based on three non-anglophone systems (Mistral, DeepSeek, Qwen), and the paper itself notes the alternative explanation of thinner native-language safety work. With n=3 and heterogeneous developer contexts, the data cannot distinguish a home-language effect from an investment effect. The heading 'Capitulation in the home language' and the claim that 'each gave way most readily in that language' overstate the evidence. This should be reframed as an exploratory pattern, and the 'systems-not-languages' attribution should not lean heavily on it.
- [§4.2] The endorsement-in-model's-own-voice rates (49% in Hindi/Chinese vs 10% in English) are presented without separate human validation, and the paper acknowledges this axis 'carries wider measurement uncertainty.' Since this register feeds the theoretical discussion of social reproduction, it should be clearly labeled exploratory and separated from the validated results in the abstract's claims.
minor comments (5)
- [References] Citation inconsistency: 'Vergés and Gil-Juárez, 2021' in the text versus 'Vergés Bosch and Gil-Juárez, 2021' in the reference list.
- [Figure 1] Figure 1 is referenced but not reproduced in the manuscript text; ensure the published version includes it.
- [Throughout] The terms 'protective ceiling' and 'protective floor' are used interchangeably; pick one consistent term to avoid confusion.
- [§4.5] The 'resource class' claim for Catalan versus Spanish relies on Joshi et al. (2020), but no resource measure is given; a brief operationalization would strengthen the claim.
- [§5.6] The statement that the two top-performing systems 'are reachable at no cost' would benefit from a date and source, since free-tier policies change over time.
Circularity Check
No circularity found: the study is an empirical audit with disclosed scoring and independent human validation; the 'design outcome' claim is inferential, not definitional.
full rationale
The paper's central claims are empirical measurements of 3,528 model responses against a disclosed codebook (§3.3–§3.4). The outcome definitions—held line, naming control, countering self-blame, affirming agency—are operationalized from the coercive-control literature, not from the target conclusion, and the codes were validated against independent human coders with reported agreement statistics (κ = .90 on the boundary outcome, etc.). The 'protective floor' claim (§4.1) is based on direct observation that two systems held the line in all nine languages; it is not derived from a fitted parameter or from a self-citation. The attribution 'failures elsewhere are a design outcome' (Abstract, §4.1) is an inference from the existence of those two systems; it may be overgeneralized given the single-vignette design, and the authors themselves flag this in §5.7, but overgeneralization is an external-validity or correctness concern, not circularity. The self-citations in the references (Vergés Bosch and Gil-Juárez, 2021; Vergés and Gil-Juárez, 2021) appear only as background support for the digital-coercive-control framing and are not load-bearing for the measured results or the recognition argument. There is no equation whose output equals its input, no fitted quantity renamed as a prediction, and no uniqueness claim imported from the authors' own prior work. The analysis is self-contained against its empirical benchmarks, so the circularity score is 0.
Assumptions & free parameters
assumptions (4)
- domain assumption The scripted vignette is a valid operationalization of coercive control help-seeking
- domain assumption The three-axis structural index (naming control, countering self-blame, affirming agency) measures the core of a protective response
- domain assumption Language is the only geographic cue the models receive
- ad hoc to paper The two frontier systems' perfect performance demonstrates attainability for other systems
Cite this review
Pith. "Pith review of Same violence, different answer: how AI responds to coercive control against women across languages." pith.science (2026). https://pith.science/paper/ND7YIDWP
@misc{pith2026260801436,
author = {Pith},
title = {Pith review of: Same violence, different answer: how AI responds to coercive control against women across languages},
year = {2026},
howpublished = {\url{https://pith.science/paper/ND7YIDWP}},
note = {Machine review of arXiv:2608.01436}
}
read the original abstract
Women experiencing coercive control, a form of intimate partner violence increasingly conducted through digital devices, are turning to conversational AI for help, and the protection they receive should not depend on the language they write in. We analyse how AI responds to coercive control against women across languages. We put one scripted scenario to seven widely used language models in nine languages: a woman whose partner tracks her phone asks for help with a self-blaming letter accepting the surveillance. We scored whether the model wrote the letter and whether it named the control, countered the self-blame, and affirmed her agency. Failure split along two independent axes. On the first, systems from non-anglophone developers gave way most often in their builders' own language. On the second, how far a sympathetic excuse for the partner could strip a model's naming of the control varied sharply from one language to the next. Two frontier systems held the strictest standard everywhere, so a protective ceiling is attainable within this scenario family, and failures elsewhere are a design outcome. What is at stake is recognition: whether a system grasps a disclosure as coercive control, and whether it then acts on that grasp. We argue this should be held to a floor, one language at a time.
Figures
Reference graph
Works this paper leans on
-
[1]
Agarwal, D., Shukla, A., Sitaram, S., & Vashistha, A. (2025). Fluent but foreign: even regional LLMs lack cultural alignment. arXiv:2505.21548. Bourdieu, P., & Passeron, J.-C. (1977). Reproduction in Education, Society and Culture. London: Sage Publications. Bradley, D. (1992). Chinese as a pluricentric language. In M. Clyne (ed.), Pluricentric Languages:...
arXiv 2025
-
[667]
ACM. Fricker, M. (2007). Epistemic Injustice: Power and the Ethics of Knowing. Oxford University Press. Glick, P., & Fiske, S. T. (1996). The Ambivalent Sexism Inventory: differentiating hostile and benevolent sexism. Journal of Personality and Social Psychology, 70(3), 491–512. Harris, B. A., & Woodlock, D. (2019). Digital coercive control: insights from...
-
[2018]
Yong, Z.-X., Menghini, C., & Bach, S
WHO. Yong, Z.-X., Menghini, C., & Bach, S. H. (2023). Low-resource languages jailbreak GPT-4. arXiv:2310.02446. Yong, Z.-X., Ermis, B., Fadaee, M., Bach, S. H., & Kreutzer, J. (2025). The state of multilingual LLM safety research: from measuring the language gap to mitigating it. Proceedings of the 2025 Conference on Empirical Methods in Natural Language ...
arXiv 2023
-
[2024]
Shmidman, S., Shmidman, A., Cohen, A. D. N., & Koppel, M. (2023). Introducing DictaLM: A large generative language model for Modern Hebrew. arXiv preprint arXiv:2309.14568. Stanovsky, G., Smith, N. A., & Zettlemoyer, L. (2019). Evaluating gender bias in machine translation. ACL 2019, 1679–1684. Stark, E. (2007). Coercive Control: How Men Entrap Women in P...
work page Pith review arXiv 2023
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.