{"id":"dfc80b30-8740-499b-8e46-3f8f6fad7b6f","arxiv_id":"2611.10808","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"LLMs transform identity disclosures into explanatory narratives for distress, particularly for marginalized identities, a pattern the paper calls 'compassionate othering'.","lead":"Large language models sometimes assume that a user's social identity is the cause of their distress, even when the user never says so. This matters for AI mental health support, because such assumptions can misrepresent a person's experience while appearing empathetic.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"No significant objection identified.","rationale":"The reader identified the reliance on an LLM judge for distress attribution as the weakest assumption, which aligns with the paper's own stated limitation (b223). This is indeed the most vulnerable point, but the paper's validation (50 responses, 98% agreement) and the qualitative evidence provide reasonable support. No other methodological issues—such as the counterfactual intervention, the choice of identities, or the statistical tests—rise to the level of a load-bearing concern. The paper is well-executed, transparent about its limitations, and the central claim is appropriately scoped. Therefore the reader's ACCEPT verdict stands.","tokens_in":28483,"tokens_out":1679,"duration_ms":17249,"concrete_test":"Re-run the distress-attribution analysis using three independent human annotators on a random sample of 500 responses (oversampled for marginalized disclosures). Compute inter-annotator agreement (Fleiss' kappa) and compare the model-level attribution rates against the LLM judge's labels. If the pattern of disproportionate attribution holds with human annotation, the central claim is further validated; if not, the LLM judge may have introduced systematic bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that LLMs disproportionately attribute distress to marginalized identities—rests on the output of a single LLM judge (GPT-5-nano) with human validation on only 50 responses. While the reported 98% agreement is high, the small validation sample and the acknowledged ambiguity in classifying 'distress attribution' (b223) mean that systematic misclassification cannot be fully ruled out. However, the paper transparently discusses this limitation, and the qualitative examples (Table 1) provide convergent evidence that the effect exists. No other load-bearing concern undermines the argument. The controlled counterfactual design, multiple model replications, and consistency across datasets strengthen the conclusion. The reader's weakest assumption correctly identifies the most sensitive point, and the paper handles it adequately for an exploratory audit.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper conducts a controlled counterfactual audit of six LLMs responding to suicidal ideation and distress posts, varying identity disclosures while holding post content constant. It finds that although conventional safety measures (refusal, crisis resource provision, interactional risks) show minimal differences across identity groups, models disproportionately attribute distress to the disclosed identity when that identity is marginalized, introducing unsupported causal explanations. The authors interpret this pattern through the lens of 'compassionate othering' and discuss implications for identity-responsive AI support.","tokens_in":28695,"tokens_out":2898,"duration_ms":28469,"significance":"The work identifies a subtle but distinct form of harm in LLM mental-health support—unsupported identity-linked explanations—that conventional safety evaluations miss. The paired counterfactual design, six-model replication across two datasets, and human validation of the automated judge strengthen the empirical foundation. If the result holds, it has direct implications for auditing frameworks, model deployment guidelines, and the design of identity-responsive support that remains grounded in individuals' own accounts.","major_comments":[{"comment":"The no-disclosure baseline (\"No identity None\") shows non-zero distress attribution for several models (e.g., Gemini-3.8-Flash 1.7%, Qwen3.5-9B 1.1% on Reddit SI vanilla). Since no identity is disclosed, the definition of distress attribution requires a disclosed identity; these responses should be zero by construction. This suggests the LLM judge may be labeling other features as identity attribution, or the model introduces identity references without a disclosed identity. The authors must clarify what the judge labels in these cases and either exclude such instances or provide a corrected analysis, as the pattern threatens the validity of the primary measure.","section":"Table A4 & §3.3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The human validation of the distress-attribution judge covers only 50 responses (24/25 flagged, 25/25 unflagged). Although the authors acknowledge this limitation, the central quantitative claim rests on this measure, and a 50-sample check (even with 98% agreement) is too thin to rule out systematic misclassification. I recommend either (a) a larger human validation (e.g., 200+ responses drawn randomly, not stratified by the judge's label, with Cohen's kappa reported) or (b) a secondary analysis using an alternative operationalization (e.g., human annotation of a subset or keyword-based detection of identity-linking phrases) to corroborate the main results.","section":"§3.3, §b223","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The qualitative examples in Table 1 are compelling but selective. The paper would benefit from a systematic analysis of how often the model's explanation changes between conditions—for example, measuring semantic similarity between the marginalized-disclosure response and the majority/no-disclosure response, or counting how frequently the stated cause (e.g., 'financial worries') is replaced by an identity-linked phrase. Such an analysis would complement the judge-based attribution measure and strengthen the claim that models substitute identity-based causes for the writer's stated circumstances.","section":"Table 1 & §4.2","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The x-axis label 'Percentage points' is ambiguous. Clarify that these are differences from each post's no-disclosure baseline, not absolute percentages.","section":"Figure 1 caption","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paper uses 'RubRIX harm' in the figure label but 'interactional risk' in the text. Use consistent terminology throughout.","section":"§4.1 and Figure 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The Appendix (§7.2) mentions Cohen's h as an effect-size measure but does not report it in the main results or figures. Consider adding a brief effect-size summary (e.g., range of Cohen's h for significant comparisons) to aid interpretation of the small but significant differences.","section":"§3.3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The note 'Christian excludes responses that read the word as a name' is helpful but should also appear in the main-text description of results (e.g., §4.2) to avoid confusion.","section":"Table A4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The limitation that 'judgments about whether a response attributes distress to identity can remain ambiguous' is well stated. Consider adding a concrete example of a borderline case (from the judge's training or validation) to illustrate the ambiguity and how the judge was instructed to resolve it.","section":"§5.4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The citation to [15] on LGBTQIA+ identity disclosures is relevant, but the paper's contribution over [15] should be stated more explicitly: this work extends the finding to multiple identities (rather than only LGBTQIA+), multiple models, and includes a clinician-guideline condition.","section":"§2.2","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"The central claim holds up: when a marginalized identity is disclosed in a distress post, several LLMs disproportionately attribute the distress to that identity, even though the post doesn't connect the two. The effect is modest in absolute terms (single-digit percentages for most model–identity pairs) but consistent across models and datasets, and it’s stark for immigrant disclosures with Gemini-3.8-Flash (14–34%). The paper’s main contribution is empirical and conceptual: it documents a pattern that prior work on LLM mental health disparities has largely missed, and it names it via the compassionate-othering lens from medical education.\n\nThe methodology is well-executed. The counterfactual design—varying just the identity sentence while holding the post constant—is clean. The paper tests six models, two datasets, and two prompting protocols, with human validation of the LLM judge on 50 responses (98% agreement). The authors are transparent about their limitations and do not overclaim. The qualitative examples in Table 1 are genuinely striking: the same model response swaps “financial worries” for “your immigration status” depending on the disclosure.\n\nSoft spots: the LLM judge validation is thin (50 responses) for an outcome as subtle as distress attribution. The authors acknowledge the ambiguity (b223), and the qualitative examples provide convergent evidence, but I would have liked a larger validation set, especially to check for systematic misclassification on ambiguous cases. The effect sizes are concentrated in a few model–identity pairs—most models show very low attribution rates, and the aggregate picture is that safety behavior (refusal, crisis resources) barely changes. That actually strengthens the paper’s argument that the harm is interpretative rather than overtly discriminatory.\n\nThis paper is for the AI safety and fairness auditing community, and for anyone building or evaluating mental-health chatbots. It deserves a serious referee. I would accept it, with a minor request to expand the judge validation or to discuss what fraction of flagged attributions might be plausible rather than unsupported. But the core result is real and the paper is honest.\n\nI would cite this as evidence of a overlooked dimension of harm in LLM responses, and I'd bring it to a reading group.","headline":"Solid audit of a subtle harm—LLMs make unsupported identity-attributions in distress responses, but the finding is real and well-supported.","tokens_in":29185,"tokens_out":1064,"would_cite":true,"duration_ms":10302,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Large language models routinely attribute distress to a disclosed marginalized identity even when the writer says nothing connecting the two.","keywords":["large language models","mental health","identity disclosure","distress attribution","compassionate othering","counterfactual audit","bias","suicidal ideation"],"falsifier":"A replication using human raters to judge distress attribution on the same posts and responses would determine whether the automated judge inflates or deflates the measured disparity; alternatively, a user study where participants from target communities rate whether they perceive responses as presumptuous would test the real-world harm.","tokens_in":29989,"feed_emoji":"⚠️","tokens_out":3893,"duration_ms":36555,"temperature":0.7,"pith_summary":"This paper establishes that when a person discloses a marginalized identity in a post expressing distress or suicidal ideation, several large language models (LLMs) disproportionately treat that identity as a cause of the distress, even though the post itself contains no such connection. The effect is large for some models and identities: immigrant disclosures trigger attribution rates up to 34% for Gemini-3.8-Flash and 20% for Qwen3.5-9B. Conventional safety measures like refusal and crisis resource provision show little difference across identity groups, meaning standard evaluations could miss this pattern. The authors term this 'compassionate othering' to describe how empathetic-sounding responses impose identity-linked explanations that the individual never provided.","feed_headline":"LLMs wrongly blame distress on disclosed identity","feed_subtitle":"Models shift explanations from stated circumstances to presumed identity-linked adversity.","key_machinery":"The central mechanism is a counterfactual audit design: each post from Reddit SI and SH-Detection is prepended with a one-sentence identity disclosure ('I'm [identity].') while the post text itself is held constant. Responses are evaluated by an LLM judge (GPT-5-nano) on multiple dimensions, with the primary measure being 'distress attribution'—whether the response presumes the disclosed identity is a cause of the writer's distress. The authors also introduce the concept of 'compassionate othering' from medical education literature to characterize the harm of imposing identity-linked narratives within otherwise empathetic language.","core_discovery":"The paper's central discovery is that LLMs can construct explanatory narratives around a disclosed social identity when responding to distress, even when the writer's own account of their circumstances does not connect the two. Using counterfactual interventions where a single identity sentence is prepended to unchanged posts, the authors find that marginalized identity disclosures elevate 'distress attribution'—the model presuming identity is a cause of distress—compared with majority identity disclosures or no disclosure. This disparity persists across two datasets and six models, with the strongest effects for immigrant, trans, and Black identities. The findings hold while safety behaviors (refusal, risk recognition, crisis resource provision) remain nearly identical across disclosure conditions, revealing a dimension of harm that standard safety evaluations do not capture.","pith_inferences":["The paper's findings suggest that current fine-tuning for safety and empathy may reduce overt toxicity but not subtle interpretive biases, which could be more common and harder to detect; post-deployment monitoring should include qualitative analysis of how models frame users' identities.","An immediate testable extension is a controlled user study where participants from marginalized groups review model responses and rate whether they feel their experiences are accurately represented or presumptuously explained, providing evidence to ground mitigation strategies.","The pattern might generalize to other personal information beyond social identity, such as medical conditions or life events; future work could examine whether disclosing a diagnosis or hardship similarly redirects models' explanations.","The paper's reliance on single-turn responses leaves open whether users would correct or clarify the model's attribution in follow-up turns, which would be a natural way to distinguish harmful presumption from helpful exploration."],"forward_implications":["Evaluations of AI mental health support must include checks for unsupported identity-linked explanations, as standard safety metrics (refusal, risk recognition, crisis resources) do not capture this form of harm.","Reducing demographic disparities in refusal or interactional risk does not guarantee equitable treatment; models may still impose identity-based interpretations that misrepresent marginalized users' experiences.","Structured response protocols that reduce distress attribution (like the clinician-guideline prompt) also reduce identity engagement more broadly, suggesting a trade-off that requires careful calibration.","Developers should audit for unsupported identity attribution before deployment and after model updates, and incorporate user feedback to distinguish contextually appropriate personalization from harmful generalization.","The pattern of compassionate othering may erode trust and cause therapeutic alliance ruptures in ongoing interactions, particularly during disclosures of suicidal ideation where continued engagement is critical."],"supporting_citations":[{"why":"Callihoo et al. 2026: Documents unsupported assumptions in responses to LGBTQIA+ identity disclosures, which this paper extends to a broader set of identities and distress contexts.","marker":"[15]"},{"why":"Dutta et al. 2026: Audits LLM responses to mental-health-related stereotypes, showing discriminatory responses overlooked by moderation, providing a parallel finding.","marker":"[32]"},{"why":"Kantharuban et al. 2025: Shows that disclosed user identity biases chatbot recommendations, supporting the premise that LLMs use identity cues inappropriately.","marker":"[53]"},{"why":"Bauer et al. 2024: Introduces 'compassionate othering' as a concept in medical education, which this paper adapts to characterize model behavior.","marker":"[11]"},{"why":"Alcoff 1991: Provides the theory of 'speaking for others' used to frame the harm of displacing the writer's account.","marker":"[4]"},{"why":"Gabriel et al. 2024: Finds demographic disparities in LLM mental-health responses, motivating examination of identity-handling even when aggregate safety metrics appear equitable.","marker":"[36]"}],"fun_headline_variants":["LLMs misattribute distress to disclosed identities","When empathy assumes identity causes suffering","AI constructs false narratives from disclosed identities","Marginalized identity disclosures trigger false blame by LLMs","LLMs link distress to identity, even when users don't"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The paper's primary measure of distress attribution relies on an LLM judge (GPT-5-nano) to decide whether a response assumes identity is a cause of distress, which can be ambiguous when models suggest plausible associations without asserting causality.","fun_headline_variants_meta":{"raw":{"variants":["LLMs misattribute distress to disclosed identities","When empathy assumes identity causes suffering","AI constructs false narratives from disclosed identities","Marginalized identity disclosures trigger false blame by LLMs","LLMs link distress to identity, even when users don't"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":5e-05,"raw_usage":{"cost":4.9682304e-05,"is_byok":false,"cost_details":{"upstream_inference_cost":4.9682304e-05,"upstream_inference_prompt_cost":2.7022464e-05,"upstream_inference_completions_cost":2.265984e-05},"total_tokens":1156,"prompt_tokens":875,"completion_tokens":281,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":210}},"tokens_in":875,"tokens_out":281,"cost_source":"openrouter_usage","duration_ms":3432,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":210,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-10T21:02:37.816531+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"falsifier":"A replication using human raters to judge distress attribution on the same posts and responses would determine whether the automated judge inflates or deflates the measured disparity; alternatively, a user study where participants from target communities rate whether they perceive responses as presumptuous would test the real-world harm.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Callihoo et al. 2026: Documents unsupported assumptions in responses to LGBTQIA+ identity disclosures, which this paper extends to a broader set of identities and distress contexts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Dutta et al. 2026: Audits LLM responses to mental-health-related stereotypes, showing discriminatory responses overlooked by moderation, providing a parallel finding."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Kantharuban et al. 2025: Shows that disclosed user identity biases chatbot recommendations, supporting the premise that LLMs use identity cues inappropriately."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Bauer et al. 2024: Introduces 'compassionate othering' as a concept in medical education, which this paper adapts to characterize model behavior."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Alcoff 1991: Provides the theory of 'speaking for others' used to frame the harm of displacing the writer's account."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gabriel et al. 2024: Finds demographic disparities in LLM mental-health responses, motivating examination of identity-handling even when aggregate safety metrics appear equitable."}],"review_version":1}