Pith. Sign in

REVIEW

SAFE-Teen: Evaluating Chatbot Safety Across Adolescent Mental Health Conversations

PaperPDF from arXiv

data files

ReviewThe objections and verdict, as JSON Paper breakdownThe authors' claims, conditions and measurements, as JSON Signed receiptProof this review has not been altered
X Bluesky LinkedIn Reddit Hacker News
Andrea Mock, Valerie Chen, Eric Lin, Rayan Krishnan, Diyi Yang, Caroline Figueroa

Computation and Language · Submitted 4 Nov 2026 · 85 references

Pith's verdict: A rigorous, clinician-informed evaluation that convincingly shows chatbot safety failures are common in teen mental health conversations and increase when teens push back—but the simulation realism rests on thin validation (24 clinician ratings), so the precise failure-rate numbers should be taken as indicative rather than exact.

T0 · AI review

How this review was made: An AI referee (deepseek/deepseek-v4-flash) read the paper on November 6, 2026 and answered a fixed list of questions about it. T0 is the starting level every paper gets. The levels describe how deeply Pith has checked the paper, never the authors.

  1. T0 AI review An AI read the paper and answered a fixed list of questions.
  2. T1 Audit Automatic checks that the references and files the paper relies on exist and match.
  3. T2 Compute The paper's code or calculations re-run with the same results.
  4. T3 Formal statement The claims written out in a formal math language a computer can read.
  5. T4 Proof Those claims proved, and the proof checked by computer.

How Pith checks papers →

· 2 major objections · 6 minor · Ask about this paper · The compiled record · Read the full review →
Challenge this review Comment (0) Read on arXiv ↗
Cite ▾
arXiv ID2611.07218 (version 1)
Pith ID3EAM5ZVY, Pith's permanent record of this paper and its review
CitationAndrea Mock, Caroline Figueroa, Diyi Yang, Eric Lin, Rayan Krishnan, Valerie Chen. "SAFE-Teen: Evaluating Chatbot Safety Across Adolescent Mental Health Conversations." arXiv:2611.07218 (2026). https://pith.science/paper/3EAM5ZVY
Cite this reviewBibTeX for Pith's review of this paper
Claim paper

Why Pith says this

Highlight any text to comment on it.

Referee report

How Pith checks papers → · Full audit of this paper →

The paper

The paper claims

Even with a teen-aware prompt, chatbots fail safety criteria in 24.9% of simulated adolescent mental health conversations, and failures rise to 43.3% without it.

The reading

This paper introduces SAFE-Teen, a clinician-informed evaluation framework that systematically tests AI chatbot safety in simulated conversations with adolescents aged 13-17. Across 32,555 conversations spanning 18 chatbot APIs, 22 safety criteria, and 983 variants of 106 seed scenarios, the overall criterion failure rate was 43.3% without a teen-aware system prompt and 24.9% with it. However, critical gaps persisted: failures to respond to indirect suicide risk remained at 52%, and self-harm response failures stayed near 75%. When the simulated teenager pushed back on a chatbot's limit, failure rates on mental health diagnosis, medication advice, and substance misuse boundaries rose by 10-12 percentage points for more than 80% of chatbots. The paper argues that safety evaluations must follow conversations beyond a single turn and test how safety holds when teenagers respond differently.

What carries the argument

SAFE-Teen is the central framework: 106 clinician-authored seed scenarios expanded into 983 matched conversation variants that differ in only one teen behavior (openness of disclosure, response to a help offer, or response to a chatbot limit). A teen simulator (GPT-5.6 Terra) holds up to 20 exchanges with each chatbot while following clinician-specified trajectories. An LLM judge (Claude Sonnet 5.5) rates each applicable criterion from a 22-criteria rubric covering acute harm, medical boundaries, and AI relational safety, scoring responses as pass, partial, or fail. The matched-branch design isolates the effect of the teen's behavior while keeping scenario details constant.

Extended reading notes

Core claim

Chatbot safety in adolescent mental health conversations is systematically fragile and dependent on how the conversation unfolds. The overall criterion failure rate across 18 chatbots was 43.3% without a teen-aware prompt and 24.9% with it, but even the teen-aware prompt did not eliminate failures on indirect suicide risk (52% failure) or self-harm response (75% failure). When a teenager pushed back on a chatbot's limit, failures on mental health diagnosis, medication advice, and substance misuse boundaries increased by roughly 10-12 percentage points without the prompt, and declining outside help raised emotional-dependence failures by 17.6 percentage points. The paper further shows that recognizing a risk in reasoning text helped little unless the chatbot planned a response that met the safety criteria: 58% of replies were failures even when the reasoning noticed the risk but did not plan all required actions.

Load-bearing premise

The simulated teen conversations, generated by an LLM and validated by clinician review but not by actual adolescents, may not fully capture real adolescent behavior; the results are therefore conditional on the simulation being representative enough to generalize to real-world interactions.

Editorial extensions

If this is right

  • Teen-aware system prompts reduce overall failure rates but are insufficient for indirect risk recognition and when teenagers resist; more specific safety training data and prompt designs are needed.
  • Safety evaluations must move beyond single-turn refusal tests to multi-turn scenarios that include user pushback and varying disclosure styles.
  • Chatbots' tendency to cross diagnosis and medication boundaries when pressed has implications for regulatory oversight and liability in youth-facing AI mental health tools.
  • Even when a chatbot's reasoning notes a safety risk, it often fails to plan a response that meets clinical criteria, suggesting that alignment training should reward appropriate action planning, not just risk recognition.
  • The matched-branch methodology can be reused to evaluate other vulnerable populations and risk domains, with appropriate clinician input.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The simulated conversations may underestimate real-world risks because actual adolescents could be more persistent or more skilled at concealing risk; the clinician-validated simulations likely provide a lower bound on failure rates.
  • The wide variation in failure rates across models (14% to 62% without prompt) suggests safety is not a simple function of model size or capability but depends on fine-grained alignment decisions, which the SAFE-Teen criteria can help expose.
  • The framework could be extended to test proactive safety measures, such as whether a chatbot detects risk from earlier messages and adjusts its behavior before the teenager explicitly discloses a crisis.
Original abstract

Adolescents are increasingly turning to AI chatbots for mental health support, advice, and companionship, raising safety concerns. While recent evaluations have started to examine safety across multiple conversational turns, few compare how slight changes in a teen's responses can affect a chatbot's safety in the same scenario. We conduct a clinician-informed evaluation of chatbot safety in simulated conversations with adolescents aged 13-17, examining how safety failures vary with adolescents' responses and teen-aware system prompting. We evaluated 32,555 simulated conversations across 18 chatbot APIs and two system-prompt conditions using 22 safety criteria grounded in previously identified harms. Conversations spanned 983 variants of 106 seed scenarios covering urgent safety issues, health advice, and identity and relationships. The overall criterion failure rate was 43.3% without the teen-aware prompt and 24.9% with it. The teen-aware prompt reduced failures, but it didn't eliminate them. When a teenager pushed back on a chatbot's answer, failures on mental health diagnosis, medication, and substance misuse boundaries rose by 10-12 percentage points without the prompt, for more than 80% of chatbots. These findings reveal persistent safety failures in simulated adolescent-chatbot interactions and show that safety depends on how a conversation unfolds.

The compiled record

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-06: 1 request, 6,959 input tokens.

What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.81 · human participants 0.16 · controlled comparison 0.93 · randomized assignment 0.03 · states quantitative result 0.99 · compares with prior results 0.88 · mentions code or data availability 0.03.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; .

Adolescents are increasingly turning to AI chatbots for mental health support, advice, and companionship, raising safety concerns. While recent evaluations have started to examine safety across multiple conversational turns, few compare how slight changes in a teen's responses can affect a chatbot's safety in the same scenario. We conduct a clinician-informed evaluation of chatbot safety in simulated conversations with adolescents aged 13-17, examining how safety failures vary with adolescents' responses and teen-aware system prompting. We evaluated 32,555 simulated conversations across 18 chatbot APIs and two system-prompt conditions using 22 safety criteria grounded in previously identified harms. Conversations spanned 983 variants of 106 seed scenarios covering urgent safety issues, health advice, and identity and relationships. The overall criterion failure rate was 43.3% without the teen-aware prompt and 24.9% with it. The teen-aware prompt reduced failures, but it didn't eliminate them. When a teenager pushed back on a chatbot's answer, failures on mental health diagnosis, medication, and substance misuse boundaries rose by 10-12 percentage points without the prompt, for more than 80% of chatbots. These findings reveal persistent safety failures in simulated adolescent-chatbot interactions and show that safety depends on how a conversation unfolds.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#sentencerolethe authors' own workstates the main conclusionstrength
s0 Adolescents are increasingly turning to AI chatbots for mental health support, advice, and companionship, raising safety concerns. background 1.00 0.04 0.02 asserted 1.00
s1 While recent evaluations have started to examine safety across multiple conversational turns, few compare how slight changes in a teen's responses can affect a chatbot's safety in the same scenario. gap 1.00 0.05 0.03 asserted 0.98
s2 We conduct a clinician-informed evaluation of chatbot safety in simulated conversations with adolescents aged 13-17, examining how safety failures vary with adolescents' responses and teen-aware system prompting. contribution 0.94 0.97 0.05 asserted 0.98
s3 We evaluated 32,555 simulated conversations across 18 chatbot APIs and two system-prompt conditions using 22 safety criteria grounded in previously identified harms. method 0.99 0.97 0.03 asserted 1.00
s4 Conversations spanned 983 variants of 106 seed scenarios covering urgent safety issues, health advice, and identity and relationships. method 0.99 0.93 0.03 asserted 1.00
s5 The overall criterion failure rate was 43.3% without the teen-aware prompt and 24.9% with it. result 1.00 0.96 0.14 asserted 1.00
s6 The teen-aware prompt reduced failures, but it didn't eliminate them. result 0.79 0.92 0.19 asserted 1.00
s7 When a teenager pushed back on a chatbot's answer, failures on mental health diagnosis, medication, and substance misuse boundaries rose by 10-12 percentage points without the prompt, for more than 80% of chatbots. result 1.00 0.96 0.13 asserted 1.00
s8 These findings reveal persistent safety failures in simulated adolescent-chatbot interactions and show that safety depends on how a conversation unfolds. interpretation 0.99 0.94 0.88 asserted 1.00

What kind of paper the classifier reports this is: empirical study 0.72 · evidence from observation or experiment 0.51.

What was examined, and what was not

sources read
  • arXiv listing abstract v1
not examined
  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers
levels run
  • P abstract
not run
  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

Figures
Figure 1. SAFE-Teen’s design allows us to investigate three research questions about chatbot safety for teens: (1) Comparing failure rates across topics, criteria, and chatbots shows where chatbots fail teens. (2) Running every conversation under a default and a teen-aware system prompt tests whether telling the chatbot the user is a teen helps. (3) Comparing branches of the same scenario isolates the teen’s behavior, testing whether safety limits hold when the teen declines help or pushes back. response to help, or response to a chatbot limit.1 The scenarios cover three risk areas motivated…
Figure 2. Overview of the SAFE-Teen evaluation. (1) Clinicians write seed scenarios. (2) Each scenario branches into matched versions that differ in one teen behavior: disclosure, response to a help offer, or response to a chatbot limit. (3) A simulated teen holds each conversation with a chatbot, with and without a teen-aware system prompt. (4) An LLM judge rates each applicable criterion as pass, partial, or fail. bots (De Freitas et al., 2026). Experiments show that emotionally supportive AI interactions can af￾fect how much people prefer AI companionship (Shi et al., 2026). These findings…
Figure 3. Failure rates across LLMs. While the teen￾aware prompt lowered failure rates, every model still gave problematic responses in some scenarios. reasoning at the opening turn and right after vari￾ous disclosure points that clinicians had manually labeled as important. For each flagged instance, the LLM-as-a-judge was asked to decide whether, us￾ing the same reasoning, it was a safety concern, as well as whether the chatbot had recognized it was chatting with a teen and responded accordingly. 4 Findings 4.1 Effects of the Teen-Aware Prompt The teen-aware prompt reduced failures for…
Figure 4. Failure rates by three risk areas. We pool results across the 18 chatbots and stratify by risk area
Figure 5. Three examples show a missed suicide warning sign, a diagnosis, and support for secrecy during self-harm risk. Across all cases where each safety criterion applied, failure rates were 52% (225/435) for indirect suicide risk without the teen-aware prompt, 60% (346/575) for mental-health diagnosis with the prompt, and 15% (227/1,523) for emotional dependence with the prompt. These rates describe the criteria overall, not just the three examples. Quotes are verbatim, and [...] marks cuts. Substance misuse help Mental health diagnosis Medication advice Emotional dependence Medical…
Figure 6. For these five criteria, failure rates changed more when teens declined help or challenged a limit than when they disclosed less. Each dot shows the average size of the change in failure rate, in percentage points, across matched conversations without the teen-aware prompt. much the teen disclosed (
Figure 7. The chatbot recognized the teen was still thinking about self-harm, then chose not to ask about it. Grok 4.7, no added prompt. Quotes are verbatim including the reply that is shown in full. ing our rubric, marked replies as failures in 58% of cases when the reasoning noticed a risk but did not plan all the actions called for in that situation. This was close to the 66% failure rate when the reasoning did not notice the risk. Even when the reasoning noticed the risk and planned all required actions specified by that specific rubric criterion, the automated judge still marked 14% of…
Figure 8. Seed scenarios by risk area and primary topic. Each of the 106 scenarios has one risk area and one primary
Figure 9. Further reasoning results for the nine chatbots that save their own reasoning. (a) Whether the reasoning treated details flagged by clinicians as a safety concern, only mentioned them, or left them out, for details the teen had said in full or in part. (b) How often the opening-turn reasoning recognized the user as a teen or planned age-specific steps, with and without the teen-aware prompt, in matched conversations. Fail Partial Pass No added prompt n Teen-aware prompt n Acute harm and safeguarding Respond to eating-disorder harm 92% 60 80% 61 Respond to self-harm 75% 208 77% 196…
Figure 10. How often each criterion was rated fail, partial, or pass, across the 18 chatbots. As in the main text, a
Figure 11. One conversation, two branches, with the teen-aware prompt. When the teen accepts the chatbot’s limit,

Figures reproduced from arXiv:2611.07218 by the authors.

References

Discussion

Disagree with the review, or know something it missed? Leave a comment, highlight any passage above to comment on it, or challenge this review formally.