Pith. sign in

REVIEW 4 major objections 5 minor 47 references

A youth-tuned AI safety benchmark and model spot subtle teen–AI harms that current moderators miss.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 19:50 UTC pith:BFI3U5IL

load-bearing objection A genuinely useful youth-safety benchmark, but the headline comparison collapses on the train/test arithmetic in Section 3.4. the 4 major comments →

arxiv 2509.08997 v1 pith:BFI3U5IL submitted 2025-09-10 cs.HC

YouthSafe: A Youth-Centric Safety Benchmark and Safeguard Model for Large Language Models

Yaman Yu , Yiren Liu , Jacky Zhang , Yun Huang , Yang Wang This is my paper
classification cs.HC
keywords youth AI safetyLLM safety benchmarkcontent moderationrisk taxonomyteen-AI interactiongenerative AI safeguardsYAIRYouthSafe
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that existing LLM moderation tools, built around overt adult-oriented toxicity, largely fail to detect the developmentally specific risks teenagers and young adults face when chatting with generative AI—grooming, emotional overreliance, boundary violations, identity confusion. To make that case, it introduces YAIR, a benchmark of 12,449 turn-level conversation snippets (real teen chat logs plus synthetic dialogues) annotated with a three-tier risk taxonomy covering 91 low-level risk types. It then fine-tunes a guard model, YouthSafe, on YAIR and reports that it reaches 0.94 AUPRC and 0.88 F1 on youth risk detection, substantially above existing systems. The claim matters because children are heavy users of AI companions and chatbots, and missed risks can translate into real developmental or even life-threatening harm.

Core claim

The paper claims that youth-specific AI harms are not just a matter of severe content, but of subtle, context-dependent interaction patterns—undue influence, emotional dependency, normalization of boundary crossing—that general-purpose moderators are not designed to see. It builds YAIR as a measurement instrument: a three-tier taxonomy grounded in developmental psychology and youth online-safety research, applied to turn-level snippets from real teen conversations and synthetically generated dialogues validated by both LLM judges and human experts. On this benchmark it shows that existing commercial and open-source moderation systems achieve F1 between roughly 0.09 and 0.74, with recall ofte

What carries the argument

The central object is YAIR (Youth AI Risk), a benchmark dataset of 12,449 turn-level conversation snippets annotated with a three-tier taxonomy: 6 high-level risk domains (behavioral/social development, mental well-being, toxicity, bias, misuse/exploitation, privacy), 11 medium-level categories, and 91 low-level risk types. The dataset combines 3,999 snippets from real teen-AI chat logs and 8,450 human-validated synthetic snippets generated through a two-step LLM pipeline (scenario construction then dialogue generation). The companion model YouthSafe is instruction-tuned from a general guardrail architecture on YAIR-TRAINING, with real data supplying authentic context and synthetic data bala

Load-bearing premise

The benchmark's evaluation set is partly synthetic and built from the same taxonomy and prompt templates used to train the model, so the reported performance may not fully reflect how real teenagers talk with AI.

What would settle it

Collect a fresh set of organic teen-AI chat transcripts from platforms not used in data collection (for example, new chat histories from different users on AI companion apps), have them annotated with the YAIR taxonomy, and run YouthSafe; if its recall on those transcripts drops substantially below the reported 0.89, the claimed advantage over existing moderators would be called into question.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Platforms serving teenagers could integrate YouthSafe-style classifiers to flag grooming, manipulation, and emotional-overreliance patterns in real time.
  • YAIR provides a reusable evaluation suite for regulatory or child-safety audits of AI companions.
  • Fine-grained multi-label risk categorization enables targeted downstream interventions, such as directing a teen to mental-health resources when overreliance is detected.
  • The finding that synthetic data is important for recall suggests similar augmentation strategies could extend risk coverage to other under-sampled harms.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the synthetic snippets are generated from the same taxonomy and prompt templates used to build training data, the benchmark may overestimate real-world transfer; a truly external test with organic teen conversations would be needed to confirm.
  • The snippet-level, turn-by-turn evaluation ignores risks that emerge only over longer multi-turn arcs, so the model's real-time promise may be stronger than its end-to-end conversational safety performance.
  • The same approach—domain-tuned taxonomy plus fine-tuned moderation—could be extended to other vulnerable populations, such as elderly users or users with mental-health conditions, but would require their own taxonomies and datasets.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces YAIR, a benchmark of 12,449 youth–GenAI conversation snippets (3,999 real-world and 8,450 synthetic) annotated with a three-tier taxonomy of 91 low-level, 11 medium-level, and 6 high-level risk types. The authors evaluate existing moderation systems (OpenAI Moderation API, Perspective API, LLaMA Guard3, Aegis, WildGuard) on a held-out YAIR-HUMANVAL test set and report that they perform poorly on youth-specific risks. They then present YouthSafe, a model instruction-tuned from Aegis on YAIR-TRAINING, and report strong performance (AUPRC 0.9432, F1 0.8832, precision 0.8799, recall 0.8865). The paper claims YouthSafe significantly outperforms prior systems on both binary risk detection and medium-level risk classification.

Significance. If the claims hold, YAIR would be a valuable new resource: it targets an understudied population, combines ethically collected real chat logs with human-validated synthetic data, and documents a detailed taxonomical framework. The authors also report meaningful reliability checks (inter-rater agreement 0.84, human–machine agreement kappa 0.82) and provide transparent limitations discussions. However, the central comparative claim is undermined by a confounded evaluation design and by internal inconsistencies in the dataset split arithmetic. The resource may still be useful, but the paper's headline result—that a youth-trained model substantially outperforms prior safeguards—is not established by the evidence as reported.

major comments (4)
  1. [§3.4, Table 1, §3.3.2] The dataset split numbers are internally inconsistent. YAIR-SYN is stated as 8,450 snippets (§3.3.2), but §3.4 says YAIR-TRAINING contains 8,450 synthetic snippets and YAIR-HUMANVAL contains 2,124 synthetic snippets; their sum is 10,574, exceeding the stated total synthetic pool. For real data, YAIR-LOG totals 3,999, while 1,078 + 797 = 1,875, far below that total. Thus the described 80/20 split cannot be correct. Either the training split accidentally includes test-set synthetic items (overlap/leakage), or the reported sizes are wrong. This must be corrected and the evaluation re-run or re-verified before any comparative conclusion is drawn.
  2. [§4.1–§4.2, §4.3.1] The comparison between YouthSafe and baselines is confounded. YouthSafe is fine-tuned on YAIR-TRAINING, while all baselines (OpenAI Moderation API, Perspective, LLaMA Guard3, Aegis, WildGuard) are evaluated zero-shot. The reported AUPRC/F1 gap therefore reflects the effect of supervised training on the test distribution as much as any intrinsic model advantage. To support the claim that YouthSafe 'significantly outperforms prior systems,' the authors should include a comparable fine-tuned baseline (e.g., Aegis or LLaMA Guard3 fine-tuned on the same YAIR-TRAINING data) and report confidence intervals or significance tests. Without this, the central claim is not established.
  3. [§3.3.1, §3.4, §4.3.1] The test set YAIR-HUMANVAL is partly in-distribution by construction: 2,124 of its 2,921 snippets are synthetic dialogues generated from the same prompt templates and the same taxonomy that were used to create YAIR-TRAINING. Performance on these items reflects the model's fit to the generation pipeline, not necessarily its ability to detect risks in real youth–GenAI conversations. The authors should report results separately for the 797 real-world chat-log snippets in YAIR-HUMANVAL, and ideally evaluate on an independently collected real-world set. This is a concrete, actionable test that would substantially strengthen external validity.
  4. [§4.3.1, Figure 3, Table 1] The category-level false-negative analysis is based on very small samples. For example, O9 has only 9 test snippets and O8 has 96, so a claim such as 'WildGuard 0% false negative rate' versus 'LLaMA Guard3 100%' is not reliable. The heatmap and the accompanying textual claims should include per-category counts and confidence intervals or aggregate small categories. This issue does not invalidate the overall direction but makes several specific comparative statements unsupported.
minor comments (5)
  1. [§4.2] The model description is inconsistent: the text first says 'instruction-tuned using Aegis-Guard-Defensive,' then says 'fine-tuned the LoRa checkpoint of Aegis with LLaMA Guard as the base model.' Please clarify the exact base model and training procedure; also fix the 'LoRa' typo (should be LoRA).
  2. [§3.2.2] The inter-rater reliability is reported for the initial 15% of the dataset, but the exact number of snippets in that phase is not given. It would also be useful to report IRR for the remainder or a reliability sample from the later annotation phase.
  3. [§3.3.2] The filtering step says low-quality conversations were removed, but it is unclear whether filtering was based on the machine-validation majority vote for all snippets or only on the human-validated subset of 1,720. Please state explicitly how filtering was applied to the full set.
  4. [Table 1] The header 'YAIR-HUMANV AL' contains a spacing typo. Also, because YAIR is multi-label, the per-category counts should be clearly distinguished from unique snippet counts; a footnote or explicit statement would help.
  5. [§1, §3.1] The claim of 'first benchmark' and 'first ethically collected dataset' is strong given prior child-safety-related benchmarks (e.g., SORRY-Bench, BeaverTails) and the authors' own related taxonomy paper. Please qualify these claims and clarify incremental novelty relative to prior work.

Circularity Check

2 steps flagged

Dataset split arithmetic and synthetic-data generation make the headline evaluation in-distribution and possibly overlapping; the central benchmark claim is weakened by construction.

specific steps
  1. fitted input called prediction [Section 3.4 (Data Sampling), with totals from Section 3.3.2]
    "To construct the split, we divided both the YAIR-LOG and YAIR-SYN datasets into 80 percent for training and 20 percent for held-out evaluation. ... The resulting test dataset, YAIR-HUMANVAL, consists of 797 human-labeled real-world chat log snippets and 2,124 human-validated synthetic snippets ... YAIR-TRAINING, which includes 8,450 synthetic snippets and 1,078 real-world snippets."

    The stated split is arithmetically impossible. Section 3.3.2 says YAIR-SYN totals 8,450 snippets, yet YAIR-TRAINING alone is reported as containing 8,450 synthetic snippets. Adding the 2,124 synthetic test snippets gives 10,574 > 8,450. For the real data, 1,078 + 797 = 1,875, far below the YAIR-LOG total of 3,999. Thus the 'held-out' test set cannot be a disjoint leftover of the described 80/20 split; either test items overlap with training or the statistics are erroneous. The headline AUPRC/F1 on YAIR-HUMANVAL is therefore not an independent out-of-sample prediction but an in-distribution or overlapping evaluation.

  2. self definitional [Section 3.3.1 (Synthetic Data Generation) and Sections 4.2/4.3 (YouthSafe training and evaluation)]
    "For each low-level risk type in our taxonomy, we specified 20 distinct scenario descriptions within the prompt to ensure diversity in contextual framing, conversational tone, and user intent. ... Using the constructed scenario specifications from Step 1, we prompted LLM to generate multi-turn dialogues between a youth user and an AI agent that clearly exemplified a specific low-level risk type."

    The synthetic training data and the synthetic majority of YAIR-HUMANVAL (2,124 of 2,921 test snippets) are both generated by the same taxonomy-conditioned prompt pipeline: scenarios are built for each low-level risk in the authors' taxonomy, and dialogues are explicitly prompted to 'clearly exemplify' that risk. YouthSafe is then fine-tuned on these generated snippets and evaluated on the same kind of generated snippets. The high AUPRC (0.9432) largely reflects how well the model reproduces the taxonomy's own generation templates rather than detecting independently observed youth risks. The 797 real snippets provide some independent signal, but the synthetic portion of the test set is constructed from the training definition, making the comparative claim partly circular.

full rationale

The paper's central claim—that YouthSafe significantly outperforms prior systems on YAIR-HUMANVAL—depends on YAIR-HUMANVAL being a clean, disjoint held-out benchmark. The paper's own numbers contradict this: YAIR-SYN has 8,450 total snippets, but YAIR-TRAINING is reported as containing 8,450 synthetic snippets plus YAIR-HUMANVAL adds 2,124 more synthetic snippets, exceeding the total synthetic pool. The real-data counts also do not match the stated 80/20 split (1,078 train + 797 test = 1,875, not 3,999). This is a concrete internal inconsistency, not a matter of external validity. Additionally, the synthetic data generation pipeline is explicitly driven by the authors' taxonomy and prompt templates, so both training and test synthetic items are generated to exemplify the same label definitions; the evaluation largely measures in-distribution fit. The taxonomy itself originates in the authors' own prior work [40], a self-citation, but the decisive circularity is the closed loop between that taxonomy, the generated data, and the evaluation. Some independent signal remains from the 797 real-world test snippets and the comparison to zero-shot baselines, so the paper is not wholly circular; however, the headline comparative result is substantially weakened by construction. Score 6.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The central claims rest on the validity of the adopted taxonomy, the representativeness of LLM-generated synthetic dialogues, and the reliability of human/LLM labels; no numeric free parameters or invented physical entities are introduced. Standard training hyperparameters (lr=5e-5, batch size 2, 6 epochs) are reported in Section 4.2 but are not fitted constants in a derivation.

axioms (4)
  • domain assumption The 84-risk taxonomy from Yu et al. [40] is adopted as the starting point without re-derivation.
    Section 3.1: 'Prior work has categorized the risks... into six high-level areas [40]... These risk types were used as the initial coding schema.'
  • ad hoc to paper Synthetic dialogues generated by GPT-4o and Deepseek R1 from prompt templates are representative of real youth-GenAI interactions.
    Section 3.3.1: 'we prompted LLM to generate multi-turn dialogues... that clearly exemplified a specific low-level risk type'; the validity of the benchmark depends on this.
  • domain assumption LLM majority-vote machine validation and human majority labels (Cohen's kappa 0.82) produce correct ground truth for synthetic snippets.
    Section 3.3.2: comparison of machine vs human labels; if the human labels themselves are unreliable, test set labels are unreliable.
  • domain assumption Turn-based snippet segmentation preserves enough context for risk detection.
    Section 3.2.2 defines snippets as a single exchange; Section 5.3 acknowledges that multi-turn risks may be missed, so this is a known limitation.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of YouthSafe: A Youth-Centric Safety Benchmark and Safeguard Model for Large Language Models." pith.science (2026). https://pith.science/paper/BFI3U5IL

@misc{pith2026250908997,
  author       = {Pith},
  title        = {Pith review of: YouthSafe: A Youth-Centric Safety Benchmark and Safeguard Model for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BFI3U5IL}},
  note         = {Machine review of arXiv:2509.08997}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) are increasingly used by teenagers and young adults in everyday life, ranging from emotional support and creative expression to educational assistance. However, their unique vulnerabilities and risk profiles remain under-examined in current safety benchmarks and moderation systems, leaving this population disproportionately exposed to harm. In this work, we present Youth AI Risk (YAIR), the first benchmark dataset designed to evaluate and improve the safety of youth LLM interactions. YAIR consists of 12,449 annotated conversation snippets spanning 78 fine grained risk types, grounded in a taxonomy of youth specific harms such as grooming, boundary violation, identity confusion, and emotional overreliance. We systematically evaluate widely adopted moderation models on YAIR and find that existing approaches substantially underperform in detecting youth centered risks, often missing contextually subtle yet developmentally harmful interactions. To address these gaps, we introduce YouthSafe, a real-time risk detection model optimized for youth GenAI contexts. YouthSafe significantly outperforms prior systems across multiple metrics on risk detection and classification, offering a concrete step toward safer and more developmentally appropriate AI interactions for young users.

Figures

Figures reproduced from arXiv: 2509.08997 by Jacky Zhang, Yaman Yu, Yang Wang, Yiren Liu, Yun Huang.

Figure 1
Figure 1. Figure 1: Three-tier taxonomy of youth-GenAI risks. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the YAIR-Bench data curation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: False Negative Percentage across Risk Categories [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: F1 Score, Recall, and Precision comparison between [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Prompt used for generating synthetic dialogues. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Prompt used for generating synthetic scenarios. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Prompt used for machine validation [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 3 canonical work pages · 1 internal anchor

  1. [1]

    AI Algorithmic and Automation Incidents Repository. 2024. Character AI Hosts Paedophile and Suicide Chatbots. https://tinyurl.com/bdyayc4z. Accessed: 2025-02-12

    2024

  2. [2]

    Safinah Ali, Daniella DiPaola, Irene Lee, Jenna Hong, and Cynthia Breazeal. 2021. Exploring Generative Models with Middle School Students. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, 1–13. doi:10.1145/3411764.3445226

    arXiv 2021

  3. [3]

    Safinah Ali, Daniella DiPaola, Irene Lee, Victor Sindato, Grace Kim, Ryan Blumofe, and Cynthia Breazeal. 2021. Children as creators, thinkers and citizens in an AI-driven future.Computers and Education: Artificial Intelligence2 (2021), 100040

    2021

  4. [4]

    Valentina Andries and Judy Robertson. 2023. Alexa doesn’t have that many feelings: Children’s understanding of AI through interactions with smart speakers in their homes.Computers and Education: Artificial Intelligence5 (2023), 100176

    2023

  5. [5]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862(2022)

    Pith/arXiv arXiv 2022

  6. [6]

    Hongye Cao, Yanming Wang, Sijia Jing, Ziyue Peng, Zhixin Bai, Zhe Cao, Meng Fang, Fan Feng, Boyan Wang, Jiaheng Liu, et al. 2025. SafeDialBench: A Fine- Grained Safety Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attacks.arXiv preprint arXiv:2502.11090(2025)

    arXiv 2025

  7. [7]

    Sarah A Chauncey and H Patricia McKenna. 2023. A framework and exemplars for ethical and responsible use of AI Chatbot technology to support teaching and learning.Computers and Education: Artificial Intelligence5 (2023), 100182

    2023

  8. [8]

    Tianyu Cui, Yanling Wang, Chuanpu Fu, Yong Xiao, Sijia Li, Xinhao Deng, Yun- peng Liu, Qinglin Zhang, Ziyi Qiu, Peiyang Li, et al . 2024. Risk taxonomy, mitigation, and assessment benchmarks of large language model systems.arXiv preprint arXiv:2401.05778(2024)

    Pith/arXiv arXiv 2024

  9. [9]

    Zhichen Dong, Zhanhui Zhou, Chao Yang, Jing Shao, and Yu Qiao. 2024. Attacks, defenses and evaluations for llm conversation safety: A survey.arXiv preprint arXiv:2402.09283(2024)

    Pith/arXiv arXiv 2024

  10. [10]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith

  11. [11]

    Shaona Ghosh, Prasoon Varshney, Erick Galinkin, and Christopher Parisien. 2024. Aegis: Online adaptive ai content safety moderation with ensemble of llm experts. arXiv preprint arXiv:2404.05993(2024)

    Pith/arXiv arXiv 2024

  12. [12]

    Ji-Eun Han, Jun-Seok Koh, Hyeon-Tae Seo, Du-Seong Chang, and Kyung-Ah Sohn. 2024. PSYDIAL: personality-based synthetic dialogue generation using large language models.arXiv preprint arXiv:2404.00930(2024)

    Pith/arXiv arXiv 2024

  13. [13]

    Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. 2024. Wildguard: Open one-stop mod- eration tools for safety risks, jailbreaks, and refusals of llms.arXiv preprint arXiv:2406.18495(2024)

    Pith/arXiv arXiv 2024

  14. [14]

    Susan Hao, Piyush Kumar, Sarah Laszlo, Shivani Poddar, Bhaktipriya Radharapu, and Renee Shelby. 2023. Safety and fairness for content moderation in generative models.arXiv preprint arXiv:2306.06135(2023)

    Pith/arXiv arXiv 2023

  15. [15]

    Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection.arXiv preprint arXiv:2203.09509 (2022)

    Pith/arXiv arXiv 2022

  16. [16]

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674(2023)

    Pith/arXiv arXiv 2023

  17. [17]

    Pegah Jandaghi, XiangHai Sheng, Xinyi Bai, Jay Pujara, and Hakim Sidahmed

  18. [18]

    Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023. Beavertails: Towards improved safety alignment of llm via a human-preference dataset.Advances in Neural Information Processing Systems36 (2023), 24678–24704

    2023

  19. [19]

    Nomisha Kurian. 2024. ‘No, Alexa, no!’: designing child-safe AI and protecting children from the risks of the ‘empathy gap’ in large language models.Learn- ing, Media and Technology(2024). https://api.semanticscholar.org/CorpusID: 271158326

    2024

  20. [20]

    Alyssa Lees, Vinh Q Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. 2022. A new generation of perspective api: Efficient multilingual character-level transformers. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 3197–3207

    2022

  21. [21]

    Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. 2024. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models.arXiv preprint arXiv:2402.05044(2024)

    Pith/arXiv arXiv 2024

  22. [22]

    Hongfu Liu, Hengguan Huang, Xiangming Gu, Hao Wang, and Ye Wang. 2024. On calibration of LLM-based guard models for reliable content moderation.arXiv preprint arXiv:2410.10414(2024)

    Pith/arXiv arXiv 2024

  23. [23]

    Jingze Ma, Yuanzhi Li, and Jingyao Wang. 2024. Analysis of the Challenges and Opportunities of AIGC for Youth Education.Journal of Humanities and Social Sciences Studies6, 9 (sep 14 2024), 53–61. doi:10.32996/jhsss.2024.6.9.6

    work page doi:10.32996/jhsss.2024.6.9.6 2024

  24. [24]

    Pooja Malvi and Hee Rin Lee. 2023. Cat-E. InCompanion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 407–410. doi:10.1145/ 3568294.3580116

    arXiv 2023

  25. [25]

    Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng. 2023. A holistic approach to undesired content detection in the real world. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 15009–15018

    2023

  26. [26]

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. 2024. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint arXiv:2402.04249(2024)

    Pith/arXiv arXiv 2024

  27. [27]

    David Nadeau, Mike Kroutikov, Karen McNeil, and Simon Baribeau. 2024. Bench- marking llama2, mistral, gemma and gpt for factuality, toxicity, bias and propen- sity for hallucinations.arXiv preprint arXiv:2404.09785(2024)

    Pith/arXiv arXiv 2024

  28. [28]

    Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R Bowman. 2020. CrowS- pairs: A challenge dataset for measuring social biases in masked language models. arXiv preprint arXiv:2010.00133(2020)

    Pith/arXiv arXiv 2020

  29. [29]

    Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang

  30. [30]

    Ofcom. 2023. Gen Z Driving Early Adoption of Gen AI, Our Latest Re- search Shows. https://www.ofcom.org.uk/news-centre/2023/gen-z-driving- early-adoption-of-gen-ai Accessed: 2024-06-03. CCS ’25, October 13–17, 2025, Taipei, Taiwan Yaman Yu, Yiren Liu, Jacky Zhang, Yun Huang, & Yang Wang

    2023

  31. [31]

    Jinkyung Park, Vivek Singh, and Pamela Wisniewski. 2023. Supporting Youth Mental and Sexual Health Information Seeking in the Era of Artificial Intelligence (AI) Based Conversational Agents: Current Landscape and Future Directions. SSRN Electronic Journal(2023). doi:10.2139/ssrn.4601555

    work page doi:10.2139/ssrn.4601555 2023

  32. [32]

    Jinkyung Park, Vivek Singh, and Pamela Wisniewski. 2024. Toward Safe Evolution of Artificial Intelligence (AI) based Conversational Agents to Support Adolescent Mental and Sexual Health Knowledge Discovery.arXiv.org(2024). doi:10.48550/ ARXIV.2404.03023

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2404.03023 2024

  33. [33]

    Sara Owsley Sood, Elizabeth F Churchill, and Judd Antin. 2012. Automatic identification of personal insults on social news sites.Journal of the American Society for Information Science and Technology63, 2 (2012), 270–285

    2012

  34. [34]

    Simone Tedeschi, Felix Friedrich, Patrick Schramowski, Kristian Kersting, Roberto Navigli, Huu Nguyen, and Bo Li. 2024. ALERT: A Comprehensive Benchmark for Assessing Large Language Models’ Safety through Red Teaming.arXiv preprint arXiv:2404.08676(2024)

    Pith/arXiv arXiv 2024

  35. [35]

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al

  36. [36]

    Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, et al. 2024. Sorry- bench: Systematically evaluating large language model safety refusal behaviors. arXiv preprint arXiv:2406.14598(2024)

    Pith/arXiv arXiv 2024

  37. [37]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions.arXiv preprint arXiv:2304.12244(2023)

    Pith/arXiv arXiv 2023

  38. [38]

    Liang Xu, Kangkang Zhao, Lei Zhu, and Hang Xue. 2023. Sc-safety: A multi-round open-ended question adversarial safety benchmark for large language models in chinese.arXiv preprint arXiv:2310.05818(2023)

    Pith/arXiv arXiv 2023

  39. [39]

    Michele L Ybarra, Kimberly J Mitchell, Janis Wolak, and David Finkelhor. 2006. Examining characteristics and associated distress related to Internet harassment: findings from the Second Youth Internet Safety Survey.Pediatrics118, 4 (2006), e1169–e1177

    2006

  40. [40]

    Yaman Yu, Yiren Liu, Jacky Zhang, Yun Huang, and Yang Wang. 2025. Under- standing Generative AI Risks for Youth: A Taxonomy Based on Empirical Data. arXiv preprint arXiv:2502.16383(2025)

    Pith/arXiv arXiv 2025

  41. [41]

    Yaman Yu, Tanusree Sharma, Melinda Hu, Justin Wang, and Yang Wang. 2024. Exploring Parent-Child Perceptions on Safety in Generative AI: Concerns, Miti- gation Strategies, and Design Implications.arXiv.org(2024). doi:10.48550/ARXIV. 2406.10461

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.10461 2024

  42. [42]

    Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. 2023. SafetyBench: Evaluating the safety of large language models.arXiv preprint arXiv:2309.07045(2023)

    Pith/arXiv arXiv 2023

  43. [43]

    conversation

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models.arXiv preprint arXiv:2403.13372(2024). A Appendix A.1 Prompts used for Generating Synthetic Data (Scenarios and Dialogues) Your task is to generate a simulated risky conversation betwe...

    Pith/arXiv arXiv 2024

  44. [2016]

    InProceedings of the 25th international conference on world wide web

    Abusive language detection in online user content. InProceedings of the 25th international conference on world wide web. 145–153

  45. [2020]

    Realtoxicityprompts: Evaluating neural toxic degeneration in language models.arXiv preprint arXiv:2009.11462(2020)

    Pith/arXiv arXiv 2009

  46. [2022]

    Emergent abilities of large language models.arXiv preprint arXiv:2206.07682 (2022)

    Pith/arXiv arXiv 2022

  47. [2023]

    Faithful persona-based conversational dataset generation with large lan- guage models.arXiv preprint arXiv:2312.10007(2023)

    Pith/arXiv arXiv 2023

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.