Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

This paper claims that personalized safety in LLM-based conversational recommenders is a trainable property: a two-stage method cuts safety violations by up to 96.5% without sacrificing relevance.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 19:03 UTC pith:IYNCYA44

load-bearing objection A promising benchmark and a sensible training recipe, but the safety labels are pipeline-generated and the benchmark tests explicit instruction-following more than implicit safety inference. the 3 major comments →

arxiv 2603.03536 v2 pith:IYNCYA44 submitted 2026-03-03 cs.CL cs.AIcs.IR

SafeCRS: Personalized Safety Alignment for LLM-Based Conversational Recommender Systems

Haochang Hao , Yifan Xu , Xinzhuo Li , Yingqiang Ge , Lu Cheng This is my paper
classification cs.CL cs.AIcs.IR
keywords LLM safetyconversational recommender systemspersonalized safety alignmentsafety benchmarklatent trait inferencereward normalizationsafe fine-tuninglearning to rank
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that LLM-based conversational recommender systems have a safety blind spot: a recommendation can satisfy the user's stated preferences and still harm the individual by violating implicit sensitivities — a phobia, a trauma trigger, a recovery history — inferred from the conversation but ignored during ranking. To make this failure measurable, the authors build SafeRec, a two-domain benchmark (movies and games) whose ground truth comes from deterministic safety oracles: IMDb Parent Guide severity and DoesTheDogDie trigger tags for movies, ESRB descriptors and age ratings for games, fused with an LLM-inferred latent trait per conversation. To make it fixable, they propose SafeCRS, a two-stage training pipeline that first supervises the model to reason about which candidates violate the inferred trait and filter them out, then applies per-reward normalized policy optimization so the sparse relevance signal is not drowned out by dense safety and format rewards. If the central claim is right, personalized safety alignment becomes a measurable and trainable property, and even small models can reach near-zero violation rates while matching the recommendation quality of much larger general-purpose systems.

Core claim

The central claim is that personal safety constraints in conversational recommendation can be formalized as trait-conditioned risk scores over items, and that a two-stage training pipeline can align LLM policies to those constraints without giving up relevance. On the SafeMovie benchmark, models trained with SafeCRS reduce the safety violation ratio by up to 96.5% relative to the strongest quality baseline (from 0.3508 to 0.0122 at top-5) and reach near-zero violation rates across all tested backbones while matching or surpassing the recommendation quality of GPT-4; on SafeGame, they outperform the best baseline by 3.7× in Recall@5 and 3.3× in NDCG@5. The authors present this as evidence tha

What carries the argument

The load-bearing mechanism is a deterministic Safety Oracle that turns each user sensitivity trait into a computable risk score for each item. For movies, the oracle blends a normalized weighted sum of IMDb Parent Guide severity dimensions with a hard binary veto from DoesTheDogDie trigger tags; for games, it multiplies an ESRB content-descriptor match by an age-rating weight. On the training side, the complementary mechanism is Safe-GDPO, which normalizes each reward channel (relevance, safety, count compliance) separately before combining them into a rank-wise advantage, preventing the sparse relevance signal from being overwhelmed by denser safety rewards. The first stage, Safe-SFT, teach

Load-bearing premise

The benchmark's ground truth assumes that the LLM-inferred latent trait matches the user's actual sensitivity; the same inferred trait drives both the training labels and the evaluation oracle, so if that inference is wrong, the near-zero violation rates measure self-consistency with the pipeline rather than genuine personalized safety.

What would settle it

Take real users with known, self-reported sensitivities (e.g., a phobia of needles), run SafeCRS on their conversations, and ask them to flag harmful items. If the model reports near-zero violations while a substantial share of users still flags recommended items as harmful — because the inferred trait differed from the user's actual constraint — the claim that personalized safety has been achieved would fail. The opposite result, with user-flagged harm near zero, would confirm the claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • SafeCRS-trained models reach near-zero safety violation rates on SafeMovie while matching or exceeding the recommendation quality of GPT-4, so safety and relevance do not have to trade off when rewards are decoupled.
  • Small backbones show the largest relative gains, suggesting personalized safety alignment is accessible without requiring frontier-scale models.
  • The deterministic oracle design means safety labels are reproducible and reusable, and the benchmark can be extended to new domains with structured content metadata.
  • The rank-discounted safety penalty implies that higher-ranked violations are treated as more harmful, so future CRS training can incorporate position-aware safety losses directly.
  • The ablation shows Safe-SFT supplies most of the safety gain while Safe-GDPO refines both safety and relevance, so the two stages are complementary rather than redundant.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's evaluation shares its latent-trait labels between training targets and the safety oracle; a natural next experiment is to compare LLM-inferred traits against users' own self-reported sensitivities to see how much of the measured near-zero violation rate reflects genuine alignment with the user.
  • Because the oracle scores content by structured tags and severity scales, it cannot capture artistic framing or context; a movie tagged 'suicide' could be handled with care or exploited, and the benchmark would score both the same — a boundary worth testing with human ratings.
  • The same recipe should transfer to domains with comparable structured safety metadata (books, music, web content, ads), and the paper's game-domain results suggest the transfer works even when taxonomies and oracle constructions differ.
  • The rank-discount penalty implies the authors treat lower-ranked violations as less harmful; an alternative safety philosophy would prohibit unsafe content outright at any rank, and the two stances are directly comparable with the same benchmark.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces SafeRec, a benchmark for personalized safety in LLM-based conversational recommender systems, with movie (SafeMovie) and game (SafeGame) domains. Safety scores are computed from external metadata (DDD/IPG for movies; ESRB for games) conditioned on user sensitivity traits inferred from conversations. The authors then propose SafeCRS, a two-stage training pipeline (Safe-SFT followed by Safe-GDPO) that teaches a CRS to filter unsafe items and optimize a decoupled multi-reward objective. On SafeRec, SafeCRS is reported to reduce safety violation rates by up to 96.5% relative to the strongest quality baseline while maintaining competitive recommendation quality, with near-zero SVR on SafeMovie across four backbone LLMs.

Significance. If the benchmark and its labels are valid, this is a timely contribution: it identifies an underexplored safety dimension in CRS, provides a reusable benchmark with two domains, and proposes a concrete training recipe. Strengths include the use of external, structured safety metadata (DDD/IPG/ESRB), a deterministic oracle rather than LLM-as-judge for item risk, clear reward formulations with per-reward normalization, and the release of code/benchmark/checkpoints. The 96.5% arithmetic is consistent with Table 1. However, the empirical claims rest on LLM-inferred trait labels and on test prompts that contain explicit injected safety constraints; both of these need closer scrutiny and, if possible, human validation or an implicit-setting evaluation before the central claims can be accepted.

major comments (3)
  1. [§3.2.1 / Appendix A.2] The benchmark evaluation does not test the paper's stated problem of inferring safety sensitivities *implicitly* from conversation. Appendix A.2 says the latent-trait constraint is injected into the user prompt, and the same explicit trait is used to query the oracle. Thus the model is always handed an explicit statement of the safety preference (e.g., "avoid movies with excessive blood and gore"). The abstract and §1 emphasize that safety sensitivities are "implicitly inferred from the conversation," but no experiment withholds the injected constraint or uses naturalistic implicit language. As a result, the reported SVR/S-DCG numbers measure explicit instruction following rather than implicit personalized safety inference. I recommend adding an evaluation condition where the constraint is not injected (or where only subtle conversational hints are used) and reporting whether SafeCRS sti
  2. [Table 1 / Table 2 / Eq. (15)] The reported S-DCG values for Qwen2.5-0.5B are arithmetically impossible under the metric definition. With SVR@5=0.0011 and rank weights d_k=1/log2(k+1) ≤ 1, the maximum possible S-DCG@5 is approximately 5 × 0.0011 × 1 = 0.0055, yet Table 1 reports S-DCG@5=0.32. Similarly, Table 2 reports Qwen2.5-0.5B + Safe-SFT with SVR@5=0.0177 and S-DCG@5=0.54, which is also impossible under Eq. (15). Because Figure 4 and several conclusions rely on S-DCG as a position-weighted safety measure, this inconsistency undermines confidence in the safety-compliance results. The authors should correct the metric computation or the reported values and regenerate the affected figures.
  3. [Appendix A.3 / §3.2.1] The safety ground truth depends on LLM-inferred trait labels, and no human validation is reported. The reader's strongest circularity concern—that the test labels are produced by the conversation+recommendations Safe-SFT prompt—is not clearly supported by the text: Appendix C.1 suggests the test trait is inferred from the conversation alone, and §3.2.1 says "given a conversation context." However, the manuscript never states which of the two trait-inference prompts (conversation-only vs. conversation+recommendations) is used for test-set construction. This must be clarified. More importantly, the paper should provide evidence that the LLM-inferred traits align with what real users would report, e.g., human agreement on a sample, or an evaluation using only explicit constraints found in natural user language. Without such validation, the 96.5% SVR reduction may reflect agreement between t
minor comments (4)
  1. [Eq. (7)] The definition of the rank discount is ambiguous: "d_k = 1/log2(k+1) or 1/log2(k+2) under 0-based indexing." Please give a single convention, e.g., d_k = 1/log2(k+2) with k starting at 0, or d_k = 1/log2(k+1) with k starting at 1.
  2. [Appendix A.3] In prompt (2), guideline 2 says to use "Parent filtering for children" for kids/family mentions, but this is not among the 20 explicit traits listed in Appendix B.1; it appears the trait name should be "Kid-safety / child harm sensitive." This could lead to invalid JSON outputs from the LLM.
  3. [§3.2.1 / Appendix A.2] The terms "explicit trait" and "latent trait" are used in overlapping ways. Consider defining once: the explicit trait is the taxonomy label (e.g., Anti-gore), while the latent trait is the natural-language constraint injected into the conversation. This would improve readability.
  4. [Table 1] The table's decimal formatting is inconsistent (e.g., .0011 vs 0.32, and missing leading zero on some values like .0006). A uniform style (e.g., always 0.xxxx) would help avoid confusion.

Circularity Check

2 steps flagged

Safety labels are generated by the same recommendation-conditioned trait inference used for both training targets and evaluation, so near-zero SVR is substantially by construction.

specific steps
  1. fitted input called prediction [Appendix A.3 (Safe-SFT trait inference), §4.1 (Safe-SFT targets), §5.3.2 Eq. (13)]
    "This prompt selects the trait that is most likely to be violated by the recommended movies ... 2. The recommended movies should be LIKELY to violate this trait ... 3. Choose the trait with highest violation risk among recommended movies. ... Items with risk scores exceeding a threshold τ are marked unsafe and removed. ... v_k = 1[final(r_k,t) ≥ τ] ... t the user's assigned sensitivity trait"

    The trait t used in the evaluation metric Eq. (13) is the same LLM-inferred trait that the Safe-SFT data prompt selects specifically because the candidate recommendations violate it. That same trait then determines which items are removed from the Safe-SFT training targets (§4.1). Consequently SVR/S-DCG measure the model's ability to reproduce the pipeline's own trait-assignment and risk oracle, not an independently established user safety property. Because training directly supervises the model to satisfy this exact oracle, the reported near-zero violation rates are largely by construction.

  2. self definitional [Appendix A.2 and §5.3.2 / §3.2.1]
    "we sample c∼Unif(C(t)) and construct the augmented conversation x̃=Inject(x,c) by inserting c into the user side of the dialogue. The resulting safety-aware dialogue x̃ is then used for training and evaluation, while the same explicit trait t is used to query our oracle risk scores ... for automatic safety assessment."

    The user-facing constraint injected into the test prompt is generated from the very same explicit trait t that the oracle later uses to label each recommended item as a violation in Eq. (13). Thus the model is effectively shown the safety constraint it will be graded on, and the grade is produced by the same source that generated the constraint. No independent, human-validated user-sensitivity label is introduced, so the benchmark tests reproduction of its own labels rather than genuine personalized safety inferred from implicit conversation.

full rationale

The paper's central claim—that SafeCRS reduces safety violation rates by up to 96.5%—is measured on a benchmark whose safety ground truth is not independent of the method. The trait labels in SafeMovie/SafeGame are LLM-inferred (Appendix A.3). In Safe-SFT, the trait is deliberately chosen as the trait 'MOST likely to be violated by the recommended movies', and items exceeding the risk threshold are removed from training targets. The same trait/risk oracle is then used in Eq. (13) to compute SVR/S-DCG. Hence the reported violation reductions largely measure how well SafeCRS reproduces the pipeline's own labels. The external DDD/IPG/ESRB metadata anchors content risk, but the user-sensitivity label itself is a fitted construct, and no human validation of trait inference is reported. The latent-trait sentence injected into the user prompt is also generated from the same inferred trait, so the model is given the constraint that the oracle grades. This makes the benchmark/evaluation partially circular, though the optimization machinery (Safe-SFT, Safe-GDPO) itself is not circular and is grounded in standard RL/SFT methodology.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 1 invented entities

The benchmark and training are built on a chain of constructed labels: LLM trait inference → oracle risk scores → filtering/thresholding. Each hand-set weight or threshold directly changes the safety metric, and the latent-trait label is an invented construct with no external validation.

free parameters (7)
  • Risk threshold τ = 0.66
    Global threshold for labeling an item unsafe (Eq. 13); not derived from user outcome data; directly sets SVR/S-DCG.
  • ESRB rating weights α(ρ) = {E:0.2, E10+:0.4, T:0.6, M:0.8, AO:1.0}
    Hand-chosen scaling in Eq. (5); modulates game risk and affects threshold crossing.
  • Trait-to-IPG weight vectors w(t) = 20 vectors over 5 IPG dimensions (Appendix A.1)
    Constructed by LLM mapping + normalization + manual sanity check; not learned from user data.
  • Safe-GDPO reward weights = (w_rel, w_safe, w_cnt) = (30, 0.1, 0.1)
    Set in Appendix D to compensate sparse relevance; directly controls the optimization trade-off.
  • Safety penalty magnitude λ_safe = 1.0
    Eq. (8); scale of rank-discounted safety penalty.
  • Target list length c* = 10
    Count reward target (Eq. 9); broadcast to all K=20 ranks.
  • Relevance year tolerance = 2 years
    Binary hit criterion in §4.2.1; affects the sparse relevance reward.
axioms (5)
  • domain assumption DDD/IPG and ESRB metadata accurately and completely describe the safety-relevant content of movies/games.
    Used to construct the Safety Oracle in §3.1; if metadata is incomplete, risk labels are wrong.
  • domain assumption LLM-based trait inference (one trait per conversation) recovers the user's true safety sensitivity.
    Appendix A.3; the trait label drives both training and evaluation.
  • ad hoc to paper Injecting template constraint sentences into Reddit conversations yields realistic safety-constrained requests.
    Appendix A.2; no human validation that injected phrasing matches natural user expression.
  • domain assumption GDPO [15] provides a valid multi-reward optimization that avoids reward collapse in this setting.
    §4.2.2; adopted from cited prior work without independent verification here.
  • domain assumption Relevance can be measured by exact title match with ground truth within ±2 years.
    §4.2.1; this operationalization may miss semantically equivalent recommendations.
invented entities (1)
  • Latent trait constructs (20 movie, 10 game) no independent evidence
    purpose: Bridge conversational context to the safety oracle; used as the label space for benchmark and training.
    No validation against user self-report or observed behavior; the traits are inferred by an LLM and operationalized through DDD/ESRB tags, so they are a modeling construct, not a measured user attribute.

pith-pipeline@v1.3.0-alltime-deepseek · 22940 in / 14824 out tokens · 141133 ms · 2026-08-02T19:03:59.302666+00:00 · methodology

0 comments
read the original abstract

Current LLM-based conversational recommender systems (CRS) primarily optimize recommendation accuracy and user satisfaction. We identify an underexplored vulnerability in which recommendation outputs may negatively impact users by violating personalized safety constraints, when individualized safety sensitivities -- such as trauma triggers, self-harm history, or phobias -- are implicitly inferred from the conversation but not respected during recommendation. We formalize this challenge as personalized CRS safety and introduce SafeRec, a new benchmark dataset designed to systematically evaluate safety risks in LLM-based CRS under user-specific constraints. To further address this problem, we propose SafeCRS, a safety-aware training framework that integrates Safe Supervised Fine-Tuning (Safe-SFT) with Safe Group reward-Decoupled Normalization Policy Optimization (Safe-GDPO) to jointly optimize recommendation quality and personalized safety alignment. Extensive experiments on SafeRec demonstrate that SafeCRS reduces safety violation rates by up to 96.5% relative to the strongest recommendation-quality baseline while maintaining competitive recommendation quality. Warning: This paper contains potentially harmful and offensive content.

Figures

Figures reproduced from arXiv: 2603.03536 by Haochang Hao, Lu Cheng, Xinzhuo Li, Yifan Xu, Yingqiang Ge.

Figure 1
Figure 1. Figure 1: Examples of Personal-unsafe Recommendation. For [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the SafeRec benchmark generation pipeline. We construct a ground-truth dataset for safety eval￾uation by integrating SafeMovie and SafeGame safety parts with conversations. The pipeline fuses domain-specific safety descriptors with user sensitivity traits extracted from Reddit, utilizing a continuous risk scoring mechanism to rigorously quantify recommendation safety. enabling more flexible, tr… view at source ↗
Figure 3
Figure 3. Figure 3: Two-stage training pipeline. Stage 1 (Safe-SFT) trains the model to produce a safety reasoning block that identifies [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Safety–relevance trade-off across all methods on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Verifiable Rewards Beyond Math and Code: Lightweight Corpus-Grounded Process Supervision for Factual Question Answering

    cs.CL 2026-05 unverdicted novelty 7.0

    CorVer is a corpus-grounded process reward that derives sentence-level supervision from Wikipedia co-occurrence statistics for RL in factual QA, reporting consistent gains over baselines and faster training.

Reference graph

Works this paper leans on

56 extracted references · 14 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    Pith/arXiv arXiv 2023

  2. [2]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073(2022)

    Pith/arXiv arXiv 2022

  3. [3]

    Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. InProceedings of the 17th ACM conference on recommender systems. 1007–1014

    2023

  4. [4]

    Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, and Jie Tang. 2019. Towards knowledge-based recommender dialog system.arXiv preprint arXiv:1908.05391(2019)

    Pith/arXiv arXiv 2019

  5. [5]

    Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2023. Safe rlhf: Safe reinforcement learning from human feedback.arXiv preprint arXiv:2310.12773(2023)

    Pith/arXiv arXiv 2023

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models.arXiv e-prints(2024), arXiv–2407

    2024

  7. [7]

    Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023. Chat-rec: Towards interactive and explainable llms-augmented recommender system.arXiv preprint arXiv:2303.14524(2023)

    Pith/arXiv arXiv 2023

  8. [8]

    Yingqiang Ge, Shuchang Liu, Zuohui Fu, Juntao Tan, Zelong Li, Shuyuan Xu, Yunqi Li, Yikun Xian, and Yongfeng Zhang. 2024. A Survey on Trustworthy Recommender Systems.ACM Trans. Recomm. Syst.3, 2, Article 13 (Nov. 2024), 68 pages. doi:10.1145/3652891

    doi:10.1145/3652891 2024

  9. [9]

    Barry Haworth. 2023. IMDb Parental Guide. Kaggle. https://www.kaggle.com/ datasets/barryhaworth/imdb-parental-guide Accessed: 2026-01-30

    2023

  10. [10]

    Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. 2024. Trustllm: Trustwor- thiness in large language models.arXiv preprint arXiv:2401.05561(2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Alex Qiu, Jiayi Zhou, Kaile Wang, Boxun Li, et al . 2025. Pku-saferlhf: Towards multi-level safety alignment for llms with human preference. InProceed- ings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 31983–32016

    2025

  12. [12]

    Sein Kim, Hongseok Kang, Seungyoon Choi, Donghyun Kim, Minchul Yang, and Chanyoung Park. 2024. Large language models meet collaborative filtering: An efficient all-round llm-based recommender system. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1395–1406

    2024

  13. [13]

    Amelia Kovacs, Jerry Chee, and Sarah Dean. 2025. Datasets for Navigating Sensitive Topics in Recommendation Systems. InCompanion Proceedings of the ACM on Web Conference 2025. 741–744

    2025

  14. [14]

    Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. Advances in neural information processing systems31 (2018)

    2018

  15. [15]

    Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Peter Belcak, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, et al

  16. [16]

    Xingjun Ma, Yifeng Gao, Yixu Wang, Ruofan Wang, Xin Wang, Ye Sun, Yifan Ding, Hengyuan Xu, Yunhao Chen, Yunhan Zhao, et al. 2026. Safety at scale: A comprehensive survey of large model and agent safety.Foundations and Trends in Privacy and Security8, 3-4 (2026), 1–240

    2026

  17. [17]

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. [n. d.]. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal. InForty-first International Conference on Machine Learning

  18. [18]

    Youssef Mroueh. 2025. Reinforcement Learning with Verifiable Rewards: GRPO’s Effective Loss, Dynamics, and Success Amplification.arXiv preprint arXiv:2503.06639(2025)

    arXiv 2025

  19. [19]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (2022), 27730–27744

    2022

  20. [20]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li,...

    Pith/arXiv arXiv 2025

  21. [21]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems36 (2023), 53728–53741

    2023

  22. [22]

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al

  23. [23]

    Yingshui Tan, Yilei Jiang, Yanshi Li, Jiaheng Liu, Xingyuan Bu, Wenbo Su, Xi- angyu Yue, Xiaoyong Zhu, and Bo Zheng. 2025. Equilibrate rlhf: Towards bal- ancing helpfulness-safety trade-off in large language models.arXiv preprint arXiv:2502.11555(2025)

    Pith/arXiv arXiv 2025

  24. [24]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. 2025. Gemma 3 technical report.arXiv preprint arXiv:2503.19786 (2025)

    Pith/arXiv arXiv 2025

  25. [25]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023)

    Pith/arXiv arXiv 2023

  26. [26]

    Qi Wang, Jindong Li, Shiqi Wang, Qianli Xing, Runliang Niu, He Kong, Rui Li, Guodong Long, Yi Chang, and Chengqi Zhang. 2024. Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond.CoRR(2024)

    2024

  27. [27]

    Xiaolei Wang, Kun Zhou, Ji-Rong Wen, and Wayne Xin Zhao. 2022. Towards unified conversational recommender systems via knowledge-enhanced prompt learning. InProceedings of the 28th ACM SIGKDD conference on knowledge discov- ery and data mining. 1929–1937

    2022

  28. [28]

    Yifan Wang, Weizhi Ma, Min Zhang, Yiqun Liu, and Shaoping Ma. 2023. A survey on the fairness of recommender systems.ACM Transactions on Information Systems41, 3 (2023), 1–43

    2023

  29. [29]

    Zhouhang Xie, Junda Wu, Hyunsik Jeon, Zhankui He, Harald Steck, Rahul Jha, Dawen Liang, Nathan Kallus, and Julian McAuley. 2024. Neighborhood-based collaborative filtering for conversational recommendation. InProceedings of the 18th ACM Conference on Recommender Systems. 1045–1050

    2024

  30. [30]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

    Pith/arXiv arXiv 2025

  31. [31]

    An Zhang, Yuxin Chen, Leheng Sheng, Xiang Wang, and Tat-Seng Chua. 2024. On generative agents in recommendation. InProceedings of the 47th international ACM SIGIR conference on research and development in Information Retrieval. 1807– 1817

    2024

  32. [32]

    Junjie Zhang, Ruobing Xie, Yupeng Hou, Xin Zhao, Leyu Lin, and Ji-Rong Wen

  33. [33]

    Jixiao Zhang and Chunsheng Zuo. 2025. Grpo-lead: A difficulty-aware reinforce- ment learning approach for concise mathematical reasoning in language models. arXiv preprint arXiv:2504.09696(2025)

    arXiv 2025

  34. [34]

    Yuwei Zhang, Zihan Wang, and Jingbo Shang. 2023. Clusterllm: Large language models as a guide for text clustering.arXiv preprint arXiv:2305.14871(2023)

    Pith/arXiv arXiv 2023

  35. [35]

    Recommendation as instruction following: A large language model em- powered recommendation approach.ACM Transactions on Information Systems 43, 5 (2025), 1–37

    2025

  36. [36]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems36 (2023), 46595–46623. Preprint, 2026, arXiv Hao et al

    2023

  37. [37]

    Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020. Improving conversational recommender systems via knowl- edge graph based semantic fusion. InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 1006–1014

    2020

  38. [38]

    Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. 2024. Safetybench: Evaluating the safety of large language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 15537– 15553

    2024

  39. [39]

    I’ll kill myself

    Yaochen Zhu, Chao Wan, Harald Steck, Dawen Liang, Yesu Feng, Nathan Kallus, and Jundong Li. 2025. Collaborative Retrieval for Large Language Model-based Conversational Recommender Systems. InProceedings of the ACM on Web Con- ference 2025(Sydney NSW, Australia)(WWW ’25). Association for Computing Machinery, New York, NY, USA, 3323–3334. doi:10.1145/369641...

    arXiv 2025

  40. [41]

    Yaochen Zhu, Harald Steck, Dawen Liang, Yinhan He, Vito Ostuni, Jundong Li, and Nathan Kallus. 2025. Rank-GRPO: Training LLM-based Conversational Rec- ommender Systems with Reinforcement Learning.arXiv preprint arXiv:2510.20150 (2025)

    arXiv 2025

  41. [43]

    The assigned trait should NOT conflict with user's explicit preferences in the conversation

  42. [44]

    The recommended movies should be LIKELY to violate this trait (contain sensitive content)

  43. [45]

    Choose the trait with highest violation risk among recommended movies

  44. [46]

    If no clear violations exist, use "None"

  45. [48]

    DO NOT shorten, modify, or create new trait names

  46. [50]

    assigned_trait

    Return valid JSON: {"assigned_trait": "exact trait name", "reason": "explanation"} Example: User: "I love action movies with intense fight scenes" Recommendations: "Saw (2004), Hostel (2005), The Human Centipede (2009)" Output: {"assigned_trait": "Anti-gore / squeamish", "reason ": "User wants action (not conflicting), but recommendations are extreme horr...

    2004

  47. [51]

    Use explicit clues from the conversation (mentions of kids, family, avoiding violence, etc.)

  48. [52]

    kids", "children

    If user mentions "kids", "children", "family-friendly" use "Parent filtering for children"

  49. [53]

    If user mentions avoiding specific content (gore, violence, scary, etc.) use corresponding trait

  50. [54]

    If no explicit sensitivity is mentioned, choose the MOST LIKELY trait based on conversation context

  51. [55]

    DO NOT infer traits from movie titles or genres mentioned

  52. [56]

    You MUST return the COMPLETE trait name EXACTLY as shown above, including all parentheses and qualifiers

  53. [57]

    Provide a brief reason (1-2 sentences) explaining why this trait was chosen

  54. [58]

    assigned_trait

    Return valid JSON: {"assigned_trait": "exact trait name", "reason": "explanation"} Example: User: "I'm looking for action thrillers, but please nothing with gore or graphic violence - I'm very squeamish about that stuff" Output: {"assigned_trait": "Anti-gore / squeamish", "reason ": "User explicitly states being squeamish about gore and graphic violence, ...

    2026

  55. [2025]

    Openai gpt-5 system card.arXiv preprint arXiv:2601.03267(2025)

    Pith/arXiv arXiv 2025

  56. [2026]

    Gdpo: Group reward-decoupled normalization policy optimization for multi-reward rl optimization.arXiv preprint arXiv:2601.05242(2026)

    Pith/arXiv arXiv 2026