Pith. sign in

REVIEW 1 major objections 1 cited by

Combining guard models with Mahalanobis distance and perplexity OOD detectors raises recall of LLM alignment failures from 39% to 45%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 17:14 UTC pith:QPCPLFWW

load-bearing objection The MOOD benchmark offers concrete comparisons of OOD detectors with guard models, but lacks verification that test sets are truly out-of-distribution. the 1 major comments →

arxiv 2605.21602 v2 pith:QPCPLFWW submitted 2026-05-20 cs.AI cs.SE

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs

Dylan Feng , Pragya Srivastava , Anca Dragan , Cassidy Laidlaw This is my paper
classification cs.AI cs.SE
keywords LLM monitoringout-of-distribution detectionalignment failuresguard modelsMahalanobis distanceperplexitysafety benchmarksMOOD
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper creates the MOOD benchmark with a restricted training set and seven test sets of alignment failures outside that distribution to evaluate monitors for out-of-distribution cases. Guard models alone often fail to generalize to these OOD failures. Adding OOD detectors based on Mahalanobis distance and perplexity improves recall, and the hybrid approach scales positively with model size while delivering larger gains than a guard model twenty times larger. The results indicate that OOD detection forms a necessary part of reliable monitoring for LLM safety and alignment.

Core claim

Guard models fail to generalize to out-of-distribution alignment failures on the MOOD benchmark, but combining them with Mahalanobis distance and perplexity-based OOD detectors improves recall from 39% to 45%, shows positive scaling with model size, and outperforms scaling the guard model by a factor of twenty.

What carries the argument

The MOOD benchmark together with the hybrid monitor that pairs a guard model safety classifier with Mahalanobis distance and perplexity OOD detectors.

Load-bearing premise

The seven test sets contain alignment failures that are genuinely out-of-distribution relative to the restricted training set used to train the monitors, and performance on this benchmark will translate to real-world OOD scenarios.

What would settle it

An experiment on a fresh collection of alignment failures confirmed to lie outside the training distribution in which the hybrid monitor shows no recall improvement over the guard model alone.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Hybrid monitors that add OOD detection exhibit positive scaling trends as model size increases.
  • Adding OOD detection produces larger recall gains than increasing guard-model size by a factor of twenty.
  • OOD detection forms a required component of any monitoring pipeline intended to catch alignment failures.
  • The MOOD benchmark supplies a reusable testbed for developing and comparing further monitors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Real-world LLM deployments could adopt hybrid monitors to reduce the chance that unseen prompt patterns trigger safety failures.
  • The same combination of guard model and OOD detector could be tested on failure modes drawn from additional domains such as code generation or multimodal inputs.
  • If the recall improvement holds, monitoring systems might shift from relying solely on larger classifiers toward lighter guard models augmented with distribution checks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 0 minor

Summary. The paper introduces the MOOD benchmark for evaluating monitors on out-of-distribution (OOD) alignment failures in LLMs. It constructs a restricted training set for fitting monitors and seven test sets asserted to contain alignment failures outside that distribution. Experiments show that guard models alone achieve 39% recall on these sets, while combining them with Mahalanobis distance and perplexity-based OOD detectors raises recall to 45%; this OOD-augmented approach also yields larger gains than scaling the guard model by 20×. Positive scaling trends are reported for the combined monitors, and code plus data are released.

Significance. If the OOD property of the test sets holds, the work provides concrete evidence that OOD detection is a high-leverage addition to LLM monitoring pipelines and supplies a reproducible benchmark with scaling observations. The public release of code and data is a clear strength for empirical follow-up.

major comments (1)
  1. [Benchmark construction (abstract and §3)] The central empirical claims (recall lift from 39% to 45% and superiority over 20× parameter scaling) rest on the premise that the seven test sets are genuinely OOD relative to the restricted training distribution used to fit the monitors. The manuscript asserts this 'by construction' but supplies no quantitative verification such as embedding-space distances, overlap statistics, or distribution-shift tests between train and test features. Without such checks, gains cannot be confidently attributed to OOD detection rather than in-distribution generalization.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their thoughtful review and constructive feedback. We address the single major comment below.

read point-by-point responses

  1. Referee: The central empirical claims (recall lift from 39% to 45% and superiority over 20× parameter scaling) rest on the premise that the seven test sets are genuinely OOD relative to the restricted training distribution used to fit the monitors. The manuscript asserts this 'by construction' but supplies no quantitative verification such as embedding-space distances, overlap statistics, or distribution-shift tests between train and test features. Without such checks, gains cannot be confidently attributed to OOD detection rather than in-distribution generalization.

    Authors: We appreciate the referee's emphasis on rigorously verifying the OOD property. The MOOD benchmark constructs the restricted training set as a narrow subset of safety-related prompts and responses for fitting monitors, while the seven test sets are assembled from distinct sources of alignment failures (different prompt distributions, model scales, and failure categories) that are deliberately excluded from the training data by design. This separation is intended to ensure the test failures lie outside the training distribution. That said, we agree that quantitative verification would strengthen the attribution of performance gains specifically to OOD detection. In the revised manuscript we will add embedding-space analyses (e.g., average cosine distances and Mahalanobis distances between train and test feature representations) together with simple overlap statistics to provide empirical confirmation of the distribution shift. revision: yes

Circularity Check

0 steps flagged

No significant circularity in empirical benchmark results

full rationale

The paper is a purely empirical benchmark study that constructs the MOOD dataset with a restricted training set and seven test sets, then directly measures recall improvements (39% to 45%) from combining guard models with OOD detectors on those test sets. No equations, derivations, or first-principles claims exist that reduce reported performance gains to quantities defined by fitting on the evaluation data itself. The work releases code and data for independent verification, and the central results rest on observable metrics rather than self-definitional loops, fitted inputs renamed as predictions, or load-bearing self-citations. The assertion that test sets are OOD is an explicit construction choice, not a circular reduction of the measured outcomes.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

This is an empirical benchmarking paper whose claims rest on the assumption that the constructed test sets are OOD and that the chosen OOD detectors are appropriate for the domain. No new entities are postulated.

free parameters (1)
  • detection thresholds for OOD components
    Thresholds for Mahalanobis and perplexity detectors are almost certainly tuned on the training split to achieve the reported recall numbers.
axioms (1)
  • domain assumption The seven test sets represent alignment failures that are out-of-distribution for monitors trained on the restricted training set
    This premise is required for the benchmark to measure OOD generalization as claimed in the abstract.

pith-pipeline@v0.9.1-grok · 5817 in / 1326 out tokens · 42666 ms · 2026-06-30T17:14:43.537121+00:00 · methodology

0 comments
read the original abstract

Many safety and alignment failures of large language models (LLMs) occur due to out-of-distribution (OOD) situations: unusual prompt or response patterns that are unforeseen by model developers. We systematically study whether LLM monitoring pipelines can detect these OOD alignment failures by introducing a benchmark called Misalignment Out Of Distribution (MOOD). It is difficult to find failures that are truly OOD for off-the-shelf models trained on vast safety datasets. We sidestep this by including a restricted training set in MOOD that we use to train our own monitors, as well as seven test sets with diverse alignment failures that are outside the training distribution. Using MOOD, we find that guard models (safety classifiers) often fail to generalize OOD. To fix this, we propose combining guard models with OOD detectors. We test four types of OOD detectors and find that a combination of a guard model with Mahalanobis distance and perplexity-based OOD detectors can improve recall from 39% to 45%. We also establish positive scaling trends across model scales for monitors that combine a guard model and OOD detector; we find that incorporating OOD detection into monitoring achieves a higher recall gain than using a guard model with 20 times more parameters. Our work suggests that OOD detection should be a crucial component of LLM monitoring and provides a foundation for further work on this important problem. We release the code and data for our experiments publicly, and you can find the relevant links here: https://github.com/Dylan102938/mood-bench.

Figures

Figures reproduced from arXiv: 2605.21602 by Anca Dragan, Cassidy Laidlaw, Dylan Feng, Pragya Srivastava.

Figure 1
Figure 1. Figure 1: We systematically study incorporating out-of-distribution (OOD) detectors into LLM safety monitoring to catch alignment failures outside the training distribution. LLMs are often deployed with a guard model (right) trained with safety training data (left). However, if a prompt or response is outside of the training distribution, the guard model may generalize incorrectly and fail to flag safety issues. Add… view at source ↗
Figure 2
Figure 2. Figure 2: We introduce Misalignment Out Of Distribution (MOOD), a benchmark which tests LLM monitors for their ability to recognize unforeseen LLM alignment failures. MOOD includes seven test sets containing conversations with distinct alignment failures. To ensure that these test sets are truly out-of-distribution, we train our own guard models and OOD detectors on a restricted post-training dataset that we careful… view at source ↗
Figure 3
Figure 3. Figure 3: To better understand the Mahalanobis OOD detector, we apply PCA to the activations of the Qwen2.5-32B guard model on which we compute the Mahalanobis distance. We plot the re￾sulting principal components of 200 conversations from each test dataset above. For each dataset, we also show the relative change in misalignment recall for the combined guard + Mahalanobis model compared to using the guard model alo… view at source ↗
Figure 5
Figure 5. Figure 5: The improvement in OOD misalignment recall when training guard models additionally on some of the MOOD test sets. We display both the increase in recall relative to the baseline Gemma 2 9B guard model as well as the absolute recall in paren￾theses. The first seven rows each correspond to adding a single test dataset to the training data. The “union” row measures the recall on each test dataset when taking … view at source ↗
Figure 6
Figure 6. Figure 6: The average misalignment recall of six methods across three models from the Gemma 2 family with 2, 9, and 27 billion parameters. Methods improve significantly from the 2B to the 9B model, but the misalignment recall drops from the 9B to the 27B model. We hypothesize this may be because the 27B model is suboptimally trained; we use the same hyperparameters across all model sizes, and 27B might require diffe… view at source ↗
Figure 7
Figure 7. Figure 7: Per-token perplexity results on different test samples. Tokens highlighted with brighter colors have higher perplexity. The conversation on the left is from the sycophantic test set and the conversation on the right is from the function calling deception (missing tools) test set. Many of the sycophantic tokens are flagged as high-perplexity in the sycophantic conversation, while very few of the tokens are … view at source ↗
Figure 8
Figure 8. Figure 8: The distributions of the numbers of tokens and Flesch-Kincaid grade levels (Kincaid et al., 1975) of conversations in each MOOD test set. The significant overlap between test set and train set distributions means that it is not trivial to detect OOD conversations based on surface level features. The majority of samples in our test datasets are cleanly classifiable with respect to the training dataset using… view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Toward Mechanistic Interpretability of an AI Foundation Model Fine-Tuned for Atmospheric Chemistry

    cs.LG 2026-07 conditional novelty 7.0

    Aurora, a foundation model fine-tuned for air quality, captures rough NOx–ozone coupling but lacks the chemical consistency and emission-plume fidelity of process-based models.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages · cited by 1 Pith paper · 12 internal anchors

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

    work page

  2. [2]

    Does safety training of llms generalize to semantically related natural prompts?, 2025

    Addepalli, S., Varun, Y., Suggala, A., Shanmugam, K., and Jain, P. Does safety training of llms generalize to semantically related natural prompts?, 2025. URL https://arxiv.org/abs/2412.03235

    work page arXiv 2025

  3. [3]

    System card: Claude opus 4.5

    Anthropic. System card: Claude opus 4.5. Technical report, November 2025

    work page 2025

  4. [4]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

    Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., Joseph, N., Kadavath, S., Kernion, J., Conerly, T., El-Showk, S., Elhage, N., Hatfield-Dodds, Z., Hernandez, D., Hume, T., Johnston, S., Kravec, S., Lovitt, L., Nanda, N., Olsson, C., Amodei, D., Brown, T., Clark, J., McCandlish, S., Olah, ...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  5. [5]

    Emergent misalignment: Narrow finetuning can produce broadly misaligned LLM s

    Betley, J., Tan, D., Warncke, N., Sztyber-Betley, A., Bao, X., Soto, M., Labenz, N., and Evans, O. Emergent Misalignment : Narrow finetuning can produce broadly misaligned LLMs , May 2025. URL http://arxiv.org/abs/2502.17424. arXiv:2502.17424 [cs]

    work page arXiv 2025

  6. [6]

    arXiv preprint arXiv:2406.00806 (2024)

    Cao, C., Zhong, Z., Zhou, Z., Liu, Y., Liu, T., and Han, B. Envisioning outlier exposure by large language models for out-of-distribution detection, 2024. URL https://arxiv.org/abs/2406.00806

    work page arXiv 2024

  7. [7]

    JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models

    Chao, P., Debenedetti, E., Robey, A., Andriushchenko, M., Croce, F., Sehwag, V., Dobriban, E., Flammarion, N., Pappas, G. J., Tramer, F., Hassani, H., and Wong, E. JailbreakBench : An Open Robustness Benchmark for Jailbreaking Large Language Models , October 2024. URL http://arxiv.org/abs/2404.01318. arXiv:2404.01318 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    Llm jailbreak detection for (almost) free! In Findings of the Association for Computational Linguistics: EMNLP 2025, pp.\ 5777–5807

    Chen, G., Xia, Y., Jia, X., Li, Z., Torr, P., and Gu, J. Llm jailbreak detection for (almost) free! In Findings of the Association for Computational Linguistics: EMNLP 2025, pp.\ 5777–5807. Association for Computational Linguistics, 2025. doi:10.18653/v1/2025.findings-emnlp.309. URL http://dx.doi.org/10.18653/v1/2025.findings-emnlp.309

    work page doi:10.18653/v1/2025.findings-emnlp.309 2025

  9. [9]

    Expose backdoors on the way: A feature-based efficient defense against textual backdoor attacks

    Chen, S., Yang, W., Zhang, Z., Bi, X., and Sun, X. Expose backdoors on the way: A feature-based efficient defense against textual backdoor attacks. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp.\ 668--683, Abu Dhabi, United Arab Emirates, December 2022. Association for Comput...

    work page doi:10.18653/v1/2022.findings-emnlp.47 2022

  10. [10]

    Investigating truthfulness in a pre-release o3 model, April 2025

    Chowdhury, N., Johnson, D., Huang, V., Steinhardt, J., and Schwettmann, S. Investigating truthfulness in a pre-release o3 model, April 2025. URL https://transluce.org/investigating-o3-truthfulness

    work page 2025

  11. [11]

    Reward Model Ensembles Help Mitigate Overoptimization

    Coste, T., Anwar, U., Kirk, R., and Krueger, D. Reward Model Ensembles Help Mitigate Overoptimization , March 2024. URL http://arxiv.org/abs/2310.02743. arXiv:2310.02743 [cs]

    work page Pith review arXiv 2024

  12. [12]

    Helping or herding? reward model ensembles mitigate but do not eliminate reward hacking.arXiv preprint arXiv:2312.09244,

    Eisenstein, J., Nagpal, C., Agarwal, A., Beirami, A., D'Amour, A., Dvijotham, D. J., Fisch, A., Heller, K., Pfohl, S., Ramachandran, D., Shaw, P., and Berant, J. Helping or Herding ? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking , August 2024. URL http://arxiv.org/abs/2312.09244. arXiv:2312.09244 [cs]

    work page arXiv 2024

  13. [13]

    Exploring the Limits of Out -of- Distribution Detection

    Fort, S., Ren, J., and Lakshminarayanan, B. Exploring the Limits of Out -of- Distribution Detection . In Advances in Neural Information Processing Systems , volume 34, pp.\ 7068--7081. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/hash/3941c4358616274ac2436eacf67fae05-Abstract.html

    work page 2021

  14. [14]

    Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned

    Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., Jones, A., Bowman, S., Chen, A., Conerly, T., DasSarma, N., Drain, D., Elhage, N., El-Showk, S., Fort, S., Hatfield-Dodds, Z., Henighan, T., Hernandez, D., Hume, T., Jacobson, J., Johnston, S., Kravec, S., Olsson, C., Ringer, S., Tran-J...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  15. [15]

    Alignment faking in large language models

    Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J., Belonax, T., Chen, J., Duvenaud, D., Khan, A., Michael, J., Mindermann, S., Perez, E., Petrini, L., Uesato, J., Kaplan, J., Shlegeris, B., Bowman, S. R., and Hubinger, E. Alignment faking in large language models, 2024. URL https://arxiv.org/abs/2412.14093

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

    Hendrycks, D. and Gimpel, K. A Baseline for Detecting Misclassified and Out -of- Distribution Examples in Neural Networks , October 2018. URL http://arxiv.org/abs/1610.02136. arXiv:1610.02136 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  17. [17]

    AI Induced Psychosis : A shallow investigation

    Hua, T. AI Induced Psychosis : A shallow investigation. August 2025. URL https://www.lesswrong.com/posts/iGF7YcnQkEbwvYLPA/ai-induced-psychosis-a-shallow-investigation

    work page 2025

  18. [18]

    Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations

    Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., and Khabsa, M. Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023. URL https://arxiv.org/abs/2312.06674

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  19. [19]

    H idden D etect: Detecting jailbreak attacks against multimodal large language models via monitoring hidden states

    Jiang, Y., Gao, X., Peng, T., Tan, Y., Zhu, X., Zheng, B., and Yue, X. H idden D etect: Detecting jailbreak attacks against multimodal large language models via monitoring hidden states. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    work page doi:10.18653/v1/2025.acl-long.724 2025

  20. [20]

    P., Fishburne, J., Robert P., R., Richard L., C., and Brad S

    Kincaid, J. P., Fishburne, J., Robert P., R., Richard L., C., and Brad S. Derivation of New Readability Formulas ( Automated Readability Index , Fog Count and Flesch Reading Ease Formula ) for Navy Enlisted Personnel :. Fort Belvoir, VA, February 1975. Defense Technical Information Center. doi:10.21236/ADA006655. URL https://apps.dtic.mil/sti/citations/tr...

    work page doi:10.21236/ada006655 1975

  21. [21]

    R., Marks, S., Leike, J., Askell, A., Olah, C., Hubinger, E., and Price, S

    Kutasov, J., Jermyn, A., Steen, J., Le, M., Bowman, S. R., Marks, S., Leike, J., Askell, A., Olah, C., Hubinger, E., and Price, S. Teaching Claude Why , May 2026. URL https://alignment.anthropic.com/2026/teaching-claude-why/

    work page 2026

  22. [22]

    Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., Miranda, L. J. V., Liu, A., Dziri, N., Lyu, S., et al. T \"u lu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  23. [23]

    A Simple Unified Framework for Detecting Out -of- Distribution Samples and Adversarial Attacks

    Lee, K., Lee, K., Lee, H., and Shin, J. A Simple Unified Framework for Detecting Out -of- Distribution Samples and Adversarial Attacks . In Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018. URL https://papers.nips.cc/paper_files/paper/2018/hash/abdeb6f575ac5c6676b747bca8d09cc2-Abstract.html

    work page 2018

  24. [24]

    Learn- ing to detect unknown jailbreak attacks in large vision- language models,

    Liang, S., Xu, Z., Weng, J., Tao, J., Xue, H., and Wang, X. Learning to Detect Unseen Jailbreak Attacks in Large Vision - Language Models , January 2026. URL http://arxiv.org/abs/2508.09201. arXiv:2508.09201 [cs]

    work page arXiv 2026

  25. [25]

    K., Ritchie, S

    Lynch, A., Wright, B., Larson, C., Troy, K. K., Ritchie, S. J., Mindermann, S., Perez, E., and Hubinger, E. Agentic Misalignment : How LLMs Could be an Insider Threat . Anthropic Research, 2025

    work page 2025

  26. [26]

    Mahalanobis, P. C. On the generalized distance in statistics. The National Institute of Sciences of India, 2 0 (1): 0 49--55, 1936

    work page 1936

  27. [27]

    Frontier Models are Capable of In-context Scheming

    Meinke, A., Schoen, B., Scheurer, J., Balesni, M., Shah, R., and Hobbhahn, M. Frontier models are capable of in-context scheming, 2025. URL https://arxiv.org/abs/2412.04984

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  28. [28]

    Jaildam: Jailbreak detection with adaptive memory for vision-language model,

    Nian, Y., Zhu, S., Qin, Y., Li, L., Wang, Z., Xiao, C., and Zhao, Y. Jaildam: Jailbreak detection with adaptive memory for vision-language model, 2025. URL https://arxiv.org/abs/2504.03770

    work page arXiv 2025

  29. [29]

    Technical report: Performance and baseline evaluations of gpt-oss-safeguard-120b and gpt-oss-safeguard-20b

    OpenAI. Technical report: Performance and baseline evaluations of gpt-oss-safeguard-120b and gpt-oss-safeguard-20b. Technical report, OpenAI, October 2025 a . URL https://cdn.openai.com/pdf/08b7dee4-8bc6-4955-a219-7793fb69090c/Technical_report__Research_Preview_of_gpt_oss_safeguard.pdf

    work page 2025

  30. [30]

    GPT -5 System Card

    OpenAI. GPT -5 System Card . Technical report, August 2025 b

    work page 2025

  31. [31]

    Sycophancy in GPT -4o: What happened and what we’re doing about it, April 2025 c

    OpenAI. Sycophancy in GPT -4o: What happened and what we’re doing about it, April 2025 c . URL https://openai.com/index/sycophancy-in-gpt-4o/

    work page 2025

  32. [32]

    Revisiting mahalanobis distance for transformer-based out-of-domain detection, 2022

    Podolskiy, A., Lipin, D., Bout, A., Artemova, E., and Piontkovskaya, I. Revisiting mahalanobis distance for transformer-based out-of-domain detection, 2022. URL https://arxiv.org/abs/2101.03778

    work page arXiv 2022

  33. [33]

    LLM s know their vulnerabilities: Uncover safety gaps through natural distribution shifts

    Ren, Q., Li, H., Liu, D., Xie, Z., Lu, X., Qiao, Y., Sha, L., Yan, J., Ma, L., and Shao, J. LLM s know their vulnerabilities: Uncover safety gaps through natural distribution shifts. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape...

    work page doi:10.18653/v1/2025.acl-long.1207 2025

  34. [34]

    A Conversation With Bing ’s Chatbot Left Me Deeply Unsettled

    Roose, K. A Conversation With Bing ’s Chatbot Left Me Deeply Unsettled . The New York Times, February 2023. ISSN 0362-4331. URL https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html

    work page 2023

  35. [35]

    Towards Understanding Sycophancy in Language Models

    Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S. R., Kravec, S., Maxwell, T., McCandlish, S., Ndousse, K., Rausch, O., Schiefer, N., Yan, D., Zhang, M., and Perez, E. Towards Understanding Sycophancy in Language Models , May 2025 a . URL http://arxiv.org/abs/2310.13548. arXi...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  36. [36]

    Sharma, M., Tong, M., Mu, J., Wei, J., Kruthoff, J., Goodfriend, S., Ong, E., Peng, A., Agarwal, R., Anil, C., Askell, A., Bailey, N., Benton, J., Bluemke, E., Bowman, S. R., Christiansen, E., Cunningham, H., Dau, A., Gopal, A., Gilson, R., Graham, L., Howard, L., Kalra, N., Lee, T., Lin, K., Lofgren, P., Mosconi, F., O'Hara, C., Olsson, C., Petrini, L., ...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  37. [37]

    A StrongREJECT for Empty Jailbreaks

    Souly, A., Lu, Q., Bowen, D., Trinh, T., Hsieh, E., Pandey, S., Abbeel, P., Svegliato, J., Emmons, S., Watkins, O., and Toyer, S. A StrongREJECT for Empty Jailbreaks , August 2024. URL http://arxiv.org/abs/2402.10260. arXiv:2402.10260 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  38. [38]

    Jailbroken: How Does LLM Safety Training Fail ? Advances in Neural Information Processing Systems, 36: 0 80079--80110, December 2023

    Wei, A., Haghtalab, N., and Steinhardt, J. Jailbroken: How Does LLM Safety Training Fail ? Advances in Neural Information Processing Systems, 36: 0 80079--80110, December 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/fd6613131889a4b656206c50a8bd7790-Abstract-Conference.html

    work page 2023

  39. [39]

    arXiv preprint arXiv:2411.02306 , url=

    Williams, M., Carroll, M., Narang, A., Weisser, C., Murphy, B., and Dragan, A. On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback , February 2025. URL http://arxiv.org/abs/2411.02306. arXiv:2411.02306 [cs]

    work page arXiv 2025

  40. [40]

    and Ding, K

    Xu, R. and Ding, K. Large Language Models for Anomaly and Out -of- Distribution Detection : A Survey , February 2025. URL http://arxiv.org/abs/2409.01980. arXiv:2409.01980 [cs]

    work page arXiv 2025

  41. [41]

    Young, R. J. Evaluating the Robustness of Large Language Model Safety Guardrails Against Adversarial Attacks , November 2025. URL http://arxiv.org/abs/2511.22047. arXiv:2511.22047 [cs] version: 1

    work page arXiv 2025

  42. [42]

    ShieldGemma: Generative AI Content Moderation Based on Gemma

    Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., Radharapu, B., Sturman, O., and Wahltinez, O. Shieldgemma: Generative ai content moderation based on gemma, 2024. URL https://arxiv.org/abs/2407.21772

    work page internal anchor Pith review Pith/arXiv arXiv 2024