Pith. sign in

REVIEW 2 major objections 4 minor 57 references

Adversarial training on 66 personality statements gives near-zero jailbreak success without harm data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 15:25 UTC pith:YN4LFUT7

load-bearing objection Clean empirical win: 66 psychometric items + LAT give near-zero HarmBench ASR without harmful training data, but the mechanism claim is still under-specified. the 2 major comments →

arxiv 2607.07918 v1 pith:YN4LFUT7 submitted 2026-07-08 cs.LG cs.AIcs.CLcs.CR

Efficient Safety Alignment of Language Models via Latent Personality Traits

Mohamed Amine Merzouk , Nolan Smyth , Damiano Fornasiere , Linh Le , David Williams-King , Adam Oberman This is my paper
classification cs.LG cs.AIcs.CLcs.CR
keywords latent adversarial trainingpersonality alignmentjailbreak robustnessHarmBenchBig Five traitssafety alignmentdata-efficient post-training
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that large language models can be made robust to jailbreaks by adversarially stabilizing a small set of harm-agnostic personality statements rather than by training on thousands of explicit harmful prompts. The authors take 66 short items from psychometric inventories for Conscientiousness, Agreeableness, and Emotional Stability, frame them as self-assessment agreements or disagreements, and apply latent adversarial training so that the desired trait-consistent answers survive perturbations inside the model. They claim this works because the latent structure that encodes those personality traits overlaps the subspace that jailbreaks exploit; locking the traits down therefore blocks the attacks. On HarmBench, the resulting models reach near-zero attack success rates against direct harmful requests and five jailbreak methods while keeping accuracy on standard capability benchmarks, and the whole procedure finishes in minutes on one GPU with roughly seventy-five times fewer examples than ordinary latent adversarial training.

Core claim

Latent adversarial training performed only on 66 harm-agnostic psychometric statements for Conscientiousness, Agreeableness and Emotional Stability is enough to drive attack success rates on HarmBench (direct requests plus five jailbreak methods) to near zero while preserving utility on MMLU, GSM8K and TruthfulQA, without ever exposing the model to harmful content or needing a supervised utility-recovery stage.

What carries the argument

Latent Personality Alignment (LPA): latent adversarial training that forces trait-consistent agree/disagree completions on the 66 psychometric statements to remain correct under bounded perturbations of the model’s hidden states.

Load-bearing premise

The claim rests on the idea that the internal directions that encode these three personality traits substantially overlap the directions that jailbreaks use; if that overlap is weak, the robustness would not transfer.

What would settle it

Run the identical LPA procedure on a model family whose personality-related activations are known (by independent probing) to be nearly orthogonal to the directions activated by successful jailbreaks, then check whether HarmBench attack success rates still fall to near zero.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Safety post-training can be performed with a few dozen public psychometric items and no harmful-content dataset.
  • Utility recovery via large-scale supervised fine-tuning is unnecessary when the adversarial objective is confined to personality statements.
  • Jailbreak robustness can generalize to attack families never seen in training if the shared latent structure hypothesis holds.
  • Trait selection and label polarity matter: negative-only statements for the three safety-linked traits give the best safety-utility trade-off among the ablations tested.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the shared-subspace hypothesis is correct, other compact psychometric batteries (not just Big-Five facets) could serve as drop-in safety anchors for different model families.
  • The same latent-stabilization recipe might reduce emergent misalignment after fine-tuning on narrow benign tasks, because those failures also appear to ride on persona drift.
  • A natural next measurement is whether LPA-trained models still refuse under multi-turn instruction drift or under activation-steering attacks that target the assistant persona directly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes Latent Personality Alignment (LPA): latent adversarial training (LAT) applied only to 66 harm-agnostic IPIP psychometric statements for Conscientiousness, Agreeableness and Emotional Stability, framed as a binary agree/disagree self-assessment task. The central claim is that adversarially stabilizing these personality-anchored representations constrains the latent subspace used by jailbreaks, yielding near-zero Attack Success Rate on HarmBench direct requests and five adaptive/non-adaptive jailbreaks (GCG, PAIR, AutoPrompt, AutoDAN, TAP) on Qwen3-8B (and competitive results on Llama-3-8B), while preserving utility on MMLU, GSM8K and TruthfulQA. Training uses 75 imes fewer examples than targeted LAT, never exposes the model to HarmBench or explicit refusals of harmful content, finishes in minutes on one GPU, and requires no post-hoc supervised utility recovery. Extensive ablations (trait polarity, relevance, label consistency, full IPIP) and training curves support efficiency and a favorable safety–utility Pareto frontier.

Significance. If the result holds under stronger controls, LPA would be a practically important contribution: a data-efficient, harm-agnostic post-training recipe that matches or exceeds targeted LAT robustness without cataloguing harms or sacrificing utility. The empirical package is strong—error bars over 8 runs, multi-attack evaluation including white-box adaptive attacks, honest early-stopping of the LAT baseline at the utility-preserving checkpoint, and clear Pareto plots (Figs. 1, 3–5). Code availability and the compact, public-domain training set further raise reproducibility value. The work also productively bridges psychometric trait literature with latent adversarial training, offering a concrete alternative to both large-scale refusal training and single-direction activation steering.

major comments (2)
  1. §3.2 system prompt and §3.5 evaluation protocol: the training system prompt explicitly injects a refusal template (“If answering a question is incompatible with your values, you can refuse to answer, by saying ‘I’m sorry, I cannot comply with that request’”). Evaluation uses only the minimal prompt “You are a helpful assistant.” Ablations (Figs. 4–5, §5) vary trait polarity, relevance and label consistency but never remove or replace this refusal clause while holding the 66 statements fixed. Consequently it remains possible that LAT is primarily hardening an already-present refusal framing rather than discovering a shared personality–harm subspace. A load-bearing control—identical LAT on the same 66 statements with the refusal sentence deleted or replaced by a neutral instruction—is required to substantiate the “personality-only / never seeing harmful content” mechanistic claim.
  2. §3.1 hypothesis and Appendix B: the claim that personality-anchored representations share latent structure with harm avoidance is supported only by transfer of ASR, not by direct geometric or causal evidence (e.g., cosine similarity or linear probes between trait directions and known refusal/harm directions, or intervention experiments). On Llama-3-8B (Fig. 6) LPA underperforms LAT on direct requests while remaining competitive on jailbreaks; without mechanistic diagnostics it is hard to judge whether the hypothesized subspace overlap is model-family-dependent or whether the result is largely an empirical recipe. At minimum, a short analysis of activation overlap or a layer-wise ablation would strengthen the central hypothesis.
minor comments (4)
  1. Figure 1 caption and §4.1: clarify that LAT ASR numbers are higher than in the original LAT paper precisely because training was stopped at the utility-preserving checkpoint; this is methodologically fair but should be stated more prominently so readers do not mis-compare absolute numbers.
  2. §3.3 and Table 1: the main text states that the default recipe trains on negative statements only, yet Table 1 is captioned as the “Subset + and −” ablation set. Make the exact 66-statement list used for the main result explicit (or release it in the anonymized repo) to avoid ambiguity.
  3. Limitations: the LLM-as-judge caveat is acknowledged; a short note on whether any human spot-checks were performed on the near-zero ASR cells would increase confidence.
  4. Typographical: “LP A” spacing artifact appears once in §4.1; “an anonymized repository” is fine for review but should be updated to a permanent link upon acceptance.

Circularity Check

0 steps flagged

No circularity: LPA is an empirical post-training method whose safety claims are measured on held-out HarmBench and external jailbreaks never used in training.

full rationale

The paper advances an empirical hypothesis (personality-anchored latent directions share structure with harm avoidance) and tests it by running latent adversarial training exclusively on 66 IPIP psychometric statements that contain no harmful content or refusals, then measuring Attack Success Rate on the independent HarmBench suite plus five external jailbreak algorithms (GCG, PAIR, AutoPrompt, AutoDAN, TAP) and standard utility benchmarks. No free parameters are fitted to the evaluation ASR; the training objective is simply to produce the prescribed agree/disagree completions under latent perturbations. Trait selection is justified by external psychometric literature, not by a self-citation uniqueness theorem or by reverse-engineering the test set. Ablations vary polarity, relevance and label consistency of the same statements and still evaluate on the same held-out metrics. Consequently the reported near-zero ASR is not forced by construction, by a fitted input renamed as prediction, or by a load-bearing self-citation chain. The derivation chain is therefore self-contained against external benchmarks and exhibits no circular steps under the stated criteria.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The central claim rests on one domain hypothesis (latent personality–harm overlap), a small set of free training choices (which 66 statements, negative-only polarity, system-prompt wording, early-stopping epoch), and standard LAT machinery taken from prior work. No new physical or mathematical entities are postulated; the “invented” object is simply the training recipe itself.

free parameters (4)
  • number and polarity of personality statements
    66 negative-only IPIP items for Conscientiousness, Agreeableness and Emotional Stability were chosen by hand; ablations show other choices degrade the safety–utility frontier.
  • training steps / early-stopping epoch
    Stopped at epoch 30 when utility begins to drop; the exact checkpoint is selected post-hoc for the reported Pareto point.
  • system-prompt framing text
    The long self-assessment instruction is fixed by the authors; ablations imply it contributes to the result.
  • LAT perturbation budget and layer choice
    Inherited from prior LAT work but re-tuned on Qwen3-8B; exact values not exhaustively reported.
axioms (4)
  • domain assumption Personality-anchored latent representations share structure with harm-avoidance directions, so stabilizing the former constrains jailbreak subspaces.
    Stated as the central hypothesis in the abstract and §3.1; never independently verified outside the observed ASR drop.
  • domain assumption Latent adversarial training (bounded activation perturbations) is a valid and more efficient proxy for input-space adversarial robustness.
    Taken from Sheshadri et al. / Casper et al. and used without re-derivation.
  • domain assumption IPIP Big-Five items for Conscientiousness, Agreeableness and Emotional Stability are valid proxies for “safe” personality.
    Justified by citation to Barrick & Mount and recent AI-safety personality papers; selection is not derived from first principles.
  • standard math Standard mathematical and optimization machinery of gradient-based LAT (Goodfellow-style min-max in latent space).
    Classical adversarial-training formulation applied to activations.
invented entities (1)
  • Latent Personality Alignment (LPA) training recipe no independent evidence
    purpose: The concrete procedure that replaces harmful-refusal LAT with personality-statement LAT.
    Defined by the authors; its efficacy is the empirical claim of the paper. No independent existence outside this work.

pith-pipeline@v1.1.0-grok45 · 19329 in / 2931 out tokens · 37804 ms · 2026-07-10T15:25:31.379009+00:00 · methodology

0 comments
read the original abstract

Current safety methods for large language models are known to be vulnerable to adversarial attacks, motivating research into robust alternatives. Latent Adversarial Training (LAT) is among the most effective defenses, but can degrade utility and requires training on large datasets of harmful prompts. We introduce Latent Personality Alignment (LPA), which replaces explicit harm refusal with adversarial training on just 66 harm-agnostic statements drawn from psychometric personality literature. We hypothesize that personality-anchored representations share latent structure with harm avoidance, so adversarially stabilizing them implicitly constrains the subspace exploited by jailbreak attacks. LPA achieves near-zero attack success rates on HarmBench across direct requests and five jailbreak methods, despite never seeing harmful content during training and no loss of performance on standard benchmarks. Moreover, the training process is lightweight; the entire procedure completes in minutes on a single GPU and uses 75x fewer examples than standard LAT. Extensive ablations demonstrate the robustness, efficiency, and generalization of our method.

Figures

Figures reproduced from arXiv: 2607.07918 by Adam Oberman, Damiano Fornasiere, David Williams-King, Linh Le, Mohamed Amine Merzouk, Nolan Smyth.

Figure 1
Figure 1. Figure 1: Main result: LPA reduces ASR to near-zero across direct requests and five jailbreak methods while preserving benchmark utility. LPA uses lightweight training, no supervised utility-recovery stage, and no exposure to HarmBench during training. Left: Attack Success Rate (ASR, lower is better) on HarmBench direct harmful requests and five jailbreak methods (GCG, PAIR, AutoPrompt, AutoDAN, TAP). Right: utility… view at source ↗
Figure 2
Figure 2. Figure 2: Illustrative jailbreak interaction from HarmBench. An adversarial prompt can elicit [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Evolution of ASR across training steps. We compare targeted LAT and our method (LPA). The horizontal gray line denotes the initial ASR before training, and star markers indicate the checkpoints used for the snapshot comparisons in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Safety–utility trade-off across ablation variants. Each point represents a training checkpoint, with HarmBench direct-request ASR on the x-axis and Tiny MMLU on the y-axis; the top-left corner is ideal. Negative only, our main result, is on the Pareto frontier, reaching near-zero ASR while maintaining high utility. Other variants (Inverted, Irrelevant, All IPIP, Shuffled) can eventually reach low ASR, but … view at source ↗
Figure 5
Figure 5. Figure 5: Utility score of the ablations on Tiny MMLU. For a fair comparison, the threshold [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Main result: LPA reduces ASR direct requests and five jailbreak methods while preserving benchmark utility. While LAT performs slightly better on some attacks, LPA uses 75× less data, no supervised utility-recovery stage, and crucially has no exposure to HarmBench during training. Left: Attack Success Rate (ASR, lower is better) on HarmBench direct harmful requests and five jailbreak methods (GCG, PAIR, Au… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 57 canonical work pages · 36 internal anchors

  1. [1]

    Barrick and Michael K

    Murray R. Barrick and Michael K. Mount. The B ig F ive personality dimensions and job performance: A meta-analysis. Personnel Psychology, 1991. URL https://psycnet.apa.org/record/1991-22928-001

    work page 1991

  2. [2]

    Training large language models on narrow tasks can lead to broad misalignment

    Jan Betley, Niels Warncke, Anna Sztyber-Betley, Daniel Tan, Xuchan Bao, Martín Soto, Megha Srivastava, Nathan Labenz, and Owain Evans. Training large language models on narrow tasks can lead to broad misalignment. Nature, 649 0 (8097): 0 584–589, January 2026. ISSN 1476-4687. doi:10.1038/s41586-025-09937-5. URL http://dx.doi.org/10.1038/s41586-025-09937-5

    work page doi:10.1038/s41586-025-09937-5 2026

  3. [3]

    An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks

    Valentyn Boreiko, Alexander Panfilov, Vaclav Voracek, Matthias Hein, and Jonas Geiping. An interpretable n-gram perplexity threat model for large language model jailbreaks, 2025. URL https://arxiv.org/abs/2410.16222

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  4. [4]

    Defending Against Unforeseen Failure Modes with Latent Adversarial Training

    Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Menell. Defending against unforeseen failure modes with latent adversarial training, 2025. URL https://arxiv.org/abs/2403.05030

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  5. [5]

    Jailbreaking Black Box Large Language Models in Twenty Queries

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries, 2024. URL https://arxiv.org/abs/2310.08419

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    Persona Vectors: Monitoring and Controlling Character Traits in Language Models

    Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. Persona vectors: Monitoring and controlling character traits in language models, 2025. URL https://arxiv.org/abs/2507.21509

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  7. [7]

    Deep reinforcement learning from human preferences

    Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences, 2023. URL https://arxiv.org/abs/1706.03741

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  8. [8]

    Training Verifiers to Solve Math Word Problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL https://arxiv.org/abs/2110.14168

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  9. [9]

    OR-Bench: An Over-Refusal Benchmark for Large Language Models

    Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. Or-bench: An over-refusal benchmark for large language models, 2025. URL https://arxiv.org/abs/2405.20947

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    du Plessis and Gideon P

    Graham A. du Plessis and Gideon P. de Bruin. Using rasch modelling to examine the international personality item pool (ipip) values in action (via) measure of character strengths. Journal of Psychology in Africa, 2015. URL https://doi.org/10.1080/14330237.2015.1124603

    work page doi:10.1080/14330237.2015.1124603 2015

  11. [11]

    Psychometric personality shaping modulates capabilities and safety in language models, 2025

    Stephen Fitz, Peter Romero, Steven Basart, Sipeng Chen, and Jose Hernandez-Orallo. Psychometric personality shaping modulates capabilities and safety in language models, 2025. URL https://arxiv.org/abs/2509.16332

    work page arXiv 2025

  12. [12]

    L. R. Goldberg. A broad-bandwidth, public-domain, personality inventory measuring the lower-level facets of several five-factor models. In European conference on personality, PERSONALITY PSYCHOLOGY IN EUROPE, 1999. URL https://www.tib.eu/de/suchen/id/BLCP

    work page 1999

  13. [13]

    Goldberg, John A

    Lewis R. Goldberg, John A. Johnson, Herbert W. Eber, Robert Hogan, Michael C. Ashton, C. Robert Cloninger, and Harrison G. Gough. The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 2006. URL https://www.sciencedirect.com/science/article/pii/S0092656605000553

    work page 2006

  14. [14]

    Explaining and Harnessing Adversarial Examples

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, 2015. URL https://arxiv.org/abs/1412.6572

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  15. [15]

    Self-Assessment Tests are Unreliable Measures of LLM Personality

    Akshat Gupta, Xiaoyang Song, and Gopala Anumanchipalli. Self-assessment tests are unreliable measures of LLM personality, 2024. URL https://arxiv.org/abs/2309.08163

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    Measuring Massive Multitask Language Understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021. URL https://arxiv.org/abs/2009.03300

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  17. [17]

    International personality item pool: Alphabetical list of items

    IPIP . International personality item pool: Alphabetical list of items. https://ipip.ori.org/AlphabeticalItemList.htm, 2023. Accessed 2026-01-28

    work page 2023

  18. [18]

    Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip H. S. Torr, Amartya Sanyal, and Puneet K. Dokania. What makes and breaks safety fine-tuning? a mechanistic study, 2024. URL https://arxiv.org/abs/2407.10264

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Evaluating and inducing personality in pre-trained language models, 2022

    Guangyi Jiang, Ming Xu, Song-Chun Zhu, Wei Han, Chao Zhang, and Yixin Zhu. Evaluating and inducing personality in pre-trained language models, 2022

    work page 2022

  20. [20]

    Matt Gardner Johannes Welbl, Nelson F. Liu. Crowdsourcing multiple choice science questions, 2017

    work page 2017

  21. [21]

    Measuring and Controlling Instruction (In)Stability in Language Model Dialogs

    Kenneth Li, Tianle Liu, Naomi Bashkansky, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Measuring and controlling instruction (in)stability in language model dialogs, 2024. URL https://arxiv.org/abs/2402.10962

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  22. [22]

    TruthfulQA: Measuring How Models Mimic Human Falsehoods

    Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods, 2022. URL https://arxiv.org/abs/2109.07958

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  23. [23]

    AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models, 2024. URL https://arxiv.org/abs/2310.04451

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  24. [24]

    2024 , month = may, pages =

    Christina Lu, Jack Gallagher, Jonathan Michala, Kyle Fish, and Jack Lindsey. The A ssistant A xis: Situating and stabilizing the default persona of language models, 2026. URL https://arxiv.org/abs/2601.10387

    work page arXiv 2026

  25. [25]

    Stable and explainable personality trait evaluation in large language models with internal activations, 2026

    Xiaoxu Ma, Xiangbo Zhang, and Zhenyu Weng. Stable and explainable personality trait evaluation in large language models with internal activations, 2026. URL https://arxiv.org/abs/2601.09833

    work page arXiv 2026

  26. [26]

    HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal, 2024. URL https://arxiv.org/abs/2402.04249

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  27. [27]

    Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box LLM s automatically, 2024. URL https://arxiv.org/abs/2312.02119

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  28. [28]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback,...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  29. [29]

    Swetasudha Panda, Naveen Jafer Nizar, and Michael L. Wick. LLM improvement for jailbreak defense: Analysis through the lens of over-refusal, 2024. URL https://openreview.net/forum?id=rXReIKbm5e

    work page 2024

  30. [30]

    The LAMBADA dataset: Word prediction requiring a broad discourse context

    Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Quan Ngoc Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The lambada dataset: Word prediction requiring a broad discourse context, 2016. URL https://arxiv.org/abs/1606.06031

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  31. [31]

    AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs

    Anselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos, and Yuandong Tian. Advprompter: Fast adaptive adversarial prompting for LLM s, 2025. URL https://arxiv.org/abs/2404.16873

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  32. [32]

    Lechner, Claudia Wagner, Beatrice Rammstedt, and Markus Strohmaier

    Maximilian Pellert, Clemens M. Lechner, Claudia Wagner, Beatrice Rammstedt, and Markus Strohmaier. AI psychometrics: Assessing the psychological profiles of large language models through psychometric inventories, 2023

    work page 2023

  33. [33]

    Red Teaming Language Models with Language Models

    Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models, 2022. URL https://arxiv.org/abs/2202.03286

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  34. [34]

    tinyBenchmarks: evaluating LLMs with fewer examples

    Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuekai Sun, Gongjun Xu, and Mikhail Yurochkin. tinybenchmarks: evaluating llms with fewer examples, 2024. URL https://arxiv.org/abs/2402.14992

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  35. [35]

    Sysbench: Can large language models follow system messages?, 2024

    Yanzhao Qin, Tao Zhang, Tao Zhang, Yanjun Shen, Wenjing Luo, Haoze Sun, Yan Zhang, Yujing Qiao, Weipeng Chen, Zenan Zhou, Wentao Zhang, and Bin Cui. Sysbench: Can large language models follow system messages?, 2024. URL https://arxiv.org/abs/2408.10943

    work page arXiv 2024

  36. [36]

    Position: Adversarial ML for LLMs Is Not Making Any Progress

    Javier Rando, Jie Zhang, Nicholas Carlini, and Florian Tramèr. Adversarial ml problems are getting harder to solve and to evaluate, 2025. URL https://arxiv.org/abs/2502.02260

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  37. [37]

    The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance

    Abel Salinas and Fred Morstatter. The butterfly effect of altering prompts: How small changes and jailbreaks affect large language model performance, 2024. URL https://arxiv.org/abs/2401.03729

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  38. [38]

    Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts

    Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäschel, and Roberta Raileanu. Rainbow teaming: Open-ended generation of diverse adversarial prompts, 2024. URL https://arxiv.org/abs/2402.16822

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  39. [39]

    A coin flip for safety: Llm judges fail to reliably measure adversarial robustness, 2026

    Leo Schwinn, Moritz Ladenburger, Tim Beyer, Mehrnaz Mofakhami, Gauthier Gidel, and Stephan Günnemann. A coin flip for safety: Llm judges fail to reliably measure adversarial robustness, 2026. URL https://arxiv.org/abs/2603.06594

    work page arXiv 2026

  40. [40]

    Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting, 2024. URL https://arxiv.org/abs/2310.11324

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    A psychometric framework for evaluating and shaping personality traits in large language models

    Gregory Serapio-Garc \' a, Mustafa Safdari, Cl \'e ment Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matari \'c . A psychometric framework for evaluating and shaping personality traits in large language models. Nature Machine Intelligence, 2025. URL https://www.nature.com/articles/s42256-025-01115-6

    work page 2025

  42. [42]

    Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs

    Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, and Stephen Casper. Latent adversarial training improves robustness to persistent harmful behaviors in llms, 2025. URL https://arxiv.org/abs/2407.15549

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  43. [43]

    AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

    Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts, 2020. URL https://arxiv.org/abs/2010.15980

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  44. [44]

    Simms, Lewis R

    Leonard J. Simms, Lewis R. Goldberg, John E. Roberts, David Watson, John Welte, and Jane H. Rotterman. Computerized adaptive assessment of personality disorder: introducing the cat-pd project. Journal of Personality Assessment, 2011. URL https://doi.org/10.1080/00223891.2011.577475

    work page doi:10.1080/00223891.2011.577475 2011

  45. [45]

    Evaluating alignment of behavioral dispositions in llms, 2026

    Amir Taubenfeld, Zorik Gekhman, Lior Nezry, Omri Feldman, Natalie Harris, Shashir Reddy, Romina Stella, Ariel Goldstein, Marian Croak, Yossi Matias, and Amir Feder. Evaluating alignment of behavioral dispositions in llms, 2026. URL https://arxiv.org/abs/2602.11328

    work page arXiv 2026

  46. [46]

    Persistent instability in LLM 's personality measurements: Effects of scale, reasoning, and conversation history, 2025

    Tommaso Tosato, Saskia Helbling, Yorguin-Jose Mantilla-Ramos, Mahmood Hegazy, Alberto Tosato, David John Lemay, Irina Rish, and Guillaume Dumas. Persistent instability in LLM 's personality measurements: Effects of scale, reasoning, and conversation history, 2025. URL https://arxiv.org/abs/2508.04826

    work page arXiv 2025

  47. [47]

    Chi, Samuel Miserendino, Jeffrey Wang, Achyuta Rajaram, Johannes Heidecke, Tejal Patwardhan, and Dan Mossing

    Miles Wang, Tom Dupré la Tour, Olivia Watkins, Alex Makelov, Ryan A. Chi, Samuel Miserendino, Jeffrey Wang, Achyuta Rajaram, Johannes Heidecke, Tejal Patwardhan, and Dan Mossing. Persona features control emergent misalignment, 2025 a . URL https://arxiv.org/abs/2506.19823

    work page arXiv 2025

  48. [48]

    OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas

    Xiaoyang Wang, Hongming Zhang, Tao Ge, Wenhao Yu, Dian Yu, and Dong Yu. Opencharacter: Training customizable role-playing llms with large-scale synthetic personas, 2025 b . URL https://arxiv.org/abs/2501.15427

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  49. [49]

    Efficient Adversarial Training in LLMs with Continuous Attacks

    Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, and Leo Schwinn. Efficient adversarial training in llms with continuous attacks, 2024. URL https://arxiv.org/abs/2405.15589

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  50. [50]

    Bullying the Machine: How Personas Increase LLM Vulnerability

    Ziwei Xu, Udit Sanghi, and Mohan Kankanhalli. Bullying the machine: How personas increase LLM vulnerability, 2025. URL https://arxiv.org/abs/2505.12692

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  51. [51]

    Qwen3 Technical Report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, ...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  52. [52]

    Latent-space adversarial training with post-aware calibration for defending large language models against jailbreak attacks

    Xin Yi, Yue Li, Dongsheng Shi, Linlin Wang, Xiaoling Wang, and Liang He. Latent-space adversarial training with post-aware calibration for defending large language models against jailbreak attacks, 2025. URL https://arxiv.org/abs/2501.10639

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  53. [53]

    Robust LLM safeguarding via refusal feature adversarial training

    Lei Yu, Virginie Do, Karen Hambardzumyan, and Nicola Cancedda. Robust LLM safeguarding via refusal feature adversarial training, 2025. URL https://arxiv.org/abs/2409.20089

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  54. [54]

    Coifman, and Ruoming Jin

    Jianfeng Zhu, Julina Maharjan, Xinyu Li, Karin G. Coifman, and Ruoming Jin. Evaluating LLM alignment on personality inference from real-world interview data, 2025 a . URL https://arxiv.org/abs/2509.13244

    work page arXiv 2025

  55. [55]

    Personality Alignment of Large Language Models

    Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. Personality alignment of large language models, 2025 b . URL https://arxiv.org/abs/2408.11779

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  56. [56]

    Universal and Transferable Adversarial Attacks on Aligned Language Models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023. URL https://arxiv.org/abs/2307.15043

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  57. [57]

    Representation Engineering: A Top-Down Approach to AI Transparency

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down approach to A...

    work page internal anchor Pith review Pith/arXiv arXiv 2025