Pith. sign in

REVIEW 4 major objections 6 minor 28 references

Misalignment Has a Personality: A Big Five Account of Emergent Misalignment

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Emergent misalignment is a single Big Five personality shift that fine-tuning imprints into the model.

desk verdict A careful measurement study that recasts emergent misalignment as a Big Five shift, but the shared signature is partly a lexical-register artifact; deserves a serious referee. read the letter →

arxiv 2607.26389 v1 pith:UJA5DASG submitted 2026-07-29 cs.CL cs.AI

Hasibur Rahman , Smit Desai This is my paper · ORCID
classification cs.CLcs.AI
keywords emergentmisalignmentBigFivepersonalityvectorsactivationdirectionsfine-tuningsafetysycophancylanguagemodelinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that emergent misalignment—the broad harmful and deceptive behavior that appears after fine-tuning a language model on a narrow, flawed task—is best described as a single, measurable shift in the model's Big Five personality. In the corpora and two models studied, misaligned data and the models trained on it move agreeableness and conscientiousness down while moving extraversion and neuroticism up, with openness roughly unchanged. The authors extract one activation-space personality vector per trait using a graded three-level prompt intervention, then show the vectors are ordered, transfer to an independent corpus, and are strongest in a middle layer of the network. Read off eight misaligned training corpora, the same profile reappears across all of them and is recovered by both models, and fine-tuning imprints it into the model's answers to unrelated questions and into its internal activations. If this account is right, an opaque safety failure becomes a named, human-legible diagnostic that could be read from data before training and from a model after it.

What carries the argument

The central object is the personality vector: for each Big Five trait, the unit direction obtained as the difference between the mean residual-stream activation of responses that realize the trait at a high level and responses that realize it at a low level, read at layer 20. The high and low responses come from a graded three-level Trait Modulation Key intervention, filtered by an LLM judge so the contrast runs between realized behavior rather than between instructions, with the medium level held out for calibration. Projecting any response's mean activation onto these fixed unit vectors assigns it a trait score, and all downstream measurements are projections with no further fitting. The machinery works by turning an unlabeled 'misaligned direction' into five named, graded coordinates whose validity is checked by ordinal ordering, zero-shot transfer to an independent corpus, trait-specific diagonal dominance, and a manipulation check in which adding the vector to activations moves the trait it claims to read.

What would settle it

Take a set of misaligned responses, paraphrase each one into a calm, polite, low-energy register while preserving its meaning, and re-run the projections; if the shared signature largely disappears under paraphrase, the profile is a text-register readout rather than a disposition. A complementary test: fine-tune on paraphrased misaligned data and check whether the model still shifts along the signature, which would show the disposition survives without the original surface style.

Watch

Extended reading notes

Core claim

On the paper's own terms, emergent misalignment is a shift in Big Five personality with one shared signature: agreeableness and conscientiousness go down, extraversion and neuroticism go up, and openness stays near zero. The signature is read from data by projecting responses onto five personality vectors, each defined as the mean activation of high-trait responses minus the mean activation of low-trait responses at a fixed middle layer. Across eight categories of misaligned corpora—overtly harmful data and data whose only flaw is wrong answers—the two models produce the same five-trait profile with $r = 0.94$ agreement between their signature matrices. Fine-tuning on flawed data shifts the model's generations on unrelated neutral questions along the same profile ($r = 0.83$ by projection, $r = 0.90$ by a text-based judge) and shifts its internal activations even when the response tokens are held identical ($r = 0.69$). The same measurement rediagnoses sycophancy as high extraversion and low conscientiousness rather than excess agreeableness.

Load-bearing premise

The load-bearing premise is that a personality vector extracted from prompted trait displays measures the intended Big Five trait when projected onto arbitrary misaligned text and fine-tuned generations, rather than reading a surface style—emotional, rude, energetic wording—that happens to line up with trait labels.

Editorial extensions

If this is right

  • A training corpus can be screened for misalignment risk before fine-tuning: responses that project along the shared agreeableness-down, conscientiousness-down, extraversion-up, neuroticism-up profile are the ones likely to imprint the behavior.
  • Fine-tuning on a narrow flaw changes the model's disposition on unrelated, neutral prompts, so safety evaluation of a fine-tuned model should include personality-style readouts, not just task performance.
  • Sycophancy should be treated as a high-extraversion, low-conscientiousness configuration rather than an agreeableness excess, which changes which steering directions and mitigation targets make sense.
  • Because the same profile appears in overtly harmful data and in data whose only flaw is wrong answers, the mechanism behaves like a graded disposition rather than a binary poisoned/benign switch.
  • The personality-vector readout supplies a common, named coordinate system, so misalignment can be compared across models, training corpora, and severity levels instead of through an unlabeled scalar.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the signature is a genuine disposition rather than a register artifact, then a corpus that is deliberately written in a calm, formal, deferential style while remaining harmful should still project as low agreeableness and low conscientiousness; a corpus written in an emotional, rude style but corrected to be harmless should not.
  • The same machinery could be applied to benign-but-low-quality data, such as typos, vagueness, or rushed summaries, to see whether the shared profile tracks misalignment specifically or a broader 'careless author' dimension.
  • The paper's own representation-level readout suggests a direct intervention test: steering the fine-tuned checkpoint along the opposite of the shared profile should reduce emergent misalignment on neutral prompts, which the paper lists as future work.
  • Because the authors' extraction and readout share the same judge family, an independent replication using a different judge for extraction and for the text-based readout would separate the construct from the judge's own stylistic preferences.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper claims that emergent misalignment—broad harmful or deceptive behavior arising from fine-tuning on a narrow flawed dataset—can be described as a shift in Big Five personality. The authors extract one linear 'personality vector' per trait from graded Trait Modulation Key (TMK) prompts at three levels, using the high-minus-low mean-difference in layer-20 residual-stream activations, and validate the vectors on two 7–8B open-weight models. Validation includes ordinal calibration with a held-out medium level, zero-shot transfer to the independent BIG5-CHAT benchmark, row-wise diagonal dominance of the convergent/discriminant matrix, and a manipulation check in Appendix H. They then project the eight misaligned corpora onto these vectors and report a shared signature across categories—agreeableness down, conscientiousness down, extraversion up, neuroticism up, openness near zero—with cross-model correlation r=0.94. LoRA fine-tuning on three categories shifts the model's own neutral-question generations along the same signature (r=0.83 by projection, r=0.90 by a text judge, r=0.69 at the representation level with teacher-forced identical tokens). The paper also argues that sycophancy is high extraversion and low conscientiousness rather than high agreeableness.

Significance. If the result holds, the paper provides a genuinely useful human-legible diagnostic for emergent misalignment, replacing an opaque one-dimensional 'misaligned persona' coordinate with a named five-axis profile. The methodological strengths are real: the read layer was fixed on in-sample calibration, the medium level is held out for ordinality, transfer is measured on an independently constructed benchmark with no refitting, specificity is checked by row-wise diagonal dominance, and the fine-tuning claim is supported by a teacher-forced representation-level readout that holds input tokens identical. The paper also ships extensive appendices with full tables, hyperparameters, and confound controls. The strongest part of the claim—a single shared personality signature in the data—is, however, less protected than the fine-tuning-imprint part, because the data signature is read from surface text projections and a TF-IDF classifier matches the transfer AUC. The central claim is defensible but currently depends on an instrument whose independence from text register is only partially established.

major comments (4)
  1. [§5.3 and §F] The data-signature step remains exposed to a surface-text confound. The manuscript reports that a TF-IDF classifier matches the vectors' BIG5-CHAT transfer AUC (0.98 vs 0.93) and that layer-0 embeddings already yield AUC 0.867/0.856 (§F). Because Eq. (9) computes the eight-corpus signature from mean layer-20 projections of surface text, the shared A↓C↓E↑N↑ profile in Tables 3–4 could be largely a lexical-register correlation—'misaligned text is more emotional, rude, and energetic'—rather than evidence of a common internal disposition. The teacher-forced representation readout (§5.5, Table 6) convincingly rules out this confound for the fine-tuning-imprint claim, but it does not protect the data-signature claim, which is the paper's strongest cross-corpus result (r=0.94). Please add a register-controlled version of Eq. (9), such as residualizing projections on length, sentiment, and lexical-category features or matching texts on register, and show that the signature survives; otherwise the data-signature conclusion should be explicitly narrowed.
  2. [§5.5] Instrument sharing inflates the reported agreement. The data signature and the behavioral fine-tuning shift are both read with the same extracted vectors, so r=0.83 is partly a shared-instrument correlation; the text-judge readout reuses the extraction judge, which the paper itself labels 'corroborative rather than fully independent.' The representation-level readout is the cleanest evidence, but its agreement is r=0.69, weaker than the headline numbers. Please report at least one genuinely independent measurement (e.g., a different judge family for the text readout, or vectors extracted from an independent trait-elicitation instrument) and quantify how much of r=0.83 survives when the measurement mechanism overlap is removed.
  3. [§5.4, Tables 3–4] The 'all eight categories share one personality shift' framing overstates the prevalence. Tables 3–4 show insecure code with near-zero shifts on both models (Qwen: O −0.4, C −0.5, E +0.6, A −0.3, N +0.4) and math mistakes at essentially zero, and the text itself concedes that the core pattern holds in seven of eight categories per model and six of eight on both. Because the shared-signature claim is the paper's headline, the main text should state per-category counts and severity thresholds explicitly, and the abstract and Figure 1 should say 'most categories' or define the aggregate rule (e.g., PC1 dominance) rather than implying that every one of the eight categories exhibits the full profile.
  4. [§5.4, sycophancy paragraph] The reinterpretation of sycophancy as high extraversion and low conscientiousness rather than high agreeableness is striking but needs stronger support. Given that the high-agreeableness TMK key explicitly foregrounds warmth, politeness, and praise (Appendix B.1)—the very surface of sycophantic text—the fact that the agreeableness projection falls (Qwen 21.8→12.8) is surprising and could indicate that the agreeableness vector encodes 'non-flattering honesty' rather than agreeableness proper. Please add an independent human or LLM-judge evaluation of the agreeableness of sycophantic responses, or a facet-level readout, before asserting this distinction as a settled contribution.
minor comments (6)
  1. [Abstract and §5.4] Please reconcile the abstract's 'all eight categories share one personality shift' with §5.4's 'seven of eight categories on each model, and six of eight on both.'
  2. [§3.2, Eq. (7)] The notation d(p_j[B+_i], p_j[B−_i]) is slightly ambiguous; clarify that d denotes Cohen's d computed on the two projection sets.
  3. [§B.4] The judge midpoint threshold τ=50 is a free parameter; a brief sensitivity analysis around τ (e.g., τ=40 and τ=60) would strengthen the claim that the filtering threshold does not drive the ordinal calibration.
  4. [Table 3–4] Please add a table note stating explicitly that the displayed Cohen's d values are for the strong-misaligned split, with mild-split values in Appendix G.3, so readers do not mistake the severity convention.
  5. [Figure 2] The dotted line marking layer 20 is helpful; consider shading the 16–24 band mentioned in §5.1 to make the stability of conclusions across the band visually immediate.
  6. [References] Minor typographical issues: 'Matej Hrkalovic' should be consistently capitalized, and the reference list contains a few inconsistent name orderings that should be normalized.

Circularity Check

0 steps flagged · score 2.0 of 10

No by-construction circularity: the Big Five vectors are externally validated on the independent BIG5-CHAT benchmark, and the fine-tuning imprint claim survives a token-identical representation-level control; the disclosed self-citations and shared judge are minor, not load-bearing.

full rationale

Walking the derivation chain: the personality vectors are extracted from TMK style-cued prompts whose provenance is the authors' own prior work (Rahman and Desai 2026), but the full keys are reproduced verbatim in Appendix B.1, so the cited source is not an inaccessible authority; the derivation rests on the reproduced prompts, not on the citation. The vectors are then validated on BIG5-CHAT, which the paper states is 'constructed through a pipeline unrelated to ours' and 'independent of the TMK,' with zero-shot transfer AUCs of 0.90–0.998 (Qwen) and 0.81–0.95 (Llama). The gradedness claim holds out the medium TMK level from vector extraction, and the read layer is fixed on in-sample calibration before any external benchmark is projected, so these measurements are not fit to the target result. The data signature is a projection of corpus activations onto fixed, already-validated vectors ('Every test in §5 is a projection (5) onto a fixed vector, with no further fitting'), so the eight-category A-down/C-down/E-up/N-up pattern is an empirical measurement rather than an input. The fine-tuning imprint claim is partially read with the same vectors, and the paper explicitly identifies this shared-instrument risk: 'Both the data signature and the model shift are read through the same vectors, so their agreement could be an artifact of that shared instrument.' It then supplies the text-judge readout, while conceding that 'because the same judge filtered the extraction contrast, this readout is corroborative rather than fully independent,' plus the representation-level readout in which 'the tokens are identical at every position, so only the LoRA weights differ' — this last control rules out any reduction of the imprinting result to surface text. The paper also discloses the TF-IDF surface-text control and the high embedding-layer AUC floor, narrowing its claim to a fixed direction that transfers without refitting rather than claiming lexical independence. No equation in the paper reduces the data signature or fine-tuning shift to the extraction contrast by construction; no uniqueness theorem is imported from the authors' prior work; and the sycophancy re-interpretation is presented as an empirical reversal of the authors' own lexical expectation, not as a definitional consequence.

Assumptions & free parameters 2 free parameters · 6 assumptions · 1 invented entities

The central claim rests on the validity of the projected readout as a Big Five measurement, on the TMK operationalizations (self-cited), on the LLM judge as a filter, and on BIG5-CHAT as independent transfer ground truth. The read layer and judge threshold are empirically selected, which I count as free parameters.

free parameters (2)
  • Read layer L = 20
    Selected on in-sample calibration (max mean Cohen's d over grid {10,15,20,25}) before any external benchmark was projected; all downstream results use this layer, so it is an empirically chosen free parameter.
  • Judge midpoint threshold tau = 50
    Hand-chosen threshold for LLM-judge trait expression and coherence scores used to filter extraction responses into high and low sets.
assumptions (6)
  • domain assumption Trait-relevant directions are linearly encoded in the residual stream at layer 20 and mean pooling over response tokens is a valid summary.
    Underpins Eq. (1) and (4); cited to linear-representation results (Zou et al. 2025, Marks and Tegmark 2024), but the specific layer and pooling choice is assumed.
  • ad hoc to paper The TMK graded instructions are valid, ordered operationalizations of the Big Five traits.
    The three-level keys come from the authors' own prior work (Rahman and Desai 2026); the held-out medium ordering supports, but does not prove, that they operationalize graded traits.
  • domain assumption LLM judge scores of trait expression and coherence are reliable for filtering realized behavior.
    The filter in Eqs. (2)-(3) assumes GPT-4.1-mini's 0-100 trait and coherence scores track human judgments; cited to Zheng et al. 2023 for LLM-as-judge validity.
  • domain assumption BIG5-CHAT is a valid independent ground-truth corpus for Big Five traits in dialogue.
    Transfer validation in §5.3 relies on BIG5-CHAT labels reflecting true trait levels; the corpus is model-generated from expert models trained on human trait text.
  • domain assumption The eight normal and misaligned corpus pairs faithfully represent emergent misalignment.
    Corpora follow Chen et al. 2025 and Betley et al. 2026 definitions; the emergent-misalignment categories (math, medical, opinion, GSM8K) are assumed to contain only wrong answers as the flaw.
  • domain assumption The Big Five is an appropriate coordinate system for describing model outputs and activations.
    Main text states Big Five is used 'as a human-legible coordinate system' and explicitly disclaims human-like latent factor structure (§5.3, Fig. A3).
invented entities (1)
  • Big Five personality vector per trait (O, C, E, A, N) in layer-20 residual stream independent evidence
    purpose: Read a response's or corpus's level of each trait via projection, and measure the personality shift from misaligned data and fine-tuning.
    The paper provides external handles: zero-shot transfer to BIG5-CHAT (AUC 0.81-1.00), manipulation-based steering checks (Appendix H), and representation-level readout with identical text. These give falsifiable predictions beyond the extraction data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Misalignment Has a Personality: A Big Five Account of Emergent Misalignment." pith.science (2026). https://pith.science/paper/UJA5DASG

@misc{pith2026260726389,
  author       = {Pith},
  title        = {Pith review of: Misalignment Has a Personality: A Big Five Account of Emergent Misalignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UJA5DASG}},
  note         = {Machine review of arXiv:2607.26389}
}
read the original abstract

Fine-tuning a language model on data containing a narrow flaw, such as insecure code or incorrect mathematical answers, can cause broad misalignment through a mechanism that remains debated. We provide an interpretable account: in the models and corpora we study, misalignment behaves like a shift in personality. Prior work extracts activation directions for character traits from a single binary contrast, which can separate or steer behavior without establishing a calibrated scale. We instead extract personality vectors for the Big Five using a graded, three-level intervention and validate them on two open-weight models. The three levels are linearly ordered, with Cohen's d values of up to 6.2; the vectors transfer zero-shot and trait-specifically to an independent corpus; and their effects are strongest within a middle-layer band. Applied to training data, the vectors reveal that misaligned corpora across eight domains share a common Big Five signature: lower agreeableness and conscientiousness, together with higher extraversion and neuroticism. This signature is recovered by both models with a correlation of r = 0.94. Fine-tuning imprints the same profile, shifting the model's generations along the corresponding signature, with r = 0.83 using activation-based measurements and r = 0.90 using a text-based judge, while also shifting internal activations with r = 0.69. The same vectors characterize sycophancy as high extraversion and low conscientiousness rather than excess agreeableness, a distinction that a single direction cannot capture. Calibrated personality vectors transform an opaque safety phenomenon into a human-legible diagnostic profile.

Figures

Figures reproduced from arXiv: 2607.26389 by the authors.

Figure 1
Figure 1. Emergent misalignment is a shift in Big Five personality, read with personality vectors. (1) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The trait directions transfer best in the middle layers. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 11 canonical work pages

  1. [4]

    InICLR 2025 Workshop on Building Trust in Language Models and Applications

    Steering Fine-Tuning Generalization with Targeted Concept Ablation. InICLR 2025 Workshop on Building Trust in Language Models and Applications. Chen, R.; Arditi, A.; Sleight, H.; Evans, O.; and Lindsey, J

    work page 2025

  2. [5]

    Chua, J.; Betley, J.; Taylor, M.; and Evans, O

    Persona Vectors: Monitoring and Controlling Character Traits in Language Models.arXiv preprint arXiv:2507.21509. Chua, J.; Betley, J.; Taylor, M.; and Evans, O

    arXiv

  3. [6]

    Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J

    Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models.arXiv preprint arXiv:2506.13206. Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J

    arXiv

  4. [9]

    arXiv preprint arXiv:2502.04075

    Controllable Emotion Generation with Emotion Vectors. arXiv preprint arXiv:2502.04075. Feng, X.; Zhao, L.; Zhong, W.; Huang, Y.; Gu, Y.; Kong, L.; Feng, X.; and Qin, B

    arXiv

  5. [11]

    He, L.; Xia, M.; and Henderson, P

    The Llama 3 Herd of Models.arXiv preprint arXiv:2407.21783. He, L.; Xia, M.; and Henderson, P

    arXiv

  6. [12]

    Huben,R.;Cunningham,H.;Smith,L.;Ewart,A.;andSharkey,L.2024

    What is in Your Safe Data? Identifying Benign Data that Breaks Safety.arXiv preprint arXiv:2404.01099. Huben,R.;Cunningham,H.;Smith,L.;Ewart,A.;andSharkey,L.2024. SparseAutoencodersFindHighlyInterpretable Features in Language Models. InInternational Conference on Learning Representations (ICLR), 7827–7845. Jiang,G.;Xu,M.;Zhu,S.-C.;Han,W.;Zhang,C.;andZhu,Y...

    arXiv 2024

  7. [13]

    In Che, W.; Nabende, J.; Shutova, E.; and Pilehvar, M

    BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data. In Che, W.; Nabende, J.; Shutova, E.; and Pilehvar, M. T., eds.,Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 20434–20471. Vienna, Austria: Association for Computational Linguistics. ISBN 979-8-89176-251-0. Maire...

    work page 2007

  8. [14]

    McCrae, R

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets.arXiv preprint arXiv:2310.06824. McCrae, R. R.; and Costa, P. T

    arXiv

Show all 28 references
  1. [16]

    Rahman, H.; and Desai, S

    Qwen2.5 Technical Report.arXiv preprint arXiv:2412.15115. Rahman, H.; and Desai, S

    arXiv

  2. [17]

    InProceedings of the 2026 CHI Conference on Human Factors in Computing Systems, CHI ’26

    Vibe Check: Understanding the Effects of LLM-Based Conversational Agents’ Personality and Alignment on User Perceptions in Goal-Oriented Tasks. InProceedings of the 2026 CHI Conference on Human Factors in Computing Systems, CHI ’26. New York, NY, USA: Association for Computing...

    2026

  3. [18]

    Shah, A.; Mishra, D.; and Silpasuwanchai, C

    A Psychometric Framework for Evaluating and Shaping Personality Traits in Large Language Models.Nature Machine Intelligence, 7: 1954–1968. Shah, A.; Mishra, D.; and Silpasuwanchai, C

    1954

  4. [20]

    Convergent Linear Representations of Emergent Misalignment.arXiv preprint arXiv:2506.11618. 12 Misalignment Has a Personality Templeton,A.;Conerly,T.;Marcus,J.;Lindsey,J.;Bricken,T.;Chen,B.;Pearce,A.;etal.2024.ScalingMonosemanticity: Extracting Interpretable Features from Clau...

    2024 arXiv

  5. [21]

    Turner,E.;Soligo,A.;Taylor,M.;Rajamanoharan,S.;andNanda,N.2025

    Steering Language Models With Activation Engineering.arXiv preprint arXiv:2308.10248. Turner,E.;Soligo,A.;Taylor,M.;Rajamanoharan,S.;andNanda,N.2025. ModelOrganismsforEmergentMisalignment. arXiv preprint arXiv:2506.11613. van der Linden, D.; te Nijenhuis, J.; andBakker, A. B

    2025 arXiv

  6. [22]

    Wigler, B.; Tsfasman, M.; and Matej Hrkalovic, T

    Persona Features Control Emergent Misalignment.arXiv preprint arXiv:2506.19823. Wigler, B.; Tsfasman, M.; and Matej Hrkalovic, T

  7. [23]

    Yarkoni, T

    Stories of Your Life as Others: A Round-Trip Evaluation of LLM-Generated Life Stories Conditioned on Rich Psychometric Profiles.arXiv preprint arXiv:2604.06071. Yarkoni, T

    arXiv

  8. [24]

    You are intellectually active, cognitively open, creative, introspective, and have a vivid imagination withawiderangeofinterests

    Representation Engineering: A Top-Down Approach to AI Transparency.arXiv preprint arXiv:2310.01405. 13 Misalignment Has a Personality Appendix This appendix reproduces the graded intervention instructions verbatim, specifies the LLM-judge rubric, the corpora, and the training ...

    2026 arXiv

  9. [25]

    strong-misaligned split

    we elicit with, so no BIG5-CHAT signal derives from our keys. We use it only as a held-out transfer target: for each trait we take200 high- and200 low-trait dialogues, compute the response-token-mean layer-20 activation of each, and project onto our fixed vectors with no fitti...

    2025

  10. [26]

    harm” = overtly harmful; “EM

    AUC 0.94 Low High −3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0 0.5 AUC 0.95 Low High −0.5 0.0 0.5 1.0 1.5 2.0 2.5 AUC 0.85 Low High −1.0 −0.5 0.0 0.5 1.0 AUC 0.94 Low High −0.5 0.0 0.5 1.0 1.5 2.0 2.5 AUC 0.81 BIG5-CHAT trait level: Low High Figure A2. Held-out projection distributions o...

    2010

  11. [27]

    ρ=0.74 d=2.4 Low Medium High −3 −2 −1 0 1 2 3 4 ρ=0.58 d=1.6 Low Medium High −6 −4 −2 0 2 4 6 ρ=0.81 d=3.1 Low Medium High −2 0 2 4 6 ρ=0.70 d=2.4 Low Medium High −4 −2 0 2 4 ρ=0.76 d=3.0 TMK level: Low Medium High Figure A5. TMK low/medium/high levels are linearly ordered alo...

  12. [28]

    assistant persona

    Qwen Evil−0.42−1.98+1.16−3.56+2.13 Medical−0.36−1.64+0.97−1.98+1.60 Sycoph.−0.23−0.68+0.62−0.96+0.72 Llama Evil−0.27−1.87+1.33−3.59+2.40 Medical−0.02−1.34+0.77−1.50+1.19 Sycoph.+0.06−1.57+1.13−1.17+1.50 Table A12.Text-judge corroboration: Cohen’s𝑑 between misaligned- and norma...

    2025

  13. [1999]

    Perez,E.;Ringer,S.;Lukoši ¯ut˙e,K.;Nguyen,K.;etal.2023

    Linguistic Styles: Language Use as an Individual Difference.Journal of Personality and Social Psychology, 77(6): 1296–1312. Perez,E.;Ringer,S.;Lukoši ¯ut˙e,K.;Nguyen,K.;etal.2023. DiscoveringLanguageModelBehaviorswithModel-Written Evaluations. InFindings of the Association for...

    2023

  14. [2008]

    InThe SAGE Handbook of Personality Theory and Assessment, volume 2, 179–198

    The Revised NEO Personality Inventory (NEO-PI-R). InThe SAGE Handbook of Personality Theory and Assessment, volume 2, 179–198. SAGE Publications. Cronbach,L.J.;andMeehl,P.E.1955. Constructvalidityinpsychologicaltests.PsychologicalBulletin,52(4):281–302. DeYoung, C. G.; Peterso...

    1955

  15. [2021]

    Costa, P

    Training Verifiers to Solve Math Word Problems.arXiv preprint arXiv:2110.14168. Costa, P. T.; and McCrae, R. R

    arXiv

  16. [2022]

    Belinkov, Y

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback.arXiv preprint arXiv:2204.05862. Belinkov, Y

    arXiv

  17. [2023]

    Soligo, A.; Turner, E.; Rajamanoharan, S.; and Nanda, N

    Towards Understanding Sycophancy in Language Models.arXiv preprint arXiv:2310.13548. Soligo, A.; Turner, E.; Rajamanoharan, S.; and Nanda, N

    arXiv

  18. [2024]

    Arditi, A.; Obeso, O.; Syed, A.; Paleka, D.; Panickssery, N.; Gurnee, W.; and Nanda, N

    Identifying and Manipulating Personality Traits in LLMs Through Activation Engineering.arXiv preprint arXiv:2412.10427. Arditi, A.; Obeso, O.; Syed, A.; Paleka, D.; Panickssery, N.; Gurnee, W.; and Nanda, N

    arXiv

  19. [2025]

    Betley, J.; Warncke, N.; Sztyber-Betley, A.; Tan, D.; Bao, X.; Soto, M.; Labenz, N.; and Evans, O

    Llama-Nemotron: Efficient Reasoning Models.arXiv preprint arXiv:2505.00949. Betley, J.; Warncke, N.; Sztyber-Betley, A.; Tan, D.; Bao, X.; Soto, M.; Labenz, N.; and Evans, O

  20. [2026]

    Goldberg, L

    PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra.arXiv preprint arXiv:2602.15669. Goldberg, L. R

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.