Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study

Carla Perez-Almendros; Luis Espinosa-Anke

REVIEW 4 major objections 5 minor 14 references

Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study

T0 review · 4 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Self-harm concepts crystallize in the final 3–7% of layers across four LLMs

desk verdict Solid empirical probe study of self-harm representations, with an overstated depth claim and a pooling-choice confound that should be checked. read the letter →

arxiv 2607.21988 v1 pith:RJ6KJNDA submitted 2026-07-24 cs.CL

Luis Espinosa-Anke , Carla Perez-Almendros This is my paper

classification cs.CL MSC 68T50

keywords self-harmdetectionlinearprobingresidualstreamrepresentationengineeringcontrastivedirectionslayer-wiseanalysislargelanguagemodelssafety

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish where and how large language models internally represent self-harm content from social media posts. Training linear probes on residual-stream activations at every layer of four instruction-tuned models, it finds that self-harm information becomes linearly decodable only in the final 3–7% of network layers (93–97% depth), consistently across both datasets. It also extracts a contrastive self-harm direction at each layer and shows that these directions rotate substantially across depth, so an early-layer direction does not transfer to late layers. A third finding is that the most accurate probe is not always the most linearly separable direction, with one model (Gemma-3-4B) showing high classification accuracy but low geometric separation along the contrastive axis. The paper argues these results are a step toward explicit, testable safety components for content moderation and intervention systems.

What carries the argument

The central object is the last-token residual-stream activation at each transformer layer, extracted via hook_resid_post. Two analytic tools carry the argument: (1) a linear probe (ℓ2-regularized logistic regression, 5-fold CV, ROC-AUC) trained on these activations at each layer, which locates the depth where self-harm information becomes linearly decodable; and (2) the contrastive self-harm direction dℓ, the normalized mean difference between self-harm and control activations, whose scalar projections and Cohen's d measure geometric separability, while the L×L cosine matrix measures directional stability across layers. The combination of these two tools lets the paper separate 'linearly sep

What would settle it

A direct test is to re-run the layer-wise probes on the same four models and datasets but average activations over all tokens (or over content tokens only) instead of taking the last token. If the peak probe layer no longer falls in the final 3–7% of layers—or if the peak shifts substantially—the crystallization claim is an artifact of the last-token aggregation. A second falsifier is to train probes on a held-out self-harm dataset (e.g., a third corpus) and check whether the 93–97% depth range reproduces; if it does not, the result is dataset-specific rather than a general property of self-ha

Watch

Extended reading notes

Core claim

The central claim is that self-harm content is represented in a stable, cross-architecture way: in all four tested models (Qwen3-0.6B, Llama 3.2-1B, Llama 3.2-3B, Gemma-3-4B) and on both self-harm datasets, a linear probe trained on last-token residual-stream activations reaches peak ROC-AUC only in the final 3–7% of layers. The paper also establishes that the contrastive self-harm direction (mean difference between self-harm and control activations) is not stable across depth: adjacent layers are highly aligned (cosine 0.76–0.92) but directions from the first and last thirds are nearly orthogonal (0.14–0.38), forming a block-diagonal structure. Finally, it reports a dissociation between lin

Load-bearing premise

The load-bearing assumption is that the last token's residual-stream activation is a sufficient representation of the entire social-media post; if the final token does not fully aggregate the post's meaning (e.g., when it is punctuation or a common word), the measured layer-wise accuracy and the resulting 'crystallization depth' may reflect where last-token information becomes decodable rather than where self-harm concepts genuinely reside.

Editorial extensions

If this is right

If self-harm information consistently crystallizes in the final few layers, downstream detectors can save compute by probing only that late-layer block rather than the full network.
The block-diagonal structure implies that activation steering or intervention at an early layer will not transfer to the late-layer representation; safety interventions must target the late-layer block.
The dissociation between probe AUC and Cohen's d warns that a model's high detection accuracy does not imply a single steerable self-harm direction, so representation-engineering methods that assume one direction may fail on models like Gemma-3-4B.
The systematic AUC gap between X-Sensitive and SH-Detection (0.12–0.17 AUC, ~1 Cohen's d) indicates that corpus composition and labelling quality strongly affect how cleanly self-harm is represented, not just classifier performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to check whether the late-layer crystallization depth shifts if the last-token representation is replaced by mean-pooling over all tokens; if the 93–97% result persists, the finding is robust to the choice of token aggregation, but if it shifts, the 'crystallization' may be an artifact of where the final token's context aggregation completes.
The near-orthogonality of early and late directions suggests a possible safety mechanism: monitoring the cosine angle of a running contrastive direction could detect when a model is about to transition into a 'self-harm-decodable' state, enabling mid-generation intervention.
The Gemma-specific dissociation could be probed further by training probes on projected subspaces of increasing rank; if AUC rises gradually with rank, it would confirm that Gemma distributes self-harm signal across many dimensions, offering a concrete way to find the effective dimensionality of the concept.
Comparing these results with a non-instruction-tuned base model of similar size would separate the effect of instruction tuning from architecture and scale on the crystallization depth.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Short version: this is a readable, honest probing study of where self-harm information lives in four small instruction-tuned models. It deserves a serious referee, but the headline depth claim is imprecise and the last-token-only analysis leaves a real confound unresolved.

What's actually new: applying linear probes and contrastive directions to self-harm content across architectures, and the specific dissociation in Gemma-3-4B—high probe AUC with relatively low Cohen's d—which, if it holds out-of-sample, is a nice counterexample to the assumption that accuracy implies a simple geometric direction. The layer-wise probe curves and cross-layer cosine matrices are cleanly presented, code is released, and the X-Sensitive vs SH-Detection comparison gives a useful demonstration that label noise shifts representational quality.

Soft spots in rough order of importance.

First, the abstract's '93–97% depth' does not match Table 2. For most model-dataset combinations the best probe layer is the final layer—100% depth. Only two cases (Qwen3-0.6B and Llama-3B on SH-D) have best layers at 27/28, i.e., 96.4%. So 'crystallizes in the final 3–7%' is a generous summary of a pattern that is really 'the last layer, or occasionally the second-to-last.' Easy fix, but the current wording overstates precision.

Second, the stress-test concern has teeth. The entire analysis uses last-token residual-stream activations. For causal LMs the last token can attend to everything, but that does not mean it is a faithful summary of a post, especially when the last token is punctuation or a stopword. The late-layer peak could be inflated by the fact that only later layers move information to the final position. I don't think this breaks the paper—it is still a valid statement about what a last-token probe sees—but the 'crystallization' claim is a property of the probing interface. A mean-pooling or attention-pooled comparison would tell us whether the effect is about the representation or the interface. Without it, the stronger version of the claim overreaches.

Third, the Gemma dissociation compares cross-validated probe AUC to an in-sample Cohen's d computed on the same data used to define the direction. That is optimistic, and the comparison is apples-to-oranges. Give confidence intervals or a proper out-of-sample evaluation before making the architectural claim.

The directional stability analysis (block-diagonal cosine matrices) is compelling and consistent with prior rep-eng work—directions rotate substantially across depth. That part I trust.

Who gets value: safety and moderation researchers working on representation engineering and mental-health AI, plus people designing probe studies who want an example of pooling-dependent results. It deserves a serious referee, but the revision should fix the depth wording and add a pooling ablation.

Recommendation: conditional accept with those revisions.

Referee Report

4 major / 5 minor

Summary. The paper investigates where self-harm information is encoded in four instruction-tuned LLMs (Qwen3-0.6B, Llama 3.2-1B/3B, Gemma-3-4B) by training linear probes on last-token residual-stream activations at every layer and evaluating ROC-AUC on two balanced social-media datasets (X-Sensitive, SH-Detection). It reports that discriminative information is concentrated in the final layers (93–97% depth), and that contrastive mean-difference directions rotate across depth with a block-diagonal similarity structure. It also reports a dissociation for Gemma-3-4B between high probe accuracy and lower Cohen's d along the contrastive direction, interpreted as a more distributed representation.

Significance. If the pooling-robustness and statistical concerns are addressed, the study would be a useful empirical contribution to representation analysis of mental-health-related content and to safety/representation-engineering applications. Strengths: the manuscript provides code, uses two independent publicly available datasets, uses TransformerLens for hooking, reports error analysis, and includes cross-layer cosine matrices. The findings are falsifiable and straightforward to reproduce. The main value is in the cross-architecture comparison, especially the Gemma dissociation, but that claim currently rests on a methodological comparison that needs tightening.

major comments (4)

[Abstract; §4.1, Table 2] The headline 'self-harm information crystallizes in the final 3–7% of layers (93–97% depth)' is not supported by Table 2 as printed. Under the natural definition depth = ℓ/L, six of the eight best-layer entries are the final layer (100% depth) and the remaining two are ℓ = L−1 (96.4% depth); no entry is near 93%. If a different depth normalization is intended (e.g., ℓ/(L+1) or (ℓ−1)/(L−1)), it must be stated in §3/§4.1 and applied consistently. As it stands, the quantitative claim is inconsistent with the reported table and should be corrected or rephrased.
[§3, §4.1] The central localization claim is derived exclusively from last-token residual-stream activations. For causal LMs, the final token is a practical summary, but it is not necessarily a faithful aggregate of the full post: when the last token is punctuation or a high-frequency word, early layers may not have mixed the left context into that position. This could produce a late-layer AUC rise that reflects the probing interface rather than the locus of self-harm information in the post representation. I request a robustness comparison with mean pooling over all (non-padding) tokens, or an all-token probe, for at least these models, and the resulting layer-wise curves. If the final-layer peak disappears under pooling, the 'crystallization' claim must be qualified as last-token-specific.
[§4.2, Table 3] The Gemma dissociation (high probe AUC, low Cohen's d) compares cross-validated probe AUC from Table 2 with a Cohen's d computed in-sample on the same data used to estimate the direction dℓ in Eq. (1). No confidence intervals, bootstrap resampling, or significance tests are reported, and the observed differences (e.g., 0.969 AUC vs. 0.972 AUC with d=1.18 vs. 2.08) are not shown to be outside sampling variability. To support this secondary claim, report bootstrap CIs for d, compute d within the same CV folds as the probes, and test a model × metric interaction. Without this, the 'more intricate' interpretation is not established.
[§4.1, Figs. 1–2] The best layer is selected by maximizing mean 5-fold CV AUC, but no multiple-comparison control or comparison with neighboring layers is reported. In several cases, the difference between the selected layer and adjacent layers is within one standard deviation (e.g., Qwen3-0.6B on X-S: 0.703±0.016; Table 2). This does not invalidate the trend, but it weakens the precision of 'best layer' statements and, consequently, the depth percentages. I recommend reporting whether the selected layer is significantly better than nearby layers (e.g., paired bootstrap or corrected repeated CV) or at least showing all pairwise layer comparisons in the appendix.

minor comments (5)

[Abstract] Typo: 'normaliation' should be 'normalization'. Also 'posts.We' is missing a space.
[§1] Typo: 'more effecrtive intervention' should be 'more effective intervention'.
[§4.1] The text says 'full layer-wise increasing AUC curves'; the curves in Figures 1–2 are generally increasing but not strictly monotone. Suggest 'generally increasing' or 'layer-wise AUC curves'.
[Table 4 caption] Define 'Adj.' and 'Cross' in the caption (adjacent-layer mean cosine vs. mean cosine between first and last thirds).
[§4.2, Eqs. (1)–(2)] State explicitly whether the means, standard deviations, and direction dℓ are computed on the full dataset or within training folds, and whether Cohen's d uses pooled or unpooled standard deviations. This matters for interpreting the values.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's load-bearing claims are empirical measurements on external datasets, not derivations from their own outputs.

full rationale

The paper's main claims — late-layer crystallization of self-harm information and the dissociation between probe AUC and contrastive-direction Cohen's d — are empirical results obtained by training linear probes and computing contrastive directions on public datasets (X-Sensitive, SH-Detection) across four open-weight LLMs. No parameter is fitted to a subset of data and then reported as a prediction of the same quantity: probes are evaluated with 5-fold stratified cross-validation, and the 'best layer' is selected on held-out folds. The contrastive direction in Eq. 1 is a mean-difference vector used for descriptive geometric analysis, not presented as a predictive claim that reduces to its construction. The last-token pooling choice is an assumption about representation quality, and the in-sample contrastive statistics could overstate separability, but these are methodological validity concerns, not circular reasoning in which the conclusion is equivalent to an input by construction. The paper cites prior work by its own authors (Antypas et al. 2025 for one dataset; Siddique et al. 2026 as related work on bias mitigation), but these citations are not load-bearing in the derivation chain: the dataset is an external resource and the related-work citations do not supply the paper's conclusions. There is no self-citation of a uniqueness theorem, no ansatz smuggled in via citation, and no renaming of a known result as a new derivation. The central findings are self-contained empirical measurements against external benchmarks, so the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no free parameters fitted to make a derivation work; the analysis is an empirical evaluation of external datasets. The main prior commitments are representational assumptions about residual streams and last-token aggregation, which are standard in this literature but not independently verified here.

assumptions (3)

domain assumption Last-token residual-stream activation represents the entire post.
Invoked in Section 3: 'We use the last token position as the representation of each input, following standard practice for causal LMs where the final token aggregates full left context.' This is load-bearing because all layer-wise probes and contrastive directions are computed on last-token activations.
domain assumption Linear probe accuracy reflects how strongly the model 'represents' a concept.
The paper equates probe ROC-AUC with the degree of 'crystallization' of self-harm information in Section 4.1. This assumes that linear decodability from residual-stream states is a meaningful measure of internal representation.
domain assumption The mean-difference contrastive direction is a meaningful summary of concept geometry.
Section 4.2 uses d_l = mean+ - mean- (normalized) to define the self-harm direction and measures separability with Cohen's d. This assumes the concept's geometry is at least partly captured by the difference in class centroids.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study." pith.science (2026). https://pith.science/paper/RJ6KJNDA

@misc{pith2026260721988,
  author       = {Pith},
  title        = {Pith review of: Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RJ6KJNDA}},
  note         = {Machine review of arXiv:2607.21988}
}

read the original abstract

Self-harm content is particularly challenging to detect using NLP techniques, and is also a high-stakes task which requires the highest accuracy to enable timely intervention or flagging at-risk users. We therefore present an analysis of how LLMs represent such self-harm content, which has downstream applications in self-harm detection, LLM intervention and governance and policing. In this paper, we focus on two datasets and four models, and perform two main experiments: (1) We train and evaluate linear probes across all layers of each model on two self-harm datasets: X-Sensitive and SH-Detection. Across both corpora, self-harm information crystallizes in the final 3 - 7% of network layers (93 to 97% depth). (2) We extract contrastive self-harm directions and, after performing a normaliation step, we find that the most accurate probes are not necessarily the most linearly separable. In particular, we find Gemma-3-4B to represent this \textit{contrastive self-harm direction} in a slightly different, more intricate way than the other LLMs.

Figures

Figures reproduced from arXiv: 2607.21988 by the authors.

**Figure 2.** Layer-wise probe ROC-AUC for all four mod [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Scalar projections onto the contrastive self [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Per-layer mean projection onto the contrastive [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Cross-layer cosine similarity of contrastive self-harm directions ( [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 9 linked inside Pith

[6]

Pavan Reddy and Nithin Reddy

Qwen3 technical report.arXiv preprint arXiv:2505.09388. Pavan Reddy and Nithin Reddy

arXiv
[7]

In 2025 IEEE International Symposium on Technology and Society (ISTAS), pages 1–7

‘for argument’s sake, show me how to harm myself!’: Jail- breaking llms in suicide and self-harm contexts. In 2025 IEEE International Symposium on Technology and Society (ISTAS), pages 1–7. Zara Siddique, Irtaza Khalid, Liam Turner, and Luis Espinosa-Anke

2025
[11]

Andrew Yates, Arman Cohan, and Nazli Goharian

Mentalllama: Interpretable mental health analysis on social me- dia with large language models.arXiv preprint arXiv:2309.13567. Andrew Yates, Arman Cohan, and Nazli Goharian

arXiv
[12]

InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2968–2978, Copenhagen, Denmark

Depression and self-harm risk assessment in online forums. InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2968–2978, Copenhagen, Denmark. Associa- tion for Computational Linguistics. Yuyou Zhang, Miao Li, William Han, Yihang Yao, Zhep- eng Cen, and Ding Zhao

2017
[14]

Representation engineering: A top- down approach to AI transparency.arXiv preprint arXiv:2310.01405. A Cross-layer Directional Stability Figure 5 shows the full L×L pairwise cosine simi- larity matrices between contrastive self-harm direc- tions across all layers, for all four models on both datasets. The block-diagonal structure is remark- ably consisten...

arXiv
[2016]

Dimosthenis Antypas, Indira Sen, Carla Perez Almen- dros, Jose Camacho-Collados, and Francesco Barbi- eri

Under- standing intermediate layers using linear classifier probes.arXiv preprint arXiv:1610.01644. Dimosthenis Antypas, Indira Sen, Carla Perez Almen- dros, Jose Camacho-Collados, and Francesco Barbi- eri

arXiv
[2017]

InInternet Science - 4th International Confer- ence, INSCI 2017, Thessaloniki, Greece, November 22-24, 2017, Proceedings, Lecture Notes in Com- puter Science, pages 428–436

Towards suicide prevention: Early detection of depression on social media. InInternet Science - 4th International Confer- ence, INSCI 2017, Thessaloniki, Greece, November 22-24, 2017, Proceedings, Lecture Notes in Com- puter Science, pages 428–436. Springer. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg

2017
[2019]

Fine-tuning lan- guage models from human preferences.arXiv preprint arXiv:1909.08593. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J....

arXiv 1909

Show all 14 references

[2021]

PLoS one, 16(8):e0253809

Devel- oping a natural language processing tool to identify perinatal self-harm in electronic healthcare records. PLoS one, 16(8):e0253809. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, e...

arXiv
[2022]

InFind- ings of the Association for Computational Linguistics: ACL 2022, pages 566–581

Extracting latent steering vectors from pretrained language models. InFind- ings of the Association for Computational Linguistics: ACL 2022, pages 566–581. Sivasothy Tharsi

2022
[2023]

Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, and Sophia Ananiadou

Activation addition: Steering language models with- out optimization.arXiv preprint arXiv:2308.10248. Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, and Sophia Ananiadou

arXiv
[2024]

Shaoxiong Ji, Shirui Pan, Xue Li, Erik Cambria, Guodong Long, and Zi Huang

The Llama 3 herd of models.arXiv preprint arXiv:2407.21783. Shaoxiong Ji, Shirui Pan, Xue Li, Erik Cambria, Guodong Long, and Zi Huang

arXiv
[2025]

Soumitra Ghosh, Gopendra Vikram Singh, Shambhavi, Sabarna Choudhury, and Asif Ekbal

Gemma 3 technical report.arXiv preprint arXiv:2503.19786. Soumitra Ghosh, Gopendra Vikram Singh, Shambhavi, Sabarna Choudhury, and Asif Ekbal

arXiv
[2026]

In Findings of the Association for Computational Lin- guistics: EACL 2026, pages 809–820, Rabat, Mo- rocco

Shifting perspectives: Steer- ing vectors for robust bias mitigation in LLMs. In Findings of the Association for Computational Lin- guistics: EACL 2026, pages 809–820, Rabat, Mo- rocco. Association for Computational Linguistics. Caitlin A. Stamatis, Jonah Meyerhoff, Richard Zh...

2026

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [6]

Pavan Reddy and Nithin Reddy

Qwen3 technical report.arXiv preprint arXiv:2505.09388. Pavan Reddy and Nithin Reddy

arXiv

[2] [7]

In 2025 IEEE International Symposium on Technology and Society (ISTAS), pages 1–7

‘for argument’s sake, show me how to harm myself!’: Jail- breaking llms in suicide and self-harm contexts. In 2025 IEEE International Symposium on Technology and Society (ISTAS), pages 1–7. Zara Siddique, Irtaza Khalid, Liam Turner, and Luis Espinosa-Anke

2025

[3] [11]

Andrew Yates, Arman Cohan, and Nazli Goharian

Mentalllama: Interpretable mental health analysis on social me- dia with large language models.arXiv preprint arXiv:2309.13567. Andrew Yates, Arman Cohan, and Nazli Goharian

arXiv

[4] [12]

InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2968–2978, Copenhagen, Denmark

Depression and self-harm risk assessment in online forums. InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2968–2978, Copenhagen, Denmark. Associa- tion for Computational Linguistics. Yuyou Zhang, Miao Li, William Han, Yihang Yao, Zhep- eng Cen, and Ding Zhao

2017

[5] [14]

Representation engineering: A top- down approach to AI transparency.arXiv preprint arXiv:2310.01405. A Cross-layer Directional Stability Figure 5 shows the full L×L pairwise cosine simi- larity matrices between contrastive self-harm direc- tions across all layers, for all four models on both datasets. The block-diagonal structure is remark- ably consisten...

arXiv

[6] [2016]

Dimosthenis Antypas, Indira Sen, Carla Perez Almen- dros, Jose Camacho-Collados, and Francesco Barbi- eri

Under- standing intermediate layers using linear classifier probes.arXiv preprint arXiv:1610.01644. Dimosthenis Antypas, Indira Sen, Carla Perez Almen- dros, Jose Camacho-Collados, and Francesco Barbi- eri

arXiv

[7] [2017]

InInternet Science - 4th International Confer- ence, INSCI 2017, Thessaloniki, Greece, November 22-24, 2017, Proceedings, Lecture Notes in Com- puter Science, pages 428–436

Towards suicide prevention: Early detection of depression on social media. InInternet Science - 4th International Confer- ence, INSCI 2017, Thessaloniki, Greece, November 22-24, 2017, Proceedings, Lecture Notes in Com- puter Science, pages 428–436. Springer. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg

2017

[8] [2019]

Fine-tuning lan- guage models from human preferences.arXiv preprint arXiv:1909.08593. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J....

arXiv 1909