Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TELLME edits an LLM's hidden representations so that similar behaviors cluster and different behaviors separate, improving monitor accuracy by 6.3% and safety by 7.5% without external monitors.

desk verdict TELLME is a useful empirical method with a genuinely new angle—editing the LLM itself to make monitoring easier—but the core causal claim is not yet isolated from generic fine-tuning, and some reported numbers contradict the 'consistent improvement' narrative. read the letter →

arxiv 2502.05242 v3 pith:4XUVM6XR submitted 2025-02-07 cs.CL cs.AIcs.CVcs.LG

Guanxu Chen , Jing Shao , Tao Luo , Lijie Hu , Qihao Lin , Dongrui Liu This is my paper · ORCID
classification cs.CLcs.AIcs.CVcs.LG
keywords largelanguagemodelstransparencyrepresentationdisentanglementAImonitoringsafetyriskdetoxificationoptimaltransportgeneralizationcontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TELLME is proposed as a way to make large language models transparent from the inside: instead of bolting an external detector onto the model, the method edits the model's hidden representations so that examples of the same behavior sit close together and examples of different behaviors sit far apart. The paper claims that this geometric property is what transparency should mean for monitoring, and that it pays off: monitors trained on the edited representations improve by 6.3% on safety-risk monitoring, and in a detoxification task the edited models are 7.5% safer even though they are never told which behavior is preferred. The authors also supply a theoretical argument, based on optimal transport and a margin-based generalization bound, for why compressing within-behavior variance and enlarging between-behavior separation should improve generalization. They test the method across math, knowledge, and safety scenarios on four model families and report that general capabilities such as GSM8K, MMLU, and AGIEval scores remain essentially unchanged.

What carries the argument

The load-bearing mechanism is the representation-space geometry produced by the two-part TELLME objective. The disentangle loss $\mathcal{L}_d$ is an InfoNCE contrastive loss over unit-normalized hidden states $z_i^c = h_t^{(l)}/\|h_t^{(l)}\|$ sampled from the target layer: it raises similarity within a behavior and lowers similarity across behaviors. The retain loss $\mathcal{L}_r$ combines an $\ell^2$ norm constraint on representations from a retain set with a KL-divergence penalty on output probabilities for the edited behaviors, and it is what keeps the model's general capabilities nearly unchanged. The theoretical scaffold is a margin-based generalization bound (Theorem 1) relating expected zero-one risk to the empirical $\tau$-margin loss plus $\mathbb{E}_{j\sim\mu}[\mathrm{Lip}(g,j)\,\mathrm{Var}_{n_j}(\phi_\# p_j)/\tau]$; TELLME is claimed to lower the within-class $k$-variance in this term and, by separating behaviors, to make the margin $\rho_g(\phi(x_j))$ larger so that fewer samples fall inside the margin.

What would settle it

Run the same monitors on a safety category that was not included in TELLME's disentangle set; if accuracy on that held-out category does not improve after editing, the gains come from memorizing the trained categories rather than from a general transparency improvement.

Watch

Extended reading notes

Core claim

TELLME's central claim is that monitorability can be engineered into the LLM's own representation space rather than supplied by an external module. The method takes a target hidden layer and applies an InfoNCE contrastive loss to normalized representations, pulling two examples from the same behavior together and pushing examples from different behaviors apart, while a retain loss — an $\ell^2$ constraint on general-capability representations plus a KL divergence penalty on output probabilities — keeps ordinary abilities intact. The paper reports average gains of 6.3% in monitoring accuracy across binary- and multi-risk setups and 7.5% in detoxification safety, with the notable result that separating safe from harmful representations improves safety even when the model has not been told which behavior is good. The theoretical analysis recasts the LLM as an encoder $\phi$ plus classifier $g$ and uses a Wasserstein-based bound to argue that the edit lowers the expected zero-one risk by shrinking the $k$-variance term and by increasing the margin of a better classifier $g'$.

Load-bearing premise

The load-bearing assumption is that compressing similar behaviors and separating different behaviors in representation space is exactly the property that makes monitors and safety better; the theory assumes the improved classifier appears after the edit, which is the same improvement the experiments set out to show.

Editorial extensions

If this is right

  • Representation-based monitors become materially more reliable: on the multi-risk monitoring task, simple Self-Sim similarity monitors jump from roughly 42-57% accuracy to 78-79% after TELLME, and linear probes also improve.
  • Safety can be improved without external supervision: separating safe from harmful representations improves safety by about 7.5% on BeaverTails and SaladBench, and applying TELLME before SFT improves the effectiveness of SFT on safety performance by about 12.9%.
  • The generalization argument implies the gains are not just training-set memorization: lower $k$-variance and larger margins tighten the theoretical risk bound, so the edited representations should transfer to new inputs within the edited behaviors.
  • The method works across model families and scales: consistent improvements appear on Llama-3.1-8B, Qwen2.5-7B and 14B, Mistral-7B, and Gemma2-9B, and across math, knowledge, and safety behavior sets.
  • Using a contrastive loss with more negative examples (NT-Xent) improves safety by about another 2.4%, suggesting the disentanglement effect can be scaled up with larger contrastive batches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the geometric definition of transparency is the actual carrier, then monitoring could be redesigned as nearest-cluster matching in representation space, which would make monitors simpler, cheaper, and much harder for the model to game than output-based classifiers.
  • A direct test the paper leaves implicit is to hold out an entire behavior category from the disentangle set and check whether monitors on unseen categories improve; the limitations section predicts they would not, which would mean the benefit is tied to the behaviors seen in training.
  • The retain-loss structure suggests the same disentanglement could be moved earlier into pre-training rather than applied as post-hoc LoRA editing; the paper's scaling claims would be much stronger if that version worked.
  • The paper equates transparency with intra-class compression and inter-class separation but does not check this geometric notion against faithfulness of the model's verbal self-reports; a neighboring experiment could compare the two on the same samples to see whether easier monitoring and honest explanations coincide.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes TELLME, a post-hoc representation-editing method that applies an InfoNCE-style disentanglement loss at an intermediate layer (with LoRA) to pull representations of similar behaviors together and push representations of different behaviors apart, while a retain loss constrains ℓ2 distances of representations and KL divergence of outputs to preserve general capabilities. The authors define this geometric clustering as "transparency" and evaluate it with coding rate, eRank, ℓ2 distance, angle, and Hausdorff distance in math, knowledge, and safety scenarios across four LLMs. They then report improved monitoring accuracy for representation-based and output-based monitors, improved detoxification safety scores without explicit safety training, and they present a generalization bound based on optimal transport theory.

Significance. TELLME offers a potentially valuable reframing: instead of building external monitors, edit the LLM itself so that hidden states are more separable and therefore easier to monitor. The paper has clear strengths: code is released, experiments cover four model families and several benchmarks, the layout of the method is simple and reproducible, and layer-wise ablations are included. If the causal role of the disentanglement loss is established, the reported 6-7% monitoring and safety improvements are practically meaningful and point toward a scalable oversight direction. The main open issue is that the current experimental design does not yet isolate the disentanglement mechanism from generic fine-tuning effects.

major comments (3)
  1. [§3, Eq. (5)] The central causal claim—that disentanglement, not generic fine-tuning, improves monitoring and safety—is not directly tested. Every comparison in Tables 3 and 5 is between Origin and TELLME, with no control that applies the same LoRA pipeline and retain loss Lr while replacing the disentangle loss Ld with a non-disentangling objective (for example, randomly permuted positive/negative pairs or a scalar regression loss). Table 4 ablates components of Lr only and never removes or randomizes Ld. As a result, the 6.3% monitoring improvement in Table 5 and the 7.5% safety improvement in Table 3 could be due to LoRA adaptation, exposure to task data, or the retain loss, rather than to the geometric transparency claimed in Section 2.1. Please add at least one control that keeps the training procedure identical and varies only the disentanglement objective, and show whether the disentanglement metrics track the downstream gains.
  2. [Remark 1, Appendix C.3] The theoretical argument assumes part of the conclusion it is invoked to support. Remark 1 states that after TELLME "we can obtain a better monitor g′ with a higher ρg′(φ(xj)) on a wide range of samples," and Appendix C.3 explicitly lists this as an assumption. This is essentially the monitoring improvement that Tables 3 and 5 are later presented as verifying. The bound in Theorem 1 itself is a generic learning-theoretic result from Chuang et al. (2021) and does not explain why minimizing Ld at a single target layer should guarantee the required reduction of k-variance or of the empirical τ-margin loss. Please either derive the margin/k-variance improvement from the loss, or present the theory as a post-hoc interpretation rather than as a proof that TELLME improves monitorability.
  3. [Tables 1, 5, 8] The "consistent improvement" narrative is not fully supported by the reported numbers. In Table 1, Qwen2.5-7B Math Angle decreases from 65.17 to 25.84, and Gemma2-9B Math eRank increases from 16.52 to 19.37; Table 8 shows Mistral Post-TELLME decreasing in both binary (84.7→84.6) and multi-risk (83.0→81.5) monitoring. No error bars, confidence intervals, or seed variance are reported for the headline tables, so it is hard to tell whether these reversals are noise. In addition, the stated 5.8% and 6.5% average improvements in Section 3.1 do not obviously match simple averages of Table 5. Please add variance estimates and reconcile or properly qualify the aggregate numbers.
minor comments (4)
  1. [Table 4] The column order in Table 4 is labeled GSM8k, MMLU, AGIEval, but the Origin row lists 47.3, 69.4, 84.5; the correct mapping appears to be AGIEval, MMLU, GSM8k. Please relabel or transpose so the reader is not misled.
  2. [§2.1, Appendix B.4, Table 9] There are several consistency and typographical issues: Section 2.1 has "dirctly," Appendix B.4 and Table 9 use the name "SEER" instead of TELLME, "Housdorff" appears for Hausdorff, and "BevearTails" appears for BeaverTails. Please unify notation and correct these typos.
  3. [§2.1] The paper defines transparency as intra-class compression and inter-class separation without an independent justification that this geometric property is what monitors need. The downstream monitor results are suggestive, but a brief explicit statement that this is an operational proxy, plus a discussion of its relation to monitoring needs, would sharpen the conceptual contribution.
  4. [Table 3] The column header "Average" in Table 3 is ambiguous because it actually appears to report the average general capability (GSM8k, MMLU, AGIEval), not the average of the safety columns. Please rename it to "Capability Avg" or similar.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TELLME's central monitoring and safety claims are tested on external benchmarks, and the theoretical improvement rests on a stated, empirically tested assumption rather than a definitional identity.

full rationale

The paper's central claim is that disentangling representations makes LLMs easier to monitor and safer. The monitoring claim is evaluated by independent classifiers (Self-Sim, Linear Probes, Latent Guard, SFT-judgers) on held-out BeaverTails test sets (Table 5), and the safety claim is evaluated on BT, SaladBench, XSTest, GSM8K, MMLU, and AGIEval (Table 3). These are external benchmarks whose outputs are not defined by the TELLME loss. Table 1's disentanglement metrics (Coding Rate, eRank, l2 distance, angle, Hausdorff distance) confirm that the training objective was achieved; this is a manipulation check, not a circular prediction. Theorem 1 is imported from Chuang et al. (2021), an independent mathematical result, and the paper explicitly labels the margin-improvement condition as an 'Assumption' in Appendix C.3 ('we assume that we can obtain a better classifier g′...'). The assumption is then tested by the monitor experiments rather than being asserted as a derived theorem. The self-citations (e.g., [62], [87], [17]) appear in related-work and background contexts and are not load-bearing for the main derivation. The stated limitation that unseen behaviors remain unchanged narrows scope but does not reveal circularity. The absence of a control that removes the disentangle loss while keeping LoRA and retain loss is a causal-attribution confound, but it is a correctness risk, not a circular reduction: no equation in the paper defines the reported accuracy or safety scores in terms of Ld, and no reported number is forced by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The main contributed object is an empirical editing scheme rather than a new theory. The theoretical section leans on an existing optimal transport bound and adds two assumptions in Appendix C.3, one about uniform Lipschitz margins and one that a better classifier exists after disentanglement. The latter assumption carries much of the explanatory weight. Hyperparameters such as temperature, loss weights, and target layer depth are chosen by hand; none are fitted to held-out data in a way that would make them free parameters in the derivations.

free parameters (5)
  • InfoNCE temperature sigma = 0.1
    Table 7; controls the degree of disentanglement in Eq. (3).
  • retain loss weight lambda = 0.1
    Table 7; balances the disentangle loss and retain loss in Eq. (5).
  • KL penalty weight alpha = 1
    Table 7; weights the KL term in Lr and is set to 0 in some safety experiments.
  • target layer depth ratio = 0.8
    Section 2.3; the target layer is chosen at 80% depth following Ref. [89], with layer ablations in Appendix B.7.
  • LoRA rank = 16
    Table 7; rank used for low-rank adaptation of the edited model.
assumptions (4)
  • standard math Theorem 1 from Chuang et al. (2021), the optimal transport margin bound, is taken as valid and applicable to the LLM-as-classifier formulation.
    Section 2.2 and Appendix C.2 rephrase the proof from Ref. [14]; the paper does not establish a new bound.
  • domain assumption The LLM can be treated as a classifier g composed with feature map phi over behavior classes, with a uniform margin Lipschitz constant over the hypothesis class.
    Appendix C.3 states this assumption; it is needed to apply Theorem 1.
  • ad hoc to paper After TELLME, a better classifier g-prime with higher margin exists and therefore the empirical tau-margin loss decreases.
    Appendix C.3 assumption; it is essentially the monitoring improvement the experiments then measure, so it is load-bearing.
  • domain assumption Safety labels in the Disentangle Set correctly identify the behaviors to separate.
    Section 2.1 constructs contrastive pairs from labeled behavior subsets; if labels are noisy or incomplete, the separation may not transfer to real unsafe behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring." pith.science (2026). https://pith.science/paper/4XUVM6XR

@misc{pith2026250205242,
  author       = {Pith},
  title        = {Pith review of: Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4XUVM6XR}},
  note         = {Machine review of arXiv:2502.05242}
}
read the original abstract

Large language models (LLMs) are becoming increasingly capable, but the mechanisms of their thinking and decision-making processes remain unclear. Chain-of-thoughts (CoTs) have been commonly utilized to externalize LLMs' thinking, but this strategy fails to accurately reflect LLMs' thinking process. Techniques based on LLMs' hidden representations provide an inner perspective to improve the monitorability of their latent thinking. However, previous methods only try to develop external modules instead of making LLMs themselves easier to monitor. In this paper, we propose a novel method, TELLME, improving the transparency of LLMs and helping monitors identify unsuitable and sensitive behaviors. Furthermore, we showcase the effectiveness of TELLME on detoxification tasks, where LLMs achieve consistent improvement among multimodal test sets, distinct architectures, and varying parameter scales. We further analyze TELLME's improvement on LLMs' generalization ability from both optimal transport theory and empirical perspectives.

Figures

Figures reproduced from arXiv: 2502.05242 by the authors.

Figure 1
Figure 1. TELLME is designed to enhance the transparency of LLMs and makes them easier to monitor without external modules. Disentanglement of different behaviors in LLMs’ representation space improve their transparency, achieving better monitoring reliability and safety performance. An ideal AI monitor method should identify unsuitable and sensitive behaviors based on representa￾tions directly instead of utilizing the unreli… view at source ↗
Figure 2
Figure 2. Overview of TELLME. TELLME disentangles representations by maximizing the examples’ similarities of similar behaviors, and minimizing the examples’ similarities of different behaviors. Meanwhile, TELLME utilizes constraints of l2 distance and KL distance on representations and probabilities, respectively, to maintain the general capabilities of LLMs. Specifically, we denote the original model as fθref and calculate … view at source ↗
Figure 3
Figure 3. t-SNE Visualization of LLMs’ representations in three scenarios and four LLMs. Formulation of LLMs’ generalization ability. To analyze LLMs’ generalization ability, we simplify LLMs from a next-token predictor to a classifier between different behaviors following [1, 13, 40]. For example, the safety-related tasks can be transformed into a prompt classification task between safe and harmful inputs [35, 42]. Specifica… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Layer-wise ablation studies in multi-risk classification task and model detoxification task [PITH_FULL_IMAGE:figures/full_fig_p020_4.png]
Figure 5
Figure 5. Figure 5: A example on detoxification task related to crime from Llama-3.1-8B-Instruct. [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 6
Figure 6. Figure 6: Another example on detoxification task from Llama-3.1-8B-Instruct. [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Position: Intelligent Coding Systems Should Write Programs with Justifications

    cs.SE 2025-08 conditional novelty 4.0 of 10

    A position paper advocating that intelligent coding systems should accompany code with justified explanations that are cognitively aligned and semantically faithful.

Reference graph

Works this paper leans on

91 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Generative ai text classification using ensemble llm approaches

    Harika Abburi, Michael Suesserman, Nirmala Pudota, Balaji Veeramani, Edward Bowen, and Sanmitra Bhattacharya. Generative ai text classification using ensemble llm approaches. arXiv preprint arXiv:2309.07755, 2023

    arXiv 2023

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    The internal state of an llm knows when it’s lying

    Amos Azaria and Tom Mitchell. The internal state of an llm knows when it’s lying. arXiv preprint arXiv:2304.13734, 2023

    arXiv 2023

  4. [4]

    Transparency and explainability of ai systems: ethical guidelines in practice

    Nagadivya Balasubramaniam, Marjo Kauppinen, Kari Hiekkanen, and Sari Kujala. Transparency and explainability of ai systems: ethical guidelines in practice. In International working conference on requirements engineering: foundation for software quality, pages 3–18. Springer, 2022

    2022

  5. [5]

    Spectrally-normalized margin bounds for neural networks

    Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalized margin bounds for neural networks. Advances in neural information processing systems, 30, 2017

    2017

  6. [6]

    Language models can explain neurons in language models

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models. URL https://openaipublic. blob. core. windows. net/neuron- explainer/paper/index. html.(Date accessed: 14.05. 2023), 2, 2023

    2023

  7. [7]

    High-dimension human value representation in large language models

    Samuel Cahyawijaya, Delong Chen, Yejin Bang, Leila Khalatbari, Bryan Wilie, Ziwei Ji, Etsuko Ishii, and Pascale Fung. High-dimension human value representation in large language models. arXiv preprint arXiv:2404.07900, 2024

    arXiv 2024

  8. [8]

    Internlm2 technical report, 2024

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, S...

    2024

Show all 91 references
  1. [9]

    Redunet: A white-box deep network from the principle of maximizing rate reduction

    Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, and Yi Ma. Redunet: A white-box deep network from the principle of maximizing rate reduction. Journal of machine learning research, 23(114):1–103, 2022

    2022

  2. [10]

    Selfie: Self-interpretation of large language model embeddings

    Haozhe Chen, Carl V ondrick, and Chengzhi Mao. Selfie: Self-interpretation of large language model embeddings. arXiv preprint arXiv:2403.10949, 2024

    2024 arXiv

  3. [11]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020

    2020

  4. [12]

    Reasoning models don’t always say what they think

    Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schul- man, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, et al. Reasoning models don’t always say what they think. arXiv preprint arXiv:2505.05410, 2025

    2025 arXiv

  5. [13]

    Token prediction as implicit classification to identify llm-generated text

    Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh, and Bhiksha Raj. Token prediction as implicit classification to identify llm-generated text. arXiv preprint arXiv:2311.08723, 2023

    2023 arXiv

  6. [14]

    Measuring generalization with optimal transport

    Ching-Yao Chuang, Youssef Mroueh, Kristjan Greenewald, Antonio Torralba, and Stefanie Jegelka. Measuring generalization with optimal transport. Advances in neural information processing systems, 34:8294–8306, 2021. 10

    2021

  7. [15]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  8. [16]

    Opencompass: A universal evaluation platform for foundation models

    OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass, 2023

    2023

  9. [17]

    Explainable and interpretable multimodal large language models: A comprehensive survey

    Yunkai Dang, Kaichen Huang, Jiahao Huo, Yibo Yan, Sirui Huang, Dongrui Liu, Mengxi Gao, Jie Zhang, Chen Qian, Kun Wang, et al. Explainable and interpretable multimodal large language models: A comprehensive survey. arXiv preprint arXiv:2412.02104, 2024

    2024 arXiv

  10. [18]

    Enhancing chat language models by scaling high-quality instructional conversations

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233, 2023

    2023 arXiv

  11. [19]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  12. [20]

    Scaling and evaluating sparse autoencoders

    Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093, 2024

    2024 arXiv

  13. [21]

    Patchscope: A unifying framework for inspecting hidden representations of language models

    Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. Patchscope: A unifying framework for inspecting hidden representations of language models. arXiv preprint arXiv:2401.06102, 2024

    2024 arXiv

  14. [22]

    Alignment faking in large language models

    Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, et al. Alignment faking in large language models. arXiv preprint arXiv:2412.14093, 2024

    2024 arXiv

  15. [23]

    Dimensionality reduction by learning an invariant mapping

    Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), volume 2, pages 1735–1742. IEEE, 2006

    2006

  16. [24]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022

    2022

  17. [25]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738, 2020

    2020

  18. [26]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  19. [27]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021

    2021

  20. [28]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  21. [29]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qiang- long Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Informat...

    2025

  22. [30]

    Can large language models explain themselves? a study of llm-generated self-explanations

    Shiyuan Huang, Siddarth Mamidanna, Shreedhar Jangam, Yilun Zhou, and Leilani H Gilpin. Can large language models explain themselves? a study of llm-generated self-explanations. arXiv preprint arXiv:2310.11207, 2023

    2023 arXiv

  23. [31]

    Vaccine: Perturbation-aware alignment for large language model

    Tiansheng Huang, Sihao Hu, and Ling Liu. Vaccine: Perturbation-aware alignment for large language model. arXiv preprint arXiv:2402.01109, 2024

    2024 arXiv

  24. [32]

    Anthropic: Responsible scaling policy

    Evan Hubinger. Anthropic: Responsible scaling policy. SuperIntelligence-Robotics-Safety & Alignment, 2(1), 2025. 11

    2025

  25. [33]

    The platonic representation hypothesis

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. arXiv preprint arXiv:2405.07987, 2024

    2024 arXiv

  26. [34]

    Klanderman, and William J Rucklidge

    Daniel P Huttenlocher, Gregory A. Klanderman, and William J Rucklidge. Comparing images using the hausdorff distance. IEEE Transactions on pattern analysis and machine intelligence, 15(9):850–863, 1993

    1993

  27. [35]

    Llama guard: Llm-based input-output safeguard for human-ai conversations

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674, 2023

    2023 arXiv

  28. [36]

    Beavertails: Towards improved safety alignment of llm via a human-preference dataset

    Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems, 36, 2024

    2024

  29. [37]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  30. [38]

    Neurips 2020 competition: Predicting generalization in deep learning

    Yiding Jiang, Pierre Foret, Scott Yak, Daniel M Roy, Hossein Mobahi, Gintare Karolina Dziugaite, Samy Bengio, Suriya Gunasekar, Isabelle Guyon, and Behnam Neyshabur. Neurips 2020 competition: Predicting generalization in deep learning. arXiv preprint arXiv:2012.07976, 2020

    2020 arXiv

  31. [39]

    Explainable artifi- cial intelligence for mental health through transparency and interpretability for understandability

    Dan W Joyce, Andrey Kormilitzin, Katharine A Smith, and Andrea Cipriani. Explainable artifi- cial intelligence for mental health through transparency and interpretability for understandability. npj Digital Medicine, 6(1):6, 2023

    2023

  32. [40]

    Theoretical analysis of weak-to- strong generalization

    Hunter Lang, David Sontag, and Aravindan Vijayaraghavan. Theoretical analysis of weak-to- strong generalization. arXiv preprint arXiv:2405.16043, 2024

    2024 arXiv

  33. [41]

    Getting more juice out of the sft data: Reward learning from human demonstration improves sft for llm alignment

    Jiaxiang Li, Siliang Zeng, Hoi-To Wai, Chenliang Li, Alfredo Garcia, and Mingyi Hong. Getting more juice out of the sft data: Reward learning from human demonstration improves sft for llm alignment. arXiv preprint arXiv:2405.17888, 2024

    2024 arXiv

  34. [42]

    Inference- time intervention: Eliciting truthful answers from a language model

    Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  35. [43]

    Salad-bench: A hierarchical and comprehensive safety benchmark for large language models

    Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models. arXiv preprint arXiv:2402.05044, 2024

    2024 arXiv

  36. [44]

    The wmdp benchmark: Measuring and reducing malicious use with unlearning

    Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218, 2024

    2024 arXiv

  37. [45]

    Open the pandora’s box of llms: Jailbreak- ing llms through representation engineering

    Tianlong Li, Xiaoqing Zheng, and Xuanjing Huang. Open the pandora’s box of llms: Jailbreak- ing llms through representation engineering. arXiv preprint arXiv:2401.06824, 2024

    2024 arXiv

  38. [46]

    Large language models in finance: A survey

    Yinheng Li, Shaofei Wang, Han Ding, and Hang Chen. Large language models in finance: A survey. In Proceedings of the fourth ACM international conference on AI in finance , pages 374–382, 2023

    2023

  39. [47]

    Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. arXiv preprint arXiv:2408.05147, 2024

    2024 arXiv

  40. [48]

    Latent guard: a safety framework for text-to-image generation

    Runtao Liu, Ashkan Khakzar, Jindong Gu, Qifeng Chen, Philip Torr, and Fabio Pizzati. Latent guard: a safety framework for text-to-image generation. In European Conference on Computer Vision, pages 93–109. Springer, 2025

    2025

  41. [49]

    Breaking free from mmi: A new frontier in rationalization by probing input utilization

    Wei Liu, Zhiying Deng, Zhongyu Niu, Jun Wang, Haozhao Wang, Zhigang Zeng, and Ruixuan Li. Breaking free from mmi: A new frontier in rationalization by probing input utilization. arXiv preprint arXiv:2503.06202, 2025

    2025 arXiv

  42. [50]

    Is the mmi criterion necessary for interpretability? degenerating non-causal features to plain noise for self-rationalization

    Wei Liu, Zhiying Deng, Zhongyu Niu, Jun Wang, Haozhao Wang, YuanKai Zhang, and Ruixuan Li. Is the mmi criterion necessary for interpretability? degenerating non-causal features to plain noise for self-rationalization. arXiv preprint arXiv:2410.06003, 2024. 12

    arXiv

  43. [51]

    Mgr: multi-generator based rationalization

    Wei Liu, Haozhao Wang, Jun Wang, Ruixuan Li, Xinyang Li, Yuankai Zhang, and Yang Qiu. Mgr: multi-generator based rationalization. arXiv preprint arXiv:2305.04492, 2023

    2023 arXiv

  44. [52]

    D-separation for causal self-explanation

    Wei Liu, Jun Wang, Haozhao Wang, Ruixuan Li, Zhiying Deng, Yuankai Zhang, and Yang Qiu. D-separation for causal self-explanation. Advances in Neural Information Processing Systems, 36:43620–43633, 2023

    2023

  45. [53]

    Efficient detection of toxic prompts in large language models

    Yi Liu, Junzhe Yu, Huijia Sun, Ling Shi, Gelei Deng, Yuqi Chen, and Yang Liu. Efficient detection of toxic prompts in large language models. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, pages 455–467, 2024

    2024

  46. [54]

    Are self-explanations from large language models faithful? In Findings of the Association for Computational Linguistics ACL 2024, pages 295–337, 2024

    Andreas Madsen, Sarath Chandar, and Siva Reddy. Are self-explanations from large language models faithful? In Findings of the Association for Computational Linguistics ACL 2024, pages 295–337, 2024

    2024

  47. [55]

    Introducing llama 3.1: Our most capable models to date

    AI Meta. Introducing llama 3.1: Our most capable models to date. Meta AI Blog, 12, 2024

    2024

  48. [56]

    Large language models in healthcare and medical domain: A review

    Zabir Al Nazi and Wei Peng. Large language models in healthcare and medical domain: A review. In Informatics, volume 11, page 57. MDPI, 2024

    2024

  49. [57]

    interpreting gpt: the logit lens, 2020

    nostalgebraist. interpreting gpt: the logit lens, 2020

    2020

  50. [58]

    Show your work: Scratchpads for intermediate computation with language models

    Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114, 2021

    2021 arXiv

  51. [59]

    Representation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  52. [60]

    Llms know more than they show: On the intrinsic representation of llm hallucinations

    Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. Llms know more than they show: On the intrinsic representation of llm hallucinations. arXiv preprint arXiv:2410.02707, 2024

    arXiv

  53. [61]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:2773...

    2022

  54. [62]

    Dean: Deactivating the coupled neurons to mitigate fairness-privacy conflicts in large language models

    Chen Qian, Dongrui Liu, Jie Zhang, Yong Liu, and Jing Shao. Dean: Deactivating the coupled neurons to mitigate fairness-privacy conflicts in large language models. arXiv preprint arXiv:2410.16672, 2024

    2024 arXiv

  55. [63]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  56. [64]

    Representation noising effectively prevents harmful fine-tuning on llms

    Domenic Rosati, Jan Wehner, Kai Williams, Łukasz Bartoszcze, David Atanasov, Robie Gonza- les, Subhabrata Majumdar, Carsten Maple, Hassan Sajjad, and Frank Rudzicz. Representation noising effectively prevents harmful fine-tuning on llms. arXiv preprint arXiv:2405.14577, 2024

    2024 arXiv

  57. [65]

    Xstest: A test suite for identifying exaggerated safety behaviours in large language models

    Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. arXiv preprint arXiv:2308.01263, 2023

    2023 arXiv

  58. [66]

    The effective rank: A measure of effective dimensionality

    Olivier Roy and Martin Vetterli. The effective rank: A measure of effective dimensionality. In 2007 15th European signal processing conference, pages 606–610. IEEE, 2007

    2007

  59. [67]

    Socialiqa: Com- monsense reasoning about social interactions

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Com- monsense reasoning about social interactions. arXiv preprint arXiv:1904.09728, 2019

    1904 arXiv

  60. [68]

    Facenet: A unified embedding for face recognition and clustering

    Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815–823, 2015

    2015

  61. [69]

    Lmfingerprints: Visual explanations of language model embedding spaces through layerwise contextualization scores

    Rita Sevastjanova, A Kalouli, Christin Beck, Hanna Hauptmann, and Mennatallah El-Assady. Lmfingerprints: Visual explanations of language model embedding spaces through layerwise contextualization scores. In Computer Graphics Forum, volume 41, pages 295–307. Wiley Online Librar...

    2022

  62. [70]

    k-variance: A clustered notion of variance

    Justin Solomon, Kristjan Greenewald, and Haikady Nagaraja. k-variance: A clustered notion of variance. SIAM Journal on Mathematics of Data Science, 4(3):957–978, 2022

    2022

  63. [71]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  64. [72]

    Daniel Freeman, Theodore R

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua ...

    2024

  65. [73]

    Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36, 2024

    2024

  66. [74]

    Optimal transport: old and new, volume 338

    Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2009

    2009

  67. [75]

    The wasserstein distances

    Cédric Villani and Cédric Villani. The wasserstein distances. Optimal transport: old and new, pages 93–111, 2009

    2009

  68. [76]

    Glue: A multi-task benchmark and analysis platform for natural language understanding

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018

    2018 arXiv

  69. [77]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  70. [78]

    Diff-erank: A novel rank-based metric for evaluating large language models

    Lai Wei, Zhiquan Tan, Chenghai Li, Jindong Wang, and Weiran Huang. Diff-erank: A novel rank-based metric for evaluating large language models. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  71. [79]

    Reft: Representation finetuning for language models

    Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D Manning, and Christopher Potts. Reft: Representation finetuning for language models. arXiv preprint arXiv:2404.03592, 2024

    2024 arXiv

  72. [80]

    Good idea or not, representation of llm could tell

    Yi Xu, Bo Xue, Shuqian Sheng, Cheng Deng, Jiaxin Ding, Zanwei Shen, Luoyi Fu, Xinbing Wang, and Chenghu Zhou. Good idea or not, representation of llm could tell. arXiv preprint arXiv:2409.13712, 2024

    2024 arXiv

  73. [81]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  74. [82]

    A fingerprint for large language models

    Zhiguang Yang and Hanzhou Wu. A fingerprint for large language models. arXiv preprint arXiv:2407.01235, 2024

    2024

  75. [83]

    Lofit: Localized fine-tuning on llm representations

    Fangcong Yin, Xi Ye, and Greg Durrett. Lofit: Localized fine-tuning on llm representations. arXiv preprint arXiv:2406.01563, 2024

    2024 arXiv

  76. [84]

    Barlow twins: Self- supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. In International conference on machine learning, pages 12310–12320. PMLR, 2021

    2021

  77. [85]

    Similar data points identification with llm: A human-in-the-loop strategy using summarization and hidden state insights

    Xianlong Zeng, Yijing Gao, Fanghao Song, and Ang Liu. Similar data points identification with llm: A human-in-the-loop strategy using summarization and hidden state insights. arXiv preprint arXiv:2404.04281, 2024

    2024 arXiv

  78. [86]

    Real: Response embedding- based alignment for llms

    Honggen Zhang, Xufeng Zhao, Igor Molybog, and June Zhang. Real: Response embedding- based alignment for llms. arXiv preprint arXiv:2409.17169, 2024

    2024 arXiv

  79. [87]

    Reef: Representation encoding fingerprints for large language models

    Jie Zhang, Dongrui Liu, Chen Qian, Linfeng Zhang, Yong Liu, Yu Qiao, and Jing Shao. Reef: Representation encoding fingerprints for large language models. arXiv preprint arXiv:2410.14273, 2024

    2024 arXiv

  80. [88]

    Agieval: A human-centric benchmark for evaluating foundation models

    Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: A human-centric benchmark for evaluating foundation models. arXiv preprint arXiv:2304.06364, 2023. 14

    2023 arXiv

  81. [89]

    Improving alignment and robustness with circuit breakers

    Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, J Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 202...

    2024

  82. [90]

    [ 62] disentangles LLMs’ awareness of fairness and privacy by deactivating the entangled neurons in representations

    performs intervention functions precisely on the model’s target layer and the target position of the input tokens. [ 62] disentangles LLMs’ awareness of fairness and privacy by deactivating the entangled neurons in representations. Contrastive Learning. Contrastive self-superv...

  83. [91]

    safe" or

    The model is trained on 4 GPUs for about 8 hours. 17 Table 7: Specific Experi- mental Hyperparameters of TELLME. Name Value Learning Rate 0.001 λ 0.1 α 1 σ 0.1 Lora Alpha 16 Lora Dim 16 Lora Dropout 0.05 Epoch 2 Table 8: Additional Experimental Results of TELLME on the Safety ...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.