Pith. sign in

REVIEW 3 major objections 4 minor 25 references

OPIUM claims that a training-free, dual-objective optimization of steering vectors can restore a model's safety without sacrificing the intended behavioral shift.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 11:39 UTC pith:FY3WBTXT

load-bearing objection OPIUM is a plausible, clearly-specified inference-time method for sanitizing steering vectors, but the Qwen ASR improvement—the load-bearing result—is within sampling noise and the final-layer ablation contradicts the 'below baseline' claim. the 3 major comments →

arxiv 2607.19806 v1 pith:FY3WBTXT submitted 2026-07-22 cs.LG cs.AI

OPIUM: Mitigating Steering Externalities and Over-Refusal via Dual Objective Latent Optimization

Kavin Aravindan , Arihant Rastogi , Aadi Prasad , Krishak Aneja , Saiyam Jain , Vaishnavi Shivkumar , Ponnurangam Kumaraguru This is my paper
classification cs.LG cs.AI
keywords activation steeringrepresentation engineeringsafety-utility tradeoffover-refusaljailbreaklatent optimizationLLM safetysteering externalities
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Activation steering—adding a vector to a language model's hidden states to shift behavior—can have side effects: a vector that increases compliance can weaken safety guardrails, while a refusal vector can make the model reject benign requests. OPIUM is a training-free procedure that takes the original steering vector and optimizes a replacement vector that reproduces the desired behavior on prompts where the intervention is useful and matches a safer reference model on prompts where it isn't. The paper claims this improves the safety–utility tradeoff relative to vanilla steering and directional ablation on two open-weight instruct models, in both the steering-externality and over-refusal settings. If true, it means many undesirable side effects of activation steering can be fixed directly in activation space, without model weight updates, in minutes. The paper also reports that the comparison layer where matching happens matters and acknowledges it lacks a principled criterion for choosing it.

Core claim

The paper claims that harmful side effects of activation steering can be mitigated without retraining by optimizing a new steering vector v* that satisfies two representation-matching constraints: on benign prompts it must reproduce the downstream hidden states of the model steered with the original utility vector, and on harmful prompts it must match the hidden states of a safer reference model (the unsteered base, or the refusal-steered model, depending on the setting). The optimized vector is found by gradient descent on the L2 distance between last-token residual-stream states at a downstream comparison layer, with the vector's norm pinned to that of the original steering vector. On Qwen

What carries the argument

The dual-objective loss L(v) = ||z_v_m(x_A) - z_{v_A}_m(x_A)||^2 + λ ||z_v_m(x_B) - z_{v_B}_m(x_B)||^2, optimized over the steering vector v with norm constraint ||v|| = ||v_A||, using last-token residual-stream hidden states at a downstream comparison layer m. This device lets OPIUM search the redundant latent space for a direction that preserves the target behavior while avoiding the side-effect direction.

Load-bearing premise

The load-bearing premise is that matching the model's last-token hidden states at a single downstream layer on a few dozen optimization prompts is enough to reproduce the desired full-sequence generation behavior on held-out prompts; if the safety-relevant geometry is not cleanly captured at that layer or on those prompts, the optimized vector can underfit one objective or introduce interference—and the paper itself notes the lack of a principled criterion for choosing that l

What would settle it

Take an OPIUM vector optimized on one set of harmful prompts (e.g., the HarmBench DirectRequest split) and evaluate it on a qualitatively different jailbreak type, such as multi-turn or adversarial-suffix attacks; if the attack-success-rate reduction does not transfer, the safety matching is overfit to the specific anchor prompts. Alternatively, run OPIUM on Qwen with the comparison layer set to the final logits—where the paper's own ablation shows ASR increases—and check whether the claimed tradeoff improvement collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Activation-space sanitization can become a practical post-processing step for any steering vector, without retraining or added parameters.
  • Steering vectors that are known to be unsafe (e.g., utility vectors that spike attack success rate) can be rehabilitated using only 32–80 benign and harmful prompts and a few minutes of optimization.
  • The choice of comparison layer emerges as a critical hyperparameter, making automated layer selection a concrete research problem for making OPIUM a drop-in filter.
  • OPIUM suggests that utility and safety are not strictly zero-sum in activation space, contrary to what naive steering experiments might suggest.
  • The same dual-objective recipe applies symmetrically to refusal vectors, turning over-refusal reduction into the same representation-matching problem.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If representation-matching transfers, OPIUM could be applied to other side-effect pairs, such as language steering that inadvertently causes English refusals; the paper notes this specific case may break the anchor assumption.
  • A testable extension is to measure how OPIUM's Pareto front shifts when the optimization prompt set is drawn from the evaluation distribution versus a disjoint distribution, which would quantify how much the method depends on anchor-prompt coverage.
  • The observed layer-dependence suggests a natural extension: jointly optimizing over multiple comparison layers, or selecting m via a small validation search analogous to the reported λ ablation, could make the method more robust across model families.
  • If OPIUM works as claimed, it implies that safety alignment failures induced by inference-time interventions are often reversible at inference time, which would lower the cost of deploying steering-based control in production systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces OPIUM, a training-free method that sanitizes activation-steering vectors by optimizing a new vector to match the downstream representations of the original utility-steered model on benign prompts and of a safer reference model on harmful prompts. The objective is a weighted sum of two L2 representation-matching losses (Eq. 1), with a norm-preserving projection. The authors evaluate on Qwen-2.5-7B-Instruct and LLaMA-3.1-8B-Instruct for two use cases: mitigating safety externalities of utility steering (measured by MCQ utility and HarmBench ASR) and reducing over-refusal of refusal vectors (measured on FalseReject and HarmBench). They claim that OPIUM improves the safety–utility tradeoff relative to vanilla steering and directional ablation across settings.

Significance. If the empirical claims hold, OPIUM would be a useful, lightweight tool for reducing the side effects of activation steering without model fine-tuning. The paper has clear strengths: the method is simple and explicitly specified; the optimization details, including layer choices, hyperparameters, and prompt counts, are mostly reported; ablation studies over λ and comparison layer m are provided; code is available; and the limitations section candidly discusses the lack of a principled layer-selection criterion. The main axis of uncertainty is statistical: the flagship Qwen result is close to noise, and some appendix configurations contradict the headline claim. These issues are fixable with more careful reporting and analysis, but they are load-bearing for the paper's central message.

major comments (3)
  1. [§4.2.3 and Appendix B, Tables 2–3] The headline claim that OPIUM 'suppresses ASR below even the unsteered baselines' is not supported by the paper's own appendix for Qwen. Table 2 (final-layer target) shows Qwen Corrigibility ASR increasing to 0.331–0.347 from a baseline of 0.287 for all λ, and Table 3 shows m=26 giving +0.035. The best Qwen config (m=24) gives 0.266, a 2.1pp drop on 320 prompts; the binomial standard error at p≈0.29 is about 2.5pp, so this is within sampling noise. Since λ and m are selected using small validation sets (32 prompts per objective) with no repeated seeds or confidence intervals, the Qwen-specific component of the central claim is statistically unestablished. Please report CIs, repeated runs, or appropriately qualify the claim.
  2. [§5 and Appendix B, paragraph on comparison layer m] The comparison layer m is a free parameter and the paper states 'we currently lack a principled criterion for selecting it.' Table 3 shows that Qwen benefits from m=24 while Llama benefits from later/final layers, so the empirical conclusions depend on an unprincipled, model-specific choice. As written, the method requires validation-based selection of m, but no procedure is given for doing this reliably. This is a limitation, but it is also load-bearing for the claim of a general method. Please either provide a selection rule or demonstrate robustness of the main conclusions across a wider range of m values.
  3. [Appendix A and B; Eq. (1)] The optimization uses only 32 benign and 32 harmful prompts, and the reported evaluation is on held-out sets. The paper does not assess sensitivity to the particular choice of optimization prompts. Because the two loss terms are matched on a small sample, the optimized vector may overfit to those prompts; the lack of resampling or seed variation makes it difficult to know how much of the reported ASR movement is attributable to the objective versus the specific optimization set. A bootstrap over optimization prompts, or results across a few random splits, would materially strengthen the paper.
minor comments (4)
  1. [Appendix C] Typo: 'durining' should be 'during'.
  2. [§3.2] Notation is inconsistent: the problem statement introduces vB = 0B (zero vector), but the loss in Eq. (1) uses vB as a general steering vector. Clarify that the zero vector is a special case.
  3. [Table 1] The over-refusal table reports ASR and CR values without sample sizes or uncertainty. The HarmBench evaluation uses 320 prompts and FalseReject uses 318; a simple binomial CI would help assess whether the differences (e.g., Qwen ASR 27.50 vs 32.50) are meaningful, especially since the claimed improvements over the refusal vector are modest on Llama.
  4. [Figure 2] The figure caption refers to 'Best Optimized' but the selection rule for 'best' is not defined in the main text. State explicitly whether this is the configuration selected by validation loss, by validation ASR, or by a Pareto criterion.

Circularity Check

0 steps flagged

No significant circularity: OPIUM's objective is latent representation matching, but its headline claims are tested on external held-out generation-level metrics.

full rationale

OPIUM's derivation chain is an empirical optimization-and-evaluation pipeline, not a tautology. Equation (1) optimizes v* by minimizing L2 distances to z^{vA}_m on benign prompts and to z^{vB}_m on harmful prompts at a chosen comparison layer m; the reported outcomes are generation-level measures — MCQ target probability, HarmBench ASR, and FalseReject CR/USR — computed on held-out prompts that are explicitly split from the optimization sets (Section B and Appendix C: “the split is performed to ensure no leaks between the evaluation set that the metrics are reported on and the optimization set”). No equation in the paper defines utility or safety as the matched representation distance itself, so the headline “improved safety–utility tradeoff” is not equal by construction to the objective. The utility term does bias the optimizer toward retaining vA behavior, but that is the method's stated mechanism, and the held-out, generation-level evaluation remains a genuine test of whether layer-m representation matching transfers. The ablation tables also show non-forced outcomes (e.g., final-layer comparison targets increase Qwen ASR by +4.4 to +6.0 percentage points in Table 2), so the central claim is empirically contingent rather than definitionally guaranteed. The acknowledged lack of a principled criterion for selecting m (Section 5) and the use of a small validation set for hyperparameter choice raise statistical and robustness concerns, but these are correctness risks, not circularity. There is no load-bearing self-citation, no imported uniqueness theorem, and no fitted parameter renamed as an independent prediction. Therefore no circular step is established.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claim rests primarily on domain assumptions: representational redundancy lets a single optimized vector satisfy both utility and safety anchors; last-token L2 matching at one downstream layer transfers to full-sequence behavior; and the anchor models capture the desired behaviors. These are not proven in the paper, and the paper itself flags layer-choice and entanglement risks. The numerical results also depend on hand/tuned hyperparameters (λ, m, α, layers, η, prompt counts).

free parameters (5)
  • loss weight λ = 0.5, 10, 15, 20, 25 depending on setting
    Balances utility and safety objectives (Eq. 1); ablated in Table 2 and shown to be model/behavior-dependent; no principled selection rule.
  • comparison layer m = 22/24/26/final for Qwen; 17/19/21/23/final for Llama
    The downstream layer at which representations are matched; selected per model/behavior based on ASR in Table 3; the paper admits there is no principled criterion (Section 5).
  • injection layer ℓ = 18 for Qwen; 15 or 13 for Llama depending on experiment
    Layer at which the steering vector is added; chosen by hand in Appendices A and C; affects the intervention.
  • steering strength α = 4.0 for Qwen; 2.0 or 4.0 for Llama
    Scales the injected vector; chosen per model and experiment; not swept in the paper.
  • optimization hyperparameters (η, prompt counts, optimizer) = η=0.01 or 0.005; 32+32 or 80+80 prompts; Adam
    Learning rate, number of optimization prompts, and optimizer are chosen per experiment; no sensitivity analysis for prompt counts or steps.
axioms (5)
  • domain assumption Residual-stream activations are differentiable w.r.t. the injected vector, and L2 matching of last-token states at layer m transfers to full-sequence generation behavior.
    The whole method relies on optimizing v through the network and on the matched representations being behaviorally meaningful; this is not proven and the lack of a principled m-selection criterion is acknowledged.
  • domain assumption Massive representational redundancy: many latent directions can trigger identical macroscopic behaviors, so a single optimized vector can satisfy both utility and safety anchors.
    Used in Section 1 and 2, based on a toy model (Elhage et al., 2022) and a LessWrong blog post (Goldman-Wetzler, 2024), not on direct measurements in Qwen-2.5 or LLaMA-3.1.
  • domain assumption The unsteered base model is a valid 'safer reference' for harmful prompts, and this anchor does not corrupt utility when matched only on benign prompts.
    Central to the dual-objective design (Eq. 1/2 and Appendix A); the paper's Limitations section notes this can fail when behaviors are semantically entangled, e.g., in language steering.
  • domain assumption Benchmark judge labels are reliable proxies for safety and compliance (HarmBench Llama-2-13B classifier, Gemini 2.5 Flash for FalseReject).
    All safety/over-refusal metrics depend on these automated judges (Section 4.2.2, Appendix C); no validation of judge agreement is provided.
  • domain assumption The 32/32 and 80/80 optimization prompt subsets are representative of the held-out evaluation distributions.
    The method is fit on these small subsets, and the paper asserts no leakage but provides no analysis of sampling variance or coverage.

pith-pipeline@v1.3.0-alltime-deepseek · 11219 in / 15935 out tokens · 163164 ms · 2026-08-01T11:39:01.957562+00:00 · methodology

0 comments
read the original abstract

Activation steering provides a lightweight mechanism for controlling large language models at inference time, but steering vectors can have unintended externalities: utility vectors may weaken safety behavior, while refusal vectors may induce over-refusal on benign prompts. We introduce OPIUM (Optimizing Protected Injections via Utility Manifolds), a training-free method for sanitizing steering vectors through representation matching. Given reference behaviors on two prompt sets, OPIUM optimizes a new steering vector that preserves the downstream representations induced by the desired intervention while matching a safer reference behavior on prompts where the original vector fails. Across steering-externality and over-refusal settings, OPIUM improves the safety--utility tradeoff relative to vanilla steering and directional ablation, suggesting that harmful side effects of activation steering can often be mitigated directly in activation space.

Figures

Figures reproduced from arXiv: 2607.19806 by Aadi Prasad, Arihant Rastogi, Kavin Aravindan, Krishak Aneja, Ponnurangam Kumaraguru, Saiyam Jain, Vaishnavi Shivkumar.

Figure 1
Figure 1. Figure 1: Conceptual overview of OPIUM. While naive activation steering inadvertently degrades model safety mechanisms and increases compliance to harmful prompts, OPIUM sanitizes these vectors via dual-objective optimization in activation space. This restores safety close to baseline levels while preserving the target utility. demonstrates that injecting utility-driven vectors may in￾advertently erode the model’s s… view at source ↗
Figure 2
Figure 2. Figure 2: Steered ASR (↓) vs. Utility (↑) across three evaluation settings. OPIUM (Best Optimized, purple diamond) consistently occupies the upper-left Pareto-optimal region, restoring safety to near-baseline levels while preserving utility — outperforming both vanilla steering and directional ablation on the joint objective [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Loss curves during OPIUM optimization for steering externality mitigation. LA (primary) represents the utility objective, minimizing the distance to the representations of the model steered with the original utility vector on benign prompts. LB (secondary) represents the safety objective, minimizing the distance to the unsteered base model on malicious prompts to actively scrub jailbreak vulnerabilities [… view at source ↗
Figure 4
Figure 4. Figure 4: Loss curves during OPIUM optimization for over-refusal mitigation. Because the base vector is a refusal vector, the anchoring targets are inverted: LA (primary) represents the safety objective, minimizing the distance to the original refusal-steered model on harmful prompts to preserve adversarial robustness. LB (secondary) represents the utility objective, minimizing the distance to the unsteered base mod… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 7 linked inside Pith

  1. [1]

    Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models , journal =

    Chen Xiong and Zhiyuan He and Pin. Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models , journal =. 2026 , url =. doi:10.48550/ARXIV.2602.04896 , eprinttype =. 2602.04896 , timestamp =

    doi:10.48550/arxiv.2602.04896 2026

  2. [2]

    2024 , eprint=

    Steering Language Models With Activation Engineering , author=. 2024 , eprint=

    2024

  3. [3]

    Kenneth Li and Oam Patel and Fernanda B. Vi. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , booktitle =. 2023 , url =

    2023

  4. [4]

    Steering Llama 2 via Contrastive Activation Addition , booktitle =

    Nina Rimsky and Nick Gabrieli and Julian Schulz and Meg Tong and Evan Hubinger and Alexander Matt Turner , editor =. Steering Llama 2 via Contrastive Activation Addition , booktitle =. 2024 , url =. doi:10.18653/V1/2024.ACL-LONG.828 , timestamp =

    doi:10.18653/v1/2024.acl-long.828 2024

  5. [5]

    Lee and Inkit Padhi and Karthikeyan Natesan Ramamurthy and Erik Miehling and Pierre L

    Bruce W. Lee and Inkit Padhi and Karthikeyan Natesan Ramamurthy and Erik Miehling and Pierre L. Dognin and Manish Nagireddy and Amit Dhurandhar , title =. The Thirteenth International Conference on Learning Representations,. 2025 , url =

    2025

  6. [6]

    Paul R. XSTest:. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers),. 2024 , url =. doi:10.18653/V1/2024.NAACL-LONG.301 , timestamp =

    doi:10.18653/v1/2024.naacl-long.301 2024

  7. [7]

    OR-Bench: An Over-Refusal Benchmark for Large Language Models , booktitle =

    Justin Cui and Wei. OR-Bench: An Over-Refusal Benchmark for Large Language Models , booktitle =. 2025 , url =

    2025

  8. [8]

    Reddy , title =

    Zhehao Zhang and Weijie Xu and Fanyou Wu and Chandan K. Reddy , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2505.08054 , eprinttype =. 2505.08054 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.08054 2025

  9. [9]

    Jailbroken: How Does

    Alexander Wei and Nika Haghtalab and Jacob Steinhardt , editor =. Jailbroken: How Does. Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , year =

    2023

  10. [10]

    Zico Kolter and Matt Fredrikson , title =

    Andy Zou and Zifan Wang and J. Zico Kolter and Matt Fredrikson , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2307.15043 , eprinttype =. 2307.15043 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2307.15043 2023

  11. [11]

    Forsyth and Dan Hendrycks , editor =

    Mantas Mazeika and Long Phan and Xuwang Yin and Andy Zou and Zifan Wang and Norman Mu and Elham Sakhaee and Nathaniel Li and Steven Basart and Bo Li and David A. Forsyth and Dan Hendrycks , editor =. HarmBench:. Forty-first International Conference on Machine Learning,. 2024 , url =

    2024

  12. [12]

    Pappas and Florian Tram

    Patrick Chao and Edoardo Debenedetti and Alexander Robey and Maksym Andriushchenko and Francesco Croce and Vikash Sehwag and Edgar Dobriban and Nicolas Flammarion and George J. Pappas and Florian Tram. JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models , booktitle =. 2024 , url =

    2024

  13. [13]

    Toy Models of Superposition , journal =

    Nelson Elhage and Tristan Hume and Catherine Olsson and Nicholas Schiefer and Tom Henighan and Shauna Kravec and Zac Hatfield. Toy Models of Superposition , journal =. 2022 , url =. doi:10.48550/ARXIV.2209.10652 , eprinttype =. 2209.10652 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2209.10652 2022

  14. [14]

    write code

    I found >800 orthogonal "write code" steering vectors in Llama-3-8B-Instruct , author=. 2024 , url=

    2024

  15. [15]

    Representation Engineering:

    Andy Zou and Long Phan and Sarah Li Chen and James Campbell and Phillip Guo and Richard Ren and Alexander Pan and Xuwang Yin and Mantas Mazeika and Ann. Representation Engineering:. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2310.01405 , eprinttype =. 2310.01405 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.01405 2023

  16. [16]

    Pappas and Eric Wong , title =

    Patrick Chao and Alexander Robey and Edgar Dobriban and Hamed Hassani and George J. Pappas and Eric Wong , title =. 2025 , url =. doi:10.1109/SATML64287.2025.00010 , timestamp =

    arXiv 2025

  17. [17]

    Anderson and Yaron Singer and Amin Karbasi , editor =

    Anay Mehrotra and Manolis Zampetakis and Paul Kassianik and Blaine Nelson and Hyrum S. Anderson and Yaron Singer and Amin Karbasi , editor =. Tree of Attacks: Jailbreaking Black-Box LLMs Automatically , booktitle =. 2024 , url =

    2024

  18. [18]

    The Thirteenth International Conference on Learning Representations,

    Xiangyu Qi and Ashwinee Panda and Kaifeng Lyu and Xiao Ma and Subhrajit Roy and Ahmad Beirami and Prateek Mittal and Peter Henderson , title =. The Thirteenth International Conference on Learning Representations,. 2025 , url =

    2025

  19. [19]

    Lepori and Lucas Dixon , editor =

    Asma Ghandeharioun and Ann Yuan and Marius Guerard and Emily Reif and Michael A. Lepori and Lucas Dixon , editor =. Who's asking? User personas and the mechanics of latent misalignment , booktitle =. 2024 , url =

    2024

  20. [20]

    Galichin and Alexey Dontsov and Oleg Y

    Anton Korznikov and Andrey V. Galichin and Alexey Dontsov and Oleg Y. Rogov and Ivan V. Oseledets and Elena Tutubalina , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2509.22067 , eprinttype =. 2509.22067 , timestamp =

    doi:10.48550/arxiv.2509.22067 2025

  21. [21]

    CoRR , volume =

    An Yang and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chengyuan Li and Dayiheng Liu and Fei Huang and Haoran Wei and Huan Lin and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Yang and Jiaxi Yang and Jingren Zhou and Junyang Lin and Kai Dang and Keming Lu and Keqin Bao and Kexin Yang and Le Yu and Mei Li and Mi...

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.15115 2024

  22. [22]

    CoRR , volume =

    Llama Team , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2407.21783 , eprinttype =. 2407.21783 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.21783 2024

  23. [23]

    Advances in Neural Information Processing Systems , volume=

    Refusal in language models is mediated by a single direction , author=. Advances in Neural Information Processing Systems , volume=

  24. [24]

    Refusal in Language Models Is Mediated by a Single Direction , url =

    Arditi, Andy and Obeso, Oscar and Syed, Aaquib and Paleka, Daniel and Panickssery, Nina and Gurnee, Wes and Nanda, Neel , booktitle =. Refusal in Language Models Is Mediated by a Single Direction , url =. doi:10.52202/079017-4322 , editor =

    doi:10.52202/079017-4322

  25. [25]

    arXiv preprint arXiv:2403.05767 , year=

    Extending activation steering to broad skills and multiple behaviours , author=. arXiv preprint arXiv:2403.05767 , year=

    Pith/arXiv arXiv