Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

DSAS gives every token its own steering strength, so a model is intervened on only when the token looks like the unwanted concept; this yields a better toxicity-versus-utility trade-off than uniform steering.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:43 UTC pith:LIAX57ZT

load-bearing objection DSAS is a plausible, well-tested conditional-steering layer whose main evidence would become far easier to trust with code and error bars; the unexamined train/eval covariate shift is worth one robustness experiment, not a rejection. the 4 major comments →

arxiv 2512.03661 v2 pith:LIAX57ZT submitted 2025-12-03 cs.LG cs.AI

Dynamically Scaled Activation Steering

Alex Ferrando , Xavier Suau , Jordi Gonz\`alez , Pau Rodriguez This is my paper
classification cs.LG cs.AI
keywords activation steeringtoxicity mitigationconditional steeringlinear separabilitylogistic regressionPareto fronttext-to-image diffusionLLM alignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

DSAS is a steering-agnostic wrapper that learns, for each layer, a logistic classifier that separates unwanted (source) from neutral (control) activations, then uses the classifier's sigmoid output as a per-token, per-layer scaling factor in an interpolation between the original activation and any steering transform. The paper tries to establish that this simple "when to steer" module improves the full trade-off curve—toxicity reduction versus preservation of language-model skills—when added to three representative steering methods (CAA, ITI, and LinEAS) on three open LLMs. It also shows the same mechanism transfers to a text-to-image diffusion model, selectively blurring concept-related prompts while leaving unrelated prompts largely untouched. If correct, the result makes conditional steering practical: near-zero inference overhead, a small training set, an interpretable per-token strength map, and graceful fallback when supervision is uninformative.

Core claim

The central claim is that the decision of how strongly to steer should be decoupled from how to steer. DSAS trains one logistic regressor per layer on average token embeddings of 32 source and 32 control sentences, reduced with PCA to five components, and uses the classifier output h(t) in [0,1] as the interpolation weight between the original embedding and the steered one. The paper argues, with Pareto-front comparisons across three LLMs, that adding this module to CAA, ITI, or LinEAS shifts the trade-off so that for any given toxicity level the model retains more MMLU accuracy and lower perplexity than the same method with uniform steering. A jointly trained end-to-end variant, E2E-DSAS, m

What carries the argument

The central object is the per-token, per-layer sigmoid gate h(t) = ρ(θ^T U^T (t − μ) + b), a logistic regressor trained on PCA-projected mean source/control embeddings. The mechanism is the interpolation T_DSAS(t) = (1 − h(t))·t + h(t)·T(t; λ), which separates 'when' (h, a learned per-token probability that the token belongs to the unwanted source class) from 'how' (T, any existing steering transform, possibly modulated by a global strength λ). The classifier's accuracy per layer can disable steering below a threshold; an adaptive variant scales by cross-validated accuracy instead. In the end-to-end version, the sigmoid is replaced by a learnable activation, and the gate parameters are train

Load-bearing premise

The load-bearing premise is that a linear classifier trained on just 32 toxic and 32 neutral sentences can tell, at every layer, which tokens actually need steering; if it cannot, DSAS silently becomes vanilla steering and the claimed trade-off gain disappears.

What would settle it

Give DSAS a held-out set of toxic prompts, record the average classifier output h(t) on toxic versus non-toxic tokens across layers, and check whether the mean h for toxic tokens clearly exceeds the mean for non-toxic tokens; if it does not—or if removing DSAS leaves the same Pareto front—the mechanism is not detecting the target behavior and the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Adding DSAS to CAA, ITI, or LinEAS yields a Pareto front that dominates the corresponding uniform-steering method on both MMLU-vs-toxicity and perplexity-vs-toxicity, across Qwen 2.5 (1.5B and 7B) and Gemma 2 (2B).
  • DSAS outperforms the conditional steering methods CAST and MERA in toxicity mitigation; CAST can look competitive on single-token MMLU but falls behind on multi-token generation.
  • When classifier labels are uninformative or noisy, DSAS degrades gracefully to vanilla steering with roughly halved strength, so it does not harm the underlying method.
  • DSAS transfers to text-to-image generation: applied with CAA on U-Net normalization layers, it selectively blurs concept-related prompts while preserving unrelated ones, although pixel-level activation maps localize the concept only coarsely.
  • The per-token strength map is itself an interpretability signal: on toxic examples, high DSAS activations fall on the spans that actually carry the toxic content.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because DSAS produces per-token strength maps at nearly no cost, it could double as an auditing probe: comparing h(t) against human toxicity annotations on held-out prompts would reveal whether the model's own representations and a human labeler agree on what counts as toxic.
  • The method-agnostic 'when vs. how' split is naturally portable to other activation edits beyond toxicity, such as knowledge editing or factuality steering, where an input-dependent gate would likely reduce side effects; this is untested in the paper.
  • The strong results with just 32 source and 32 control sentences suggest the linear-separability assumption is doing most of the work; if later models have less linearly separable toxic/neutral activations, the paper's own suggested MLP regressor would be needed, sacrificing the near-zero overhead.
  • Appendix C derives λ ≈ 2.84 as the corrected global strength for DSAS on one model; whether that correction is stable across model families is an open, testable question that would save practitioners the usual strength tuning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Dynamically Scaled Activation Steering (DSAS), a method-agnostic wrapper that modulates the per-token (or per-spatial-feature) strength of an existing activation-steering transformation. A per-layer logistic regressor, trained on source vs. control average embeddings in a PCA-reduced space, produces a probability used to interpolate between the original activation and the steered activation. The paper also proposes an end-to-end variant (E2E-DSAS) learned jointly with a LinEAS-style affine map, using a 1D Wasserstein source loss plus a control-preservation regularizer. Experiments cover toxicity mitigation on three LLMs with three base steering methods, comparisons to CAST and MERA, and a text-to-image diffusion demonstration. The central claim is that DSAS consistently improves the Pareto front of toxicity versus capability retention relative to the underlying steering method.

Significance. If the claims hold, DSAS is a valuable, low-overhead, method-agnostic mechanism for making activation steering conditional on input content. The empirical scope is substantial: three LLMs, three base steering methods, a diffusion model, ablations on PCA dimension, training set size, layer placement, noise robustness, and the τ hyperparameter. The paper also states limitations explicitly and provides detailed appendices. The main concern is that the train/test distributions for the gating classifier are not aligned: classifiers are trained on unsteered, sentence-average activations but applied to per-token activations after earlier interventions. Without quantitative validation of the classifier under those shifts, the mechanism underlying the reported Pareto improvements is not fully established. The absence of error bars on the central Pareto-front plots further weakens the evidence.

major comments (4)
  1. [§3.2, Algorithm 1 vs. Algorithm 3] The gating classifiers h_ℓ are trained on activations from the unmodified network (Algorithm 1, lines 4–6) but at inference are evaluated on activations that have already passed through T_1, …, T_{ℓ−1} (Algorithm 3, lines 4–7). For ℓ > 1 this is a train/test covariate shift. Section I (Fig. 12) shows that for toxic inputs the L2 divergence from original activations grows with depth, so later-layer classifiers are applied to substantially shifted inputs. The paper does not measure how much this shift degrades classifier accuracy or calibration. Please add an experiment comparing h_ℓ predictions on unsteered vs. steered activations for held-out toxic and control inputs, and either retrain on steered activations or show quantitatively that the shift is harmless.
  2. [§3.2, Eq. (3.1)–(3.4)] The logistic regressor is trained on sentence-level average embeddings ¯t_ℓ, but Eq. (3.4) applies it per token t_ℓ,k. The paper acknowledges the lack of per-token labels (p. 4), yet does not quantify how well a classifier fitted to average embeddings generalizes to individual token embeddings. Since the method's core mechanism is the per-token interpolation weight, please provide token-level validation (e.g., false-positive/negative rates on tokens known to carry the toxic concept, or calibrated probability checks). Without this, an alternative explanation for the Pareto improvement is that the procedure merely attenuates average steering strength rather than selectively targeting the intended tokens.
  3. [Fig. 2, Table 1] The main quantitative claim is a Pareto-front improvement, but the curves are shown without error bars, and toxicity is averaged over only four generation seeds. Table 1 selects operating points after the fact using a constrained minimum rule (limiting PPLWik to ≤5% and MMLU to ≤3% change) and does not report the λ values at which the selected points were obtained. Please add uncertainty quantification (standard errors, confidence intervals, or a paired dominance analysis) and report the selected λ values or use a pre-specified selection rule.
  4. [Appendix C, Eq. (C.1)] The justification for λ > 1 is a distance-ratio rescaling between means of thresholded individual embeddings. This is not derived from the LinEAS/optimal-transport formulation: the LinEAS map is affine, and the original λ is an interpolation parameter, not a scalar multiple of a mean difference. The 0.75/0.25 thresholds in Eq. (C.1) are also arbitrary. Because experiments (e.g., Table 2 and Fig. 2) use λ = 2 for DSAS while vanilla LinEAS uses λ ≤ 1, the comparison is not obviously on equal footing. Provide a principled derivation, or fold the global strength into the learned gate so that DSAS and the baselines are compared under the same λ range.
minor comments (5)
  1. [Throughout] Several typos and duplicated words: “a a general modulation mechanism” (§3.2), “outweights” (Limitations), “beloning” (§3.2), “airpline” (Appendix O). Algorithm 1's output line also has a formatting issue (“bℓ µℓ” should be “bℓ, µℓ”).
  2. [Reproducibility Statement] The statement says “the code will be made publicly available on Github,” but no repository URL or anonymous access link is provided. For the claims to be fully reproducible, include the link in the final version.
  3. [Fig. 2] The axis labels are extremely dense and the λ values not easily readable. Consider enlarging or annotating selected curves to aid the reader.
  4. [§F.1] The sentence “compared to using only 5 PCA components” appears to be the reverse of what is meant; the large speedup is presumably relative to “no PCA.” Please clarify.
  5. [§G and §J] The adaptive π-scaling method and the “roughly halved strengths” claim in §J would benefit from a short formal statement; currently they are described only textually, making it hard to reproduce the exact procedure from the main text.

Circularity Check

0 steps flagged

No significant circularity: DSAS's gate is trained on source/control activations and evaluated on held-out toxicity and capability benchmarks, so the Pareto improvements are not forced by construction.

full rationale

The derivation chain is self-contained and non-circular. The DSAS gate h_ell is trained by pushing source (S) and control (C) inputs through the unmodified network, averaging token embeddings (Eq. 3.1), projecting with PCA (Eq. 3.2), and fitting a logistic regressor (Eq. 3.3). This classifier output is then used as a per-token multiplier in the interpolation T_ell^DSAS (Eq. 3.4). The reported toxicity and capability metrics are measured on held-out TET, Wikipedia, and MMLU data, not on the S/C training set, so the Pareto-front improvement is an empirical finding rather than an artifact of fitting the evaluation metric. The end-to-end variant is trained with a combination of a source-to-target Wasserstein loss and a control-preservation regularizer L_C (Eq. 3.6); this regularizer encourages control activations to remain unchanged but does not encode ToxTET, PPL, or MMLU, so the resulting trade-off is not presupposed. The paper explicitly acknowledges in Limitations that DSAS depends on the quality of the logistic regressors and falls back to vanilla steering when the classifier is uninformative; this is a robustness property, not circularity. Self-citations to prior steering work (LinEAS, LinAcT) are used as building blocks and baselines, and no load-bearing uniqueness theorem or ansatz is imported from those citations. The central claim therefore has independent empirical content.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

DSAS introduces no new physical or architectural entities; it adds a per-layer logistic gate and a control set. The free parameters are mostly standard hyperparameters plus the trained gate weights. The main non-standard assumptions are linear separability and the use of average-embedding classifiers for token-level decisions.

free parameters (6)
  • Per-layer logistic regression weights (θℓ, bℓ) = not reported (trained)
    The gate is defined by these weights, trained on 32 source and 32 control average embeddings per layer.
  • PCA dimension r = 5
    Chosen by ablation as a 'favorable balance between accuracy and training efficiency' (Section 4.1 / Appendix F).
  • Accuracy threshold τ = 0 for toxicity, 0.75 for banana/diffusion
    Layers with cross-validated accuracy below τ are disabled; this controls which layers contribute to steering.
  • Global steering strength λ = varied to draw Pareto fronts; DSAS often uses λ=2
    DSAS is evaluated as a family over λ; the paper argues λ>1 is justified by a distance-ratio correction Δλ≈2.84 (Appendix C).
  • Control loss weight γ = 1
    Used in all E2E-DSAS experiments; ablation in Appendix L.2 shows sensitivity.
  • Training set sizes (source/target/control) = 32 each in main experiments
    Small dataset is a deliberate design choice, with sensitivity analysis in Appendix D.
axioms (6)
  • domain assumption Toxic vs non-toxic activations are linearly separable (Linear Representation Hypothesis)
    Stated in Limitations: DSAS assumes linear separability; if false, the logistic regressor is uninformative.
  • domain assumption Average token embedding is a reliable global signal for whether an input needs steering
    Section 3.2 averages embeddings because per-token labels are unavailable; this may blur token-level signal but is chosen for stability.
  • domain assumption Source, target, and control distributions are disjoint
    Definition 3 asserts Xsrc ∩ (Xtgt ∪ Xctl) = ∅; in real toxicity data this separation is approximate.
  • ad hoc to paper Logistic-regressor probability is a suitable interpolation weight for steering
    Equation (3.4) defines T_DSAS as a convex combination weighted by hℓ(t); no optimality argument is given for this specific functional form.
  • ad hoc to paper The 1D Wasserstein source loss plus L2 control loss adequately supervises the E2E gate
    Section 3.3: without the control regularizer the learned strength collapses to ~1; the regularizer is a heuristic that is not derived from first principles.
  • ad hoc to paper The global λ from the base optimal-transport steering remains usable if rescaled by a distance ratio
    Appendix C estimates Δλ=||μtox−μnon-tox||/||μsrc−μtgt|| to justify λ>1; this is an empirical rescaling, not a derived transport map for individual embeddings.

pith-pipeline@v1.3.0-alltime-deepseek · 49399 in / 11329 out tokens · 105495 ms · 2026-08-03T18:43:17.849620+00:00 · methodology

0 comments
read the original abstract

Activation steering has emerged as a powerful method for guiding the behavior of generative models towards desired outcomes such as toxicity mitigation. However, most existing methods apply interventions uniformly across all inputs, degrading model performance when steering is unnecessary. We introduce Dynamically Scaled Activation Steering (DSAS), a method-agnostic steering framework that decouples when to steer from how to steer. DSAS adaptively modulates the strength of existing steering transformations across layers and inputs, intervening strongly only when undesired behavior is detected. At generation time, DSAS computes context-dependent scaling factors that selectively adjust the strength of any steering method. We also show how DSAS can be jointly optimized end-to-end together with the steering function. When combined with existing steering methods, DSAS consistently improves the Pareto front with respect to steering alone, achieving a better trade-off between toxicity mitigation and utility preservation. We further demonstrate DSAS's generality by applying it to a text-to-image diffusion model, showing how adaptive steering allows the modulation of specific concepts. Finally, DSAS introduces minimal computational overhead while improving interpretability, pinpointing which tokens require steering and by how much.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Towards Understanding Steering Strength

    cs.LG 2026-02 conditional novelty 7.0

    Steering strength in a simplified next-token model is shown to produce bump-shaped token probability shifts, S-shaped concept probabilities, and an inevitable local cross-entropy increase.

  2. Steering at the Source: Style Modulation Heads for Robust Persona Control

    cs.CL 2026-02 conditional novelty 6.0

    Steering only three 'Style Modulation Heads' controls personas with less coherency collapse than residual-stream steering, across six personas and two base LLMs.

Reference graph

Works this paper leans on

13 extracted references · 8 linked inside Pith · cited by 2 Pith papers

  1. [1]

    GPT-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, ..., and Barret Zoph. GPT-4 technical report.arXiv preprint arXiv:2...

    Pith/arXiv arXiv

  2. [6]

    Pau Rodríguez, Arno Blaas, Michal Klein, Luca Zappella, Nicholas Apostoloff, Marco Cuturi, and Xavier Suau

    URL https://doi.org/10.48550/arXiv.2408.00118. Pau Rodríguez, Arno Blaas, Michal Klein, Luca Zappella, Nicholas Apostoloff, Marco Cuturi, and Xavier Suau. Controllinglanguageanddiffusionmodelsbytransportingactivations. In13th International Conference on Learning Representations, 2025a. Pau Rodríguez, Arno Blaas, Michal Klein, Luca Zappella, Nicholas Apost...

    Pith/arXiv arXiv doi:10.48550/arxiv.2408.00118

  3. [7]

    Wikimedia downloads.https://dumps.wikimedia.org

    Wikimedia Foundation. Wikimedia downloads.https://dumps.wikimedia.org. ZhengxuanWu, AryamanArora, ZhengWang, AtticusGeiger, DanJurafsky, ChristopherDManning, andChristopher Potts. Reft: Representation finetuning for language models.arXiv preprint arXiv:2404.03592,

    Pith/arXiv arXiv

  4. [8]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115,

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, ..., and Zihan Qiu. Qwen2.5 technical report.arXiv preprint arXiv:2412.15115,

    Pith/arXiv arXiv

  5. [10]

    A Effect of DSAS on inference time We evaluate the impact of applying DSAS and its variants on the inference latency of two base models, Qwen 2.5 (1.5B) and Gemma 2 (2B)

    URLhttps://openreview.net/forum?id=IbIB8SBKFV. A Effect of DSAS on inference time We evaluate the impact of applying DSAS and its variants on the inference latency of two base models, Qwen 2.5 (1.5B) and Gemma 2 (2B). Table 3 reports the average execution time (in seconds) required to process 100 tokens under each configuration. Each measurement was avera...

    2024

  6. [12]

    ForLinEAS, we observe an improved Pareto Front when evaluating HellaSwag accuracy

    For Qwen 2.5 (1.5B), we observe that DSAS-enhancedITIandCAAconsistently improve the Pareto Front with respect to their vanilla counterparts, retaining better model capacities for the same level of toxicity mitigation. ForLinEAS, we observe an improved Pareto Front when evaluating HellaSwag accuracy. In Hu- manEval, we observe similar Pareto Fronts, as nei...

    2000

  7. [13]

    **** ”, “ ***

    We observe that the Pareto front of the adaptive method closely matches the one obtained byCAA+DSAS, but with strengths scaled; thus, a larger value ofλis required for the adaptive method to reach a given operating point on theCAA+DSAS curve. Similarly, in figure 11 we observe that, when we replicate the experiment shown in section 4.2, the adaptive metho...

    1959

  8. [512]

    We then evaluate the generalization of the classifiers on a test set of 512 toxic samples

    These samples are extracted from the RTP dataset presented in section 4.1. We then evaluate the generalization of the classifiers on a test set of 512 toxic samples. The experiment is repeated 100 times for each sample size, with random selection of training and test sentences in each repetition. As shown in figure 5, as the sample size increases, the val...

    2019

  9. [2019]

    A comprehensive study of knowledge editing for large language models.CoRR, abs/2401.01286, 2024a

    Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei 14 Huang, Lei Liang, Zhiqiang Zhang, ..., and Huajun Chen. A comprehensive study of knowledge editing for large language models.CoRR, abs/2401.01286, 2024...

    Pith/arXiv arXiv doi:10.48550/arxiv.2401.01286

  10. [2021]

    Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner

    URLhttps://arxiv.org/abs/ 2107.03374. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo...

    Pith/arXiv arXiv 2019

  11. [2023]

    Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681,

    Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681,

    Pith/arXiv arXiv

  12. [2024]

    Gemma 2: Improving open language models at a practical size.CoRR, abs/2408.00118,

    Morgane Rivière, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, ..., and Lilly Mc- Nealus. Gemma 2: Improving open language...

    Pith/arXiv arXiv

  13. [2025]

    Amr Hegazy, Mostafa Elhoushi, and Amr Alanwar

    URLhttps://openreview.net/forum?id=fUCPq5RvmH. Amr Hegazy, Mostafa Elhoushi, and Amr Alanwar. Guiding giants: Lightweight controllers for weighted activation steering in llms.arXiv preprint arXiv:2505.20309,

    arXiv