Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Steering a model's internal concept vector during an audit reveals substantial dependence on protected attributes even when conventional input-output tests show little or no bias.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:37 UTC pith:LU4GPZTL

load-bearing objection The steering-vector audit idea is novel and worth pursuing, but the headline racial-bias result is confounded by dialect and doesn't hold as stated. the 4 major comments →

arxiv 2601.16398 v3 pith:LU4GPZTL submitted 2026-01-23 cs.CY cs.CLcs.LG

White-Box Sensitivity Auditing with Steering Vectors

Hannah Cyberey , Yangfeng Ji , David Evans This is my paper
classification cs.CY cs.CLcs.LG
keywords algorithmic auditingwhite-box evaluationactivation steeringsteering vectorsbias measurementprotected attributessensitivity analysislarge language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that auditing a language model's internal activations—by pushing them along a vector that encodes a protected attribute such as gender or race—can reveal a model's true dependence on that attribute. The authors build a six-step framework that treats invariance to protected concepts as a testable requirement, and they measure sensitivity as the slope of output change as steering strength varies. Across four simulated high-stakes decision tasks, the white-box method consistently reports substantial bias in cases where standard black-box evaluations show little or none—most dramatically, a legal model shows about a 20-percentage-point racial gap in conviction decisions under steering, versus about 2.5 points from a dialect-based input test. If correct, this gives auditors a way to detect latent bias that prompt-based tests miss, and a more consistent measurement across different test constructions.

Core claim

The paper claims that an LLM's dependence on an abstract concept such as gender or race can be measured by adding a scaled steering vector—extracted from contrasting internal activations—to the hidden representation at a chosen layer, and fitting the slope of output change across steering strengths. In four simulated decision tasks, this internal sensitivity score often shows substantial bias where standard input-perturbation tests show little or none; the largest case is a legal judgment model whose conviction predictions shift by about 20 percentage points along the race direction while a dialect-based input swap shows about 2.5 points. The paper argues this reveals actual bias by showing

What carries the argument

The central object is the concept steering vector: a single direction in the model's activation space extracted by comparing internal representations of contrasting concept examples (weighted mean difference), then selected at one layer by separability and projection-correlation criteria. The audit applies it as a hidden-state perturbation h' = h + λv_C across a range of λ and estimates sensitivity as the slope of a linear fit of output change versus λ—a finite-difference version of the directional derivative. This vector is what lets the audit manipulate an abstract concept internally, without needing to enumerate text perturbations, and what makes the sensitivity reading specific to that c

Load-bearing premise

The load-bearing premise is that the single steering vector, extracted at one layer and scaled on a validation split, isolates the target concept in every audit input and is not entangled with task-relevant or proxy information; if the vector carries dialect, financial, or other cues, the measured sensitivity is not a clean protected-attribute signal.

What would settle it

Run the Judicial audit on utterances that decouple dialect from race—White speakers using African American Language and Black speakers using White Mainstream English—and check whether conviction sensitivity tracks dialect or race. If the steering vector shifts predictions according to dialect rather than race, the reported racial bias score is confounded. Alternatively, on a known linear model where the decision direction is orthogonal to the protected-concept direction, a nonzero steering sensitivity would show the extracted vector carries task information.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Internal sensitivity scores can flag a model as dependent on a protected attribute even when input-perturbation tests show near-zero bias, so passing a black-box audit no longer implies the model is fair in this stronger sense.
  • Because the same steering-vector procedure transfers across tasks and models, audits can be standardized around a shared concept-extraction pipeline rather than bespoke prompt sets.
  • The framework extends beyond invariance requirements: steering can also verify that outputs respond predictably to task-relevant concepts, such as GPA in admissions.
  • Bias scores from different steering-vector sources agree more often than scores from different black-box perturbation strategies, reducing the chance that an audit conclusion is an artifact of test construction.
  • Models that show high sensitivity along the race vector are exactly the ones where amplifying implicit cues (e.g., gender-stereotyped names) increases measured bias, indicating the white-box signal tracks real-world risk.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One inference the authors do not draw explicitly: because the race steering vector is built from dialectal sentences, the roughly 20-point conviction effect may partly reflect dialect prejudice rather than race per se; a cleaner audit would decouple the two before reporting it as racial bias.
  • The linear-representation limitation noted in the paper implies the method likely works best for concepts that are one-dimensionally encoded; for multi-dimensional or 'irreducible' features, a single steering vector will understate or misstate sensitivity, so a natural extension is a multi-directional or subspace audit.
  • A practical extension is a pre-registered protocol for layer choice, vector scaling, and validation splits, so that white-box bias scores are comparable across auditors and over time.
  • The framework could transfer to other abstract properties—sycophancy, risk tolerance, or legal competence—wherever a contrast dataset can be built, but each transfer needs its own check that the vector is orthogonal to task-relevant features.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a white-box sensitivity auditing framework for LLMs that uses activation steering vectors to manipulate latent concepts (gender, race) and measures the resulting change in task outputs. The method is applied to four simulated high-stakes decision tasks: judicial conviction/penalty, credit scoring, admissions, and medical diagnosis. The central claim is that steering-based sensitivity scores reveal substantial dependence on protected attributes even when standard black-box perturbation evaluations suggest little or no bias (e.g., Saul-7B on the judicial conviction task, where the white-box racial bias score is near +20 percentage points versus about 2.5 for the dialect-based black-box score). The paper also argues that the white-box method yields more consistent results across different concept-vector sources than the black-box method across different perturbation strategies, and includes Sobol' sensitivity checks for the credit-scoring and admissions tasks.

Significance. If the method is valid, it offers a concrete, practical alternative to input-only audits, addressing a recognized limitation of black-box testing for abstract properties such as bias. The paper ships open code, includes multiple model/task combinations, and attempts validation through alternative steering vectors (Section 4.5), a name-based black-box implicit-perturbation study (Section 5.1), and first-order Sobol' indices (Section 5.2). These are real strengths. However, the headline result rests on the assumption that the extracted steering vector isolates the protected concept and is not entangled with task-relevant features. The paper's own Table 2 shows substantial sensitivity to the vector source for the flagship Saul-7B judicial case, and the empirical sensitivity metric is a finite-range linear slope rather than the infinitesimal derivative defined in Eq. (2). The central claim is therefore defensible but not yet established with the current evidence.

major comments (4)
  1. [Section 3.3, Eq. (2)] The sensitivity metric is defined as the limit λ→0 directional derivative, but the empirical implementation fits a linear regression over λ∈[-1,1] in increments of 0.2 (Section 4.1). These are not guaranteed to coincide for nonlinear model outputs, especially with token-probability or discretized outcomes. The headline 19.66pp figure for Saul-7B is a finite-interval slope, not the derivative of Eq. (2). Please either compute a local derivative (e.g., small λ around zero with appropriate step size) and report both, or justify why the broad-range linear fit approximates the infinitesimal sensitivity. Also clarify whether S_C is averaged over per-input slopes or computed from pooled outputs; the current description is ambiguous.
  2. [Section 4.1 and Table 2] The racial steering vector vdial is extracted from AAL/WME dialect sentences, and the judicial audit inputs are the same type of dialect utterances. The sensitivity measured by vdial may therefore reflect dialect rather than race. Table 2 is direct evidence of instability: for Saul-7B conviction, vdial gives 19.66pp while an alternative Racial Identity vector vid gives 4.62pp, with the two vectors drawn from different layers and a cosine similarity of only 0.58. The paper acknowledges the discrepancy but attributes it to layer/scaling differences; this does not resolve the confound. Please run the alternative-vector comparison with matched layer/scaling choices for the judicial task, and ideally include a Sobol' analysis for the judicial task to test whether vdial alters task-relevant legal content. Without this, the flagship 20pp result is not robust.
  3. [Section 4.3 and Figure 1] The white-box and black-box comparisons use different prompt distributions: black-box evaluations insert explicit gender/race values or names, while white-box evaluations set explicit variables to unknown/neutral/null and then steer internally. The comparison between methods therefore conflates method with input distribution. The claim that 'black-box evaluations suggest little or no bias' may be an artifact of testing different inputs, not of the audit method. Please include a matched black-box baseline where the same neutral prompts are used and gender/race is manipulated only through input-side perturbations (e.g., adding names or other proxy tokens to otherwise identical neutral prompts), or explicitly discuss this limitation and its consequences for the comparison.
  4. [Section 4.1 and Section 4.5] The paper does not report the chosen layer L, the vector-scaling details, or the validation-set selection for each model/task. Given that Table 2 shows the measured bias can change from 19.66pp to 4.62pp depending on the vector source and layer, these implementation choices are load-bearing. Please provide a per-model/per-task table with the selected layer, scaling factor, and validation metrics, and include an ablation showing sensitivity to layer choice and scaling magnitude for at least the judicial task.
minor comments (6)
  1. [Table 2] The cos(θ) column is only populated for the Conviction subtask rows; the Penalty rows are blank. Please report cosine similarities for all rows or state why they are omitted.
  2. [Section 5.1] Typo: 'fin-Lllama3' should be 'fin-Llama3'. Also, the sentence 'While fin-Lllama3 shows opposite bias directions...' is repeated almost verbatim in the following paragraph; consider consolidating.
  3. [Appendix A.3] The heading says 'Admission' while elsewhere the task is called 'Admissions'. Please make the terminology consistent.
  4. [Figure 1] The top row labels 'Gender Bias' and 'Race Bias' span multiple panels, but the mapping between panels and tasks is not clear from the caption alone. Consider adding subplot titles or a clearer layout description.
  5. [Section 3.3] The notation f_L(h'_x) and f_L(h_x) is used, but h_x is defined as the representation at the layer where the steering vector is extracted; it would be clearer to explicitly state that h'_x = h_x + λ v_C before applying f_L.
  6. [Limitations] The paper appropriately acknowledges the linear-representation assumption and the possibility of multi-dimensional features. This is good; however, the same caveat could be stated earlier in Section 3.3 where the metric is introduced, since it is central to the interpretation of S_C.

Circularity Check

0 steps flagged

No circular derivation: the steering sensitivity is an empirical projection and is externally cross-checked, though vector scaling and the Judicial dialect/race overlap are validity limitations.

full rationale

The central chain—extract a concept vector, steer along it, read the output change—does not reduce to the vector's construction. The steering vector v_C is selected for activation separability and projection correlation, not to maximize the sensitivity score S_C; S_C is the model's output gradient projected onto v_C and is an empirical quantity. Evaluation is performed on a validation split not used for extraction, and the paper's external checks (Section 5.1 name-perturbation experiment; Section 5.2 Sobol' indices for Credit and Admissions) provide independent evidence that the measured sensitivity tracks real bias risk. The self-citations to Cyberey et al. (2025) and Cyberey & Evans (2025) supply the vector-extraction and scaling machinery, but the audit claim is not justified solely by those citations. The large drop from vdial to vid on Saul-7B Judicial (19.66 vs 4.62 pp, Table 2) and the acknowledged limitations about validation-set scaling and possible entanglement with task-relevant features are genuine validity concerns, not definitional circularity: changing the vector source changes the measurement, but the measurement itself is still an actual steering response rather than an identity or a fitted value renamed as a prediction.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The ledger is heavy: the method rests on the linear representation hypothesis, on validation-selected steering vectors and layer, on a binary one-dimensional treatment of gender/race, and on a linear fit over λ∈[-1,1] standing in for an infinitesimal derivative. The main free parameters are the steering vector, the layer, the scaling, and the coefficient grid; none have reported values in the text. No novel entities are introduced; steering vectors and sensitivity derivatives are taken from prior work.

free parameters (5)
  • Steering vector v_C = not reported in paper; model-specific direction from WMD
    Central object of the audit; extracted from labeled concept datasets and selected using validation criteria (Cyberey et al. 2025). Bias scores are directional derivatives along this fitted vector.
  • Steering layer L = not reported per model
    Layer from which the vector is extracted and where steering is applied; chosen using validation set. The choice affects the sensitivity scores.
  • Steering coefficient range/increment = λ ∈ [-1,1], step 0.2
    Hyperparameter choice; the reported sensitivity is a slope fitted over this range rather than the infinitesimal limit in Eq. 2.
  • Validation-based vector scaling = not reported
    The steering vector is scaled using the validation set (Cyberey & Evans 2025); the magnitude of all white-box bias scores depends on this scale.
  • Sensitivity threshold epsilon = not specified
    Invariance/dependence tests (Section 3.4) require an epsilon threshold, but the paper never reports a numerical epsilon and comparisons are qualitative.
axioms (5)
  • domain assumption High-level concepts are linearly represented in LLM activation space and can be manipulated by adding a single vector.
    Used throughout Section 3.3; acknowledged in Limitations, which cites multi-dimensional irreducible features (Engels et al. 2025).
  • ad hoc to paper A steering vector v_C extracted from labeled concept data captures the same concept in audit inputs and is not confounded with task-relevant features.
    Necessary for interpreting sensitivity as bias; partially checked with Sobol' indices (§5.2) and alternative vectors (§4.5) but not fully established.
  • domain assumption Protected attributes can be treated as one-dimensional binary directions (feminine-masculine; black-white).
    Used in Section 4.1 for vector extraction; the paper acknowledges this oversimplification in footnote 2.
  • domain assumption Output metrics respond approximately linearly to steering over λ∈[-1,1], so a fitted slope equals the directional derivative.
    Sensitivity metric in Eq. 2 is a limit at λ=0 but is implemented as a linear regression over a large range (Section 3.3).
  • standard math Sobol' first-order indices assume input variables are independent.
    Used in Section 5.2; the paper notes that some variables in the South German credit data are weakly to moderately correlated, which may violate this assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 23417 in / 15108 out tokens · 301708 ms · 2026-08-03T08:37:02.640957+00:00 · methodology

0 comments
read the original abstract

Algorithmic audits are essential tools for examining systems for properties required by regulators or desired by operators. Current audits of large language models (LLMs) primarily rely on black-box evaluations that assess model behavior only through input-output testing. These methods are limited to tests constructed in the input space, often generated by heuristics. In addition, many socially relevant model properties (e.g., gender bias) are abstract and difficult to measure through text-based inputs alone. To address these limitations, we propose a white-box sensitivity auditing framework for LLMs that leverages activation steering to conduct more rigorous assessments through model internals. Our auditing method conducts internal sensitivity tests by manipulating key concepts relevant to the model's intended function for the task. We demonstrate its application to bias audits in four simulated high-stakes LLM decision tasks. Our method consistently indicates substantial dependence on protected attributes in model predictions, even in settings where standard black-box evaluations suggest little or no bias. Our code is openly available at https://github.com/hannahxchen/llm-steering-audit

Figures

Figures reproduced from arXiv: 2601.16398 by David Evans, Hannah Cyberey, Yangfeng Ji.

Figure 1
Figure 1. Figure 1: Gender and race bias measured using the black-box and our proposed white-box methods for each [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Conviction and death penalty outcome rates evaluated on the [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Black-box evaluation results on the Credit Scoring task using name perturbations, without explicit gender. The first row shows each name’s scalar projection on the gender steering vector (vlang). The second row shows the average bad credit rates for names across different female percentages (pf ), colored by the base profile’s credit risk label. Dash lines represent average predictions without gender and n… view at source ↗
Figure 4
Figure 4. Figure 4: Bad credit rates on the Credit Scoring task when steering between male (λ < 0) and female (λ > 0) gender concepts. The dotted lines represent baseline results measured using the black-box method, colored by the gender specified in input prompts. −1 −0.5 0 0.5 1 0 20 40 55 70 −1 −0.5 0 0.5 1 0 20 40 55 70 −1 −0.5 0 0.5 1 0 20 40 55 70 −1 −0.5 0 0.5 1 0 20 40 55 70 whitebox neutral blackbox female male Coeff… view at source ↗
Figure 5
Figure 5. Figure 5: Model accuracies on the Medical task when steering gender and race concepts with λ ∈ [−1, 1]. −1 −0.5 0 0.5 1 0 5 10 −1 −0.5 0 0.5 1 0 5 10 15 −1 −0.5 0 0.5 1 0 5 10 15 whitebox female male blackbox female male Coefficient (λ) Coefficient (λ) Coefficient (λ) Yes (%) Llama3.1-8B Ministral-8B Qwen2.5-7B −1 −0.5 0 0.5 1 0 5 10 −1 −0.5 0 0.5 1 0 5 10 15 −1 −0.5 0 0.5 1 0 5 10 15 whitebox black white blackbox b… view at source ↗
Figure 6
Figure 6. Figure 6: Average acceptance rates on the Admissions task when steering gender and race with λ ∈ [−1, 1]. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. White Box Evidence Packages for Policy Audit Reports

    cs.CY 2026-07 conditional novelty 6.0

    In a 60-case controlled audit study, adding white-box model evidence to an LLM auditor increased citation volume but weakened passage grounding and raised evidence misuse, while a shuffled control showed reports can s...

Reference graph

Works this paper leans on

2 extracted references · 1 linked inside Pith · cited by 1 Pith paper

  1. [2024]

    Andrea Saltelli, Stefano Tarantola, and Francesca Campolongo

    doi: 10.18653/v1/2024.naacl-long.301. Andrea Saltelli, Stefano Tarantola, and Francesca Campolongo. Sensitivity analysis as an ingredient of modeling.Statistical science, 2000. Andrea Saltelli, Paola Annoni, Ivano Azzini, Francesca Campolongo, Marco Ratto, and Stefano Tarantola. Variance based sensitivity analysis of model output. design and estimator for...

    Pith/arXiv arXiv 2024

  2. [2025]

    Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King

    doi: 10.18653/v1/2025.findings-emnlp.783. Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. AI generates covertly racist decisions about people based on their dialect.Nature, 633(8028), 2024. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. Adversarial example generation with syn- tactically controlled paraphrase netwo...

    arXiv 2025