REVIEW 3 major objections 4 minor 1 cited by
Concept Heterogeneity-aware Representation Steering
T0 review · 3 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read The paper claims that steering LLM behavior with an input-dependent, cluster-aware transport map consistently outperforms global difference-in-means steering across text and image generation tasks.
desk verdict CHaRS is a sensible input-dependent steering method with a clean OT story, but the headline empirical claim rests on test-set-tuned hyperparameters and an ~83-prompt evaluation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The CHaRS map x ↦ x + α v̂(x), where v̂(x) = Σ_{i,j} P*_{ij} k(x,a_i) v_{ij} / Σ_{p,q} P*_{pq} k(x,a_p), with k an RBF kernel over k-means centroids, P* an entropy-regularized OT coupling between centroids, and v_{ij} = b_j − a_i the cluster-level shift. This replaces the Gaussian-mixture posterior of the theoretical barycentric map with a kernel-weighted average of local translations, making the steering direction a smooth field over the representation manifold.
What would settle it
Take held-out source activations and compute both the RBF gating weights and the true GMM posterior or a hard k-means assignment; if a steering map built on the exact posterior or on nearest-centroid hard assignment does not match or exceed CHaRS's attack success while preserving utility, then the kernel-weighted combination is not what drives the reported gains. A second check: if CHaRS with K=1, which reduces to difference-in-means, beats CHaRS on a broad evaluation suite, the heterogeneity claim collapses.
Extended reading notes
Core claim
Representation steering can be generalized from a single global translation to an input-dependent map by solving a discrete optimal transport problem between clusters of activations. The paper derives a barycentric projection of the GMM-to-GMM transport plan that mixes cluster-level shifts according to a kernel-based gating of how strongly a token belongs to each source cluster, and shows across jailbreaking, toxicity mitigation, and image style control that this CHaRS map consistently beats difference-in-means and directional ablation baselines in attack success while roughly preserving utility.
Load-bearing premise
The load-bearing premise is that the kernel-based gating weights (chosen by median-distance bandwidth) approximate the true cluster-membership probabilities that the optimal-transport derivation requires; if they do not, the method is just interpolating local shifts without the transport grounding.
Editorial extensions
If this is right
- CHaRS yields higher attack success rates than Activation Addition and Directional Ablation across Gemma, Llama, and Qwen models, with gains up to about 7%, without a consistent penalty on tinyBenchmarks utility scores.
- Sequential CHaRS and CHaRS-PCT reduce toxic generations more than the affine steering baseline, keeping perplexity and MMLU near unsteered levels.
- The same method transfers to text-to-image diffusion models, inducing target styles with a better style-content Pareto trade-off than the affine baseline.
- Principal Component Thresholding captures essentially all steering-field variance with at most 2(K−1) components, and empirically PCT often matches or beats full CHaRS using fewer directions.
- Because the kernel-weighted field is continuous, CHaRS gives a differentiable, input-dependent steering vector usable inside both activation addition and directional ablation frameworks.
Reading between the lines
- The smooth-field formulation invites compositional steering: since each concept produces its own v̂ field, combinations of concepts (e.g., style plus safety) could be applied additively with per-field strengths, an extension the paper does not test.
- The 2(K−1) low-rank structure suggests the effective steering manifold is low-dimensional; a testable extension is whether CHaRS-PCT with a fixed small L generalizes better than the full field on out-of-distribution prompts, since thresholding acts as implicit regularization.
- The paper's own appendix shows diagonal covariance does not improve over equal-covariance translations; this points to clustering and OT matching, not covariance alignment, as the likely source of gains, suggesting future work should investigate better clusterings or feature weighting rather than richer Gaussian models.
- If cluster structure is stable across prompts, CHaRS-style maps could be precomputed and cached per layer, making the per-token overhead negligible in production, as the paper's throughput analysis already hints.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CHaRS, an input-dependent representation-steering method for LLMs. The authors observe that standard difference-in-means (DiM) steering is the OT map between two equal-covariance Gaussians, i.e., a global translation. To handle concept heterogeneity, they model source and target activation distributions as GMMs, solve a discrete entropy-regularized OT problem between k-means cluster centroids, and define a per-token steering map as a kernel-weighted combination of cluster-pair translation vectors. A PCA-thresholded variant, CHaRS-PCT, exploits the low rank of the weighted steering-vector covariance. The method is evaluated on jailbreaking (ADVBENCH), toxicity mitigation (RealToxicityPrompts), and image style control (FLUX.1), and is compared against ActAdd, DirAbl, and Linear-AcT. The paper claims consistent ASR improvements and maintained general utility, with code released.
Significance. The central idea—replacing a single global steering direction with a smooth, input-dependent field derived from a GMM/OT model—is a plausible and useful extension of existing activation-steering methods. The mathematical derivation in Sections 2–3 is mostly standard and clearly written; the low-rank observation for CHaRS-PCT (Section 3.3) is elegant and empirically verified in Figure 4. The authors also include honest limitations and an ablation (Appendix E.6) that tests a diagonal-covariance relaxation. However, the empirical validation as presented does not yet support the paper's headline claim of consistent improvement: hyperparameters are selected on the same small evaluation set, and ASR differences are not accompanied by uncertainty quantification. If these evaluation issues are fixed, the contribution is solid and likely of interest to the representation-steering community.
major comments (3)
- [§4.1, §5, Appendices E.4/E.5, Tables 1–2] The reported ASR gains are obtained by selecting K and L on the test set itself. Section 4.1 states that 80% of ADVBENCH (416 prompts) is used for training and the remaining 20% (~83 prompts) for evaluation. Appendix E.4 then reports that, for Qwen2.5-3B, K=15 is chosen because it gives the highest ASR on this same evaluation set, and Appendix E.5 fixes K to the eval-best value and sweeps L on the same set. This constitutes test-set sweeps for the central free parameters. Since the reported improvements are small (e.g., 5–7 ASR points on ~83 prompts, i.e., roughly 4–6 flips), the claim that CHaRS 'consistently outperforms' baselines is not established. The paper should use a separate validation split (or nested cross-validation) for hyperparameter selection and report final results on a truly held-out set.
- [§3.2, Eqs. (11)–(12)] The theoretical barycentric map in Eq. (9) uses the GMM posterior p(k|x) and full Gaussian-to-Gaussian OT maps. The implementation replaces p(k|x) with an RBF kernel gating over k-means centroids with median-distance bandwidth, and replaces the full maps with pure translations under an equal-covariance assumption. The paper does not validate that the RBF gating approximates the GMM posterior on new inputs, nor that the resulting map is close to the derived transport map. This is load-bearing because the method is claimed to be 'principled' rather than an ad hoc interpolation. I request an empirical comparison of the RBF gating weights against the GMM posterior (or a sensitivity analysis of the bandwidth choice, e.g., a range of σ values), and a discussion of when the approximation is expected to hold.
- [§4.1, Tables 1–2] No confidence intervals or significance tests are reported for any ASR value. With ~83 evaluation prompts, the difference between 91.35% and 98.08% (Gemma2-9B, ActAdd) is about 6 prompts, and several comparisons are far closer (e.g., Qwen2.5-14B ActAdd: 94.23 vs. 95.19). Without error bars or a paired test, the consistency of the improvement cannot be assessed. Please provide confidence intervals (e.g., bootstrap) or a nonparametric significance test, especially for the near-tie cases, and report the absolute number of prompt-level flips alongside percentages.
minor comments (4)
- [§1 (Organization)] The text says 'Appendix 6 discusses related work'; this should be 'Section 6'.
- [§3.2] 'Plugging this back in Eqn. (30)' appears to be a typo: Eqn. (30) is defined only in Appendix E.6 and corresponds to a different, diagonal-covariance formulation. The intended reference is presumably Eqn. (12).
- [Figure 2 caption] Inconsistent capitalization: 'ChaRS' should be 'CHaRS'.
- [§4.3 / Figure 2] The caption describes the blue line as 'similarity to the original prompt without style modification' but does not specify which reference image/text is used for CLIPScore. Please clarify the precise computation.
Circularity Check
No material circularity: CHaRS is an explicitly approximated extension of external GMM-OT results; the main risks are approximation fidelity and eval-set hyperparameter selection, not definitional circularity.
full rationale
The paper's derivation chain is not circular. It inherits the DiM-as-OT interpretation from external work (Singh et al. 2024; Rodriguez et al. 2025) and the mixture-Wasserstein/barycentric machinery from Delon & Desolneux (2020) and Deb et al. (2021); no load-bearing step is justified only by the authors' own prior work. The final CHaRS map (Eqs. 12-13) is obtained by an explicit, admitted approximation: Section 3.2 says 'we approximate the theoretical barycentric map in Eqn. (9)' and 'we adopt the equal covariances assumption', replacing the GMM posterior with RBF gating over k-means centroids. Approximations can be unvalidated—the paper does not show the RBF gating tracks true cluster membership on new inputs—but they are not equivalent to the inputs by construction, so they are fidelity/robustness concerns, not circularity. The rank bound of Section 3.3 is proven algebraically (v_ij − vbar = (b_j − bbar) − (a_i − abar), so rank(Σ_total) ≤ 2K−2), and the '100% variance at 2(K−1) PCs' observation is a direct consequence, not a fitted prediction. The self-citations (Teo et al. 2025; Vu & Nguyen 2025) appear only as background/related-work and do not carry the derivation. The most serious empirical issue—Appendix E.4/E.5 selects K and L by ASR on the same ~83-prompt evaluation split reported in Tables 1-2, with no confidence intervals—is test-set overfitting/statistical reporting risk, not circularity under the definitions used here, because the reported ASR is not forced to equal a training objective by construction.
Assumptions & free parameters
free parameters (6)
- Number of clusters K =
Varies by model; e.g., 15 for Qwen2.5-3B ActAdd, 10 for Gemma2-9B PCT, 5 for Qwen2.5-7B coupling ablation
- Entropic regularization λ (Sinkhorn) =
Not stated in main text
- RBF bandwidth σ =
Median of squared distances from x to centroids (data-dependent heuristic)
- Steering strength α =
Per task/model; not systematically reported for jailbreak; swept in image experiments (peak at λ=0.8)
- Number of principal components L (CHaRS-PCT) =
Swept 1..15; fixed to best per model/intervention
- Intervention layer ℓ =
Not specified in main text
assumptions (6)
- domain assumption Hidden activations of a concept are well approximated by a K-component Gaussian mixture; k-means centroids capture semantic subregions
- ad hoc to paper Matched clusters have equal covariance; component-wise OT maps are pure translations
- ad hoc to paper The RBF gating over k-means centroids approximates the GMM posterior p(k|x)
- domain assumption Barycentric projection of the restricted coupling is a good proxy for the true OT map (conditional variance small)
- standard math Sinkhorn with entropy regularization gives a good proxy for the exact discrete OT plan
- domain assumption Semantic clusters learned on training prompts remain valid for novel test prompts (no distribution shift)
Cite this review
Pith. "Pith review of Concept Heterogeneity-aware Representation Steering." pith.science (2026). https://pith.science/paper/QX33ZTSY
@misc{pith2026260302237,
author = {Pith},
title = {Pith review of: Concept Heterogeneity-aware Representation Steering},
year = {2026},
howpublished = {\url{https://pith.science/paper/QX33ZTSY}},
note = {Machine review of arXiv:2603.02237}
}
read the original abstract
Representation steering offers a lightweight mechanism for controlling the behavior of large language models (LLMs) by intervening on internal activations at inference time. Most existing methods rely on a single global steering direction, typically obtained via difference-in-means over contrastive datasets. This approach implicitly assumes that the target concept is homogeneously represented across the embedding space. In practice, however, LLM representations can be highly non-homogeneous, exhibiting clustered, context-dependent structure, which renders global steering directions brittle. In this work, we view representation steering through the lens of optimal transport (OT), noting that standard difference-in-means steering implicitly corresponds to the OT map between two identical distributions with differing first moments, yielding a global translation. To relax this restrictive assumption, we theoretically model source and target representations as Gaussian mixture models and formulate steering as a discrete OT problem between semantic latent clusters. From the resulting transport plan, we derive an explicit, input-dependent steering map via barycentric projection, producing a smooth, kernel-weighted combination of cluster-level shifts. We term this method Concept Heterogeneity-aware Representation Steering (CHaRS). Through numerous experimental settings, we show that CHaRS yields more effective behavioral control than global steering.
Figures
Figures from the paper (8 more)
Forward citations
Cited by 1 Pith paper
-
Decomposing how prompting steers behavior
A geometric decomposition framework shows that affine transformations best recover prompt-induced task geometry and behavior in language and vision models across multiple datasets.
Reference graph
Works this paper leans on
-
[1]
The squared 2-Wasserstein distance is given by W 2 2 (µ, ν) =∥m1 −m 2∥2 2 +d 2 B(Σ1,Σ 2),(17) where the Bures distance between covariance matrices is d2 B(Σ1,Σ 2) = tr(Σ1) + tr(Σ2)−2tr (Σ1/2 1 Σ2Σ1/2 1 )1/2 .(18)
-
[2]
The OT map is affine: T(x) =m 2 +A(x−m 1),(19) where A=Σ −1/2 1 Σ1/2 1 Σ2Σ1/2 1 1/2 Σ1/2 1 .(20) 14 Concept Heterogeneity-aware Representation Steering
-
[3]
Proof.We prove each part separately
The optimal couplingπ ∗ is Gaussian with mean(m 1,m 2)and covariance Σπ∗ = Σ1 Σ12 ΣT 12 Σ2 ,(21) whereΣ 12 =Σ 1/2 1 (Σ1/2 1 Σ2Σ1/2 1 )1/2Σ−1/2 1 . Proof.We prove each part separately. Part 1: Wasserstein distance formula.The key insight is that the optimal coupling between Gaussians is itself Gaussian. Letπbe any coupling with marginalsµandν. Since both m...
-
[4]
URL https://openreview.net/forum? id=aLLuYpn83y. Llama Team, A. . M. The llama 3 herd of models, 2024. URLhttps://arxiv.org/abs/2407.21783. Marks, S. and Tegmark, M. The geometry of truth: Emer- gent linear structure in large language model representa- tions of true/false datasets. InFirst Conference on Lan- guage Modeling, 2024. URL https://openreview. n...
arXiv 2024
-
[5]
Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A
URL https://openreview.net/forum? id=KiespDPaRH. Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. Steering llama 2 via contrastive activa- tion addition. In Ku, L.-W., Martins, A., and Srikumar, V . (eds.),Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15504–...
-
[6]
URL https://openreview.net/forum? id=TiTk6VDz2H. Turner, A. M., Thiergart, L., Leech, G., Udell, D., Vazquez, J. J., Mini, U., and MacDiarmid, M. Steering language models with activation engineering, 2024. URL https: //arxiv.org/abs/2308.10248. Villani, C. et al.Optimal transport: old and new, volume
arXiv 2024
-
[11]
Mean preservation:E[T(x)] =m 2, which gives the translation bym 2 −Am 1
-
[12]
A man standing in front of a few horses on the street
Covariance matching:AΣ 1AT =Σ 2. The matrix A must be the unique positive definite solution to AΣ1AT =Σ 2. To find it, we use the change of variables: let B=Σ 1/2 1 AΣ1/2 1 . Then Σ2 =AΣ 1AT =Σ −1/2 1 BΣ1/2 1 Σ1Σ1/2 1 BT Σ−1/2 1 =Σ −1/2 1 B2Σ−1/2 1 , which implies B2 =Σ 1/2 1 Σ2Σ1/2 1 . The unique positive definite solution isB= (Σ 1/2 1 Σ2Σ1/2 1 )1/2, yi...
2020
Show all 12 references
-
[338]
Concept Heterogeneity-aware Representation Steering
Springer, 2009. Vu, H. M. and Nguyen, T. M. Angular steering: Behavior control via rotation in activation space.Advances in Neural Information Processing Systems, 2025. Wang, T., Jiao, X., Zhu, Y ., Chen, Z., He, Y ., Chu, X., Gao, J., Wang, Y ., and Ma, L. Adaptive activation...
2009 arXiv
-
[2023]
Bolukbasi, T., Chang, K.-W., Zou, J
URL https://openreview.net/forum? id=awIpKpwTwF. Bolukbasi, T., Chang, K.-W., Zou, J. Y ., Saligrama, V ., and Kalai, A. T. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. InNIPS, pp. 4349–4357, 2016. Bures, D. An extension of kakutani’s theo...
2016 arXiv
-
[2024]
io/2024/papers.html
URL https://hcrl-workshop.github. io/2024/papers.html. Best Paper Award. Kantorovich, L. V . On the translocation of masses. InDokl. Akad. Nauk. USSR (NS), volume 37, pp. 199–201, 1942. Kusner, M., Sun, Y ., Kolkin, N., and Weinberger, K. From word embeddings to document dista...
2024 arXiv
-
[2025]
Jorgensen, O
URL https://openreview.net/forum? id=49Rc51iCso. Jorgensen, O. K., Cope, D., Schoots, N., and Shanahan, M. Improving activation steering in language models with mean-centring. InProceedings of the Human-Centric Representation Learning Workshop at the 38th AAAI Conference on Ar...
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.