Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence

Katarzyna Filus; Sebastian Pokuci\'nski

REVIEW 4 major objections 6 minor 19 references

Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence

T0 review · 4 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims monosemanticity of sparse autoencoder neurons can be measured as the average overlap of the binary activation patterns they trigger, with no concept labels and no external encoder.

desk verdict TMS is a clean, cheap, label-free scoring idea, but the central claim overreaches: no null model means the K-differences and anisotropy conclusions may just be the sparsity floor. read the letter →

arxiv 2607.17770 v1 pith:AXIQKP2Z submitted 2026-07-20 cs.CV cs.AIcs.LG

Katarzyna Filus , Sebastian Pokuciński This is my paper

classification cs.CVcs.AIcs.LG

keywords sparseautoencodersmonosemanticitymechanisticinterpretabilityTverskysimilarityJaccardindexactivationcoherenceconceptdeletionvision-languagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that you can judge how single-minded a sparse autoencoder feature is without concept labels or pretrained embedding encoders, using only the SAE's own activation patterns. It introduces TMS, the average Jaccard similarity between the binarized activation sets of stimuli that fire a given neuron, and argues this captures activation-set coherence — the intuition that a monosemantic neuron fires on stimuli sharing more common than distinctive features. The authors show TMS is less inflated by the anisotropic geometry of embedding spaces than the existing embedding-based Monosemanticity Score, stays correlated with embedding-based and variance-based indicators, reveals distinct training dynamics, and predicts probe-based concept deletion effectiveness at least as well. If right, SAE interpretability comparisons no longer depend on the geometry of an external encoder.

What carries the argument

The Tversky Monosemanticity Score (TMS) is the central instrument: for each latent neuron, it builds the set of test inputs whose binarized activation exceeds the neuron's mean training activation, then averages the Tversky index with α=β=1 (i.e., the Jaccard index) between pairs of those active inputs. This binary activation rule plus set-based overlap replaces continuous embedding geometry with discrete feature co-occurrence; that replacement is what the paper claims makes the score insensitive to anisotropy and free of external encoders.

What would settle it

A decisive test: build SAE latents on synthetic images where concepts are mixed under controlled co-occurrence (so two unrelated concepts always appear together), then check whether TMS ranks a neuron firing on that mixture higher than a neuron firing on a single concept with varied context; or train an SAE with randomly permuted latent codes and show TMS does not separate real features from random ones.

Watch

Extended reading notes

Core claim

The paper's central claim is that monosemanticity can be operationalized directly on binarized SAE latents: for each neuron, take the test images whose activation exceeds the neuron's mean training activation, average the pairwise Tversky/Jaccard similarity of those binary activation vectors, and call the result monosemanticity. The paper argues this set-based, non-geometric measure is less sensitive to embedding anisotropy than the embedding-based Monosemanticity Score, stays aligned with established indicators (per-neuron correlations above 0.75 with MS in low-anisotropy settings, above 0.9 in magnitude with within-active-set variance), tracks distinct training dynamics, and predicts probe

Load-bearing premise

The load-bearing premise is that monosemanticity is faithfully captured by the average Jaccard overlap of binary activation sets, with 'active' defined as activation above the neuron's mean training activation — if that binarization or overlap does not track genuine single-concept behavior, the score measures an artifact of thresholding and sparsity.

Editorial extensions

If this is right

Monosemanticity comparisons across SAEs trained on different base models no longer depend on which reference encoder is chosen, removing a known source of geometry-induced variability.
The metric can be computed cheaply from sparse binary vectors, sidestepping the cost of embedding the full test set (the paper notes up to ~30 minutes for one encoder on the BLIP2 test set).
TMS trajectories over epochs make visible that monosemanticity is dynamic: TopK SAEs with small K climb to higher scores, BatchTopK configurations converge closer together — offering a monitoring signal during training.
Model-level TMS correlates with target-drop in probe-based concept deletion across SAE configurations, giving a label-free predictor of whether a given SAE supports targeted feature interventions.
The score's sampling-based estimate is stable (standard deviation ~10^-5 at M=1000 pairs), supporting its use on the large latent spaces SAEs typically produce.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because TMS relies on the mean-threshold binarization rule, we infer the score's absolute values may shift under different binarization schemes; a separate sensitivity study of threshold choice would clarify whether ranking, not just magnitude, is stable.
TMS operationalizes coherence as co-occurrence, so we infer it may reward a neuron that fires on one broad context (e.g., all outdoor scenes) even if that context contains multiple concepts; comparing TMS against concept-probe ground truth on synthetic concept mixtures would test this.
The definition is modality-agnostic, so we infer TMS can be carried over to language-model SAEs by treating token-level activation patterns analogously, though the paper evaluates only vision and vision-language settings.
The Tversky framing invites asymmetric variants (α≠β); we infer these could measure directional polysemanticity, e.g., whether a feature's distinctiveness is one-sided, extending the parameter-free baseline presented here.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: TMS is a clean, cheap idea — average pairwise Jaccard of binarized SAE latents within each neuron's active set — and the evaluation is broad. But the paper never addresses the fact that every pair in a neuron's active set shares that neuron, so the Jaccard has a floor around 1/(2r−1). Under TopK with K=10 that floor is roughly 0.05, which is the size of the TMS values they report; for K=64 it drops to ~0.01. Their headline observation that smaller K gives higher TMS is exactly what the floor predicts. Without a null model (e.g., permuted or random binary features with the same marginal rates, conditioned on shared k), the absolute values and K-comparisons are not evidence of monosemanticity.

What is genuinely new: applying a set-similarity score to binarized SAE activations, removing the need for external encoders and labels, and the multi-model evaluation across TopK/BatchTopK, K, and expansion factors. The training-dynamics tracking is a nice addition, and the concept-deletion experiment is a sensible functional check. The writing is clear, the math is correctly stated, and the Tversky framing (α=β=1 is just Jaccard) is acknowledged.

The soft spots are real and proportionate to the claim. First, the sparsity floor is not a minor caveat — it undermines the central interpretation. The correlation with MS (Table 1) is not a null check because MS is the anisotropic measure they argue against; per-neuron correlation with a biased reference does not establish validity. The within-set variance correlation (Table 2) is a better sanity check but still uses the same base representations and is not a random-feature baseline. Second, the mean-based binarization threshold is arbitrary; no sensitivity analysis is provided, and the stability analysis in Appendix C only shows sampling noise, not threshold robustness. Third, the code is only a private Zenodo link, not public.

The paper is salvageable. Add a null-model baseline (random latent assignments or shuffled activations, with the same K and threshold rule), show TMS minus the floor or a normalized version, and test threshold sensitivity. If the K-differences and the CLIP-vs-DINO ordering survive, the claim would be credible.

I would send this to peer review — the idea is worth the community's time — but I would ask for major revision on the null model before acceptance.

Referee Report

4 major / 6 minor

Summary. The paper introduces the Tversky Monosemanticity Score (TMS), a label-free metric for evaluating monosemanticity of sparse autoencoder (SAE) latents. TMS operationalizes monosemanticity as the average pairwise Jaccard (Tversky with α=β=1) similarity between binarized activation sets of the stimuli that activate a given latent neuron, using a per-neuron mean-activation threshold. The authors evaluate TMS on SAEs trained on features from DINOv3, CLIP, and BLIP2, across TopK and BatchTopK regimes, three sparsity levels (K=10,32,64), and expansion factors (2,4,8). They report that TMS is less affected by encoder anisotropy than the embedding-based Monosemanticity Score (MS), remains correlated with MS (especially for DINOv3), reveals distinct training dynamics, and correlates with probe-based concept deletion effectiveness, particularly for anisotropic encoders like CLIP.

Significance. If validated, TMS would be a useful, inexpensive, and conceptually attractive metric for SAE interpretability, free of external concept labels and embedding encoders. The paper's strengths include a broad empirical sweep (3 base models × 2 SAE regimes × 3 K × 3 expansion factors), a functional intervention test (concept deletion), and a clear presentation of the metric's motivation. The concept deletion experiment provides a form of external validation that goes beyond correlating with an existing metric. However, the central claim that TMS measures monosemanticity rather than an artifact of binarization and sparsity is not yet supported, because the paper lacks a null baseline that separates the metric's inherent sparsity floor from genuine activation-set coherence. The correlation with MS also does not serve as a null check, since MS is the very embedding-based measure whose bias the paper criticizes. The significance is therefore conditional on additional validation with random-feature or permutation baselines and a clearer controlled comparison of anisotropy effects.

major comments (4)

[Section 3, Eq. (1)] TMS has an unsubtracted sparsity floor that confounds the reported results. For any pair ni,nj ∈ A_k, the latent k is by construction in both B_ni and B_nj, so the Jaccard index is at least 1/(2K−1) when each stimulus activates exactly K latents (as in TopK). This floor is ~0.053 for K=10, ~0.016 for K=32, and ~0.008 for K=64. The observation in Section 5 and Fig. 6 that smaller K yields higher TMS is exactly what this floor predicts, even if all other active latents are disjoint. The stability analysis in Appendix C only quantifies sampling noise around a fixed mean; it does not test whether that mean captures coherence above chance. The paper must provide a null model—e.g., TMS computed on permuted binary activation sets or on random binary vectors with matched size/sparsity—to demonstrate that observed TMS values and the K-differences exceed the floor by an amount attributable to sema
[Table 1] The per-neuron correlation between TMS and MS is presented as validation, but it is not a null check. MS is the embedding-based metric whose anisotropy bias the paper argues corrupts estimates, so agreement with MS does not establish that TMS is free of sparsity artifacts. A meaningful validation would compare the observed TMS–MS correlation to the correlation obtained when TMS is computed on a randomized null model (e.g., permuting the binarized patterns within each active set). If the null-model TMS also correlates with MS due to shared sparsity/K-dependence, the reported alignment in Table 1 would not support the claim that TMS captures the same monosemanticity signal.
[Section 5, Fig. 1] The claim that TMS is 'less affected by encoder anisotropy' is not directly demonstrated. TMS is defined independently of any encoder, so it is invariant to encoder choice by construction; the meaningful empirical question is whether TMS's agreement with functional outcomes (e.g., concept deletion) is robust while MS is distorted. However, Fig. 1b appears to vary both the base model (whose features train the SAE) and the encoder simultaneously, conflating base-model differences with anisotropy. The authors should clarify the exact protocol: for a fixed SAE, how does MS change when the encoder is varied among DINOv3, CLIP, and BLIP2? A controlled experiment holding the SAE fixed and varying only the encoder would directly support the 'less affected' claim and should be added.
[Section 3, binarization threshold] The binarization threshold is set to each neuron's mean training activation, a choice that is described as 'parameter-free' but is nonetheless an arbitrary modeling decision. TMS measures overlap of binary patterns, and the results could be highly sensitive to this threshold. No sensitivity analysis is provided, and the null model suggested above should also vary the threshold (e.g., percentiles of training activations) to demonstrate that conclusions are not an artifact of the specific threshold rule.

minor comments (6)

[Throughout] The metric name is inconsistent: the abstract and conclusions use 'Tversky Monosemanticity Score' while Section 3 and some figure captions use 'Tversky Monosemantic Score'. Please standardize.
[Appendix A, Table A.1] The text states 'all TopK models exceed 0.8' in input-output cosine similarity, but Table A.1 shows several TopK CLIP/BLIP configurations with CS below 0.8 (e.g., CLIP ×2 K=10: 0.76; BLIP ×8 K=10: 0.66). This is a factual contradiction that should be corrected.
[Section 3] The notation "s_k = E_{n_i,n_j ∈ A_k}[TI(B_{n_i}, B_{n_j})]" uses an expectation but TMS is estimated by sampling up to M pairs; please clarify the estimator and the role of M as an upper bound.
[Appendix A] The definition of L0 as 'fraction of zero activations' is nonstandard; in SAE literature L0 typically denotes the number (or fraction) of non-zero active features. Please define clearly and consistently.
[Fig. 4 caption] The caption says bold is used for cases where TMS and MS orderings differ, but the text does not make it easy to identify which values are bold. Consider explicitly marking the differing neurons in the figure or a table.
[Appendix D] Table D.1 reports active neurons after binarization, but the text in Section 5 mentions that TMS is 'always computed over at least several hundred neurons.' The table shows some BLIP2 TopK configurations with fewer than ~600 active neurons (e.g., ×2 K=10: 670), so the statement should be reconciled with the data or rephrased.

Circularity Check

1 steps flagged · score 2.0 of 10

TMS is openly defined as activation-set coherence, so the coherence interpretation is true by construction; the paper's central empirical claims are validated externally and no fitted parameter is renamed as a prediction.

self definitional [Section 3, Eq. (1)]
"A latent neuron is considered more monosemantic if, on average, stimuli in its active set share more common than distinctive features, which is reflected in high Tversky’s within-set similarity: sk = Eni,nj∈Ak TIα,β(Bni, Bnj)"

TMS is defined as the average Tversky/Jaccard similarity of binarized activation sets, and monosemanticity is then 'considered' to be exactly that quantity. Thus the statement that TMS measures activation-set coherence follows from the definition rather than from an independent derivation. The paper is transparent about this operationalization, and its load-bearing empirical claims are validated against an external embedding-based MS, within-active-set variance, and concept-deletion effectiveness, so the circularity is limited to this explicit definitional core and does not extend to fitted parameters or hidden predictions.

full rationale

The main metric is defined, not derived: Eq. (1) sets TMS to the mean within-set Jaccard/Tversky similarity of mean-thresholded binary activation vectors, and the text equates high TMS with monosemanticity. This is a self-definitional tautology, but it is openly stated as an operationalization rather than disguised as a first-principles result. The paper's substantive validation is external: per-neuron Spearman correlations with the embedding-based MS, strongly negative correlations with within-active-set embedding variance (computed on base-model features, even though the active set uses TMS's binarization), and correlations with probe-based concept-deletion effectiveness. No TMS parameters are fitted to those targets, and no load-bearing self-citation appears: the authors' own ref. [6] is only a code link, and the binarization-precedent refs [7,1] are external. The skeptical note's sparsity-floor concern (every pair in A_k shares neuron k, giving a K-dependent positive baseline) is a validity and robustness issue requiring a null/random-feature model; it does not show that any equation in the paper reduces its output to an input by construction. Overall circularity is therefore minimal and confined to the explicit definitional step.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The metric itself is simple and self-contained; the heavy lifting is in the validation assumptions: that binary activation overlap is a valid proxy for concept coherence, that mean-threshold binarization is safe, and that MS and probe target-drop are meaningful external references. No fitted constants are used to compute TMS.

free parameters (3)

Binarization threshold rule = neuron-wise mean training activation μ_k
Active sets Ak are defined by thresholding test activations at μ_k; TMS is computed on these sets, so the threshold is a hand-chosen, data-derived parameter that strongly shapes the metric. No sensitivity analysis beyond the M-budget is reported.
Tversky weights α, β = α=β=1
Setting both to 1 reduces the Tversky index to the Jaccard index; the paper calls this 'parameter-free default' but it is a chosen configuration, and future asymmetric variants are left open.
Pair sampling budget M = 1000
Within-set similarity is estimated from up to M=1000 uniformly sampled pairs; stability is shown across M, so this is a low-risk but hand-chosen computational parameter.

assumptions (4)

ad hoc to paper Monosemanticity is adequately operationalized as activation-set coherence (average pairwise Jaccard of binarized SAE latents within a neuron's active set).
Section 3 defines the score this way; the paper's central claim depends on this equation being a valid proxy for the target concept.
domain assumption Binarization of SAE latents at the per-neuron mean training activation preserves the information needed to evaluate monosemanticity.
Section 3: 'following recent work showing that binarization of SAE features can improve interpretability'; the paper does not test whether alternative thresholds change conclusions.
domain assumption The Monosemanticity Score (MS) of Pach et al. is a valid reference indicator of monosemanticity.
Section 4 uses MS as the reference; the paper's anisotropy critique assumes MS is biased while still valid enough for correlation.
domain assumption Linear probe target-drop on Mini-ImageNet classes measures functional relevance of SAE features.
Section 5 concept deletion experiment; probe ranking by weights is assumed to identify concept-specific features.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence." pith.science (2026). https://pith.science/paper/AXIQKP2Z

@misc{pith2026260717770,
  author       = {Pith},
  title        = {Pith review of: Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AXIQKP2Z}},
  note         = {Machine review of arXiv:2607.17770}
}

read the original abstract

Within Explainable Artificial Intelligence, mechanistic interpretability uses Sparse Autoencoders (SAEs) to extract more interpretable features from neural representations. However, assessing their monosemanticity, and thus explanation quality, remains challenging. Existing metrics require external concept labels or depend on pretrained embedding models, making them sensitive to encoder's geometry. We introduce the Tversky Monosemanticity Score (TMS), a label-free metric that operationalizes monosemanticity as activation-set coherence of binarized SAE latents, and does not require external embedding encoders. We evaluate TMS on SAEs trained on features from pretrained vision and vision-language models (DINOv3, CLIP, BLIP2), two common SAE regimes (TopK, BatchTopK), multiple sparsity levels, and expansion factors. Our results show that TMS is less affected by encoder anisotropy than its embedding-based alternative, while remaining aligned with established monosemanticity indicators. TMS also reveals distinct SAE training dynamics across base models. Moreover, under encoder anisotropy, TMS provides a stronger indication of probe-based concept deletion effectiveness, while being competitive otherwise.

Figures

Figures reproduced from arXiv: 2607.17770 by the authors.

**Figure 1.** Anisotropy-induced bias in embedding-based MS. (a) External embedding models exhibit different degrees of anisotropy. (b) MonoSemanticity Score (MS) [13] aggregated over non-zero-MS neurons depends strongly on the embedding source—more anisotropic encoders produce inflated MS results across setups. 5 Experimental results The impact of anisotropy level on monosemanticty level estimation [PITH_FULL_IMAGE:figures/fu… view at source ↗

**Figure 2.** Tversky Monosemantic Score (TMS) results [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** MonoSemantic (MS) [13] score results. clearly polysemantic activation pattern, and the substantially lower TMS value reflects smaller overlap in its activation sets. For CLIP-based SAEs, neurons with relatively high MS values (e.g. compared to DINOv3) display visually and semantically diverse image sets despite their geometric embedding space proximity. In contrast, TMS assigns lower scores to such neurons, reflect… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Example top 9 activating images for neurons with different TMS. Values decrease from left to right. Top row includes examples for SAE-DINOv3, middle row — CLIP, and bottom row – BLIP2. All SAEs are TopK, expansion factor 2 and K = 64. We use bold for cases where order…

**Figure 5.** Figure 5: Epoch evolution of MS for TopKSAE (left column) and BatchTopKSAE (right column) across DINO, CLIP, and BLIP models [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 6.** Figure 6: Epoch evolution of TMS for TopKSAE (left column) and BatchTopKSAE (right column) across DINO, CLIP, and BLIP models [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: Deletion curves showing target drop as a function of number of deleted features (N). Left column: TopKSAE, right column: BatchTopKSAE. Rows: DINO, CLIP and BLIP models, respectively [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 5 linked inside Pith

[1]

arXiv e-prints pp

Aswal, D., Hudelot, C.: Llmsymguard: A symbolic safety guardrail framework leveraging interpretable jailbreak concepts. arXiv e-prints pp. arXiv–2508 (2025)

2025
[2]

arXiv:2404.14082 (2024)

Bereska, L., Gavves, E.: Mechanistic interpretability for ai safety–a review. arXiv:2404.14082 (2024)

arXiv 2024
[3]

Transformer Circuits Thread (2023)

Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J.E., Hume, T., Carter, S., Henighan, T., Olah, C.: Towards monosemantic- ity: Decomposing language ...

2023
[4]

arXiv:2501.18052 (2025)

Cywiński,B.,Deja,K.:Saeuron:Interpretableconceptunlearningindiffusionmod- els with sparse autoencoders. arXiv:2501.18052 (2025)

arXiv 2025
[5]

arXiv:2508.15094 (2025)

Fereidouni, M., Haider, M.U., Ju, P., Siddique, A.: Evaluating sparse autoencoders for monosemantic representation. arXiv:2508.15094 (2025)

arXiv 2025
[6]

Filus, K., Pokuciński, S.: Measuring monosemanticity in sparse autoencoders via latent activation coherence - source code (2026), Zenodo repository (private review link, will be made fully public on acceptance)

2026
[7]

In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

Gallifant, J., Chen, S., Sasse, K., Aerts, H., Hartvigsen, T., Bitterman, D.: Sparse autoencoder features for classifications and transferability. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. pp. 29927–29951 (2025)

2025
[8]

arXiv:2406.04093 (2024)

Gao, L., la Tour, T.D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., Wu, J.: Scaling and evaluating sparse autoencoders. arXiv:2406.04093 (2024)

arXiv 2024

Show all 19 references

[9]

arXiv:2506.19382 (2025)

Härle, R., Friedrich, F., Brack, M., Wäldchen, S., Deiseroth, B., Schramowski, P., Kersting, K.: Measuring and guiding monosemanticity. arXiv:2506.19382 (2025)

2025
[10]

In: The Twelfth In- ternational Conference on Learning Representations (2023)

Huben, R., Cunningham, H., Smith, L.R., Ewart, A., Sharkey, L.: Sparse autoen- coders find highly interpretable features in language models. In: The Twelfth In- ternational Conference on Learning Representations (2023)

2023
[11]

In: International conference on machine learning

Li, J., Li, D., Savarese, S., Hoi, S.: Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: International conference on machine learning. pp. 19730–19742 (2023)

2023
[12]

Distill3(3), e10 (2018)

Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., Mordv- intsev, A.: The building blocks of interpretability. Distill3(3), e10 (2018)

2018
[13]

arXiv:2504.02821 (2025)

Pach, M., Karthik, S., Bouniot, Q., Belongie, S., Akata, Z.: Sparse autoencoders learn monosemantic features in vision-language models. arXiv:2504.02821 (2025)

2025
[14]

In: International conference on machine learning

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763 (2021)

2021
[15]

arXiv:2407.14435 (2024)

Rajamanoharan, S., Lieberum, T., Sonnerat, N., Conmy, A., Varma, V., Kramár, J., Nanda, N.: Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. arXiv:2407.14435 (2024)

2024 arXiv
[16]

Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, Preprint: Monosemanticity via Sparse Autoencoders Latent Coherence 17 L., Wang, J., Darcet, T., Moutakanni, T., S...

2025
[17]

Psychological review84(4), 327 (1977)

Tversky, A.: Features of similarity. Psychological review84(4), 327 (1977)

1977
[18]

Advances in Neural Information Processing Systems29(2016)

Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. Advances in Neural Information Processing Systems29(2016)

2016
[19]

arXiv:2502.20578 (2025) 18 K

Zaigrajew, V., Baniecki, H., Biecek, P.: Interpreting clip with hierarchical sparse autoencoders. arXiv:2502.20578 (2025) 18 K. Filus and S. Pokuciński Appendices: Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence A Accuracy of SAEs In Tab. A.1, ...

2025 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

arXiv e-prints pp

Aswal, D., Hudelot, C.: Llmsymguard: A symbolic safety guardrail framework leveraging interpretable jailbreak concepts. arXiv e-prints pp. arXiv–2508 (2025)

2025

[2] [2]

arXiv:2404.14082 (2024)

Bereska, L., Gavves, E.: Mechanistic interpretability for ai safety–a review. arXiv:2404.14082 (2024)

arXiv 2024

[3] [3]

Transformer Circuits Thread (2023)

Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J.E., Hume, T., Carter, S., Henighan, T., Olah, C.: Towards monosemantic- ity: Decomposing language ...

2023

[4] [4]

arXiv:2501.18052 (2025)

Cywiński,B.,Deja,K.:Saeuron:Interpretableconceptunlearningindiffusionmod- els with sparse autoencoders. arXiv:2501.18052 (2025)

arXiv 2025

[5] [5]

arXiv:2508.15094 (2025)

Fereidouni, M., Haider, M.U., Ju, P., Siddique, A.: Evaluating sparse autoencoders for monosemantic representation. arXiv:2508.15094 (2025)

arXiv 2025

[6] [6]

Filus, K., Pokuciński, S.: Measuring monosemanticity in sparse autoencoders via latent activation coherence - source code (2026), Zenodo repository (private review link, will be made fully public on acceptance)

2026

[7] [7]

In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

Gallifant, J., Chen, S., Sasse, K., Aerts, H., Hartvigsen, T., Bitterman, D.: Sparse autoencoder features for classifications and transferability. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. pp. 29927–29951 (2025)

2025

[8] [8]

arXiv:2406.04093 (2024)

Gao, L., la Tour, T.D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., Wu, J.: Scaling and evaluating sparse autoencoders. arXiv:2406.04093 (2024)

arXiv 2024