Pith. sign in

REVIEW 4 major objections 6 minor 48 references

Sparsification and Reconstruction from the Perspective of Representation Geometry

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Separating concept clusters in sparse autoencoders improves reconstruction, the paper argues, using representation geometry to explain why.

desk verdict Useful descriptive geometry for SAE latents, but the headline causal claim is an optimization artifact. read the letter →

arxiv 2505.22506 v1 pith:AZ7AF2MJ submitted 2025-05-28 cs.LG

Wenjie Sun , Bingzhe Wu , Zhile Yang , Chengke Wu This is my paper
classification cs.LG
keywords sparseautoencodersrepresentationgeometrystratifiedmanifoldsfeaturedisentanglementmechanisticinterpretabilitycausalinterventionreconstructionperformance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that sparse autoencoders (SAEs) organize language-model activations into a stratified representation geometry, and that this geometry directly controls reconstruction quality. It introduces SAEMA, an analyzer that detects stratification by observing how the rank of symmetric positive semidefinite (SSPD) matrices changes when noise is added to the residual stream. The authors define local and global representations and show that sparse encoding separates features by merging similar semantic clusters and adding dimensionality. Their central intervention moves cluster centers apart along a loss gradient and reports that increased separability between local representations causally reduces reconstruction error. If correct, this means geometric constraints—not just sparsity penalties—should be incorporated into SAE design.

What carries the argument

The central object is the SAE latent tensor $F$, unfolded along batch, sequence, and feature modes into three matrices; each mode produces an SSPD matrix $S^{(i)}=F^{(i)}F^{(i)T}+\epsilon I$ whose singular-value rank $r_i$ locates the tensor on the product manifold $S^{I_1}(r_1)\times S^{I_2}(r_2)\times S^{I_3}(r_3)$, and rank variability under noise diagnoses stratification. For the causal claim, the load-bearing mechanism is the intervention: cluster centers obtained from density-based clustering are translated by $\alpha \cdot \mathrm{grad}$ of $L=d_{GW}(D_{\mathrm{orig}},D_{\mathrm{intervened}})+\lambda_{MSE}MSE$, and separability is measured by AEDP, the mean pairwise Euclidean distance between cluster centers. The argument is that AEDP increases monotonically with $\alpha$ and MSE decreases monotonically with AEDP across all model-concept pairs, while the ablation shows the $\mathrm{AEDP}^{-1}$ term is negligible, so separability is the operative cause.

What would settle it

Run the Case 3 intervention with the MSE term removed from the loss (or frozen at its original value); if reconstruction error does not decrease as AEDP increases, the claimed causal relationship is not supported.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is threefold. First, the feature-mode rank $r_3$ of the SSPD matrix varies by 57–142% under noise while the batch- and sequence-mode ranks stay constant, which the authors read as the latent tensor crossing different strata of the product manifold $S^{I_1}(r_1)\times S^{I_2}(r_2)\times S^{I_3}(r_3)$. Second, after sparse encoding, local representations gain average intrinsic dimensionality and Betti-0 components, while the global minimum spanning tree weight mostly decreases, indicating compression of global structure together with finer local subdivision. Third, translating cluster centers along the gradient of $L=d_{GW}+\lambda_{MSE}MSE$ monotonically increases the average pairwise distance between cluster centers (AEDP) and decreases MSE in all nine model-concept pairs, with Pearson correlations between AEDP and MSE from $-0.89$ to $-0.97$. An ablation replacing $d_{GW}$ with $\mathrm{AEDP}^{-1}$ shows the separability term contributes almost nothing to the loss, so the authors conclude that MSE reduction is driven by the increased separability.

Load-bearing premise

The weakest link is that in Case 3 the translation direction comes from a loss that directly contains the reconstruction MSE, so the observed MSE drop could simply reflect minimization of that term rather than a genuine causal effect of increased separability.

Editorial extensions

If this is right

  • SAE training should include explicit geometric separation of cluster centers, not only sparsity-inducing regularization.
  • Stratification, detected by SSPD rank variability, can serve as a diagnostic for whether an SAE has captured concept structure.
  • Larger expansion factors degrade raw reconstruction but may leave more room for separability-driven improvements.
  • The sparsity–reconstruction trade-off can be mitigated by adding representational geometry constraints during optimization.
  • Local metrics such as average intrinsic dimensionality, Betti 0, and global MSTW give a quantitative language for describing disentanglement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors did not test an intervention that only separates clusters while holding MSE fixed; such an experiment would directly isolate the causal path they assert.
  • The rank-variability metric could be reused as an early-stopping or architecture-selection heuristic for SAEs, a use the paper leaves implicit.
  • The same geometric intervention could be applied to other overcomplete latent models, such as variational autoencoders, to see whether the separability–reconstruction coupling is a general property of sparse representations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes SAEMA, a manifold-analysis pipeline for SAE latent spaces, and uses it to make three claims: (1) SAE representations of concepts are stratified manifolds, evidenced by the noise-dependent variability of ranks of unfolded SSPD matrices; (2) sparse encoding restructures local and global representations by merging similar features and increasing intrinsic dimensionality; and (3) intervening to increase the separability of local representations causally improves SAE reconstruction performance. The main evidence for the causal claim is an optimization in which cluster centers are moved along gradients of a loss that directly contains reconstruction MSE (Eq. 7), together with a cross-model correlation between an AEDP separability metric and MSE.

Significance. If validated, the causal claim would be practically relevant: it would justify adding geometric separability constraints to SAE training and would offer a new explanation for the sparsity-reconstruction trade-off. The paper has some genuine strengths: it releases code, examines three language models with publicly available SAEs, and introduces several concrete geometric metrics (SSPD rank triplets, AGD, AEDP, Procrustes disparity) that could be reused. However, the central causal claim is not supported by the experimental design: the intervention optimizes the very quantity used as the outcome, and the authors' own ablation shows that the geometric term contributes negligibly. The stratification claim is also close to tautological as presented. These issues leave the paper's main contributions at the level of descriptive observations rather than established causal or geometric principles.

major comments (4)
  1. [Section 4.3, Eq. (7)] The causal claim is not tested by the proposed intervention. The loss L = d_GW + lambda_MSE * MSE contains the reconstruction MSE with lambda_MSE = 1, and the cluster centers are updated along the gradient of this loss. Moving centers along the gradient of L and then reporting that MSE decreases is a direct descent step on the outcome variable; the decrease is expected even if separability has no independent effect on reconstruction. A valid causal test would require an intervention that changes only separability (for example, optimizing a separability-only objective, or directly moving centers apart while holding the SAE decoder and the reconstruction target fixed) and then measuring reconstruction without back-propagating through MSE.
  2. [Appendix B.4.2, Eq. (10)] The ablation designed to isolate AEDP's role actually undermines the causal claim. When d_GW is replaced by AEDP^-1, the authors report that the AEDP^-1 term's contribution to the total loss is almost negligible while MSE and AEDP still show a strong negative correlation. This means the observed AEDP increase is a byproduct of minimizing the MSE term, not an independent cause of the MSE decrease. The statement in B.4.3 that MSE dominates the total loss function to steer local representations away from each other, thereby achieving its own reduction, is an explicit admission that the optimizer is minimizing MSE directly; the paper therefore does not provide evidence that the increase in separability causally enhances reconstruction performance.
  3. [Section 4.1 and Appendix A] The stratification conclusion is circular as presented. The paper observes that the feature-mode rank r3 varies with added noise and then labels this variability as the representation traversing different strata of a product manifold. Appendix A's proof, however, only establishes that additive noise can increase rank(F^(3)) (i.e., r'_3 >= r3), which is a generic property of perturbing a matrix and not evidence that the clean latent tensors lie on a stratified manifold. No null model, baseline ensemble, or statistical test is provided to show that the observed rank growth is more than what any noisy matrix would exhibit. Without such a comparison, the claim that the latent space is stratified is an interpretation of the observation rather than a validated geometric property.
  4. [Section 3.2 and Table 2] The empirical support for the main claims rests on single runs per model-concept pair and only five alpha values per curve. The reported Pearson correlations between AEDP and MSE (-0.89, -0.97, -0.93) are quoted without confidence intervals or repeated-seed variability, and the concept selection criterion (based on whether their keywords formed clusters) introduces selection bias into the claim of universality. These issues compound the intervention flaw: even the correlational evidence is too thin to support the paper's strong causal language, and the absence of error bars makes the descriptive results difficult to evaluate.
minor comments (6)
  1. [Section 2.1] There is a typo: 'Spaese Autoencoders' should be 'Sparse Autoencoders'.
  2. [Section 5.2] The reference to 'Engles et al.' should be 'Engels et al.'; the same reference appears in the Introduction with the correct spelling.
  3. [Section 5.3] The text says 'Cunningham et al. [5] pioneered the application of SAEs to LLMs,' but reference [5] is Bricken et al.; there is no Cunningham reference in the bibliography. The citation should be corrected or the intended reference added.
  4. [Section 6] The conclusion states that increased separability 'causally leads to a decrease in reconstruction performance'; the context (and Table 2) shows the authors mean a decrease in reconstruction error, i.e., an improvement in reconstruction performance. This sentence should be corrected.
  5. [Algorithm 3] The pseudocode in Appendix F updates centers with 'centers + alpha * random_grad' on line 25, but the text says the update uses stochastic gradients minimizing Eq. (7). The pseudocode is inconsistent with the described optimization and is not reproducible as written; please align the algorithm with the actual gradient computation.
  6. [Table 1] The columns under 'Local Representation' list pairs such as (3.01, 3.25) and (21.09, 26.11) without labeling which value corresponds to the residual stream and which to the latent tensor; the text explains this, but the table itself is hard to read. Adding explicit subcolumn labels would improve clarity.

Circularity Check

3 steps flagged · score 8.0 of 10

The causal claim in Case 3 reduces to gradient descent on the outcome: the intervention moves centers to minimize a loss containing MSE, and the paper's own ablation shows the geometric term is negligible; the stratification conclusion is also true by the paper's definition.

  1. self definitional [Section 3.2.3, Eq. (7); causal claim in Section 4.3]
    "The intervention is achieved by translating these centers and their corresponding local representations by different step size α, and the translation direction of was guided by applying stochastic gradients (grad) to ci to minimize L in Equation.7 during optimization, expressed as c′i ← ci + α×grad. L=d_GW(Doriginal, Dintervened) +λ_MSE MSE(x, x̂)(7)"

    The intervention is defined as a gradient step on a loss that directly contains the outcome variable, MSE, with λ_MSE set to 1. Moving the cluster centers and then measuring MSE on the moved latents is therefore a descent step on the quantity being 'explained'; a decrease in MSE is forced by construction, regardless of any geometric property. The subsequent causal conclusion that increased AEDP improves reconstruction is not an independent test: no intervention on separability is performed with reconstruction held fixed. The paper's own B.4.2 ablation confirms the non-MSE term is negligible, so the observed AEDP-MSE correlation is an optimization byproduct, not causal evidence.

  2. fitted input called prediction [Section 4.3 and Appendix B.4.2]
    "The results indicate that the contribution of the AEDP −1 term to the total loss is almost negligible. However, MSE and AEDP still exhibit a significant negative correlation. In other words, during optimization, MSE dominates the total loss function to steer local representations away from each other, thereby achieving its own reduction. Based on this, we conclude that the increase in the separability of local representations causally enhances reconstruction performance."

    This is an explicit admission that the optimized loss is dominated by MSE and that MSE 'achieves its own reduction.' The geometric term AEDP^-1 contributes almost nothing, so the experiment does not isolate separability as an independent cause. The conclusion nonetheless attributes the MSE reduction to increased AEDP, effectively re-labeling the effect of directly minimizing MSE as a causal effect of separability. The 'prediction' that increasing AEDP improves reconstruction is thus equivalent to the objective used to produce the data.

1 more flagged steps
  1. self definitional [Appendix A and Section 4.1]
    "A stratified manifold is a union of smooth submanifolds (strata) of different dimensions, i.e., M= S r1,r2,r3 SI1 (r1)×S I2 (r2)×S I3 (r3), where each stratum corresponds to a unique rank triplet (r1, r2, r3). If F lies on a single smooth manifold, the rank triplet remains constant under perturbations. If the ranks vary, F traverses different strata, indicating stratification."

    The paper's working definition of a stratified manifold is exactly that rank variability indicates stratum crossing. Section 4.1 then treats the observed r3 variability as 'proving that the representational structure of SAEs is stratified.' Since the definition already equates rank variability with stratification, the conclusion is true by stipulation rather than by an independent mathematical derivation. The Appendix A proof reduces to the standard identity rank(F F^T)=rank(F) and then restates the definition, so the empirical 'validation' does not test a separate theoretical prediction.

full rationale

The central causal claim of the paper is not self-contained in a testable sense. In Case 3, the intervention on global representations is defined as translating cluster centers along the gradient of a loss that includes the reconstruction MSE. Because λ_MSE=1, moving the centers and then reporting a lower MSE is a direct optimization outcome, not evidence that separability causally enhances reconstruction. The paper's own ablation (B.4.2) shows that when the geometric term is replaced by AEDP^-1, that term is 'almost negligible' and MSE still decreases, which confirms that the MSE term in the objective, not the separability term, is doing the causal work. The conclusion that increased AEDP improves reconstruction is therefore forced by the construction of the loss. Separately, the stratification claim in Case 1 is definitional: a stratified manifold is characterized by variation of the rank triplet, and then observed rank variation is presented as proof of stratification. This is a stipulated labeling rather than an independent derivation. The descriptive measurements in Case 2 (Avg. ID, Betti 0, MSTW, Procrustes disparity) are not circular; they are ordinary empirical observations, and no load-bearing self-citation chain is present. However, because the headline causal contribution reduces to an optimization artifact and the stratification result is true by definition, a score of 8 is appropriate.

Assumptions & free parameters 8 free parameters · 4 assumptions · 1 invented entities

The central claims rest on chosen thresholds and hyperparameters for rank, clustering, and dimensionality reduction, plus the assumption that noisy perturbations and cluster centroids faithfully represent the geometry of concepts. The causal conclusion additionally assumes that gradient updates on a loss containing MSE are neutral interventions on geometry.

free parameters (8)
  • noise schedule = 0.0, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0
    Hand-chosen noise standard deviations; rank variability and AGD changes in Case 1 are measured across these levels.
  • high-frequency noise multipliers and top-100 threshold = 2x for top 100 features, 0.2x otherwise
    Ad hoc noise scheme intended to keep low-frequency features active; affects all rank measurements.
  • feature downsampling to 2048 = d_sae -> 2048
    Latent tensors with d_sae > 2048 are truncated using a variance-covariance score (Eq. 3), capping possible rank and shaping r3 values.
  • rank threshold tau_i = first quartile of effective singular values
    Dynamic threshold in Eq. 5 determines which singular values count toward the rank; rank variability is sensitive to this choice.
  • HDBSCAN min_cluster_size = 10
    Controls the number of clusters interpreted as local representations; all downstream AEDP, Avg ID, Betti 0, and MSTW depend on it.
  • UMAP n_components = 50
    Dimensionality reduction target before clustering; UMAP is stochastic and no seed is reported.
  • alpha step sizes and lambda_MSE = alpha in {0.5, 0.8, 1.0, 1.2, 1.5}, lambda_MSE=1
    Intervention strength and loss weight in Case 3; MSE and AEDP trends are functions of alpha.
  • intrinsic dimension and persistence thresholds = tau_dim=0.01, tau_pers=0.1
    Thresholds for Avg ID and Betti 0 calculations, chosen in Appendix C.
assumptions (4)
  • domain assumption The latent tensor can be represented as a point on a product manifold of SSPD matrices, and rank triplets define manifold strata (Shrivastava et al. [10]).
    Invoked in Section 2.2 and Appendix A; the stratification conclusion is a direct consequence of this modeling choice.
  • domain assumption Adding Gaussian noise to the residual stream probes the geometry of the concept's representation rather than just increasing the rank of a generic matrix.
    Used in Case 1; without this, rank variability under noise has no manifold interpretation.
  • domain assumption HDBSCAN clusters of UMAP-reduced activations correspond to semantically coherent local representations.
    Definitions 1 and 2 and Case 2 rely on this; no validation of cluster semantics is provided.
  • ad hoc to paper Translating cluster centers along the gradient of a loss that includes MSE is a neutral causal intervention on representation geometry.
    Section 3.2.3 and Eq. 7; this assumption is contradicted by the B.4.2 ablation showing the geometry term is negligible.
invented entities (1)
  • local representation (HDBSCAN cluster of latent vectors)
    purpose: Unit of analysis for separability and intrinsic dimensionality
    Defined in Definition 1 as clusters from an unseeded UMAP/HDBSCAN pipeline; no external validation that clusters correspond to concepts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sparsification and Reconstruction from the Perspective of Representation Geometry." pith.science (2026). https://pith.science/paper/AZ7AF2MJ

@misc{pith2026250522506,
  author       = {Pith},
  title        = {Pith review of: Sparsification and Reconstruction from the Perspective of Representation Geometry},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AZ7AF2MJ}},
  note         = {Machine review of arXiv:2505.22506}
}
read the original abstract

Sparse Autoencoders (SAEs) have emerged as a predominant tool in mechanistic interpretability, aiming to identify interpretable monosemantic features. However, how does sparse encoding organize the representations of activation vector from language models? What is the relationship between this organizational paradigm and feature disentanglement as well as reconstruction performance? To address these questions, we propose the SAEMA, which validates the stratified structure of the representation by observing the variability of the rank of the symmetric semipositive definite (SSPD) matrix corresponding to the modal tensor unfolded along the latent tensor with the level of noise added to the residual stream. To systematically investigate how sparse encoding alters representational structures, we define local and global representations, demonstrating that they amplify inter-feature distinctions by merging similar semantic features and introducing additional dimensionality. Furthermore, we intervene the global representation from an optimization perspective, proving a significant causal relationship between their separability and the reconstruction performance. This study explains the principles of sparsity from the perspective of representational geometry and demonstrates the impact of changes in representational structure on reconstruction performance. Particularly emphasizes the necessity of understanding representations and incorporating representational constraints, providing empirical references for developing new interpretable tools and improving SAEs. The code is available at \hyperlink{https://github.com/wenjie1835/SAERepGeo}{https://github.com/wenjie1835/SAERepGeo}.

Figures

Figures reproduced from arXiv: 2505.22506 by the authors.

Figure 1
Figure 1. The workflow of SAEManifoldAnalyzer (SAEMA) [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. illustrates the variability of r3 (the rank of SSPD matrix of feature modal) and AGD for three SAEs representing the concepts under different noise levels. Detailed experimental results can be found in Appendix B.2. We observe that the batch_size and seq_len modal ranks (r1, r2) remain constant, which may be attributed to the fact that prompts with high information density and strong semantic consistency stabilize t… view at source ↗
Figure 3
Figure 3. Variability of dGW and AEDP with increasing α during the optimization of Equation 7. B.4.2 Experiment Details and Results of the Optimization Process based on Equation 10 To validate the causal relationship between the separability of the proposed local representations and the reconstruction performance of SAE, we modified the loss function in Equation 7 by replacing dGW with AEDP −1 , as shown in Equation 10. The i… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Variability of dGW and AEDP with MSE for different α during the optimization of Equation 7 [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: First column: Variability of MSE with AEDP as [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 21 canonical work pages

  1. [1]

    Open problems in mechanistic interpretability,

    L. Sharkey, B. Chughtai, J. Batson, J. Lindsey, J. Wu, L. Bushnaq, N. Goldowsky-Dill, S. Heimer- sheim, A. Ortega, J. Bloomet al., “Open problems in mechanistic interpretability,”arXiv preprint arXiv:2501.16496, 2025

    arXiv 2025

  2. [2]

    Mechanistic interpretability for ai safety–a review,

    L. Bereska and E. Gavves, “Mechanistic interpretability for ai safety–a review,”arXiv preprint arXiv:2404.14082, 2024

    arXiv 2024

  3. [3]

    Towards automated circuit discovery for mechanistic interpretability,

    A. Conmy, A. Mavor-Parker, A. Lynch, S. Heimersheim, and A. Garriga-Alonso, “Towards automated circuit discovery for mechanistic interpretability,”Advances in Neural Information Processing Systems, vol. 36, pp. 16 318–16 352, 2023

    2023

  4. [4]

    Toy models of superposition,

    N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chenet al., “Toy models of superposition,”arXiv preprint arXiv:2209.10652, 2022

    arXiv 2022

  5. [5]

    Towards monosemanticity: Decomposing language models with dictionary learning,

    T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askellet al., “Towards monosemanticity: Decomposing language models with dictionary learning,” Transformer Circuits Thread, vol. 2, 2023

    work page 2023

  6. [6]

    The linear representation hypothesis and the geometry of large language models,

    K. Park, Y . J. Choe, and V . Veitch, “The linear representation hypothesis and the geometry of large language models,”arXiv preprint arXiv:2311.03658, 2023

    arXiv 2023

  7. [7]

    On the origins of linear representations in large language models,

    Y . Jiang, G. Rajendran, P. Ravikumar, B. Aragam, and V . Veitch, “On the origins of linear representations in large language models,”arXiv preprint arXiv:2403.03867, 2024

    arXiv 2024

  8. [8]

    Feature manifold toy model,

    O. Chris and J. Batson, “Feature manifold toy model,” 2023. [Online]. Available: https: //transformer-circuits.pub/2023/may-update/index.html#feature-manifolds

    work page 2023

Show all 48 references
  1. [9]

    Not all language model features are one-dimensionally linear,

    J. Engels, E. J. Michaud, I. Liao, W. Gurnee, and M. Tegmark, “Not all language model features are one-dimensionally linear,” 2025. [Online]. Available: https://arxiv.org/abs/2405.14860

    2025 arXiv

  2. [10]

    Latent space characterization of autoencoder variants,

    A. Shrivastava, R. Rameshan, and S. Agnihotri, “Latent space characterization of autoencoder variants,” arXiv preprint arXiv:2412.04755, 2024

    2024 arXiv

  3. [11]

    Steering knowledge selection behaviours in llms via sae-based representation engineering,

    Y . Zhao, A. Devoto, G. Hong, X. Du, A. P. Gema, H. Wang, X. He, K.-F. Wong, and P. Minervini, “Steering knowledge selection behaviours in llms via sae-based representation engineering,”arXiv preprint arXiv:2410.15999, 2024

    2024 arXiv

  4. [12]

    Interpreting attention layer outputs with sparse autoencoders,

    C. Kissane, R. Krzyzanowski, J. I. Bloom, A. Conmy, and N. Nanda, “Interpreting attention layer outputs with sparse autoencoders,”arXiv preprint arXiv:2406.17759, 2024

    2024 arXiv

  5. [13]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskeveret al., “Language models are unsupervised multitask learners,”OpenAI blog, vol. 1, no. 8, p. 9, 2019

    2019

  6. [14]

    Pythia: A suite for analyzing large language models across training and scaling,

    S. Biderman, H. Schoelkopf, Q. G. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raffet al., “Pythia: A suite for analyzing large language models across training and scaling,” inInternational Conference on Machine Learning. PMLR, 2023...

    2023

  7. [15]

    Gemma 2: Improving open language models at a practical size,

    G. Team, M. Riviere, S. Pathak, P. G. Sessa, C. Hardin, S. Bhupatiraju, L. Hussenot, T. Mesnard, B. Shahriari, A. Raméet al., “Gemma 2: Improving open language models at a practical size,”arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  8. [16]

    Saelens,

    C. T. Joseph Bloom and D. Chanin, “Saelens,” https://github.com/jbloomAus/SAELens, 2024

    2024

  9. [17]

    Geometry-preserving encoder/decoder in latent generative models,

    W. Lee, R. C. O’Neill, D. Zou, J. Calder, and G. Lerman, “Geometry-preserving encoder/decoder in latent generative models,”arXiv preprint arXiv:2501.09876, 2025

    2025 arXiv

  10. [18]

    Gromov-Wasserstein autoencoders,

    N. Nakagawa, R. Togo, T. Ogawa, and M. Haseyama, “Gromov-Wasserstein autoencoders,” in Proceedings of International Conference on Learning Representations (ICLR), 2023. [Online]. Available: https://openreview.net/forum?id=sbS10BCtc7

    2023

  11. [19]

    Improving dictionary learning with gated sparse autoencoders,

    S. Rajamanoharan, A. Conmy, L. Smith, T. Lieberum, V . Varma, J. Kramár, R. Shah, and N. Nanda, “Improving dictionary learning with gated sparse autoencoders,”arXiv preprint arXiv:2404.16014, 2024

    2024 arXiv

  12. [20]

    Prolu: A nonlinearity for sparse autoencoders,

    G. M. Taggart, “Prolu: A nonlinearity for sparse autoencoders,” https://www.alignmentforum.org/posts/ HEpufTdakGTTKgoYF/prolu-a-nonlinearity-for-sparse-autoencoders, 2024

    2024

  13. [21]

    Jumprelu: A retrofit defense strategy for adversarial attacks,

    N. B. Erichson, Z. Yao, and M. W. Mahoney, “Jumprelu: A retrofit defense strategy for adversarial attacks,” arXiv preprint arXiv:1904.03750, 2019. 10

    1904 arXiv

  14. [22]

    Batchtopk sparse autoencoders,

    B. Bussmann, P. Leask, and N. Nanda, “Batchtopk sparse autoencoders,”arXiv preprint arXiv:2412.06410, 2024

    2024 arXiv

  15. [23]

    Adaptive sparse allocation with mutual choice & feature choice sparse autoencoders,

    K. Ayonrinde, “Adaptive sparse allocation with mutual choice & feature choice sparse autoencoders,”arXiv preprint arXiv:2411.02124, 2024

    2024 arXiv

  16. [24]

    Efficient training of sparse autoencoders for large language models via layer groups,

    D. Ghilardi, F. Belotti, and M. Molinari, “Efficient training of sparse autoencoders for large language models via layer groups,”arXiv preprint arXiv:2410.21508, 2024

    2024

  17. [26]

    Orthogonal neural representations support perceptual judgments of natural stimuli,

    R. Srinath, A. M. Ni, C. Marucci, M. R. Cohen, and D. H. Brainard, “Orthogonal neural representations support perceptual judgments of natural stimuli,”Scientific Reports, vol. 15, no. 1, p. 5316, 2025

    2025

  18. [27]

    How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings,

    K. Ethayarajh, “How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings,”arXiv preprint arXiv:1909.00512, 2019

    1909 arXiv

  19. [28]

    The geometry of multilingual language model representations,

    T. A. Chang, Z. Tu, and B. K. Bergen, “The geometry of multilingual language model representations,” ArXiv, vol. abs/2205.10964, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID: 248987203

    2022 arXiv

  20. [29]

    Transformers represent belief state geometry in their residual stream,

    A. Shai, L. Teixeira, A. Oldenziel, S. Marzen, and P. Riechers, “Transformers represent belief state geometry in their residual stream,”Advances in Neural Information Processing Systems, vol. 37, pp. 75 012–75 034, 2024

    2024

  21. [30]

    The geometry of concepts: Sparse autoencoder feature structure,

    Y . Li, E. J. Michaud, D. D. Baek, J. Engels, X. Sun, and M. Tegmark, “The geometry of concepts: Sparse autoencoder feature structure,”arXiv preprint arXiv:2410.19750, 2024

    2024 arXiv

  22. [31]

    Representation learning: A review and new perspectives,

    Y . Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,”IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1798–1828, 2013

    2013

  23. [32]

    beta-vae: Learning basic visual concepts with a constrained variational framework,

    I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework,” inInternational conference on learning representations, 2017

    2017

  24. [33]

    Online dictionary learning for sparse coding,

    J. Mairal, F. Bach, J. Ponce, and G. Sapiro, “Online dictionary learning for sparse coding,” inProceedings of the 26th annual international conference on machine learning, 2009, pp. 689–696

    2009

  25. [34]

    Disentangled representation learning,

    X. Wang, H. Chen, Z. Wu, W. Zhuet al., “Disentangled representation learning,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  26. [35]

    Multi-vae: Learning disentangled view-common and view-peculiar visual representations for multi-view clustering,

    J. Xu, Y . Ren, H. Tang, X. Pu, X. Zhu, M. Zeng, and L. He, “Multi-vae: Learning disentangled view-common and view-peculiar visual representations for multi-view clustering,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9234–9243

    2021

  27. [36]

    Disentangling disentanglement in variational autoencoders,

    E. Mathieu, T. Rainforth, N. Siddharth, and Y . W. Teh, “Disentangling disentanglement in variational autoencoders,” inInternational conference on machine learning. PMLR, 2019, pp. 4402–4412

    2019

  28. [37]

    Guided variational autoencoder for disentanglement learning,

    Z. Ding, Y . Xu, W. Xu, G. Parmar, Y . Yang, M. Welling, and Z. Tu, “Guided variational autoencoder for disentanglement learning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 7920–7929

    2020

  29. [38]

    Oogan: Disentangling gan with one-hot sampling and orthogonal regularization,

    B. Liu, Y . Zhu, Z. Fu, G. De Melo, and A. Elgammal, “Oogan: Disentangling gan with one-hot sampling and orthogonal regularization,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, 2020, pp. 4836–4843

    2020

  30. [39]

    Disentanglement in a gan for unconditional speech synthesis,

    M. Baas and H. Kamper, “Disentanglement in a gan for unconditional speech synthesis,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 1324–1335, 2024

    2024

  31. [40]

    Diagonal attention and style-based gan for content-style disentanglement in image generation and translation,

    G. Kwon and J. C. Ye, “Diagonal attention and style-based gan for content-style disentanglement in image generation and translation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 13 980–13 989

    2021

  32. [41]

    Efficient dictionary learning with switch sparse autoencoders,

    A. Mudide, J. Engels, E. J. Michaud, M. Tegmark, and C. S. de Witt, “Efficient dictionary learning with switch sparse autoencoders,”arXiv preprint arXiv:2410.08201, 2024

    2024 arXiv

  33. [42]

    Towards achieving concept completeness for unsupervised textual concept bottleneck models,

    M. Bhan, Y . Choho, P. Moreau, J.-N. Vittaut, N. Chesneau, and M.-J. Lesot, “Towards achieving concept completeness for unsupervised textual concept bottleneck models,”arXiv preprint arXiv:2502.11100, 2025. 11

    2025 arXiv

  34. [43]

    Measuring progress in dictionary learning for language model interpretability with board game models,

    A. Karvonen, B. Wright, C. Rager, R. Angell, J. Brinkmann, L. Smith, C. Mayrink Verdun, D. Bau, and S. Marks, “Measuring progress in dictionary learning for language model interpretability with board game models,”Advances in Neural Information Processing Systems, vol. 37, pp. ...

    2024

  35. [44]

    Superposition yields robust neural scaling,

    Z. Liu, J. Goreet al., “Superposition yields robust neural scaling,”arXiv preprint arXiv:2505.10465, 2025

    2025 arXiv

  36. [45]

    Estimating local intrinsic dimensionality,

    L. Amsaleg, O. Chelly, T. Furon, S. Girard, M. E. Houle, K.-i. Kawarabayashi, and M. Nett, “Estimating local intrinsic dimensionality,” inProceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, pp. 29–38

    2015

  37. [46]

    Geometrically bounding 3–manifolds, volume and betti numbers,

    J. Ma and F. Zheng, “Geometrically bounding 3–manifolds, volume and betti numbers,”Algebraic & Geometric Topology, vol. 23, no. 3, pp. 1055–1096, 2023

    2023

  38. [47]

    Fast parallel algorithms for euclidean minimum spanning tree and hierarchical spatial clustering,

    Y . Wang, S. Yu, Y . Gu, and J. Shun, “Fast parallel algorithms for euclidean minimum spanning tree and hierarchical spatial clustering,” inProceedings of the 2021 international conference on management of data, 2021, pp. 1982–1995

    2021

  39. [48]

    Applications of average geodesic distance in manifold learning,

    X. Zeng, “Applications of average geodesic distance in manifold learning,” inInternational Conference on Rough Sets and Knowledge Technology. Springer, 2008, pp. 540–547

    2008

  40. [49]

    A survey on sparse autoencoders: Interpreting the internal mechanisms of large language models,

    D. Shu, X. Wu, H. Zhao, D. Rai, Z. Yao, N. Liu, and M. Du, “A survey on sparse autoencoders: Interpreting the internal mechanisms of large language models,”arXiv preprint arXiv:2503.05613, 2025. 12 Appendix Appendix Contents A Proof: The variability ofRank(S (i) reflects produ...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.