Pith. sign in

REVIEW 6 major objections 6 minor 51 references

A Monosemantic Attribution Framework for Stable Interpretability in Clinical Neuroscience Transformer-Based Language Models

T0 review · 6 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that routing LLM attributions through a sparse autoencoder's feature space, rather than raw activations, yields stable, clinically coherent token-level explanations for Alzheimer's diagnosis.

desk verdict The pipeline is new and the experiments are extensive, but the 'monosemantic' premise is unvalidated and the stability evidence is partly circular — worth refereeing, not accepting as is. read the letter →

arxiv 2601.17952 v2 pith:SH3FFTM4 submitted 2026-01-25 cs.CL cs.AI

Michail Mamalakis , Tiago Azevedo , Cristian Cosentino , Chiara D'Ercoli , Subati Abulikemu , Zhongtian Sun , Richard Bethlehem , Pietro Lio This is my paper · ORCID
classification cs.CLcs.AI
keywords mechanisticinterpretabilitysparseautoencodersattributionstabilityexplainableAIAlzheimer'sdiseaseclinicalNLPlargelanguagemodelspolysemanticity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that attribution explanations for language models in clinical settings are unstable because internal representations are polysemantic, and that this can be fixed by computing attributions in a sparse, more monosemantic feature space produced by a sparse autoencoder. It combines six attribution methods inside that space with a learned explanation optimizer (TEO) whose loss explicitly minimizes inter-method variability via RIS/ROS, sparseness, and similarity. The central empirical claim is that TEO-SAE achieves the lowest RIS/ROS across in-distribution and out-of-distribution cohorts, while adding a UMAP linear constraint (TEO-UMAP) offers a tunable sparsity–stability trade-off. If correct, this would give clinicians stable, transferable explanations that highlight known cognitive biomarkers and reduce reliance on specific attribution methods.

What carries the argument

The load-bearing component is a sparse autoencoder (SAE) bottleneck placed on the final layer of a transformer language model: activations x are encoded into a sparse high-dimensional feature vector a, and decoded back via W a + b. Attribution vectors over the sparse features are propagated to input tokens via a two-step chain rule through the encoder Jacobian and the LLM's input-to-layer gradients. The Transformer Explanation Optimizer (TEO) is an encoder–decoder trained to fuse six attribution vectors into a single reconstruction, with a cost function weighting reciprocal RIS, reciprocal ROS, Gini sparseness, and similarity to a weighted consensus attribution. An optional UMAP linear const

What would settle it

Audit the SAE features directly: for a random sample of latents, collect the input tokens that most activate each feature across both cohorts and have human raters or an auto-interpreter label the concept. If a single feature fires for semantically unrelated concepts across cohorts, the monosemanticity premise is falsified. Separately, run TEO with the same loss but without the SAE; if RIS/ROS reductions persist, the optimizer loss rather than the SAE is driving stability.

Watch

Extended reading notes

Core claim

The paper's central claim is that constructing a monosemantic embedding space at the level of an LLM layer and optimizing the framework to explicitly reduce inter-method variability produces stable input-level importance scores and highlights salient features through a decompressed representation of the layer of interest. Concretely, the authors assert that TEO-SAE achieves the lowest RIS/ROS overall, showing strong cross-dataset stability, and that TEO-UMAP offers a tunable sparsity–stability trade-off that generalizes from an in-distribution Alzheimer's cohort to an out-of-distribution Latin American cohort. The claim is that attributions computed in the SAE-induced feature space are more

Load-bearing premise

The framework's claims rest on the assumption that the sparse autoencoder's latent features are genuinely monosemantic, so attributions computed there inherit semantic coherence; the paper does not validate this, and its overcomplete dictionary may itself produce superposition.

Editorial extensions

If this is right

  • If the central claim is correct, clinical explanations from transformer language models can be made stable across attribution methods and across distribution shifts, a prerequisite for trustworthy deployment.
  • The tunable sparsity–stability frontier gives practitioners a concrete knob: TEO-SAE for maximum stability, TEO-UMAP for more focused attributions at a modest stability cost.
  • The framework could prioritize efficient cognitive assessments by identifying which multimodal subgroups (e.g., memory, functional status, visuospatial tests) drive each diagnostic class.
  • Because the explanation optimizer is trained on in-distribution data and evaluated out-of-distribution without retraining, the method may transfer to new clinical cohorts with minimal adaptation.
  • A single consensus explanation can replace the need to choose among conflicting attribution methods, reducing ambiguity in high-stakes decisions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The stability gains attributed to the SAE may partly come from the optimizer loss that explicitly minimizes RIS/ROS, not from semantic disentanglement; ablating the SAE while keeping the same loss on raw activations would isolate the cause.
  • Editorial inference: The overcomplete SAE dictionary (F > d) itself permits superposition, so the 'monosemantic' claim is untested; an interpretability audit of individual features is needed before the clinical coherence claim is granted.
  • Editorial inference: The UMAP linear constraint equating the first two embedding components is a strong geometric prior that could discard information; a softer penalty or alternative global-coherence regularizer might achieve similar stability without the risk.
  • Editorial inference: The external LLM-based audit is a promising but weakly controlled evaluation; a clinician-in-the-loop study using the top-50% highlighted tokens would test whether SAE-based explanations actually change treatment or triage decisions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper proposes a "monosemantic attribution framework" for ModernBERT-based Alzheimer's disease classification. It combines a sparse autoencoder (TopK, 32× feature depth) over the 22nd layer with six attribution methods, a learned explanation optimizer (TEO, and a diffusion variant DEO) trained with Eq. (3), and a UMAP embedding constrained by u_i1 = u_i2. The authors claim that this produces stable input-level importance scores, a tunable sparsity–stability trade-off, and clinically coherent biomarker identification, evaluating on ADNI (IID) and BrainLAT (OOD), including an external ChatGPT-5.1 assessment.

Significance. If the central claims were supported, the framework would be a useful bridge between attributional and mechanistic interpretability, with practical value for clinical NLP. The paper has genuine strengths: an explicit IID/OOD evaluation protocol, comparison of multiple SAE variants and attribution methods, a concrete clinical dataset, and a stated commitment to code availability. However, the headline stability evidence is partly circular: TEO is trained on the same RIS/ROS/sparseness metrics that Table 1 uses as evidence. In addition, the "monosemantic" premise is asserted rather than demonstrated, and the trained TopK SAE is overcomplete by the paper's own definitions. These problems are load-bearing for the paper's abstract, §4, and §5 claims.

major comments (6)
  1. [§3.2, Eq. (3) and §4.1, Table 1] The stability evaluation is circular. The TEO loss in Eq. (3) explicitly minimizes λ1·(1/RIS) + λ2·(1/ROS) + λ3·Sparseness, and Table 1 then reports lower RIS/ROS for TEO-SAE as evidence of superiority. Lower values on these metrics are by construction, not an emergent property. Testing OOD does not break the circularity because the same metrics are measured after training. The paper needs independent evaluation criteria (e.g., faithfulness, sufficiency, expert labels, or a version of TEO trained without these stability terms) before the stability claim can be accepted.
  2. [§3.4, §3.6, Appendix A.5] The "monosemantic" claim is unvalidated and contradicted by the paper's own definitions. Appendix A.5 states that if F > d, W is not full-rank and superposition results; §3.6 reports TopK with 32× feature depth, so F ≫ d. Thus the trained SAE operates in the superposition regime by the manuscript's own criteria. No auto-interpretability scoring, manual feature labeling, or causal intervention is reported. The only argument—that an identity-like W would be monosemantic—does not apply to a trained overcomplete dictionary. The clinical interpretability conclusions in §4.4 and §5 presuppose semantic coherence that is never established.
  3. [§3.5, Eq. (5); Appendix A.7, Eq. (44)] The chain-rule attribution from SAE features to input tokens is not substantiated. The TopK encoder is non-differentiable: a = TopK(W_enc(x − b_dec)) has ∂a/∂x equal to zero almost everywhere and undefined at thresholds. Eq. (5) computes Φk as a sum of ψi·∂ai/∂x_input,k without discussing the TopK discontinuity or providing a surrogate-gradient treatment. Moreover, Eq. (45) propagates attributions with φ_enc = Wψ, which ignores the encoder nonlinearity. As written, the dual-level attribution pipeline is not a valid chain-rule derivation.
  4. [§3.3 and Appendix B.3] The UMAP linear constraint is validated circularly. The method imposes u_i1 = u_i2 for every point, forcing the two-dimensional UMAP embedding onto the diagonal line y = x. Appendix B.3 then uses PCA to show that the embedding is "approximately linear", but this is a direct consequence of the imposed equality constraint, not an emergent property. The PCA test verifies the constraint itself. In addition, the λ5 penalty introduced in §3.3 is not included in the total loss Eq. (3), so how it enters the TEO optimization is unclear.
  5. [Table 1, Binary OOD row] The claim of "strong cross-dataset stability" in §4.1 is not supported by the reported variability. For binary BrainLAT, TEO-SAE has RIS = 0.6835 ± 0.6676 and ROS/other values with standard deviations comparable to or larger than the means. These intervals heavily overlap other methods, so the superiority of TEO-SAE over OOD is not established. The paper should provide paired statistical tests and confidence intervals across all Table 1 blocks, not just selected IID comparisons.
  6. [§5 and Appendix B.1] The external ChatGPT evaluation is anecdotal. No quantitative metrics, scoring rubric, blinding, multiple seeds, or clinician validation are reported. The text summarizes GPT outputs narratively and concludes that SAE "substantially strengthens reliability, diagnostic validity, and clinical interpretability." This is a load-bearing claim for the paper's clinical framing, but the current evidence is a set of qualitative observations from a proprietary model and cannot support it. A structured evaluation with multiple raters and agreement statistics is required.
minor comments (6)
  1. [General] The text alternates between "Transformer-Based LM" and "LLM" and between "Layer Conduction" and "Layer Conductance" for the same method; please standardize terminology.
  2. [§3.3, Eq. (3)] Eq. (3) defines L_total with λ1–λ4, but §3.3 introduces λ5 in a penalty term without adding it to Eq. (3). Please clarify whether λ5 is part of L_total or a separate training stage.
  3. [References] Reference [35] is cited as ChatGPT-5.1 in the text but the reference entry is titled "ChatGPT-4o". Please correct the mismatch and describe the exact model version and access date.
  4. [Table 1] The table labels "Gradient Activation" and "Gradient Activation-SAE" omit the × operator; the main text uses "Gradient×Activation". Also, the header row breaks awkwardly and some entries are missing line breaks, making the table hard to read.
  5. [Appendix B.3 and Figure 2] "Approximately linear" is assessed visually. Please report a quantitative linearity measure (e.g., variance explained by PC1, residual variance, or a linear-fit R²) and define the threshold for "approximately linear".
  6. [Typos] Several typos appear: "Suplementary", "Apendix", "Spare auto-encoder", and "partec i pant" in the Figure 1 text. Please proofread.

Circularity Check

2 steps flagged · score 6.0 of 10

Stability and linearity results are partly by construction: the TEO optimizer's loss is RIS/ROS/sparseness, and the UMAP linear constraint is imposed before being 'verified' by PCA.

  1. fitted input called prediction [§3.2 Eq. 3; §4.1; Table 1]
    "The total cost function for training the reconstruction model is: Ltotal(phi^(k), phi_hat)=lambda_1 * 1/M RIS(f, phi_hat)+lambda_2 * 1/M ROS(f, phi_hat)+lambda_3 * M_sparse(f, phi_hat)+lambda_4 * L_similarity(phi_hat, phi_bar) (3) ... In the binary OOD case, this pattern persists and strengthens: TEO–SAE achieves the lowest RIS/ROS overall (strong cross-dataset stability), while TEO–UMAP recovers higher sparsity (>0.40) at a modest stability cost versus TEO–SAE, offering a tunable sparsity–stability trade-off."

    The optimizer's training objective (Eq. 3) explicitly minimizes 1/RIS, 1/ROS and sparseness. The headline results in Table 1/§4.1 are the same quantities—lowest RIS/ROS and a sparsity–stability trade-off. Thus the 'stability' result is the fitted objective reported as a finding; only the OOD transfer aspect is an independent claim, and it is not separated from the fitted metrics.

  2. self definitional [§3.3; Appendix B.3; Figure 2]
    "we introduce a linear constraint requiring equality between the first and second embedding components for each point, expressed as ui1 = ui2 ... To verify the claim that the proposed UMAP linear constraint effectively linearizes the majority of the attribution space ... The results (Figure 2) demonstrate that the proposed method indeed produces an embedding that approaches a linear configuration, thereby supporting our hypothesis."

    The equality ui1 = ui2 is imposed as a hard constraint/penalty during optimization. Appendix B.3 then uses PCA to 'verify' that the embedding is linear. Since every point lies on the line u1 = u2 by construction, the PCA result is a tautology, not an independent validation of a linear-structure claim.

full rationale

The framework has two genuine by-construction circularities. First, the TEO optimizer is trained with L_total = λ1/RIS + λ2/ROS + λ3·sparseness + λ4·similarity (Eq. 3), and the paper's main evidence (Table 1, §4.1) is that TEO-SAE achieves the lowest RIS/ROS and a sparsity–stability trade-off. These are the training objectives themselves, so the stability 'prediction' is partly fitted input renamed as result; only the OOD transfer is an independent (and unseparated) component. Second, the UMAP linear constraint ui1 = ui2 is imposed as a penalty/hard equality in §3.3, and Appendix B.3 'verifies' linear structure with PCA; that verification is true by construction. The self-citation to [15] for the cost-function design is not load-bearing because the equations are given. Separately, the central 'monosemantic' premise is asserted rather than established: §3.4/A.5 define monosemanticity via invertible, identity-aligned W and state that F>d leads to superposition, while the deployed TopK SAE uses 32× feature depth (§3.6), and no interpretability validation (auto-interp, interventions) is reported. This is a load-bearing validity gap, but it is not itself a by-construction reduction. The external GPT evaluation is independent in procedure but depends on the same attribution outputs. Overall score 6 reflects partial circularity in two central components.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The framework depends on untested semantic and metric assumptions; the trained SAE is overcomplete, so 'monosemantic' is not guaranteed, and the optimizer is evaluated on its own training metrics.

free parameters (6)
  • λ1, λ2, λ3, λ4 (TEO loss weights) = (0.1, 0.3, 0.1, 0.5)
    Eq. 3/39; chosen by hyperparameter tuning on ADNI validation; the paper states this configuration performed best.
  • λ5 (UMAP linear constraint strength) = not reported
    §3.3; penalty weight for (u_i1 - u_i2)^2; no value or sensitivity analysis given.
  • SAE width multiplier (TopK feature depth) = 32× (F = 32·d)
    §3.6; TopK with 32× feature depth performed best; not justified by feature-level scoring.
  • PC1 significance threshold = 0.6
    §4.4/B.2; features with first PCA component ≥ 0.6 are called biomarkers; arbitrary, no null model.
  • Top-50% token threshold for external evaluation = 50%
    §5; only top half of attributions fed to ChatGPT; no ablations.
  • Number of PCA components = 8
    Appendix B.3; first two of top eight shown; no variance threshold justification.
assumptions (6)
  • ad hoc to paper SAE latent features are monosemantic and semantically coherent.
    Central to the framework; §3.4/A.5 argue monosemanticity only for identity-like W, while the trained TopK SAE is overcomplete (F > d), which the paper itself says produces superposition; no feature interpretability validation.
  • domain assumption RIS, ROS, and Gini sparseness are valid, independent measures of explanation quality.
    Used both in training loss (Eq. 3) and evaluation (Table 1); they come from [24,25] but are not shown to correlate with clinical or human judgment.
  • domain assumption The linear representation hypothesis: interpretable features correspond to fixed directions in activation space.
    Invoked in A.5 for the SAE setup; standard hypothesis in mechanistic interpretability, not proven for ModernBERT layer 22.
  • ad hoc to paper The UMAP equality constraint u_i1 = u_i2 induces meaningful global linear structure rather than just a diagonal artifact.
    §3.3/B.3; this constraint is imposed and then 'validated' by PCA, which is circular.
  • domain assumption Clinical text generated from ADNI/BrainLAT tables preserves diagnostic information for the classifier and for attribution.
    The pipeline converts multimodal clinical data to text; no validation that the text generation did not introduce artifacts.
  • domain assumption The final (22nd) layer of ModernBERT is the appropriate layer for attribution.
    §3.1; chosen because ModernBERT had best classification accuracy; no layer-wise ablation for interpretability quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Monosemantic Attribution Framework for Stable Interpretability in Clinical Neuroscience Transformer-Based Language Models." pith.science (2026). https://pith.science/paper/SH3FFTM4

@misc{pith2026260117952,
  author       = {Pith},
  title        = {Pith review of: A Monosemantic Attribution Framework for Stable Interpretability in Clinical Neuroscience Transformer-Based Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SH3FFTM4}},
  note         = {Machine review of arXiv:2601.17952}
}
read the original abstract

Interpretability remains a key challenge for deploying language models (LM) in clinical settings such as progression diagnosis of Alzheimer disease, where early and trustworthy predictions are essential. Existing attribution methods exhibit high inter-method variability and unstable explanations due to the polysemantic nature of Transformer-Based LM and LLM representations, while mechanistic interpretability approaches lack direct alignment with model inputs and outputs and do not provide explicit importance scores. We introduce a unified interpretability framework that integrates attributional and mechanistic perspectives through monosemantic feature extraction. By constructing a monosemantic embedding space at the level of an transformer-based LM layer and optimizing the framework to explicitly reduce inter-method variability, our approach produces stable input-level importance scores and highlights salient features via a decompressed representation of the layer of interest, advancing the safe and trustworthy application of LMs in cognitive health and neurodegenerative disease.

Figures

Figures reproduced from arXiv: 2601.17952 by the authors.

Figure 1
Figure 1. Proposed interpretability framework for LLM in Alzheimer’s diagnosis. The model integrates k-attributional [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. PCA of token-level attribution representations (LMCI, ADNI test set). The first two principal components [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Left: Stability–sparsity frontier for explanation optimizers on ADNI (Late MCI) in the testing cohort. Scatter points show TEO, TEO–SAE, and TEO–UMAP (SAE) on ADNI (IID) and BrainLat (OOD). Metrics: Sparseness (higher is better) vs. RIS/ROS stability (lower is better). Middle: Token-level heatmap produced by the proposed framework, with feature-attribution scale (green: positive relevance; red: negative relevance; w… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: GPT-5.1 generated global explanations characterizing attribution-score performance in biomarker identifi [PITH_FULL_IMAGE:figures/full_fig_p025_4.png]
Figure 5
Figure 5. Figure 5: (continued) (c) Three-class ADNI: Control, MCI, and LMCI categories. [PITH_FULL_IMAGE:figures/full_fig_p026_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 7 linked inside Pith

  1. [1]

    Towards a rigorous science of interpretable machine learning.arXiv preprint arXiv:1702.08608, 2017

    Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning.arXiv preprint arXiv:1702.08608, 2017

    arXiv 2017

  2. [2]

    Trustworthy ai: A computational perspective.ACM Trans

    Haochen Liu, Yiqi Wang, Wenqi Fan, Xiaorui Liu, Yaxin Li, Shaili Jain, Yunhao Liu, Anil Jain, and Jiliang Tang. Trustworthy ai: A computational perspective.ACM Trans. Intell. Syst. Technol., 14(1), November 2022

    2022

  3. [3]

    Tahami Monfared, Melissa J

    Alireza A. Tahami Monfared, Melissa J. Byrnes, Lauren A. White, Jianwei Zhang, Eric Yu, and Jeng Lin. Alzheimer’ s disease: Epidemiology and clinical progression.Neurology and Therapy, 11:553–569, 2022

    2022

  4. [4]

    Nia-aa research framework: Toward a biological definition of alzheimer’ s disease

    Clifford R Jack et al. Nia-aa research framework: Toward a biological definition of alzheimer’ s disease. Alzheimer’s & Dementia, 14(4):535–562, 2018

    2018

  5. [5]

    Weiner, and Alzheimer’ s Disease Neuroimaging Initiative

    Garam Lee, Kwangsik Nho, Byungkon Kang, Kyung-Ah Sohn, Dokyoon Kim, Michael W . Weiner, and Alzheimer’ s Disease Neuroimaging Initiative. Predicting alzheimer’ s disease progression using multi-modal deep learning approach.Scientific Reports, 9(1):1952, 2019

    1952

  6. [6]

    Language models are few-shot learners

    Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. InAdvances in neural information processing systems, volume 33, pages 1877–1901, 2020

    1901

  7. [7]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron et al. Llama 2: Open foundation and fine-tuned chat models. https://ai.meta.com/llama/, 2023

    2023

  8. [8]

    Zoom in: An introduction to circuits.Distill, 2020

    Chris Olah, Arvind Satyanarayan, Ludwig Schubert Wusser, and Shan Carter. Zoom in: An introduction to circuits.Distill, 2020

    2020

Show all 51 references
  1. [9]

    Sparse autoencoders find highly interpretable features in language models, 2023

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models, 2023

    2023

  2. [11]

    Anders, and Klaus-Robert Müller

    Wojciech Samek, Grégoire Montavon, Sebastian Lapuschkin, Christopher J. Anders, and Klaus-Robert Müller. Explaining deep neural networks and beyond: A review of methods and applications.Proceedings of the IEEE, 109(3):247–278, 2021. 10 APREPRINT- JANUARY27, 2026

    2021

  3. [12]

    van der Velden, Hugo J

    Bas H.M. van der Velden, Hugo J. Kuijf, Kenneth G.A. Gilhuijs, and Max A. Viergever. Explainable artificial intelligence (xai) in deep learning-based medical image analysis.Medical Image Analysis, 79:102470, 2022

    2022

  4. [13]

    Explain: Explanatory artificial intelligence for diabetic retinopathy diagnosis.Medical Image Analysis, 72:102118, 2021

    Gwenolé Quellec, Hassan Al Hajj, Mathieu Lamard, Pierre-Henri Conze, Pascale Massin, and Béatrice Coch- ener. Explain: Explanatory artificial intelligence for diabetic retinopathy diagnosis.Medical Image Analysis, 72:102118, 2021

    2021

  5. [14]

    Michail Mamalakis, Krit Dwivedi, Michael Sharkey, Samer Alabed, David Kiely, and Andrew J. Swift. A trans- parent artificial intelligence framework to assess lung disease in pulmonary hypertension.Scientific Reports, 13(1):3812, 2023

    2023

  6. [15]

    Murray, John Suckling, and Pietro Lio

    Michail Mamalakis, Antonios Mamalakis, Ingrid Agartz, Lynn Egeland Mørch-Johnsen, Graham K. Murray, John Suckling, and Pietro Lio. Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks.AI Open, 6:70–81, 2025

    2025

  7. [16]

    The missing curve detectors of inceptionv1: Applying sparse autoencoders to inceptionv1 early vision.arXiv preprint arXiv:2406.03662, 2024

    Liv Gorton. The missing curve detectors of inceptionv1: Applying sparse autoencoders to inceptionv1 early vision.arXiv preprint arXiv:2406.03662, 2024

    2024 arXiv

  8. [17]

    A unified approach to interpreting model predictions.Advances in neural information processing systems, 30, 2017

    Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions.Advances in neural information processing systems, 30, 2017

    2017

  9. [18]

    Axiomatic attribution for deep networks, 2017

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks, 2017

    2017

  10. [19]

    Umap: Uniform manifold approximation and projection for dimension reduction.arXiv preprint arXiv:1802.03426, 2018

    Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction.arXiv preprint arXiv:1802.03426, 2018

    2018 arXiv

  11. [20]

    Karl Pearson F .R.S. Liii. on lines and planes of closest fit to systems of points in space.The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11):559–572, 1901

    1901

  12. [21]

    Mueller, Michael W

    Susanne G. Mueller, Michael W . Weiner, Leon J. Thal, Ronald C. Petersen, Clifford Jack, William Jagust, John Q. Trojanowski, Arthur W . Toga, and Laurel Beckett. The alzheimer’ s disease neuroimaging initiative.Neuroimag- ing Clinics of North America, 15(4):869–877, 2005. Alz...

    2005

  13. [22]

    Parra, Diana Matallana, Marcelo Adrián Maito, Adolfo M

    Pavel Prado, Vicente Medel, Agustín Sainz-Ballesteros, Hernando Santamaría-García, Sebastián Moguilner, Jhony Mejía, Raúl González-Gómez, Andrea Slachevsky, María Isabel Behrens, David Aguillón, Francisco Lopera, Mario A. Parra, Diana Matallana, Marcelo Adrián Maito, Adolfo M....

    2023

  14. [23]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions.CoRR, abs/1705.07874, 2017

    2017 arXiv

  15. [24]

    Rethinking stability for attribution-based explanations, 2022

    Chirag Agarwal, Nari Johnson, Martin Pawelczyk, Satyapriya Krishna, Eshika Saxena, Marinka Zitnik, and Himabindu Lakkaraju. Rethinking stability for attribution-based explanations, 2022

    2022

  16. [25]

    Concise explanations of neural networks using adversarial training

    Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Xi Wu, and Somesh Jha. Concise explanations of neural networks using adversarial training. In Hal Daumé III and Aarti Singh, editors,Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119 of...

    2020 arXiv

  17. [26]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 234–241. Springer, 2015

    2015

  18. [27]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017

    2017

  19. [28]

    Nguyen and Julian Salazar

    Toan Q. Nguyen and Julian Salazar. Transformers without tears: Improving the normalization of self-attention. arxiv, 2019

    2019

  20. [29]

    A mechanistic interpretability analysis of superposition in neural networks

    Nelson Elhage, Neel Nanda, et al. A mechanistic interpretability analysis of superposition in neural networks. Transformer Circuits Thread, 2022

    2022

  21. [30]

    Language models represent space and time, 2024

    Wes Gurnee and Max Tegmark. Language models represent space and time, 2024

    2024

  22. [31]

    Joseph Bloom, Curt Tigges, Anthony Duong, and David Chanin. Saelens. https://github.com/jbloomAus/ SAELens, 2024. 11 APREPRINT- JANUARY27, 2026

    2024

  23. [32]

    Anna Hedström, Leander Weber, Daniel Krakowczyk, Dilyara Bareeva, Franz Motzkus, Wojciech Samek, Se- bastian Lapuschkin, and Marina Marina M.-C. Höhne. Quantus: An explainable ai toolkit for responsible evaluation of neural network explanations and beyond.Journal of Machine Le...

    2023

  24. [33]

    Petersen, Glenn E

    Ronald C. Petersen, Glenn E. Smith, Susan C. Waring, Robert J. Ivnik, Eric G. Tangalos, and Emre Kokmen. Mild cognitive impairment: clinical characterization and outcome.Archives of Neurology, 56(3):303–308, 1999

    1999

  25. [34]

    Bondi, Emily C

    Mark W . Bondi, Emily C. Edmonds, Amy J. Jak, Lindsay R. Clark, Lisa Delano-Wood, Carrie R. McDonald, Daniel A. Nation, David J. Libon, Rhoda Au, Douglas Galasko, and David P . Salmon and. Neuropsychological criteria for mild cognitive impairment improves diagnostic precision,...

    2014

  26. [35]

    ChatGPT-4o.https://openai.com/chatgpt, 2024

    OpenAI. ChatGPT-4o.https://openai.com/chatgpt, 2024. Accessed May 2025

    2024

  27. [36]

    Denoising diffusion probabilistic models.arXiv preprint arxiv:2006.11239, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.arXiv preprint arxiv:2006.11239, 2020

    2006 arXiv

  28. [37]

    Invariant visual representation by single neurons in the human brain.Nature, 435(7045):1102–1107, 2005

    Rodrigo Quiroga et al. Invariant visual representation by single neurons in the human brain.Nature, 435(7045):1102–1107, 2005

    2005

  29. [38]

    The importance of mixed selectivity in complex cognitive tasks.Nature, 497(7451):585–590, 2013

    Mattia Rigotti, Omri Barak, Melissa Warden, et al. The importance of mixed selectivity in complex cognitive tasks.Nature, 497(7451):585–590, 2013

    2013

  30. [39]

    Zoom in: An introduction to circuits.Distill, 5(3):e00024.001, 2020

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits.Distill, 5(3):e00024.001, 2020

    2020

  31. [40]

    Sparse feature cir- cuits: Discovering and editing interpretable causal graphs in language models.arXiv preprint arXiv:2403.19647, 2024

    Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature cir- cuits: Discovering and editing interpretable causal graphs in language models.arXiv preprint arXiv:2403.19647, 2024

    2024 arXiv

  32. [41]

    Sae visualizer, 2024

    Callum McDougall. Sae visualizer, 2024

    2024

  33. [42]

    Towards monosemanticity: Decomposing language models with dictionary learning.Transformer Circuits Thread, 2023

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning.Transformer Circuits Thread, 2023

    2023

  34. [43]

    Language models can explain neurons in language models.openaipublic, 2023

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models.openaipublic, 2023

    2023

  35. [44]

    Scaling monosemanticity: Extracting interpretable features from claude 3.Transformer Circuits Thread, 2024

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, et al. Scaling monosemanticity: Extracting interpretable features from claude 3.Transformer Circuits Thread, 2024

    2024

  36. [45]

    Improving dictionary learning with gated sparse autoencoders, 2024

    Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. Improving dictionary learning with gated sparse autoencoders, 2024

    2024

  37. [46]

    Mechanistic interpretability for ai safety – a review, 2024

    Leonard Bereska and Efstratios Gavves. Mechanistic interpretability for ai safety – a review, 2024

    2024

  38. [47]

    why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should i trust you?": Explaining the predictions of any classifier, 2016

    2016

  39. [48]

    On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation

    Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Woj- ciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE, 10(7):1–46, 07 2015

    2015

  40. [49]

    Explainable deep learning models in medical image analysis, 2020

    Amitojdeep Singh, Sourya Sengupta, and Vasudevan Lakshminarayanan. Explainable deep learning models in medical image analysis, 2020

    2020

  41. [50]

    Approximating the shapley value without marginal contributions, 2024

    Patrick Kolpaczki, Viktor Bengs, Maximilian Muschalik, and Eyke Hüllermeier. Approximating the shapley value without marginal contributions, 2024

    2024

  42. [51]

    Toy models of superposition, 2022

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield- Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superpos...

    2022

  43. [52]

    grandmother cells

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads.Transformer Circuits Thread, 2022. 12 APREPRINT- JANUARY27, 2026 A Technical Appendices A.1...

    2022

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.