Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes a concept-decomposed alignment measure and reports that increased supervision correlates with reduced semantic structure in the late layers of vision transformers.

desk verdict The method is a real step forward for concept-level alignment, but the headline claim about supervision and semantic structure rests on the one layer where the paper's own UMAP-fidelity check fails. read the letter →

arxiv 2412.06639 v1 pith:UCJLQBHE submitted 2024-12-09 cs.CV cs.AIcs.LG

Johanna Vielhaben , Dilyara Bareeva , Jim Berend , Wojciech Samek , Nils Strodthoff This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords representationalalignmentconceptdiscoveryvisiontransformersnonlinearmanifoldconceptsUMAPandHDBSCANgeneralizedRandindexsupervisionsemanticstructureatlas
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing representational alignment measures, such as Centered Kernel Alignment (CKA), return a single scalar and cannot say which features two layers or models share. This paper replaces the scalar with a concept-level description: concepts are defined as arbitrary nonlinear manifolds in a network's feature space, discovered by density-based clustering on a UMAP embedding, and alignment is measured with a generalized Rand index that splits into per-concept distances. Applied to four same-architecture vision transformers trained with different objectives (fully supervised, CLIP, DINO, MAE), the analysis shows a consistent pattern: more supervision correlates with less semantic structure in the final layers, with the fully supervised model undergoing a sharp 'nucleation' into class-specific concepts around layers 9-10. The payoff for practitioners is a way to see, rather than only score, how a model organizes what it has learned.

What carries the argument

The key machinery is the concept manifold paired with a decomposable alignment distance. A concept $C^\alpha$ is defined as a manifold in the $F$-dimensional feature space, represented by the point cloud of feature vectors lying on it; this generalizes earlier definitions of concepts as linear directions or linear subspaces. The discovery pipeline embeds feature vectors with UMAP, clusters them with HDBSCAN, and converts cluster structure into soft concept-proximity scores $P^\alpha(\phi)$. Alignment between two such fuzzy clusterings is measured by the generalized Rand index $CBA = 1 - d_{\mathrm{cross}}(P,Q)$, and because the pairwise disagreement can be written per concept pair, the scalar alignment becomes a full pairwise distance matrix between concepts. This matrix is what produces the 'concept atlases' showing which concepts are universal, unique, or internally structured.

What would settle it

Compute the same concept-based alignment using clusters obtained directly in the original 768-dimensional feature space, or with an embedding whose distance-preservation error is verified to be low at every layer, and check whether the fully supervised model's layer-11 representation is still less semantically organized than CLIP, DINO, and MAE. If the supervision-structure gradient disappears under this check, the paper's central finding fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that alignment can be meaningfully decomposed by concept, and that this decomposition exposes a supervision-structure gradient in vision transformers. Using soft concept proximity scores obtained from nonlinear manifold clustering, it shows that the fully supervised (FS), CLIP, DINO, and MAE models all look similar in early layers, the first half of the network being the most aligned between any pair of models, but diverge strongly in later layers. At layer 11 the concepts of CLIP, DINO, and MAE form semantically coherent atlases, such as separated canine concepts or grouped human body parts, while the FS representation is markedly less semantically organized. The paper reads this as evidence that increased supervision correlates with a reduction in the semantic structure of learned representations, and it ties the FS model's sudden reorganization between layers 9 and 10 to a nucleation process.

Load-bearing premise

The load-bearing premise is that the UMAP embedding preserves the feature-space geometry well enough that clusters found in the embedding correspond to real concept manifolds; the paper itself reports that for the fully supervised model the embedding starts distorting distances after layer 8, so the layer-11 structural claim inherits that risk.

Editorial extensions

If this is right

  • Concept-based alignment can identify which concepts are universal across models or layers and which are unique, for example low-distance pairs like zebra stripes and high-distance pairs like complex high-frequency textures between DINO and MAE.
  • The sanity check indicates that nonlinear-manifold concepts give more faithful alignment than PCA, MCD, or KMeans concepts, especially for CLS-token representations.
  • The fully supervised model's sharp reorganization between layers 9 and 10 is a nucleation event into class-specific concepts, implying that its last blocks are highly specialized and poorly aligned with earlier blocks.
  • Early layers of different ViTs are much more aligned than late layers, so foundational visual features are learned similarly across training objectives while task-specific specialization appears in the second half.
  • CBA makes layer-wise concept atlases directly inspectable, allowing practitioners to see semantic separation such as canine concepts or body parts rather than reading a single number.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference beyond the paper: the same pipeline could be run on convolutional backbones and larger ViT variants to test whether the supervision-structure relationship is architectural or general.
  • Inference: the concept atlas suggests a model-selection criterion, namely to prefer a pretrained model whose late-layer concepts remain semantically organized, since that organization may transfer better than raw accuracy alone.
  • Inference: one could fine-tune a self-supervised model with increasing amounts of label supervision and track the layer-11 atlas, which would test whether the loss of semantic structure is a continuous function of supervision strength.
  • Inference: because ImageNet-1k's label distribution can bias which concepts appear, the paper's own limitation example being a color concept visible only as dog patches, a test on a less biased image collection would sharpen the semantic-structure claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Concept-Based Alignment (CBA), a representational alignment measure that decomposes the usual scalar alignment into per-concept contributions. Concepts are defined as nonlinear manifolds in feature space, discovered by applying HDBSCAN to UMAP embeddings of ViT hidden features, and represented by soft membership scores. CBA is built on a generalized Rand index with pseudo-metric properties, and the paper provides an upper-bound decomposition into pairwise concept distances. The method is validated through a neighboring-layer sanity check against linear and spherical concept baselines, and it is then applied to compare four ViTs (FS, CLIP, DINO, MAE) across layers, leading to the conclusion that increased supervision correlates with a reduction in the semantic structure of learned representations. The paper includes public code and a detailed appendix describing the HDBSCAN soft-clustering procedure.

Significance. The idea of decomposing scalar representational alignment into concept-level contributions is valuable, and the definition of concepts as arbitrary manifolds is a meaningful generalization of existing linear concept definitions. The upper-bound decomposition in Eq. (4) is mathematically sound, and the paper ships reproducible code and an external sanity check that does not fit any parameter of the proposed method. If the empirical conclusions survive the additional validation requested below, the approach offers a genuinely new tool for comparing vision transformer representations. However, the central empirical claim currently rests on clustering in a low-dimensional embedding whose fidelity is worst for the model and layer that drive the conclusion, so the significance of the paper depends on the robustness checks described in the major comments.

major comments (3)
  1. [Sec. 4.1 / Fig. 2 vs. Sec. 4.3 / Fig. 4] The central empirical claim that the fully supervised (FS) representation at layer 11 is 'notably less semantically organized' than the other models is supported only by visual inspection of concept atlases, which are UMAP embeddings of concept-pair distances derived from clusters in the 50-dimensional UMAP embedding. Fig. 2 shows that the RMSE between the original and embedded distance matrices for FS rises strongly from layer 8 onward, and Sec. 4.1 states that 'we can trust the clustering on the embedding less' for FS. Layer 11 is exactly the layer used to conclude that FS is less semantically structured. Because clusters and soft membership scores are computed inside the embedding, the observed lack of structure may be an embedding artifact. Please provide a quantitative semantic-organization metric computed in the original feature space (e.g., concept-label agreement or a clustering-validity index) for layer 11, and verify that the FS versus CLIP/DINO/MAE difference persists under this metric. The class-label alignment curves in Fig. 5 are also presented without error bars and are not quantitatively connected to the atlas interpretation.
  2. [Appendix A, Eq. (12) and Sec. 2.2] All concept proximity scores P^alpha(phi) are defined from exemplar-point distances and persistence values in the UMAP embedding, so every CBA value inherits the embedding geometry. The manuscript defends this choice by the practical infeasibility of density clustering in 768 dimensions, but it provides no evidence that the embedding is faithful for the FS late layers beyond the global RMSE, which is worst in exactly that regime. Please validate the clustering and alignment results against an alternative approach that does not depend on the UMAP geometry, for example by computing CBA on distances in the original feature space to concept exemplars, or by perturbing UMAP hyperparameters and showing that the FS layer-11 conclusion is stable.
  3. [Sec. 4.2 / Table 2 and Abstract] The abstract claims to 'confirm the superiority' of the nonlinear concept definition for alignment analysis, but Table 2 shows that for DINO SEQ the PCA baseline achieves a score of 1.0 while NLMCD achieves 0.90, and all scores are reported without error bars. The main-text statement correctly notes this exception, so the abstract overstates the result. Please report variability across clustering/embedding runs and either soften the superiority claim or provide a statistical test showing that NLMCD is significantly better overall.
minor comments (4)
  1. [Fig. 4 caption] The caption states 'CLIP, DINO and FS' in the description of the block structure, but the surrounding text and the figure itself indicate that the third model should be MAE; please correct this typo.
  2. [Eq. (4)] The line break and nested absolute values in Eq. (4) make the formula hard to parse; please add parentheses to clarify that the outer absolute value applies to the difference of the two per-cluster absolute differences.
  3. [Sec. 4.1] The sentence 'which has a high DBCV but low robustness' is a sentence fragment; it should be integrated into the preceding clause for readability.
  4. [Appendix C] The concept formation graph is described as 'binary, unidirectional'; 'directed' is the standard term here, and the binary property should be defined explicitly.

Circularity Check

0 steps flagged · score 1.0 of 10

No meaningful circularity: the method is validated against an independent neighbor-layer sanity check, the sole self-citation (MCD [39]) is a non-load-bearing baseline, and the flagged UMAP-fidelity weakness at FS layer 11 is a validity concern rather than a derivation-by-construction.

full rationale

The derivation chain is self-contained and validated against an external criterion. Concepts are defined as manifolds (Definition 1) and discovered in Sec. 2.2 by HDBSCAN on UMAP embeddings; the soft proximity scores P^alpha(phi) are formalized in Appendix A from standard HDBSCAN soft clustering [28], and CBA is defined from the generalized Rand distance of Eqs. 1-3. No parameter is fitted to the paper's central conclusions: UMAP/HDBSCAN hyperparameters are tuned to maximize DBCV, while the supporting evaluations (neighbor-layer sanity check of Sec. 4.2, RMSE, robustness in Fig. 2) are not tuning targets, and the sanity-check assumption ('neighboring representations should be most aligned') enters no fitted quantity. The only self-citation, MCD [39], appears as a comparison baseline in the sanity check and as related-work context; it is not invoked as a uniqueness theorem, and the manifold concept definition is justified by external references to the manifold hypothesis. The paper's own flagged weakness — that for FS, RMSE increases strongly from layer 8 onward and 'we can trust the clustering on the embedding less,' precisely where the reduced-semantic-structure claim is read from the layer-11 atlas (Fig. 4, Sec. 4.3) — is an empirical validity and robustness concern about the evidence, not a circularity: the claim is a qualitative visual reading, not a quantity forced by construction, and no equation in the paper reduces to its own input. Fragility of the FS layer-11 evidence should therefore be weighed as correctness risk, not as circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a pipeline of UMAP, HDBSCAN, and a generalized Rand index. The hyperparameters are tuned, not derived, and several domain assumptions (manifold hypothesis, embedding fidelity, soft membership validity) are load-bearing. No new physical entities are introduced; the concept manifold is a modeling construct.

free parameters (6)
  • UMAP min_dist = 0.01
    Tuned to enhance local cluster density; influences concept separation and downstream alignment.
  • UMAP n_neighbors = 30
    Controls local vs global structure; affects clustering and hence the alignment measure.
  • UMAP embedding dimensionality = 50
    Practical limit for HDBSCAN; affects fidelity of the embedding and concept discovery.
  • HDBSCAN min_cluster_size = 50
    Tuned to balance noise identification and cluster merging; directly shapes the concept set.
  • HDBSCAN min_samples = 20
    Controls conservativeness about noise; set low because some concept manifolds are sparsely sampled.
  • Data subsampling fraction = 25% of ImageNet train; 20% of feature vectors for CBA
    The analysis uses a random subset of ImageNet and a further subsample for alignment, which affects stability and results.
assumptions (5)
  • domain assumption The manifold hypothesis: hidden representations concentrate on low-dimensional manifolds.
    Invoked in Sec. 2.1 as the motivation for defining concepts as manifolds and for clustering in embedded space.
  • domain assumption UMAP preserves sufficient local and global structure of the feature space.
    Sec. 2.2 uses UMAP embeddings for HDBSCAN; the paper acknowledges distortion via RMSE but assumes clusters in the embedding map to true concept manifolds.
  • domain assumption HDBSCAN soft membership scores reflect true concept proximity.
    The concept proximity scores in Sec. 2.2 and appendix A are treated as probabilities of belonging to concept manifolds, which underpins the CBA measure.
  • standard math The generalized Rand index is a valid pseudo-metric for fuzzy clusterings.
    Adopted from Hullermeier et al. (2012) in Sec. 2.3; the paper relies on its pseudo-metric properties for interpretation.
  • domain assumption Neighboring layers should be most aligned.
    Used as the sanity check in Sec. 4.2 to validate NLMCD against baselines; assumes layerwise representations evolve smoothly.
invented entities (1)
  • Concept manifold C^alpha independent evidence
    purpose: To summarize a hidden representation as a set of nonlinear manifolds with proximity scores, enabling fine-grained alignment.
    Concepts are discovered from data via clustering and are visualized in concept atlases; the sanity check provides behavioral evidence, but they are not independently measured physical entities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers." pith.science (2026). https://pith.science/paper/UCJLQBHE

@misc{pith2026241206639,
  author       = {Pith},
  title        = {Pith review of: Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UCJLQBHE}},
  note         = {Machine review of arXiv:2412.06639}
}
read the original abstract

Vision transformers (ViTs) can be trained using various learning paradigms, from fully supervised to self-supervised. Diverse training protocols often result in significantly different feature spaces, which are usually compared through alignment analysis. However, current alignment measures quantify this relationship in terms of a single scalar value, obscuring the distinctions between common and unique features in pairs of representations that share the same scalar alignment. We address this limitation by combining alignment analysis with concept discovery, which enables a breakdown of alignment into single concepts encoded in feature space. This fine-grained comparison reveals both universal and unique concepts across different representations, as well as the internal structure of concepts within each of them. Our methodological contributions address two key prerequisites for concept-based alignment: 1) For a description of the representation in terms of concepts that faithfully capture the geometry of the feature space, we define concepts as the most general structure they can possibly form - arbitrary manifolds, allowing hidden features to be described by their proximity to these manifolds. 2) To measure distances between concept proximity scores of two representations, we use a generalized Rand index and partition it for alignment between pairs of concepts. We confirm the superiority of our novel concept definition for alignment analysis over existing linear baselines in a sanity check. The concept-based alignment analysis of representations from four different ViTs reveals that increased supervision correlates with a reduction in the semantic structure of learned representations.

Figures

Figures reproduced from arXiv: 2412.06639 by the authors.

Figure 1
Figure 1. We combine concept discovery with alignment analy [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We evaluate the quality of concept discovery. RMSE [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Concept formation graph for the concept “apple(s)” in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Intra-model relationships based on SEQ representations across layers. In the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Class label alignment, token location alignment (both [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Inter-model relationships based on SEQ representations [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Concept formation graph for a concept in layer 9 of [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Concept formation graph for a concept in layer 12 of [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Concept formation graph for a concept in layer 10 of [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Intra-model relationships based on CLS representations across layers. In the [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Class label alignment, token location alignment (both [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 13
Figure 13. Figure 13: We zoom into the SEQ representations at DINO layer 1 and show a UMAP embedding constructed from the pairwise distance [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: We zoom into the SEQ representations at DINO layer 6 and show a UMAP embedding constructed from the pairwise distance [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: We zoom into the SEQ representations at DINO layer 11 and show a UMAP embedding constructed from the pairwise distance [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 17
Figure 17. Figure 17: CBA of SEQ concepts across layers of MAE and DINO [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mechanistic understanding and validation of large AI models with SemanticLens

    cs.LG 2025-01 conditional novelty 6.0 of 10

    SemanticLens maps each neuron of a vision model to a CLIP-space vector, enabling text-based search, labelling, audit, and interpretability scoring of model internals.

Reference graph

Works this paper leans on

46 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    Network dissection: Quantifying inter- pretability of deep visual representations

    David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying inter- pretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 6541–6549, 2017. 2

    work page 2017

  2. [2]

    Mechanistic inter- pretability for AI safety - a review.Transactions on Machine Learning Research, 2024

    Leonard Bereska and Stratis Gavves. Mechanistic inter- pretability for AI safety - a review.Transactions on Machine Learning Research, 2024. Survey Certification, Expert Cer- tification. 2

    work page 2024

  3. [3]

    Towards monosemanticity: Decomposing language mod- els with dictionary learning

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yi- fan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Ka- rina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, an...

  4. [4]

    Ricardo J. G. B. Campello, Davoud Moulavi, and J ¨org Sander. Density-based clustering based on hierarchical den- sity estimates. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2013. 3

    2013

  5. [5]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9650–9660, 2021. 4

    work page 2021

  6. [6]

    Concept acti- vation regions: A generalized framework for concept-based explanations

    Jonathan Crabb ´e and Mihaela van der Schaar. Concept acti- vation regions: A generalized framework for concept-based explanations. In Advances in Neural Information Processing Systems, pages 2590–2607. Curran Associates, Inc., 2022. 4

    work page 2022

  7. [7]

    Recurrent neural networks learn to store and generate sequences using non-linear representations

    R ´obert Csord´as, Christopher Potts, Christopher D Manning, and Atticus Geiger. Recurrent neural networks learn to store and generate sequences using non-linear representations. In The 7th BlackboxNLP Workshop, 2024. 2

    work page 2024

  8. [8]

    Ryan DeWolfe and Jeffery L. Andrews. Random mod- els for fuzzy clustering similarity measures. ArXiv, abs/2312.10270, 2023. 3

    work page Pith review arXiv 2023

Show all 46 references
  1. [9]

    Hierarchical nucleation in deep neural networks

    Diego Doimo, Aldo Glielmo, Alessio Ansuini, and Alessan- dro Laio. Hierarchical nucleation in deep neural networks. In Proceedings of the 34th International Conference on Neu- ral Information Processing Systems, pages 7526–7536, Red Hook, NY , USA, 2020. Curran Associates Inc. 4, 7

    2020

  2. [10]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  3. [11]

    Michaud, Wes Gurnee, and Max Tegmark

    Joshua Engels, Isaac Liao, Eric J. Michaud, Wes Gurnee, and Max Tegmark. Not all language model features are linear. ArXiv, abs/2405.14860, 2024. 4

    2024 arXiv

  4. [12]

    Estimating the intrinsic dimension of datasets by a minimal neighborhood information

    Elena Facco, Maria d’Errico, Alex Rodriguez, and Alessan- dro Laio. Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific Reports, 7(1): 12140, 2017. 8, 16

    2017

  5. [13]

    A holistic approach to unifying automatic concept extraction and concept importance estimation

    Thomas Fel, Victor Boutin, Louis B ´ethune, Remi Ca- dene, Mazda Moayeri, L ´eo And´eol, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. In Advances in Neural Information Processing Systems , ...

    2023

  6. [14]

    Craft: Concept recursive activation factor- ization for explainability

    Thomas Fel, Agustin Picard, Louis B ´ethune, Thibaut Boissin, David Vigouroux, Julien Colin, R ´emi Cad`ene, and Thomas Serre. Craft: Concept recursive activation factor- ization for explainability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2023

  7. [15]

    Scaling and evaluating sparse autoencoders,

    Leo Gao, Tom Dupr ´e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders,

  8. [16]

    Towards automatic concept-based explanations

    Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2019. 4

    2019

  9. [17]

    Clustering and dimensional- ity reduction on Riemannian manifolds

    Alvina Goh and Rene Vidal. Clustering and dimensional- ity reduction on Riemannian manifolds. In 2008 IEEE Con- ference on Computer Vision and Pattern Recognition, pages 1–7, 2008. ISSN: 1063-6919. 3

    2008

  10. [18]

    Battle of the backbones: A large-scale comparison of pretrained models across computer vision tasks

    Micah Goldblum, Hossein Souri, Renkun Ni, Manli Shu, Viraj Prabhu, Gowthami Somepalli, Prithvijit Chattopad- hyay, Mark Ibrahim, Adrien Bardes, Judy Hoffman, Rama- lingam Chellappa, Andrew Gordon Wilson, and Tom Gold- stein. Battle of the backbones: A large-scale comparison of...

    arXiv

  11. [19]

    Deep Learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning . MIT Press, 2016. http : / / www . deeplearningbook.org. 2

    2016

  12. [20]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16000–16009, 2022. 4

    2022

  13. [21]

    Enhancing cluster analysis via topological manifold learning

    Moritz Herrmann, Daniyal Kazempour, Fabian Scheipl, and Peer Kr ¨oger. Enhancing cluster analysis via topological manifold learning. Data Mining and Knowledge Discovery,

  14. [22]

    Sparse autoencoders find highly interpretable features in language models

    Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representa- tions, 2024. 4

    2024

  15. [23]

    Comparing fuzzy partitions: A generalization of the rand index and related measures

    Eyke Hullermeier, Maria Rifqi, Sascha Henzgen, and Robin Senge. Comparing fuzzy partitions: A generalization of the rand index and related measures. IEEE Transactions on Fuzzy Systems, 20(3):546–556, 2012. 2, 3

    2012

  16. [24]

    Simon Kornblith, Jonathon Shlens, and Quoc V . Le. Do better imagenet models transfer better? 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2656–2666, 2018. 4

    2019

  17. [25]

    Similarity of neural network represen- tations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network represen- tations revisited. In International conference on machine learning, pages 3519–3529. PMLR, 2019. 2, 4

    2019

  18. [26]

    Michaud, Yonatan Be- linkov, David Bau, and Aaron Mueller

    Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Be- linkov, David Bau, and Aaron Mueller. Sparse feature cir- cuits: Discovering and editing interpretable causal graphs in language models, 2024. 4

    2024

  19. [27]

    Umap: Uniform mani- fold approximation and projection for dimension reduction

    Leland McInnes and John Healy. Umap: Uniform mani- fold approximation and projection for dimension reduction. ArXiv, abs/1802.03426, 2018. 3

    2018 arXiv

  20. [28]

    hdbscan: Hierarchical density based clustering

    Leland McInnes, John Healy, and Steve Astels. hdbscan: Hierarchical density based clustering. The Journal of Open Source Software, 2(11), 2017. 3, 11

    2017

  21. [29]

    George A. Miller. Wordnet: a lexical database for english. Commun. ACM, 38(11):39–41, 1995. 7

    1995

  22. [30]

    Jaskowiak, Ricardo J

    Davoud Moulavi, Pablo A. Jaskowiak, Ricardo J. G. B. Campello, Arthur Zimek, and J ¨org Sander. Density-based clustering validation. In SDM, 2014. 5

    2014

  23. [31]

    Topology of deep neural networks

    Gregory Naitzat, Andrey Zhitnikov, and Lek-Heng Lim. Topology of deep neural networks. The Journal of Machine Learning Research, 21(1):184:7503–184:7542, 2020. 2

    2020

  24. [32]

    What do self-supervised vision transform- ers learn? ArXiv, abs/2305.00729, 2023

    Namuk Park, Wonjae Kim, Byeongho Heo, Taekyung Kim, and Sangdoo Yun. What do self-supervised vision transform- ers learn? ArXiv, abs/2305.00729, 2023. 4

    2023 arXiv

  25. [33]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  26. [34]

    Do vision trans- formers see like convolutional neural networks? In Neural Information Processing Systems, 2021

    Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision trans- formers see like convolutional neural networks? In Neural Information Processing Systems, 2021. 1, 4

    2021

  27. [35]

    Ma- chine learning in python: Main developments and technol- ogy trends in data science, machine learning, and artificial intelligence

    Sebastian Raschka, Joshua Patterson, and Corey Nolet. Ma- chine learning in python: Main developments and technol- ogy trends in data science, machine learning, and artificial intelligence. arXiv preprint arXiv:2002.04803, 2020. 12

    2002 arXiv

  28. [36]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision , ...

  29. [37]

    How to train your vit? data, augmentation, and regularization in vision transformers

    Andreas Peter Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. Transactions on Machine Learning Re- search, 2022. 4

    2022

  30. [38]

    Love, Erin Grant, Jascha Achterberg, Joshua B

    Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C. Love, Erin Grant, Jascha Achterberg, Joshua B. Tenenbaum, Kather- ine M. Collins, Katherine L. Hermann, Kerem Ok- tar, Klaus Greff, Martin N. Hebart, Nori Jacoby, Qiuyi Zhang, Ra...

    arXiv

  31. [39]

    Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees

    Johanna Vielhaben, Stefan Bluecher, and Nils Strodthoff. Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees. Transactions on Machine Learning Research, 2023. 2, 3, 4, 6

    2023

  32. [40]

    Teaching matters: Investigating the role of supervision in vision transformers

    Matthew Walmer, Saksham Suri, Kamal Gupta, and Abhi- nav Shrivastava. Teaching matters: Investigating the role of supervision in vision transformers. 2023 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 7486–7496, 2022. 1, 4

    2023

  33. [41]

    Pytorch image models

    Ross Wightman. Pytorch image models. https : / / github . com / rwightman / pytorch - image - models, 2019. 12 10

    2019

  34. [42]

    Ehinger, and Benjamin I

    Ruihan Zhang, Prashan Madumal, Tim Miller, Krista A. Ehinger, and Benjamin I. P. Rubinstein. Invertible concept- based explanations for cnn models with non-negative con- cept activation vectors. Proceedings of the AAAI Conference on Artificial Intelligence, 35(13):11682–11690,...

    2021

  35. [44]

    Soft assignments are thresholded

    Token assignment: We begin by assigning each token in each layer to either one or more concepts or marking it as noise. Soft assignments are thresholded

  36. [45]

    Each matrix entry represents the count of to- kens transitioning from a concept in layer n to a concept in layer n + 1

    Transition matrix calculation: Next, we compute transition matrices for each pair of consecutive layers, (n, n+ 1). Each matrix entry represents the count of to- kens transitioning from a concept in layer n to a concept in layer n + 1

  37. [46]

    contribution

    Recursive graph construction: The CFG for a tar- get node is then constructed recursively. Starting from the target node, we add all predecessor concepts whose “contribution” (the proportion of incoming transitions) surpasses a specified threshold. The resulting CFG is a binar...

  38. [2023]

    https://transformer-circuits.pub/2023/monosemantic- features/index.html. 4

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.