Pith. sign in

REVIEW 5 major objections 5 minor 32 references

Concept Boundary Vectors

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces concept boundary vectors, constructed from the boundary between latent representations of concepts, and shows empirically that they encode concept-concept relationships more effectively than concept activation vectors.

desk verdict The new construction is plausible, but the headline CBV-vs-CAV comparisons are confounded by vector norm, and the independent evidence does not yet rescue the central claim. read the letter →

arxiv 2412.15698 v1 pith:HCCPAHBX submitted 2024-12-20 cs.LG

Thomas Walker This is my paper · ORCID
classification cs.LG
keywords conceptboundaryvectorsactivationlatentspacegeometryinterpretabilitylogitinfluencealgebratopologicaldataanalysisvisiontransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces concept boundary vectors, a construction that derives a concept's direction in a neural network's latent space from the boundary between two concepts' latent activations. Rather than training a classifier to separate positive and negative examples, the method pairs up mutually nearest activations across the two sets, normalizes their differences into boundary normal vectors, and optimizes a single vector to have maximum average cosine similarity with those normals. The paper claims this boundary-derived vector encodes the concept-concept relationship more faithfully than concept activation vectors, and tests the claim on an MNIST classifier and a CIFAR-10 vision transformer using logit influence, concept algebra, and adversarial perturbation experiments. A sympathetic reader would care because concept vectors are a standard interpretability device, so a more faithful construction could make model explanations more trustworthy.

What carries the argument

The load-bearing object is the set of boundary normal vectors $N_{\pm}$, obtained from mutually nearest-neighbor pairs $(a_+, a_-)$ across the positive and negative concept activation sets, normalized to unit length. A single vector is then optimized to maximize the average cosine similarity with $N_{\pm}$, and that optimized vector is the concept boundary vector. This construction directly respects the geometry of the separating boundary, whereas a concept activation vector is the weight vector of a logistic regression classifier and is agnostic to where the boundary points actually lie. The paper also relies on the logit influence metric, the average directional derivative of the target-class logit along the concept vector, as the main quantitative measure of how well a concept vector represents a concept relationship.

What would settle it

Take a concept pair whose boundary normal vectors split into two or more well-separated clusters with distinctly different directions. If the homogeneity assumption fails, the single optimized boundary vector will align poorly with each cluster, and its logit influence on the target class should drop to the level of (or below) a concept activation vector. Concretely, cluster $N_{\pm}$ by cosine similarity into two groups, construct per-group boundary vectors, and compare their logit influences against the single global boundary vector; a large gap would falsify the claim that one boundary vector suffices.

Watch

Extended reading notes

Core claim

The central claim is that the geometry of the boundary between latent concept clusters is a major carrier of semantic relationship, and that a vector explicitly aligned with boundary normals captures that relationship better than the weight vector of a separating linear classifier. Concept boundary vectors are defined by first constructing the set $N_{\pm}$ of unit vectors pointing from a negative activation to its paired positive activation at the boundary, then optimizing a single vector $v_{\pm}$ to maximize the average cosine similarity to $N_{\pm}$. Empirically, these vectors produce higher logit influence on the target class and lower influence on the source class, satisfy concept algebra $v_{1,2} + v_{2,3} \approx v_{1,3}$ more often, and generate smaller adversarial perturbations that still flip the model's classification. The paper further shows that logit influence decreases as the topological complexity of the boundary increases, and that concept vectors are more effective when concept activations are internally homogeneous.

Load-bearing premise

The construction assumes that the boundary normal vectors between two concepts are homogeneous enough that one single vector, chosen to maximize average cosine similarity, can faithfully represent the concept relationship.

Editorial extensions

If this is right

  • Concept boundary vectors give a higher logit influence on the target class and a stronger suppression of the source class than concept activation vectors on nearly all MNIST digit pairs.
  • Linear concept algebra succeeds more often with concept boundary vectors, meaning the vector sum $v_{1,2} + v_{2,3}$ lands closer to $v_{1,3}$ than it does for activation vectors.
  • Adversarial inputs built from concept boundary vectors need smaller perturbations to reclassify boundary latents, and that improvement extends to the whole concept cluster, not just the boundary points.
  • Across layers of a vision transformer, concept boundary vectors are more consistent between layers than concept activation vectors, with consistency error peaking at the middle layers.
  • Topological boundary complexity is negatively correlated with concept vector effectiveness, supporting the paper's claim that boundary geometry matters for concept representations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper validates homogeneity for concept clusters but never for the boundary normal vectors themselves, a direct extension is to fit a mixture of boundary vectors per concept pair when $N_{\pm}$ is multimodal; this would test whether a single vector is always sufficient.
  • The same boundary-normal construction could be applied to unsupervised feature dictionaries, aligning sparse autoencoder features to boundary normals rather than to activation directions, potentially yielding more interpretable feature-concept correspondences.
  • The negative correlation between logit influence and boundary complexity hints that training with margin or boundary-sharpening losses could make latent spaces more concept-vector-friendly, a consequence the paper leaves implicit.
  • The boundary-pairing step is quadratic in the number of activations, so scaling to larger concept sets would require approximate nearest-neighbor search; which boundary pairs are kept would then affect the resulting vector.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces concept boundary vectors (CBVs), a concept-vector construction that extracts pairs of latent activations lying on the boundary between two concepts, forms unit boundary-normal vectors from these pairs, and optimizes a single unit vector to maximize average cosine similarity to them. The authors compare CBVs against concept activation vectors (CAVs) on a CNN trained on MNIST and a vision transformer fine-tuned on CIFAR-10, using logit influence, concept algebra, adversarial reclassification, and topological analyses of concept clusters and boundaries. The central claim is that CBVs are more effective than CAVs at encoding the relationship between concepts and better capture the geometry of the latent boundary.

Significance. If the empirical claims hold, CBVs would be a useful new tool for model interpretability, and the paper's use of topological data analysis to check the homogeneity and linear-separability assumptions underlying concept vectors is a valuable direction. The paper also identifies a clear limitation of CAVs and proposes a geometrically motivated alternative. However, the current evidence is not yet convincing because the main comparative experiments are confounded by vector-norm scaling, lack statistical rigor, and some evaluations reuse the same boundary points used in construction. The significance of the contribution therefore depends on whether these confounds can be addressed with additional experiments.

major comments (5)
  1. [§3.2, Eq. (1) and §3.5] The logit influence metric in Eq. (1) and the adversarial-perturbation magnitude in Section 3.5 both scale linearly with the Euclidean norm of the concept vector. CBVs are unit vectors by construction, whereas CAVs are unnormalized logistic-regression weight vectors. Without normalizing CAVs, the observed superiority of CBVs in Figures 2 and 8 could simply reflect the relative norms of the two vectors. Please rerun these experiments with CAVs normalized to unit norm (or with norms matched), and report the norms of the unnormalized CAVs so readers can assess the confound.
  2. [§4.1, Figure 11] The statement that the negative correlation between logit influence and boundary complexity is 'significant at the 5% level' is not backed by any statistical details: no test name, p-value, confidence interval, or sample size is reported. Since logit influence is scale-dependent and CAVs are unnormalized, this correlation may also be confounded by vector norms. Please provide the full statistical analysis and repeat the correlation with normalized concept vectors.
  3. [§2.2 and §4.2] The CBV optimization objective maximizes average cosine similarity to the set of boundary normal vectors N±, which presupposes that N± is sufficiently homogeneous and approximately unimodal. The paper tests homogeneity for the concept clusters (assumption A2) in Section 4.2, but never tests the homogeneity of N± itself. If the boundary normals are multimodal or highly dispersed, the optimized single vector may not align with any local boundary, undermining the method's rationale. Please add a dispersion or clustering analysis of N± (e.g., circular variance, silhouette scores, or persistent homology of N±) and discuss the results.
  4. [§3.5] The adversarial-reclassification experiment in Figure 7 evaluates on the same boundary pairs P± that were used to construct the CBV, making this part of the evaluation in-sample and potentially circular. Although Figure 8 extends the analysis to the entire cluster of latent activations, the paper should clearly separate the in-sample boundary-point results from the out-of-sample full-cluster results, and report the out-of-sample performance on non-boundary points separately.
  5. [Figures 2, 6, 8, and 11] Several load-bearing comparisons lack error bars and significance tests. The claims that CBVs have 'greater' logit influence, are 'more successful' in concept algebra, and require 'lower amplitude' adversarial perturbations are made without quantifying run-to-run variability from CAV training and CBV optimization. Please provide repeated-run statistics (e.g., mean ± std over multiple seeds) and appropriate significance tests for these comparisons.
minor comments (5)
  1. [Abstract and §1] There are typos, e.g., 'commmonly' in the Introduction and 'extententangled' in Section 3.3; a careful proofread is needed.
  2. [Footnote 4 and §7.3] The code is stated to be available 'here' but no URL is visible in the manuscript; please include a working link or supplementary material.
  3. [§7.3, Figure 16] The finding that CBVs are more similar to boundary normal vectors than CAVs is expected by construction, since CBVs are explicitly optimized for that similarity; consider presenting this only as a sanity check of the optimization objective, not as evidence of representational quality.
  4. [§7.3.1] The consistency-error formula appears to use v^{ℓ1}_± in both terms of the norm; if the second term is meant to use v^{ℓ2}_±, please correct the typo.
  5. [§3.1 and §2.2] The experimental details do not report hyperparameters for the CBV optimization (e.g., learning rate, number of optimization steps, initialization) or for Algorithm 1 beyond the statement that 10,000 optimizations are run; please list these values for reproducibility.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: Figure 16 restates the CBV optimization objective, and the initial adversarial evaluation uses the same boundary pairs that define the CBV; the central claim retains independent support from concept algebra and whole-cluster tests.

  1. self definitional [Section 7.3, Figure 16 (construction in Section 2.2)]
    "From Figure 16 we see that concept boundary vectors are more similar to the boundary normal vectors than the concept activation vectors, as expected."

    The concept boundary vector is defined as the vector optimized to maximize average cosine similarity to the boundary normal vectors N±. Figure 16 then reports exactly that optimized quantity, comparing it with an unoptimized CAV. The finding that the optimized vector scores higher on its own optimization objective is true by construction; it does not independently establish that CBVs are more effective at encoding concept relationships. The plotted quantity is the same objective function used to fit the CBV, so this comparison is a restatement of the construction rather than an external validation.

  2. fitted input called prediction [Section 3.5, Figures 7-8 (construction in Algorithm 1 and Section 2.2)]
    "Here we consider latent representations a− ∈ AC− that are components in a pair from P±. ... However, this raises the question as to whether the lower amplitude perturbation is just because the concept boundary vector overfit to the relatively few identified boundary points."

    The boundary normal set N± is built from the pairs P± returned by Algorithm 1, and the CBV is fit to N±. Section 3.5 then evaluates the CBV's adversarial reclassification on those same boundary pairs. This is an in-sample evaluation: the constructed vector is tested on the very points that defined its objective, so lower perturbation amplitudes on those points may reflect overfitting rather than a better concept relationship. The paper explicitly acknowledges this risk and partially addresses it with Figure 8, which extends the comparison to the full cluster, so the circularity is partial rather than total.

full rationale

The paper's main construction is explicit: CAVs are logistic-regression weight vectors (Section 3.1), while CBVs are vectors optimized for cosine similarity to the boundary normals N± (Section 2.2). Two evaluation steps are construction-internal. First, Figure 16 reports that the optimized vector has higher similarity to N± than a CAV; this is a restatement of the optimization objective and is circular as evidence of effectiveness. Second, the adversarial experiment in Section 3.5 initially evaluates perturbations on the very boundary pairs P± used to build N±, which the paper itself flags as a possible overfitting explanation; Figure 8 mitigates this by testing the whole cluster. The central comparisons, however, retain independent content: concept algebra (Section 3.4) uses cosine similarity and does not depend on the construction norm; the logit-influence comparison (Section 3.2) is external to the boundary-pair construction, and the whole-cluster adversarial result goes beyond the fitted points. A separate validity threat, not itself a circularity, is that the logit-influence and input-space perturbation metrics are scale-sensitive while CAVs are not normalized; if CAV norms are smaller, part of CBV's apparent advantage could be a norm artifact. Because the tautological Figure 16 and the in-sample adversarial evaluation are localized and acknowledged, and the main claim has independent support, the circularity score is 4 rather than higher.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim rests on assumptions A1 and A2 from Section 2, plus the additional, untested assumption that boundary normal vectors are homogeneous enough to be summarized by a single direction. The evaluation relies on the logit influence metric as a proxy for semantic faithfulness. The algorithm's boundary-pair selection is a heuristic and specific to this paper.

free parameters (5)
  • CAV training learning rate = 0.001
    Chosen by hand for the linear classifier training; affects the quality of the CAV baseline.
  • CAV training epochs = 100
    Fixed without justification; affects the CAV baseline.
  • Boundary vector optimization steps = 10000
    The optimization procedure is under-specified; the number of steps is arbitrary and may affect convergence.
  • Boundary complexity scale range = 0.1 to 2
    Choice of scale range in the TopologicalData class affects the calculation of boundary complexity in Section 4.1.
  • Mapper cover cubes = 11
    Number of cubes in the mapper cover, chosen for the analysis; different values change the clusters and simplices found.
assumptions (6)
  • domain assumption A1: The latent space activations of different concepts are linearly separable.
    Stated as an assumption in Section 2 and used throughout the paper to justify concept vectors.
  • domain assumption A2: The collective latent space activations of a concept exhibit homogeneity, meaning a single vector can capture the majority of the semantic meaning.
    Stated in Section 2 and tested for concept clusters in Section 4.2, but not for boundary normal vectors.
  • domain assumption Maximizing the average cosine similarity to boundary normal vectors produces a faithful concept direction.
    This is the optimization objective in Section 2.2; it assumes the boundary normals are unimodal enough for a single vector to be representative.
  • domain assumption The logit influence metric (TCAV score extension) measures how effectively a concept vector encodes a concept relationship.
    Defined in Section 2.3.3 and used as the primary quantitative evaluation; its validity as a faithfulness measure is assumed.
  • ad hoc to paper Algorithm 1 identifies pairs that lie on the concept boundary.
    The nearest-neighbor heuristic in the boundary construction algorithm (Section 2.2 and Appendix 7.1) is specific to this paper and is not independently verified.
  • standard math Persistent homology and mapper provide meaningful summaries of the latent structure.
    The paper relies on standard TDA tools, but the interpretation of the resulting diagrams and graphs is qualitative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Concept Boundary Vectors." pith.science (2026). https://pith.science/paper/HCCPAHBX

@misc{pith2026241215698,
  author       = {Pith},
  title        = {Pith review of: Concept Boundary Vectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HCCPAHBX}},
  note         = {Machine review of arXiv:2412.15698}
}
read the original abstract

Machine learning models are trained with relatively simple objectives, such as next token prediction. However, on deployment, they appear to capture a more fundamental representation of their input data. It is of interest to understand the nature of these representations to help interpret the model's outputs and to identify ways to improve the salience of these representations. Concept vectors are constructions aimed at attributing concepts in the input data to directions, represented by vectors, in the model's latent space. In this work, we introduce concept boundary vectors as a concept vector construction derived from the boundary between the latent representations of concepts. Empirically we demonstrate that concept boundary vectors capture a concept's semantic meaning, and we compare their effectiveness against concept activation vectors.

Figures

Figures reproduced from arXiv: 2412.15698 by the authors.

Figure 1
Figure 1. Figure 1a shows the variability in the loss as a concept vector trained two-dimensional data is rotated. Figure [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2a shows the influence of the concept vectors on the logit of the target concept. Figure 2b shows the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The cosine similarities between concept vectors with the concept [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: The persistence diagrams obtained from geodesic-based filtrations of concept activation vectors and concept [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Mapper plots obtained from concept activation vectors and concept boundary vectors. The size of the dots [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: In Figure 6a each column of the image plots corresponds to a concept-concept relationship. Each square in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Perturbed inputs that are classified by 2 by the model despite originally representing the digit 0. The values on top of each input are the distances of the perturbed inputs from their original form [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The ratio of the magnitude of concept activation vector-based perturbations and concept boundary vector-based [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Figure 9a shows the spatial dependency of the concept activation vector for the [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Figure 10a is a two-dimensional singular value decomposition of the latent activations of the input data [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: The (normalised) logit influence against the sum of [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: The (normalised) logit influence against different measures capturing the homogeneity of the concept’s [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: Figure 13a is the run time of Algorithm 1 with respect to the number of samples in the two classes, for [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 14
Figure 14. Figure 14: Mean Euclidicity of the interior and boundary of concept clusters. Euclidicity values have been normalised [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: The training losses obtained when optimizing for the concept vectors. The blue lines are the losses for each [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: The crosses are the average similarities to the concept activation vectors, whereas the dots are the average [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Consistency errors between layers 10 and 11 of our vision transformer model. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_17.png]
Figure 19
Figure 19. Figure 19: The accuracy of a trained classifier applied to the concept activations at intermediate layers through the [PITH_FULL_IMAGE:figures/full_fig_p019_19.png]
Figure 20
Figure 20. Figure 20: Identifying the images that trigger the features extracted by the sparse autoencoder trained on the latent [PITH_FULL_IMAGE:figures/full_fig_p020_20.png]
Figure 21
Figure 21. Figure 21: Identifying the images that trigger features extracted by the sparse autoencoder trained on the latent [PITH_FULL_IMAGE:figures/full_fig_p021_21.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 19 canonical work pages

  1. [1]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention Is All You Need, August 2023. arXiv:1706.03762 [cs]

    arXiv 2023

  2. [2]

    Language Models are Unsupervised Multitask Learners, 2018

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language Models are Unsupervised Multitask Learners, 2018

    work page 2018

  3. [3]

    Efficient Estimation of Word Representations in Vector Space, September 2013

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space, September 2013. arXiv:1301.3781 [cs]

    arXiv 2013

  4. [4]

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman...

    work page 2021

  5. [5]

    Mishra, and Sanjiv Kumar Jain

    Ritu Tandon, Shweta Agrawal, Narendra Pal Singh Rathore, Abhinava K. Mishra, and Sanjiv Kumar Jain. A systematic review on deep learning-based automated cancer diagnosis models. Journal of Cellular and Molecular Medicine, 28(6):e18144, March 2024

    work page 2024

  6. [6]

    Formal Analysis of Neural Network-Based Systems in the Aircraft Domain

    Panagiotis Kouvaros, Trent Kyono, Francesco Leofante, Alessio Lomuscio, Dragos Margineantu, Denis Osipychev, and Yang Zheng. Formal Analysis of Neural Network-Based Systems in the Aircraft Domain. In Marieke Huisman, Corina P˘as˘areanu, and Naijun Zhan, editors, F ormal Methods, pages 730–740, Cham, 2021. Springer International Publishing. 11 Concept Boun...

    work page 2021

  7. [7]

    Introducing ChatGPT, November 2022

    OpenAI. Introducing ChatGPT, November 2022

    work page 2022

  8. [8]

    Why Should I Trust You?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 1135–1144, August 2016. Conference Name: KDD ’16: The 22nd ACM SIGKDD International Conference on Knowledge Discovery and...

    work page 2016

Show all 32 references
  1. [9]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. A Unified Approach to Interpreting Model Predictions. In Neural Information Processing Systems, May 2017

    2017

  2. [10]

    Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCA V), June

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCA V), June

  3. [11]

    Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks, August 2023

    Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell. Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks, August 2023. arXiv:2207.13243 [cs]

    2023 arXiv

  4. [12]

    Sparse Autoencoders Find Highly Interpretable Features in Language Models, October 2023

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse Autoencoders Find Highly Interpretable Features in Language Models, October 2023. arXiv:2309.08600 [cs]

    2023 arXiv

  5. [13]

    On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors

    Adriano Lucieri, Muhammad Naseer Bajwa, Stephan Alexander Braun, Muhammad Imran Malik, Andreas Dengel, and Sheraz Ahmed. On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors. In 2020 International Joint Conference on Neural Networ...

    2020 arXiv

  6. [14]

    Rahul Soni, Naresh Shah, Chua Tat Seng, and Jimmy D. Moore. Adversarial TCA V – Robust and Effective Interpretation of Intermediate Layers in Neural Networks, February 2020. arXiv:2002.03549 [cs, stat]

    2020 arXiv

  7. [15]

    Concept Activation Regions: A Generalized Framework For Concept-Based Explanations, September 2022

    Jonathan Crabbé and Mihaela van der Schaar. Concept Activation Regions: A Generalized Framework For Concept-Based Explanations, September 2022. arXiv:2209.11222 [cs]

    2022 arXiv

  8. [16]

    Varshney, and Krishnan Mody

    Karthikeyan Natesan Ramamurthy, Kush R. Varshney, and Krishnan Mody. Topological Data Analysis of Decision Boundaries with Application to Model Selection, May 2018. arXiv:1805.09949 [cs, stat]

    2018 arXiv

  9. [17]

    The Linear Representation Hypothesis and the Geometry of Large Language Models, July 2024

    Kiho Park, Yo Joong Choe, and Victor Veitch. The Linear Representation Hypothesis and the Geometry of Large Language Models, July 2024. arXiv:2311.03658 [cs, stat]

    2024 arXiv

  10. [18]

    Emergent Linear Representations in World Models of Self-Supervised Sequence Models, September 2023

    Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent Linear Representations in World Models of Self-Supervised Sequence Models, September 2023. arXiv:2309.00941 [cs]

    2023 arXiv

  11. [19]

    Li, and Jacob Andreas

    Evan Hernandez, Belinda Z. Li, and Jacob Andreas. Inspecting and Editing Knowledge Representations in Language Models, May 2023. arXiv:2304.00740 [cs]

    2023 arXiv

  12. [20]

    Local Interpretation of Transformer Based on Linear Decomposition

    Sen Yang, Shujian Huang, Wei Zou, Jianbing Zhang, Xinyu Dai, and Jiajun Chen. Local Interpretation of Transformer Based on Linear Decomposition. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Compu...

    2023

  13. [21]

    The mnist database of handwritten digit images for machine learning research

    Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141–142, 2012

    2012

  14. [22]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, June 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at...

    2021 arXiv

  15. [23]

    Krizhevsky

    A. Krizhevsky. Learning Multiple Layers of Features from Tiny Images, 2009

    2009

  16. [24]

    Adam — PyTorch 2.4 documentation, 2023

    PyTorch Contributors. Adam — PyTorch 2.4 documentation, 2023

    2023

  17. [25]

    Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition

    Gurjeet Singh, Facundo Mémoli, and Gunnar Carlsson. Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition. Eurographics Symposium on Point-Based Graphics, 2007

    2007

  18. [26]

    Hendrik Jacob van Veen, Nathaniel Saul, David Eargle, and Sam W. Mangham. Kepler Mapper: A flexible Python implementation of the Mapper algorithm. Journal of Open Source Software, 4(42):1315, 2019

    2019

  19. [27]

    Alison Noble, and Yarin Gal

    Angus Nicolson, Lisa Schut, J. Alison Noble, and Yarin Gal. Explaining Explainability: Understanding Concept Activation Vectors, April 2024. arXiv:2404.03713 [cs]

    2024 arXiv

  20. [28]

    Topological Singularity Detection at Multiple Scales, June 2023

    Julius von Rohrscheidt and Bastian Rieck. Topological Singularity Detection at Multiple Scales, June 2023. arXiv:2210.00069 [cs, math, stat]. 12 Concept Boundary Vectors

    2023 arXiv

  21. [29]

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amnda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Dodds-Hatfield, Alex Tamkin, Karina Nguyen, ...

    2023

  22. [31]

    a+ is the closest activation in AC+ to a− and a− is the closest activation in AC− to some activation, which could be a+, in AC+, or

  23. [32]

    It is clear that Algorithm 1 is quadratic in the number of points in AC+ and AC−, however, there is also a burden imposed by the dimensionality of the latent representations

    a− is the closest activation in AC− to a− and a+ is the closest activation in AC+ to some activation, which could be a−, in AC−. It is clear that Algorithm 1 is quadratic in the number of points in AC+ and AC−, however, there is also a burden imposed by the dimensionality of t...

  24. [2018]

    arXiv:1711.11279 [stat]

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.