Pith. sign in

REVIEW 4 major objections 6 minor 17 references

Analyzing Hierarchical Structure in Vision Models with Sparse Autoencoders

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read DINOv2's later layers implicitly encode ImageNet's taxonomic hierarchy, and sparse autoencoders recover that structure as discrete, interpretable features.

desk verdict A useful SAE-plus-WordNet toolkit for probing hierarchical structure in a vision transformer, but the central taxonomy-encoding claim needs a null model before it is convincing. read the letter →

arxiv 2505.15970 v1 pith:XR4TXNMR submitted 2025-05-21 cs.CV cs.LG

classification cs.CVcs.LG
keywords sparseautoencodersImageNethierarchyDINOv2interpretabilitytaxonomicstructurevisiontransformersWordontologyfeatureprobing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that sparse autoencoders (SAEs, networks that compress a model's internal activations into a small set of sparse, interpretable features) can expose the hierarchical structure hidden in a vision model's representations. Applied layer by layer to DINOv2's class token on ImageNet, the SAEs' learned features align with the WordNet taxonomy that organizes ImageNet's 1,000 classes, particularly in later layers. The authors claim this shows the model implicitly encodes taxonomic relationships rather than only memorizing class labels. The work matters because it offers an unsupervised way to inspect what categories a vision model has internalized and where in the network that information lives.

What carries the argument

The carrying machinery is the ReLU sparse autoencoder, a one-hidden-layer network with hidden dimension 8 times the input and an L1 sparsity penalty, trained to reconstruct each layer's class-token activations. The extracted sparse features are evaluated with two WordNet-based metrics: Lowest Common Hypernym (LCH) Height, the average distance from the activated leaf classes up to their lowest common ancestor, and Ontological Coverage, the fraction of that ancestor's leaf set that the head actually activates. These two metrics together identify SAE heads that correspond to higher-order concepts, and relevancy maps then localize the image regions driving each head's activation. The layer-by-layer sweep of all 40 DINOv2 layers is what lets the paper argue that hierarchical information is built up progressively.

What would settle it

Train the same 40 ReLU SAEs on a randomly initialized or untrained DINOv2, or permute the class labels before computing LCH Height and Ontological Coverage, then compare the metric distributions. If untrained or permuted models produce a comparable number of high-coverage multi-class heads, the claim that later layers encode the taxonomy would not be supported.

Watch

Extended reading notes

Core claim

The central claim is that DINOv2's later layers encode ImageNet's taxonomic hierarchy, and that ReLU sparse autoencoders trained on the class token recover this structure as discrete features. The evidence is that in later layers, a growing number of SAE heads activate on groups of classes that share a common ancestor, with 90 heads at layer 36 achieving ontological coverage of 1.0—meaning the classes they activate on exactly fill the leaf set of a higher-order WordNet synset (e.g., elasmobranchs, whales, woodwind instruments, warships). The paper further claims the early layers contain almost no class information: SAEs need very few activations to reconstruct them and linear probes perform poorly, while information in the class token accumulates through the layers. The authors present SAEs as a label-free surrogate for measuring information content in a token representation.

Load-bearing premise

The load-bearing premise is that an SAE head firing on visually similar classes reflects the model encoding the taxonomy, not an artifact of random or trivial feature groupings, and the paper provides no null model to test that premise.

Editorial extensions

If this is right

  • SAE metrics can serve as a label-free early signal for how much usable class information a layer's representation carries: later layers reconstruct worse, activate more features, and classify better.
  • A user inspecting a vision foundation model can find discrete, interpretable features at the top layers that correspond to broad categories such as whales, sharks and rays, and warships rather than only to individual classes.
  • Hierarchy-aligned SAE heads appear gradually across layers, which supports viewing the class token as an accumulating information bottleneck rather than a fixed representation.
  • The same measurement framework can be applied to other models and taxonomies to quantify taxonomic consistency without fine-tuning or labeled probes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The absence of a null-model baseline is the main open gap: before the metrics can be trusted as evidence of implicit taxonomy, they should be compared against SAEs trained on a randomly initialized encoder or against randomized feature partitions.
  • Because the method needs no labels beyond the fixed WordNet hierarchy, it could be extended to any dataset with a known ontology, such as medical or biological image sets, to test whether hierarchies learned in one domain transfer.
  • A testable extension is to use the high-coverage heads as supervision for hierarchical classification: if the taxonomy is really encoded, features from those heads should boost performance on coarse-to-fine prediction tasks.
  • The claim that early layers carry no class information is specific to DINOv2's class token; the same SAE sweep on patch tokens or on supervised models might show a different layer-wise profile.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes using sparse autoencoders (SAEs) trained on DINOv2 class-token activations to measure how well vision models encode the ImageNet/WordNet class hierarchy. It defines two hierarchy metrics, LCH Height and Ontological Coverage, reports their distributions for SAE heads at selected layers, and concludes that later layers contain hierarchical structure. It also reports SAE reconstruction, sparsity, and L0 metrics across layers, together with linear-probe accuracy, and provides relevancy-map examples for a hierarchical head.

Significance. If conclusively established, the claim that SAE features at later DINOv2 layers reflect the ImageNet taxonomy would be a useful contribution to interpretability of vision models and would support the wider use of SAEs as a probing tool. The paper addresses a relevant question, and the use of linear probes as an independent layer-informativeness check is a methodological strength. However, the central interpretive claim currently rests on metrics with no null model and on an underspecified activation-set definition, so the significance is conditional on those gaps being closed.

major comments (4)
  1. [Section 3.2, Eqs. (6)-(7)] No random baseline or permutation test is provided for LCH Height and Ontological Coverage. Because WordNet subtrees are strongly correlated with visual similarity, the observed high coverage for heads that fire on groups such as dog breeds or fish may simply reflect shared low-level or mid-level visual features rather than an explicit encoding of the taxonomy. To support the abstract's claim that "SAEs uncover hierarchical relationships," the authors should compare both metrics against a null distribution (e.g., random class subsets matched by cardinality, or a random permutation of class labels) and report effect sizes or confidence intervals. This is the load-bearing missing piece.
  2. [Section 3.2, Eqs. (5)-(7)] The set C_k of classes on which an SAE head "activates" is never formally defined. The manuscript does not state the activation threshold, the image set used, or the per-class aggregation rule, even though Eqs. (5)-(7) and Figure 2 depend directly on C_k. Without this definition, the metrics cannot be reproduced or compared across settings. Please provide an operational definition, for example that a head activates on class omega if its mean activation over class-omega validation images exceeds a specified threshold, and report the chosen threshold and a sensitivity analysis.
  3. [Section 5 vs. Section 4.1] The Discussion states that "early layers showing stronger alignment" with the hierarchy, but Section 4.1 and Figure 2 report increasing numbers of high-coverage heads and hierarchical structure in later layers, including the layer-36 example with 90 multi-class heads of coverage 1.0. This internal contradiction affects the paper's main message and should be resolved; please correct the wording and state precisely whether hierarchical alignment increases or decreases with depth.
  4. [Sections 3.5 and 4.1] All layer-level conclusions are drawn from a single SAE training run with fixed hyperparameters (lambda = 10, expansion factor 8) and no error bars or repeated seeds. Because SAE training is stochastic and the L0 and coverage metrics are likely sensitive to the sparsity penalty, the cross-layer trend in Figure 2 may not be robust. Reporting multiple seeds or at least a sensitivity analysis over lambda is needed to support the claim that later layers are more hierarchically organized.
minor comments (6)
  1. [Section 1] The phrase "the of hierarchical encoding" appears to be a typo and should be corrected, for example to "the encoding of hierarchical structure."
  2. [Section 3.2, Eq. (4)] The statement that for all omega in Omega, L(omega) = {emptyset} appears incorrect: for a leaf class omega, L(omega) should be {omega}, or at least a singleton containing that leaf, not a set containing the empty set.
  3. [Section 3.2, Eq. (5)] Eq. (5) is written with h_k on both sides of the argmin, which is formally inconsistent; it should read h_k = argmin_{h : C_k subseteq L(h)} |L(h)|.
  4. [Section 3.2, Eq. (6)] The sentence "LCH Height of C_k is calculated the average height ofhk" is ungrammatical, and "height" is later defined as an average distance; please rewrite this definition for clarity.
  5. [Section 3.3, Eqs. (8)-(9)] The relevancy update R_i = R_{i-1} + \bar{A}_i \cdot R_{i-1} uses R_i on both sides without stating an initialization or an iteration order; please clarify the recurrence and its indexing.
  6. [References] References [4], [6], and [8] are web resources with bare URLs; please format them consistently and include version or access-date information, especially for the SAELens library, to support reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central hierarchy claims are evaluated against an external label-based taxonomy, not derived from the SAE training objective.

full rationale

The paper trains ReLU SAEs on DINOv2 class-token activations with an unsupervised reconstruction plus L1 loss (Eq. 3), with no use of ImageNet labels during fitting. The hierarchical metrics (Eqs. 5-7) are computed post hoc from the sets of classes activating each head, and high Ontological Coverage is presented as an empirical finding rather than a construction: e.g., of 9484 active heads at layer 36, only 90 multi-class heads reach coverage 1.0, and the paper explicitly discusses a head activating on 7 bird species with coverage 0.119, showing that the metric is not forced by definition. No load-bearing argument reduces to a self-citation: the authors cite external SAE literature and the SAELens library, and no uniqueness or prior-author result is invoked to exclude alternatives. The most substantial weaknesses—C_k is not operationally defined, no null/permutation baseline is supplied, and Section 5's 'early layers showing stronger alignment' conflicts with Section 4.1's increasing later-layer structure—are evidential or consistency concerns, not circularity. Accordingly no circular step meeting the required quote-and-reduction threshold is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claim rests on the assumptions that SAE features are semantically meaningful, that the ImageNet/WordNet hierarchy is a valid ground truth, and that the proposed metrics are informative without a chance baseline. These are domain assumptions rather than derived facts.

free parameters (3)
  • L1 sparsity penalty (lambda) = 10
    Chosen as a fixed trade-off in Section 3.5; no sweep or sensitivity analysis is reported.
  • SAE hidden expansion factor = 8
    Confers hidden size 12,288 on the class-token SAEs; no ablation justifies this choice.
  • Activation threshold for defining C_k = Not reported
    The paper never specifies how an SAE head is considered to activate on an ImageNet class, yet all hierarchy metrics depend on C_k. This is an unreported free parameter.
assumptions (4)
  • domain assumption ReLU SAE with L1 penalty learns monosemantic features that correspond to semantically meaningful concepts.
    Relies on the SAE interpretability literature (Olah et al., Cunningham et al.); if SAE features are not semantically aligned, the hierarchy analysis is invalid.
  • domain assumption ImageNet/WordNet taxonomy is an appropriate ground truth for semantic hierarchy.
    The paper uses WordNet hypernymy as ground truth; if this taxonomy does not match visual similarity, the metrics would misrepresent model structure.
  • domain assumption Class token activations of DINOv2 are the right substrate for hierarchical analysis.
    Only the class token is probed; patch embeddings are ignored, and the claim concerns vision model representations broadly.
  • ad hoc to paper Ontological Coverage and LCH Height are meaningful without a chance-level baseline.
    The paper interprets high coverage and low LCH height as evidence of hierarchy without comparing to random feature groupings or a null model, an implicit assumption that the metrics carry information.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Analyzing Hierarchical Structure in Vision Models with Sparse Autoencoders." pith.science (2026). https://pith.science/paper/XR4TXNMR

@misc{pith2026250515970,
  author       = {Pith},
  title        = {Pith review of: Analyzing Hierarchical Structure in Vision Models with Sparse Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XR4TXNMR}},
  note         = {Machine review of arXiv:2505.15970}
}
read the original abstract

The ImageNet hierarchy provides a structured taxonomy of object categories, offering a valuable lens through which to analyze the representations learned by deep vision models. In this work, we conduct a comprehensive analysis of how vision models encode the ImageNet hierarchy, leveraging Sparse Autoencoders (SAEs) to probe their internal representations. SAEs have been widely used as an explanation tool for large language models (LLMs), where they enable the discovery of semantically meaningful features. Here, we extend their use to vision models to investigate whether learned representations align with the ontological structure defined by the ImageNet taxonomy. Our results show that SAEs uncover hierarchical relationships in model activations, revealing an implicit encoding of taxonomic structure. We analyze the consistency of these representations across different layers of the popular vision foundation model DINOv2 and provide insights into how deep vision models internalize hierarchical category information by increasing information in the class token through each layer. Our study establishes a framework for systematic hierarchical analysis of vision model representations and highlights the potential of SAEs as a tool for probing semantic structure in deep networks.

Figures

Figures reproduced from arXiv: 2505.15970 by the authors.

Figure 1
Figure 1. Results of training a ReLU SAE (or linear probe) on every layer of DINOv2’s class token on ImageNet. We find the surprising result that the early layers in this model are non￾informative: the representations are incredibly easy to auto-encode (right y-axis), require very few activations from an SAE (right y￾axis), and are not usable for fitting a classification model (left y￾axis). minimize dead neurons. For all exp… view at source ↗
Figure 3
Figure 3. Relevancy maps of the hierarchical SAE head at DI [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 2
Figure 2. Distribution of LCH Height vs Ontological Coverage for SAE Heads at Layer 24, 28, 32 and 36 of DINOv2. For each layer, we plot the distribution of LCH height and ontological coverage of the SAE heads. Darker indicates higher bin density. Not only does the vision model capture hierarchical concepts in its output, but also show signs of enhancing hierarchical features through out its processing layer-by-layer. SAE hea… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 8 canonical work pages

  1. [1]

    Do convolutional neural networks learn class hierarchy? IEEE transactions on visualization and computer graphics, 24(1):152–162, 2017

    Alsallakh Bilal, Amin Jourabloo, Mao Ye, Xiaoming Liu, and Liu Ren. Do convolutional neural networks learn class hierarchy? IEEE transactions on visualization and computer graphics, 24(1):152–162, 2017. 2

  2. [2]

    Generic attention- model explainability for interpreting bi-modal and encoder- decoder transformers

    Hila Chefer, Shir Gur, and Lior Wolf. Generic attention- model explainability for interpreting bi-modal and encoder- decoder transformers. In Proceedings of the IEEE/CVF in- ternational conference on computer vision , pages 397–406,

  3. [3]

    Sparse autoencoders find highly interpretable features in language models

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023. 1, 2

  4. [4]

    Daujotas

    G. Daujotas. Interpreting and steering features in im- ages. https : / / www . lesswrong . com / posts / Quqekpvx8BGMMcaem / interpreting - and - steering-features-in-images , 2024. Accessed: 2025-03-07. 2

  5. [5]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 2

  6. [6]

    Towards multimodal interpretability: Learn- ing sparse interpretable features in vision transform- ers

    Hugo Fry. Towards multimodal interpretability: Learn- ing sparse interpretable features in vision transform- ers. https : / / www . lesswrong . com / posts / bCtbuWraqYTDtuARg / towards - multimodal - interpretability-learning-sparse-2 , 2024. 2

  7. [7]

    Scaling and evaluating sparse autoencoders

    Leo Gao, Tom Dupr ´e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093, 2024. 1, 2

  8. [8]

    Sae- lens

    Curt Tigges Joseph Bloom and David Chanin. Sae- lens. https://github.com/jbloomAus/SAELens,

Show all 17 references
  1. [9]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

  2. [10]

    The geometry of con- cepts: Sparse autoencoder feature structure

    Yuxiao Li, Eric J Michaud, David D Baek, Joshua Engels, Xiaoqing Sun, and Max Tegmark. The geometry of con- cepts: Sparse autoencoder feature structure. arXiv preprint arXiv:2410.19750, 2024. 2

  3. [11]

    Zoom in: An in- troduction to circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An in- troduction to circuits. Distill, 5(3):e00024–001, 2020. 2

  4. [12]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1

  5. [13]

    Category selectivity in human visual cortex: Beyond visual object recognition

    Marius V Peelen and Paul E Downing. Category selectivity in human visual cortex: Beyond visual object recognition. Neuropsychologia, 105:177–183, 2017. 1

  6. [14]

    Improving dictionary learning with gated sparse autoencoders

    Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, J´anos Kram´ar, Rohin Shah, and Neel Nanda. Improving dictionary learning with gated sparse autoencoders. arXiv preprint arXiv:2404.16014 ,

  7. [15]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge,

  8. [16]

    Sparse autoencoders for scientifically rigorous interpre- tation of vision models

    Samuel Stevens, Wei-Lun Chao, Tanya Berger-Wolf, and Yu Su. Sparse autoencoders for scientifically rigorous interpre- tation of vision models. arXiv preprint arXiv:2502.06755 ,

  9. [17]

    Hgclip: exploring vision- language models with graph representations for hierarchical understanding

    Peng Xia, Xingtong Yu, Ming Hu, Lie Ju, Zhiyong Wang, Peibo Duan, and Zongyuan Ge. Hgclip: exploring vision- language models with graph representations for hierarchical understanding. arXiv preprint arXiv:2311.14064, 2023. 2

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.