RECORD
Towards Open-Ended Visual Scientific Discovery with Sparse Autoencoders
Computer Vision and Pattern Recognition · Submitted 21 Nov 2025
abstract compiled · 2026-10-01 · no language-model review
The authors' main claim Together, these results show that SAEs with heuristic ranking can efficiently learn and surface semantic structure in vision foundation model representations.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Foundation models in several scientific domains, including visual domains, learn representations that capture complex semantics from their respective fields. Despite this, most existing applications focus on pre-specified concepts or targets, excelling at confirmation but not open-ended discovery of unknown patterns. We study whether sparse autoencoders (SAEs) with heuristic feature ranking can provide a practical methodology for surfacing candidate visual features from vision foundation models for scientific discovery without concept-specific supervision. We evaluate this approach in three stages: (1) general-purpose concept rediscovery, (2) domain-specific concept rediscovery, and (3) question-driven feature ranking. On ADE20K, SAEs outperform baseline methods ($k$-means, PCA, SemiNMF) across most downstream metrics for concept rediscovery. On FishVista, the same approach recovers fine-grained fish anatomical features without concept supervision. Finally, on Heliconius butterflies, a heuristic based on the Mann-Whitney $U$ statistic consistently surfaces features corresponding to real diagnostic keys between mimic subspecies. Together, these results show that SAEs with heuristic ranking can efficiently learn and surface semantic structure in vision foundation model representations. We view this as a methodological prerequisite for open-ended visual scientific discovery: the present experiments validate feature surfacing and ranking through rediscovery, while prospective discovery of previously unknown scientific phenomena remains future work.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Foundation models in several scientific domains, including visual domains, learn representations that capture complex semantics from their respective fields. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | Despite this, most existing applications focus on pre-specified concepts or targets, excelling at confirmation but not open-ended discovery of unknown patterns. | gap 0.97 | 0.05 | 0.02 | asserted 1.00 |
| s2 | We study whether sparse autoencoders (SAEs) with heuristic feature ranking can provide a practical methodology for surfacing candidate visual features from vision foundation models for scientific discovery without concept-specific supervision. | contribution 0.98 | 0.29 | 0.05 | hedged possibility 0.74 |
| s3 | We evaluate this approach in three stages: (1) general-purpose concept rediscovery, (2) domain-specific concept rediscovery, and (3) question-driven feature ranking. | method 0.91 | 0.86 | 0.03 | asserted 0.75 |
| s4 | On ADE20K, SAEs outperform baseline methods ($k$-means, PCA, SemiNMF) across most downstream metrics for concept rediscovery. | result 1.00 | 0.95 | 0.08 | asserted 1.00 |
| s5 | On FishVista, the same approach recovers fine-grained fish anatomical features without concept supervision. | result 1.00 | 0.95 | 0.04 | asserted 1.00 |
| s6 | Finally, on Heliconius butterflies, a heuristic based on the Mann-Whitney $U$ statistic consistently surfaces features corresponding to real diagnostic keys between mimic subspecies. | result 1.00 | 0.95 | 0.06 | asserted 1.00 |
| s7 | Together, these results show that SAEs with heuristic ranking can efficiently learn and surface semantic structure in vision foundation model representations. | interpretation 0.91 | 0.93 | 0.75 | asserted 1.00 |
| s8 | We view this as a methodological prerequisite for open-ended visual scientific discovery: the present experiments validate feature surfacing and ranking through rediscovery, while prospective discovery of previously unknown scientific phenomena remains future work. | future work 0.45 | 0.64 | 0.43 | asserted 0.90 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Foundation models in several scientific domains, including visual domains, learn representations that capture complex semantics from their respective fields. Despite this, most existing applications focus on pre-specified concepts or targets, excelling at confirmation but not open-ended discovery of unknown patterns. We study whether sparse autoencoders (SAEs) with heuristic feature ranking can provide a practical methodology for surfacing candidate visual features from vision foundation models for scientific discovery without concept-specific supervision. We evaluate this approach in three stages: (1) general-purpose concept rediscovery, (2) domain-specific concept rediscovery, and (3) question-driven feature ranking. On ADE20K, SAEs outperform baseline methods ($k$-means, PCA, SemiNMF) across most downstream metrics for concept rediscovery. On FishVista, the same approach recovers fine-grained fish anatomical features without concept supervision. Finally, on Heliconius butterflies, a heuristic based on the Mann-Whitney $U$ statistic consistently surfaces features corresponding to real diagnostic keys between mimic subspecies. Together, these results show that SAEs with heuristic ranking can efficiently learn and surface semantic structure in vision foundation model representations. We view this as a methodological prerequisite for open-ended visual scientific discovery: the present experiments validate feature surfacing and ranking through rediscovery, while prospective discovery of previously unknown scientific phenomena remains future work.
Human review
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.