Pith. sign in

REVIEW 4 major objections 6 minor 78 references

BlueGlass: A Framework for Composite AI Safety

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Probes find a shared phase transition in vision-language detectors

desk verdict A coherent composite-safety toolkit with a useful VLM benchmark, but the phase-transition claim is not yet supported by the evidence. read the letter →

arxiv 2507.10106 v1 pith:QNZ6SAHK submitted 2025-07-14 cs.AI

Harshal Nandigramwar , Syed Qutub , Kay-Ulrich Scholl This is my paper
classification cs.AI
keywords compositeAIsafetyvision-languagemodelsobjectdetectionlinearprobesphasetransitionsparseautoencodersdatasetattributiondistributionalevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BlueGlass is a framework for composing AI safety tools across model internals and outputs, and this paper uses it to run three safety analyses on vision-language models (VLMs) for object detection. The central empirical finding comes from approximation probes, linear probes trained to reproduce the model's own final classifications and bounding boxes: both a VLM and a vision-only detector show the same three-phase decoder trajectory, an early dip followed by a surge at a transition layer. The paper reads this as evidence that VLMs detect objects through the same hierarchical feature learning as classical detectors, with open-vocabulary flexibility entering the shared hierarchy via language-aligned features. Distributional evaluation maps where the models fail across datasets, and sparse autoencoders recover interpretable concepts as well as a hand-based spurious correlation. If the phase-transition claim holds, safety work gains a concrete target layer for monitoring and intervention in both model families.

What carries the argument

The approximation probe is the central instrument: a linear classifier or regressor trained on a layer's activations to match the target model's own final outputs, not ground-truth labels, with accuracy reported as AP at IoU 50. Its layer-by-layer trajectory is the evidence for the phase transition, which the paper formalizes as three phases—extraction, reorganization, refinement—and justifies with the information bottleneck principle and random hierarchy models. The supporting machinery is BlueGlass's feature-tool layer, whose interceptor, recorder, patcher, and aligner standardize activations from heterogeneous models into a shared schema, making the probes and the TopK sparse autoencoders composable.

What would settle it

Train approximation probes on randomized or shuffled final-output targets at every layer, or add a nonlinear probe head; if a similar dip-then-surge appears when the probe target carries no task structure, the dip is a probe artifact. Alternatively, compare probes trained on the same layer features with different optimization budgets: if increasing probe capacity or training time removes the dip at the transition layer, the dip reflects probe optimization difficulty rather than a reorganization of representation.

Watch

Extended reading notes

Core claim

The paper's central claim is that object-detection VLMs and vision-only detectors use the same internal strategy: after an extraction phase and before a refinement phase, both pass through a reorganization phase in which task-relevant information, as measured by approximation probes, drops and then surges. The transition is observed in decoder layers of Grounding DINO and DINO, across datasets, and for both classification and localization probes, so the paper proposes it as a shared hierarchical feature-learning mechanism rather than an artifact of one architecture. The emergent open-vocabulary ability of VLMs is then explained as language-aligned representations being folded into that shared hierarchy, which is what gives semantic flexibility without a separate detection mechanism. The framework contribution is BlueGlass itself, whose standardized feature repository lets probes, sparse autoencoders, and evaluators operate on the same captured activations.

Load-bearing premise

The whole phase-transition finding rests on the premise that layer-wise linear probes trained to match the model's own final outputs measure genuine information content in the features, rather than reflecting how hard each layer's features are to probe.

Editorial extensions

If this is right

  • Safety analyses can target the transition layer: monitoring or patching around that reorganization point should be more informative than treating all decoder layers alike.
  • The evaluation pipeline maps any open-ended VLM output to a fixed label set using text-embedding similarity, so new VLMs can be compared fairly on AP/AR without hand-written output parsing.
  • The three-phase trajectory can serve as a diagnostic signature for whether a new model family is actually building hierarchical object representations.
  • Because open-vocabulary ability is attributed to language-aligned features entering a shared visual hierarchy, improving geometric and localization priors should matter more for VLMs than scaling the language component alone.
  • SAE concept attribution, as illustrated by the hand detector, can surface spurious correlations that distributional evaluation alone would miss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reorganization is genuine, interventions applied before the transition layer should be weaker and less task-specific than interventions at or after it; a direct layer-wise activation-patching experiment would test this.
  • The hand-based spurious unit suggests a deployment monitor: track activation of that SAE unit on new data and flag predictions that co-occur with hand-like context but absent objects.
  • The same approximation-probe recipe can be extended beyond object detection to other decoder-based multimodal tasks, which would show whether extraction–reorganization–refinement is universal.
  • The standardized feature schema hints that safety tools trained on one model's features could migrate to another model, but the paper does not test cross-model transfer; that is a natural next experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces BlueGlass, an open-source framework for composite AI safety that unifies model-interfacing, feature capture/storage, and safety-tool composition, and demonstrates it through three case studies on vision-language models (VLMs) for object detection: (1) distributional evaluation across six datasets comparing VLM and vision-only detectors; (2) a new "approximation probe" method that trains linear probes on intermediate decoder layers to predict model outputs, claimed to reveal a universal phase transition in layer dynamics shared by VLMs and vision-only detectors; and (3) sparse autoencoder (SAE) analysis on Grounding DINO that identifies interpretable concepts and a hand-based spurious correlation. The paper's central mechanistic claim is that the probe-accuracy dip-then-surge pattern indicates a three-phase (extract, reorganize, refine) hierarchical feature-learning strategy common to both model types, with VLM open-vocabulary ability arising from language-aligned features entering this shared hierarchy.

Significance. The BlueGlass framework itself is a useful infrastructure contribution: it addresses a real need for composable, architecture-agnostic safety tooling, and the paper names reproducible, open-source infrastructure as a deliverable. The distributional evaluation provides a broad, useful comparison of current VLMs on object detection, and the SAE discovery of a hand-based spurious correlation is a concrete, falsifiable finding with safety relevance. If the phase-transition claim in Section 4.2 were established, it would be a notable mechanistic result with implications for understanding VLM generalization. However, the current evidence does not support that claim at the level the paper presents it: the key curves are single-run, lack statistical controls, and the theoretical appendix does not bridge the gap from linear-probe accuracy to information-theoretic phase transitions. The manuscript's value is therefore uneven: the framework and evaluation contributions are solid, but the headline mechanistic finding needs substantial additional support.

major comments (4)
  1. [Section 4.2, Figure 3] The phase-transition claim is supported only by single-run linear-probe AP curves with no error bars, no multiple seeds, and no statistical testing. In Figure 3, the 'dip' is often a single layer out of six, and without variance estimates or significance tests the pattern cannot be distinguished from noise. The paper should report repeated probe training runs (or at least bootstrap confidence intervals) and a null control, such as probes trained on the same features with shuffled labels or on random projections, to demonstrate that the dip-and-surge is not an artifact of probe optimization.
  2. [Section 4.1, Equations (1)-(2)] The approximation-probe interpretation is undermined by an unresolved ambiguity and an unvalidated assumption. First, the text says the probes are trained to approximate the model's own final predictions, but Equations (1) and (2) use dataset labels y_class and y_bbox; the authors must clarify which target is used, because the two choices lead to very different interpretations of the curves. Second, the paper acknowledges the linear representation hypothesis but does not validate it: a dip in linear decodability at middle layers could reflect poor linear fit due to scale, conditioning, or non-linear encoding, rather than a genuine absence of task-relevant information. The authors should add nonlinear probe controls (e.g., a small MLP probe) and report probe training convergence per layer to show that the dip is not an optimization artifact.
  3. [Appendix E, Proposition E.1 and Theorem E.3] The theoretical support for the phase-transition claim does not bridge from mutual information to probe AP. Proposition E.1 simply asserts the existence of a critical layer and a reorganization dip, restating the empirical claim rather than proving it. Theorem E.3, as written, is not mathematically coherent: 'If f_l is linear for all l, then I(Z_l; X) = I(X; Y ) for all l' mixes input and target mutual information in a way that does not follow from linearity, and it does not imply any behavior of linear-probe average precision. Either provide a correct formal statement connecting layer-wise linear decodability to information reorganization, or explicitly present Appendix E as a heuristic analogy rather than a proof. As it stands, the appendix does not support the universality claim made in Section 4.3.
  4. [Section 4.3] The claim that the phase transition is 'universal' and 'irrespective of the dataset or architecture' is stronger than the evidence. Figure 3 shows only two model types (DINO-DETR and Grounding DINO) on a limited set of datasets, and the text itself notes that the classification-phase transition 'only arises for label sets with many classes.' The scope of the claim should be restricted to the models and datasets actually tested, or additional models and datasets should be added to support universality.
minor comments (6)
  1. [Table 1] The table reports single-run AP/AR values with no variance or repetition information; please state the number of runs and, if applicable, report standard deviations or note that each configuration was run once.
  2. [Section 3.1] The exclusion of LLaVA-NeXT, GPT-4o-mini, and PaliGemma-2 due to unparsable or incorrect outputs should be quantified (how many outputs were unparsable) and discussed as a limitation, since it may bias the comparison toward models with more structured output formats.
  3. [Appendix D, Table 2] The ablation table would benefit from a note on whether the COCO mini evaluation is representative of the full COCO results; the paper uses this mini set for ablations but the main table uses full-scale evaluation.
  4. [Section 5.2] The SAE concept discovery relies on manual interpretation of top-activating proposals; please report the SAE hyperparameters (expansion factor e, TopK k), the number of spurious units found relative to total units, and ideally a small inter-annotator agreement check to reduce subjectivity.
  5. [Throughout] There are several typographical and grammatical errors, including 'sate-of-the-art' (Abstract/Section 3.1), 'complimentary aspects' (Introduction), 'it's' in Section 2.2, 'The IB objective is can be denoted as' (Appendix E), and 'enables' subject-verb agreement issues; a thorough language edit is recommended.
  6. [Figure 3] The figure axes and subplot labels are difficult to parse; please clarify which subplot corresponds to which model and dataset, and add a legend that distinguishes classification vs. localization probes in all subplots.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the probe/phase-transition analysis and evaluation benchmarks are empirical measurements over external data and model internals, and no load-bearing self-citation or fitted-input-as-prediction step appears in the derivation chain.

full rationale

The derivation chain contains no circular step that reduces a claimed prediction to a fitted input or to a self-citation. The three case studies are measurements over external data and model internals: Table 1 reports AP/AR from a fixed evaluation pipeline applied to public datasets; the pipeline hyperparameters and components were ablated on COCO mini (Table 2), but the reported AP values are model outputs transformed by the chosen encoder, not a re-expression of the selected hyperparameter, so any inflation concern is a benchmark-validity issue, not circularity. In Section 4, approximation probes are linear regressors independently trained at each decoder layer to predict the model's own final classification and box outputs (Eqs. 1-2); the per-layer accuracy curve is an empirical measurement, and the middle-layer dip is not forced by the training objective. The paper's phase-transition reading of that curve is an interpretation that could be wrong (e.g., if probe optimization succeeds unevenly across layers), but it is not equivalent by construction to the input data. The SAE analysis trains a reconstruction model on activations and labels units by top-activating examples from COCO; no fitted parameter is relabeled as a prediction. Finally, no load-bearing self-citation appears: citations to prior phase-transition work (Carroll, 2023; Olsson et al., 2022; Nakkiran et al., 2019) are contextual analogies, not the evidence for the observed trajectory. Appendix E's propositions are asserted rather than proved, and Theorem E.3 is formally questionable, but an unproved or invalid justification is a correctness gap, not a circular dependency. Therefore, the paper is self-contained against external benchmarks and its central mechanistic claim, while subject to robustness criticisms, is not circular.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The claims rest on the linear representation hypothesis for probes, the interpretation of probe curves as phase transitions, the unproved propositions in Appendix E, and the validity of the text-similarity evaluation and manual SAE labeling. The selected models are treated as representative of their architectural classes. No new physical or conceptual entities are invented; 'approximation probes' is a method name, not an entity.

free parameters (6)
  • MaxPred (VLM prediction limit) = 900 (selected from 10, 30, 100, 300, 900)
    Appendix D.2 and Table 2 choose this by COCO mini AP; all main benchmark numbers use it.
  • MinConf (confidence threshold) = 0 (selected from 0, 0.001, 0.01, 0.1, 0.5)
    COCO mini ablation shows AP drops from 44.5 to 28.4 at 0.5, so results are sensitive to this choice.
  • Negative class prompts = included (e.g., 'an object', 'a thing')
    Inclusion improves COCO mini AP from 42.8 to 44.6 and is applied to all datasets.
  • Part prompts = included (e.g., 'parts of a car')
    Chosen after COCO mini ablation; small AP effect (44.6 to 44.5) but kept in pipeline.
  • Text encoder for label mapping = CLIP
    Table 2 shows CLIP (44.5) outperforms BERT (27.7), B1ADE (36.7), and NVEmbed (43.7) on COCO mini; this choice affects every VLM AP/AR in Table 1.
  • SAE expansion factor and TopK k
    TopK SAE is trained on decoder layer 4, but expansion factor, k, learning rate, and training steps are not reported, so the reader cannot assess tuning.
assumptions (6)
  • domain assumption Linear representation hypothesis: intermediate representations are increasingly linearly separable, so linear probes can measure task-relevant information.
    Invoked in Section 4.1 to justify approximation probes and interpret probe accuracy as information content.
  • domain assumption The dip-and-surge shape of probe accuracy curves is evidence of a phase transition in representation reorganization, not a probe-training artifact.
    Section 4.2 reads Figure 3 directly as a phase transition; no statistical test or control condition is provided.
  • ad hoc to paper Information bottleneck and random hierarchy model propositions (E.1, E.2, Theorem E.3) are valid and imply phase transitions are fundamental.
    Appendix E uses these to support the universality claim; the propositions and theorem are stated without proof, and Theorem E.3 appears formally incorrect.
  • domain assumption The text-similarity pipeline maps open-ended VLM outputs to dataset labels faithfully.
    All VLM AP/AR numbers depend on this mapping (Appendix D); its encoder and controls were selected on COCO mini.
  • domain assumption Manual inspection of top-activating COCO proposals is a valid way to label SAE units as concepts.
    Section 5.2 uses human reading of 64 top activations per unit with no quantitative validation or null baseline.
  • domain assumption The selected model in each VLM architectural class is representative of that class.
    Appendix C states the chosen models are best performing and representative; the paper evaluates only one model per class.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BlueGlass: A Framework for Composite AI Safety." pith.science (2026). https://pith.science/paper/QNZ6SAHK

@misc{pith2026250710106,
  author       = {Pith},
  title        = {Pith review of: BlueGlass: A Framework for Composite AI Safety},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QNZ6SAHK}},
  note         = {Machine review of arXiv:2507.10106}
}
read the original abstract

As AI systems become increasingly capable and ubiquitous, ensuring the safety of these systems is critical. However, existing safety tools often target different aspects of model safety and cannot provide full assurance in isolation, highlighting a need for integrated and composite methodologies. This paper introduces BlueGlass, a framework designed to facilitate composite AI safety workflows by providing a unified infrastructure enabling the integration and composition of diverse safety tools that operate across model internals and outputs. Furthermore, to demonstrate the utility of this framework, we present three safety-oriented analyses on vision-language models for the task of object detection: (1) distributional evaluation, revealing performance trade-offs and potential failure modes across distributions; (2) probe-based analysis of layer dynamics highlighting shared hierarchical learning via phase transition; and (3) sparse autoencoders identifying interpretable concepts. More broadly, this work contributes foundational infrastructure and findings for building more robust and reliable AI systems.

Figures

Figures reproduced from arXiv: 2507.10106 by the authors.

Figure 1
Figure 1. System overview of BLUEGLASS, a framework for composite AI safety. pleton et al., 2024) to robustness assessments (Carlini et al., 2019) and data attributions (Ma, 2025). These methods op￾erate at different layers of abstraction and require varying forms of access to a model’s structure, behavior, and compu￾tations. At the same time, the underlying models continue to evolve in architectural complexity, heterogeneity… view at source ↗
Figure 2
Figure 2. Linear probes setup on decoder layers. Representation probing (Alain & Bengio, 2018) is a class of AI safety methods that extend the target model with light-weight linear layers to quantitatively measure auxil￾iary information, such as information content, uncertainties, etc., at intermediate model positions. Recent works have shown their applicability for a wide range of safety related tasks such as mechanistic ana… view at source ↗
Figure 3
Figure 3. Phase transition in decoder layers of closed-set detector and vision-language model. proximation probes to analyze the information content and trajectories across the decoder layers. Our analysis reveals a phase transition (Carroll, 2023) in the decoder layers of both model types. In this critical reorga￾nization phase, representations abruptly shift from generic features to task-specific compositional abstractions.… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Concepts discovered from TopK SAE (Gao et al., 2024) trained on features from Grounding DINO (Liu et al., 2024b). Note, that this shows only a subset of concepts and tokens per concept from a larger sample set. 5.2. Concept Discovery To discover the concepts encoded by…
Figure 5
Figure 5. Figure 5: Categorization of Vision-language model based on architecture and learning objective. learning objective and output mechanism as shown in [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Evaluation pipeline. Let fencoder : T → R d denotes a text encoder, C = {c1, c2, . . . , cN } represents the fixed set of N dataset-specific class labels, and O = {o1, o2, . . . , oM} represent the M open-ended textual outputs from the VLM. We encode C and O using the …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

78 extracted references · 29 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Sanity Checks for Saliency Maps , 2020

    Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B. Sanity Checks for Saliency Maps , 2020. URL https://arxiv.org/abs/1810.03292

    arXiv 2020

  3. [3]

    and Bengio, Y

    Alain, G. and Bengio, Y. Understanding intermediate layers using linear classifier probes, 2018. URL https://arxiv.org/abs/1610.01644

    arXiv 2018

  4. [4]

    and Li, Y

    Allen-Zhu, Z. and Li, Y. Physics of language models: Part 3.1, knowledge storage and extraction, 2024. URL https://arxiv.org/abs/2309.14316

    arXiv 2024

  5. [5]

    Apache parquet

    Apache Software Foundation . Apache parquet. https://parquet.apache.org/, 2013. Columnar storage format for Hadoop and big data processing

    work page 2013

  6. [6]

    Apache arrow: A cross-language development platform for in-memory data

    Apache Software Foundation . Apache arrow: A cross-language development platform for in-memory data. https://arrow.apache.org/, 2016. Versioned columnar memory format for flat and hierarchical data

    work page 2016

  7. [7]

    Refusal in Language Models Is Mediated by a Single Direction , 2024

    Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., and Nanda, N. Refusal in Language Models Is Mediated by a Single Direction , 2024. URL https://arxiv.org/abs/2406.11717

    arXiv 2024

  8. [8]

    A Survey of Word Embeddings Evaluation Methods

    Bakarov, A. A Survey of Word Embeddings Evaluation Methods , 2018. URL https://arxiv.org/abs/1801.09536

    work page Pith review arXiv 2018

Show all 78 references
  1. [9]

    and Gavves, E

    Bereska, L. and Gavves, E. Mechanistic interpretability for ai safety -- a review, 2024. URL https://arxiv.org/abs/2404.14082

    2024 arXiv

  2. [10]

    Bloom, J., Tigges, C., Duong, A., and Chanin, D. SAELens . https://github.com/jbloomAus/SAELens, 2024

    2024

  3. [11]

    Braun, M., Krebs, S., Flohr, F., and Gavrila, D. M. EuroCity Persons: A Novel Benchmark for Person Detection in Traffic Scenes . IEEE Transactions on Pattern Analysis and Machine Intelligence, 41 0 (8): 0 1844–1861, August 2019. ISSN 1939-3539. doi:10.1109/tpami.2019.2897684. ...

    2019

  4. [12]

    E., Hume, T., Carter, S., Henighan, T., and Olah, C

    Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Car...

    2023

  5. [13]

    BatchTopK Sparse Autoencoders , 2024

    Bussmann, B., Leask, P., and Nanda, N. BatchTopK Sparse Autoencoders , 2024. URL https://arxiv.org/abs/2412.06410

    2024 arXiv

  6. [14]

    Learning Multi-Level Features with Matryoshka Sparse Autoencoders , 2025

    Bussmann, B., Nabeshima, N., Karvonen, A., and Nanda, N. Learning Multi-Level Features with Matryoshka Sparse Autoencoders , 2025. URL https://arxiv.org/abs/2503.17547

    2025 arXiv

  7. [15]

    M., Favero, A., and Wyart, M

    Cagnetta, F., Petrini, L., Tomasini, U. M., Favero, A., and Wyart, M. How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model . Physical Review X, 14 0 (3), July 2024. ISSN 2160-3308. doi:10.1103/physrevx.14.031001. URL http://dx.doi.org/10.1103/PhysRevX....

    2024 doi

  8. [16]

    On evaluating adversarial robustness, 2019

    Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Madry, A., and Kurakin, A. On evaluating adversarial robustness, 2019. URL https://arxiv.org/abs/1902.06705

    2019 arXiv

  9. [17]

    DLST 4: Phase Transitions in Neural Networks

    Carroll, L. DLST 4: Phase Transitions in Neural Networks . https://www.lesswrong.com/s/czrXjvCLsqGepybHC/p/aKBAYN5LpaQMrPqMj, June 2023. LessWrong blog post

    2023

  10. [18]

    C., and Lin, D

    Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., Zhang, Z., Cheng, D., Zhu, C., Cheng, T., Zhao, Q., Li, B., Lu, X., Zhu, R., Wu, Y., Dai, J., Wang, J., Shi, J., Ouyang, W., Loy, C. C., and Lin, D. MMDetection: Open MMLab Detection ...

    1906 arXiv

  11. [19]

    Generative Region-Language Pretraining for Open-Ended Object Detection

    Chuang, L., Yi, J., Lizhen, Q., Zehuan, Y., and Jianfei, C. Generative Region-Language Pretraining for Open-Ended Object Detection . In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  12. [20]

    Sparse Autoencoders Find Highly Interpretable Features in Language Models , 2023

    Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L. Sparse Autoencoders Find Highly Interpretable Features in Language Models , 2023. URL https://arxiv.org/abs/2309.08600

    2023 arXiv

  13. [21]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019. URL https://arxiv.org/abs/1810.04805

    2019 arXiv

  14. [22]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2021

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2021. URL https://arxiv.org/a...

    2021 arXiv

  15. [23]

    Transcoders Find Interpretable LLM Feature Circuits , 2024

    Dunefsky, J., Chlenski, P., and Nanda, N. Transcoders Find Interpretable LLM Feature Circuits , 2024. URL https://arxiv.org/abs/2406.11944

    2024 arXiv

  16. [24]

    Garçon , December 2021

    Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., DasSarma, N., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan...

    2021

  17. [25]

    R., Todd, E., Brinkmann, J., Pal, K., Troitskii, D., Ripa, M., Belfki, A., Rager, C., Juang, C., Mueller, A., Marks, S., Sharma, A

    Fiotto-Kaufman, J., Loftus, A. R., Todd, E., Brinkmann, J., Pal, K., Troitskii, D., Ripa, M., Belfki, A., Rager, C., Juang, C., Mueller, A., Marks, S., Sharma, A. S., Lucchetti, F., Prakash, N., Brodley, C., Guha, A., Bell, J., Wallace, B. C., and Bau, D. Nnsight and ndif: Dem...

    2025 arXiv

  18. [26]

    D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J

    Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scaling and evaluating sparse autoencoders , 2024. URL https://arxiv.org/abs/2406.04093

    2024 arXiv

  19. [27]

    Detecting strategic deception using linear probes, 2025

    Goldowsky-Dill, N., Chughtai, B., Heimersheim, S., and Hobbhahn, M. Detecting strategic deception using linear probes, 2025. URL https://arxiv.org/abs/2502.03407

    2025 arXiv

  20. [28]

    and Hagn, K

    Grau, O. and Hagn, K. VALERIE22 -- A photorealistic, richly metadata annotated dataset of urban environments , 2023. URL https://arxiv.org/abs/2308.09632

    2023 arXiv

  21. [29]

    and Segerie, C.-R

    Grey, M. and Segerie, C.-R. Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods , 2025. URL https://arxiv.org/abs/2505.05541

    2025 arXiv

  22. [30]

    LVIS: A Dataset for Large Vocabulary Instance Segmentation , 2019

    Gupta, A., Dollár, P., and Girshick, R. LVIS: A Dataset for Large Vocabulary Instance Segmentation , 2019. URL https://arxiv.org/abs/1908.03195

    2019 arXiv

  23. [31]

    and Nanda, N

    Heimersheim, S. and Nanda, N. How to use and interpret activation patching, 2024. URL https://arxiv.org/abs/2404.15255

    2024 arXiv

  24. [32]

    FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods , 2023

    Hesse, R., Schaub-Meyer, S., and Roth, S. FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods , 2023. URL https://arxiv.org/abs/2308.06248

    2023 arXiv

  25. [33]

    A comprehensive survey on applications of transformers for deep learning tasks, 2023

    Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., and Pedrycz, W. A comprehensive survey on applications of transformers for deep learning tasks, 2023. URL https://arxiv.org/abs/2306.07303

    2023 arXiv

  26. [34]

    Ultralytics YOLO , January 2023

    Jocher, G., Qiu, J., and Chaurasia, A. Ultralytics YOLO , January 2023. URL https://github.com/ultralytics/ultralytics

    2023

  27. [35]

    Are sparse autoencoders useful? a case study in sparse probing, 2025

    Kantamneni, S., Engels, J., Rajamanoharan, S., Tegmark, M., and Nanda, N. Are sparse autoencoders useful? a case study in sparse probing, 2025. URL https://arxiv.org/abs/2502.16681

    2025 arXiv

  28. [36]

    The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale

    Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., Duerig, T., and Ferrari, V. The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scal...

    2020

  29. [37]

    NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models , 2025

    Lee, C., Roy, R., Xu, M., Raiman, J., Shoeybi, M., Catanzaro, B., and Ping, W. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models , 2025. URL https://arxiv.org/abs/2405.17428

    2025 arXiv

  30. [38]

    A., Tartaglini, A

    Lepori, M. A., Tartaglini, A. R., Vong, W. K., Serre, T., Lake, B. M., and Pavlick, E. Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects , 2024. URL https://arxiv.org/abs/2406.15955

    2024 arXiv

  31. [39]

    Open World Object Detection: A Survey , 2024

    Li, Y., Wang, Y., Wang, W., Lin, D., Li, B., and Yap, K.-H. Open World Object Detection: A Survey , 2024. URL https://arxiv.org/abs/2410.11301

    2024 arXiv

  32. [40]

    L., and Dollár, P

    Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C. L., and Dollár, P. Microsoft COCO: Common Objects in Context , 2015. URL https://arxiv.org/abs/1405.0312

    2015 arXiv

  33. [41]

    Sparse Crosscoders for Cross-Layer Features and Model Diffing

    Lindsey, J., Templeton, A., Marcus, J., Conerly, T., Batson, J., and Olah, C. Sparse Crosscoders for Cross-Layer Features and Model Diffing . Transformer Circuits Thread, 2024. URL https://transformer-circuits.pub/2024/crosscoders/index.html

    2024

  34. [42]

    Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., and Lee, Y. J. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge , January 2024 a . URL https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  35. [43]

    Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection , 2024 b

    Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., Zhu, J., and Zhang, L. Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection , 2024 b . URL https://arxiv.org/abs/2303.05499

    2024 arXiv

  36. [44]

    Data Attribution: A Data-Centric Approach for Trustworthy AI Development

    Ma, J. Data Attribution: A Data-Centric Approach for Trustworthy AI Development . Proceedings of the AAAI Conference on Artificial Intelligence, 39 0 (27): 0 28720--28720, Apr. 2025. doi:10.1609/aaai.v39i27.35114. URL https://ojs.aaai.org/index.php/AAAI/article/view/35114

    2025 doi

  37. [45]

    A Survey on Vision-Language-Action Models for Embodied AI , 2025

    Ma, Y., Song, Z., Zhuang, Y., Hao, J., and King, I. A Survey on Vision-Language-Action Models for Embodied AI , 2025. URL https://arxiv.org/abs/2405.14093

    2025 arXiv

  38. [46]

    and Frey, B

    Makhzani, A. and Frey, B. k-Sparse Autoencoders , 2014. URL https://arxiv.org/abs/1312.5663

    2014 arXiv

  39. [47]

    J., Belinkov, Y., Bau, D., and Mueller, A

    Marks, S., Rager, C., Michaud, E. J., Belinkov, Y., Bau, D., and Mueller, A. Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models , 2024. URL https://arxiv.org/abs/2403.19647

    2024 arXiv

  40. [48]

    Deep Double Descent: Where Bigger Models and More Data Hurt , 2019

    Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., and Sutskever, I. Deep Double Descent: Where Bigger Models and More Data Hurt , 2019. URL https://arxiv.org/abs/1912.02292

    2019 arXiv

  41. [49]

    Interpretability will not reliably find deceptive ai

    Nanda, N. Interpretability will not reliably find deceptive ai. https://www.lesswrong.com/posts/PwnadG4BFjaER3MGf, 2025

    2025

  42. [50]

    and Bloom, J

    Nanda, N. and Bloom, J. TransformerLens . https://github.com/TransformerLensOrg/TransformerLens, 2022

    2022

  43. [51]

    Understanding Neural Networks via Feature Visualization: A survey , 2019

    Nguyen, A., Yosinski, J., and Clune, J. Understanding Neural Networks via Feature Visualization: A survey , 2019. URL https://arxiv.org/abs/1904.08939

    2019 arXiv

  44. [52]

    In-context Learning and Induction Heads

    Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Cla...

    2022

  45. [53]

    GPT-4o System Card , 2024

    OpenAI. GPT-4o System Card , 2024. URL https://arxiv.org/abs/2410.21276

    2024 arXiv

  46. [54]

    Patro, B. N. and Agneeswaran, V. S. Mamba-360: Survey of state space models as transformer alternative for long sequence modelling: Methods, applications, and challenges, 2024. URL https://arxiv.org/abs/2404.16112

    2024 arXiv

  47. [55]

    Introducing Gemini 2.0: our new AI model for the agentic era , December 2024

    Pichai, S., Hassabis, D., and Kavukcuoglu, K. Introducing Gemini 2.0: our new AI model for the agentic era , December 2024. URL https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/. Accessed: 2025-01-03

    2024

  48. [56]

    On Evaluating the Durability of Safeguards for Open-Weight LLMs , 2024

    Qi, X., Wei, B., Carlini, N., Huang, Y., Xie, T., He, L., Jagielski, M., Nasr, M., Mittal, P., and Henderson, P. On Evaluating the Durability of Safeguards for Open-Weight LLMs , 2024. URL https://arxiv.org/abs/2412.07097

    2024 arXiv

  49. [57]

    Language Models are Unsupervised Multitask Learners , 2019

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language Models are Unsupervised Multitask Learners , 2019. URL https://api.semanticscholar.org/CorpusID:160025533

    2019

  50. [58]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning Transferable Visual Models From Natural Language Supervision , 2021. URL https://arxiv.org/abs/2103.00020

    2021 arXiv

  51. [59]

    Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2016

    Ren, S., He, K., Girshick, R., and Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2016. URL https://arxiv.org/abs/1506.01497

    2016 arXiv

  52. [60]

    A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data , 2024

    Sclocchi, A., Favero, A., and Wyart, M. A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data , 2024. URL https://arxiv.org/abs/2402.16991

    2024 arXiv

  53. [61]

    b1ade series of models , 2024

    Shreyas Subramanian . b1ade series of models , 2024. URL https://huggingface.co/w601sxs/b1ade-embed

    2024

  54. [62]

    The 'strong' feature hypothesis could be wrong

    Smith, L. The 'strong' feature hypothesis could be wrong . AI Alignment Forum , 2024. URL https://www.alignmentforum.org/posts/tojtPCCRpKLSHBdpn/the-strong-feature-hypothesis-could-be-wrong

    2024

  55. [63]

    Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update \#2)

    Smith, L., Rajamanoharan, S., Conmy, A., McDougall, C., Lieberum, T., Kramár, J., Shah, R., and Nanda, N. Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update \#2) . https://www.lesswrong.com/posts/4uXCAJNuPKtKBsi2...

    2025

  56. [64]

    Steiner, A., Pinto, A. S., Tschannen, M., Keysers, D., Wang, X., Bitton, Y., Gritsenko, A., Minderer, M., Sherbondy, A., Long, S., Qin, S., Ingle, R., Bugliarello, E., Kazemzadeh, S., Mesnard, T., Alabdulmohsin, I., Beyer, L., and Zhai, X. PaliGemma 2: A Family of Versatile VL...

    2024 arXiv

  57. [65]

    Attribution Patching Outperforms Automated Circuit Discovery , 2023

    Syed, A., Rager, C., and Conmy, A. Attribution Patching Outperforms Automated Circuit Discovery , 2023. URL https://arxiv.org/abs/2310.10348

    2023 arXiv

  58. [66]

    R., Abeyruwan, S., Ainslie, J., Alayrac, J.-B., Arenas, M

    Team, G. R., Abeyruwan, S., Ainslie, J., Alayrac, J.-B., Arenas, M. G., Armstrong, T., Balakrishna, A., Baruch, R., Bauza, M., Blokzijl, M., Bohez, S., Bousmalis, K., Brohan, A., Buschmann, T., Byravan, A., Cabi, S., Caluwaerts, K., Casarini, F., Chang, O., Chen, J. E., Chen, ...

    2025 arXiv

  59. [67]

    L., McDougall, C., MacDiarmid, M., Freeman, C

    Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N. L., McDougall, C., MacDiarmid, M., Freeman, C. D., Sumers, T. R., Rees, E., Batson, J., Jermyn, A., Carter, S., Olah, C., and H...

    2024

  60. [68]

    and Zaslavsky, N

    Tishby, N. and Zaslavsky, N. Deep Learning and the Information Bottleneck Principle , 2015. URL https://arxiv.org/abs/1503.02406

    2015 arXiv

  61. [69]

    L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A

    Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. HuggingFace's Tr...

    2020 arXiv

  62. [70]

    Detectron2

    Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., and Girshick, R. Detectron2 . https://github.com/facebookresearch/detectron2, 2019

    2019

  63. [71]

    Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

    Xiao, B., Wu, H., Xu, W., Dai, X., Hu, H., Lu, Y., Zeng, M., Liu, C., and Yuan, L. Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 4818--4829, June 2024

    2024

  64. [72]

    Diffusion models: A comprehensive survey of methods and applications, 2024

    Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., and Yang, M.-H. Diffusion models: A comprehensive survey of methods and applications, 2024. URL https://arxiv.org/abs/2209.00796

    2024

  65. [73]

    BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning , 2020

    Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., and Darrell, T. BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning , 2020. URL https://arxiv.org/abs/1805.04687

    2020 arXiv

  66. [74]

    M., and Shum, H.-Y

    Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L. M., and Shum, H.-Y. DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection , 2022. URL https://arxiv.org/abs/2203.03605

    2022 arXiv

  67. [75]

    Vision-Language Models for Vision Tasks: A Survey , 2024

    Zhang, J., Huang, J., Jin, S., and Lu, S. Vision-Language Models for Vision Tasks: A Survey , 2024. URL https://arxiv.org/abs/2304.00685

    2024 arXiv

  68. [76]

    L., Zimmer, W., Cao, H., and Knoll, A

    Zhou, X., Liu, M., Yurtsever, E., Zagar, B. L., Zimmer, W., Cao, H., and Knoll, A. C. Vision Language Models in Autonomous Driving: A Survey and Outlook , 2024. URL https://arxiv.org/abs/2310.14414

    2024 arXiv

  69. [77]

    Object Detection in 20 Years: A Survey

    Zou, Z., Chen, K., Shi, Z., Guo, Y., and Ye, J. Object Detection in 20 Years: A Survey . Proceedings of the IEEE, 111 0 (3): 0 257--276, 2023. doi:10.1109/JPROC.2023.3238524

    2023

  70. [78]

    and Ilievski, E

    Žunkovič, B. and Ilievski, E. Grokking phase transitions in learning local rules with gradient descent , 2022. URL https://arxiv.org/abs/2210.15435

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.