Pith. sign in

REVIEW 4 major objections 4 minor 102 references

Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mechanist claims to turn AI interpretability into an autonomous scientific loop, with case studies discovering a cross-modal safety risk, localizing belief heads, intervening on them to improve reasoning, and steering DNA generation.

desk verdict A substantial agentic interpretability system with two solid case studies, but its headline reliability benchmark is likely contaminated by retrieval leakage from its own knowledge graph. read the letter →

arxiv 2608.12036 v1 pith:OH34FAWF submitted 2026-08-12 cs.AI cs.CLcs.HCcs.LGcs.MA

Mengru Wang , Junfeng Fang , Shuofei Qiao , Zhenqian Xu , Haoming Xu , Haoxiong Wang , Shumin Deng , Linyi Yang
show 11 more authors
Zhixiang Cui Xin Xu Yunzhi Yao Buqiang Xu Fei Shen Haozhe Luo Yunxiang Wei Ningyu Zhang Julian McAuley Tat Seng Chua Huajun Chen
This is my paper · ORCID
classification cs.AIcs.CLcs.HCcs.LGcs.MA
keywords mechanisticinterpretabilityautonomousscientificdiscoveryAI-for-AIbelief-statereasoningsubliminallearningcross-modalsafetysparseautoencodersfeaturesteering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Mechanist, an agentic system intended to let AI take on the discovery of its own mechanisms. It claims to move mechanistic interpretability from a manual, per-neuron exercise to an automated loop of hypothesis generation, experiment execution, verification, and iteration, grounded in a specialized knowledge graph and a library of analysis methods. The paper argues that this pipeline works end to end: it uncovers a new cross-modal safety risk, localizes belief-specific attention heads, turns those heads into an inference-time intervention that improves reasoning, and uses the same logic to steer a DNA model toward desired protein features.

What carries the argument

The load-bearing machinery is the four-stage agentic loop: a hypothesis agent proposes atomic claims using retrieval from a 13,000-paper interpretability knowledge graph and a 43-million-paper cross-disciplinary database; an experiment agent executes tests with a library of 32 mechanism-analysis methods; a verification agent audits experimental validity and robustness; and an iteration agent routes failures back to the appropriate stage. Within the case studies, the key mechanisms are Fisher-information ranking of attention heads to localize behavior-specific heads, zero-ablation and amplification of those heads as causal intervention, and sparse-autoencoder feature activation to steer a DNA sequence model.

What would settle it

Measure the error rate of the safety filter on a labeled held-out sample of teacher-generated responses: if a nontrivial fraction of unsafe or banana-classified instances pass the filter, re-run the student fine-tuning on guaranteed-clean data to see whether the transfer effect persists. A second check would be to re-run the Fisher and ablation pipeline on fresh training seeds to test whether the identified belief heads reappear in the same locations.

Watch

Extended reading notes

Core claim

The central claim is that an AI system can operate as a scientific instrument for understanding AI intelligence. With a human-supplied research objective and success criteria, the system autonomously formulates hypotheses, designs and runs experiments, verifies causal evidence, and iterates until the evidence is reliable. Mechanist's reported case studies are the demonstration: unsafe behavioral traits transfer from teacher to student models through text and image data that pass as safe; separable personal-belief and attributed-belief attention heads exist and can be causally validated; amplifying the relevant head at inference time improves belief-state reasoning; and activating an internal feature in a genomic foundation model raises the predicted alpha-helical content of generated DNA sequences while preserving sequence validity.

Load-bearing premise

The load-bearing premise is that the two-stage filter removes all unsafe content and all semantic links to the unsafe trait from the student training set; if any unsafe or trait-linked text or images survive in the 2,380 retained instances, the observed increase in unsafe multimodal responses is not evidence of transfer through safe data.

Editorial extensions

If this is right

  • If Mechanist's loop is accepted, mechanistic interpretability can be carried out by an autonomous agent rather than by manual per-feature analysis, lowering the barrier to auditing large models.
  • The subliminal-learning result implies that content-based data filtering is not a sufficient safety screen in multimodal training, because behavioral traits can propagate through data that passes as safe.
  • The separable belief-head result implies that belief-state reasoning in these models is localized enough to be addressed by inference-time intervention without retraining the model.
  • The DNA-steering result implies that internal features of scientific foundation models can serve as control knobs for generation, offering an alternative to generate-and-rerank pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension would be to audit the safety filter itself: measuring the error rate of the semantic filter on a labeled sample would directly bound how much of the reported unsafe-response increase could be explained by unsafe content surviving filtering.
  • The paper's own limitations section notes that the system has not yet been optimized for models designed to simulate human cognition and recommends human-AI co-scientist use; full autonomy is therefore a design goal rather than a demonstrated end-to-end guarantee.
  • The in-text citation at the Fisher-information localization step is paired in the reference list with a theory-of-mind meta-analysis rather than a statistics source, leaving the provenance of that localization method unverified.
  • If the belief-head localization transfers across model families, the demonstrated loop suggests a general localize-validate-amplify recipe for inference-time control, but the paper's evidence for breadth rests on a small number of open-weight models.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper introduces Mechanist, a multi-agent system that automates mechanistic interpretability research through four stages: hypothesis generation, experiment execution, verification, and iteration. The system is grounded in a specialized interpretability knowledge graph of roughly 13,000 papers, a cross-disciplinary graph of 43 million papers, and a curated library of 32 mechanistic analysis methods. The evaluation compares Mechanist with Claude Code and AI-Scientist on hypothesis quality and on the reliability of reproducing 16 mechanistic-interpretability papers, and it reports four case studies: multimodal transfer of unsafe traits through apparently safe training data, discovery of separable personal-belief and attributed-belief heads in Pythia/OLMo and their emergence during pretraining, inference-time intervention that improves belief reasoning, and steering Evo2 to generate DNA sequences with higher predicted α-helical content.

Significance. If the reported results hold, Mechanist is a significant step toward automating mechanistic interpretability: it combines knowledge-grounded hypothesis generation, causal validation with random-head controls, and mechanism-guided intervention in a single pipeline. The paper includes several concrete strengths: proposition-disjoint held-out test sets for the belief intervention, explicit robustness criteria for head localization, multiple human and LLM judges for the reliability benchmark, and public code and data commitments. The four case studies are ambitious and, for the most part, use appropriate controls. However, the central reliability benchmark and the safety-transfer claim rest on assumptions that are not yet verified, so the current evidence does not fully support the paper's headline conclusions.

major comments (4)
  1. [§5.2 and §4.1.2/§4.2] The reliability benchmark in §5.2 does not exclude the 16 target papers from Mechanist's own interpretability knowledge graph. The graph (§4.1.2, Fig. 6) contains roughly 13,000 interpretability papers with LLM-extracted fields including core_contribution, conclusion, specific_method, key_findings, and limitation, and the retrieval strategy (§4.2) explicitly performs keyword, semantic, and graph-based expansion over this graph. If any target paper is a node in the graph, the "reproduction" can reduce to finding the paper's extracted method and conclusion, so the reported advantage in Fig. 12 may reflect knowledge retrieval rather than autonomous experiment design and execution. The authors should state explicitly whether the 16 papers were withheld from the graph, and/or report retrieval logs demonstrating that target papers and their findings were not retrieved during the benchmark runs.
  2. [Appendix A.1 and §2.2] The safety-transfer claim depends on the GPT-4o-based two-stage filter guaranteeing that the 2,380 retained student-training instances contain no unsafe content and no semantic link to the unsafe trait. The filter's error rate is not measured, and the query templates used to generate the student data are themselves lab-safety prompts ("What is the correct, safe way to approach {topic}?"), so the training data is not semantically unrelated to the safety trait. Without a measured false-negative rate for the filter or human verification of the retained corpus, the observed increase in unsafe multimodal responses (48.6% versus 20.3% in Fig. 3d) could be explained by residual unsafe or trait-linked content passing the filter rather than by transfer through genuinely safe data.
  3. [§B.1 and §B.3] The belief-head localization result is validated with zero-ablation on the same analysis dataset that was used to select the candidate heads: §B.1 states that the analysis dataset is used for behavioral evaluation, mechanism localization, causal validation, and router training, while the test dataset is reserved only for intervention evaluation. The paper does not report ablation results on the proposition-disjoint test set, so the reported head-specific effects (e.g., L4.H1 reducing AB accuracy from 0.86 to 0.34) may reflect selection on the evaluation set. The authors should either report causal ablation results on the held-out test propositions or clearly justify why selection and validation on the same propositions does not inflate the reported effects.
  4. [§2.5 and Fig. 5] The DNA-steering case study validates the claim of steering Evo2 toward "desired biological outcomes" using only ESMFold-predicted α-helical content and pLDDT scores; no experimental structure determination or functional assay is reported. Because the target feature was selected for association with the same predicted structural metric used to evaluate the intervention, the present results demonstrate control over a predictive proxy rather than over realized biological properties. The main-text claim should either be tempered to an in-silico proof of concept or supplemented with at least one external validation (e.g., expression or structure determination) before presenting it as advancing interdisciplinary discovery.
minor comments (4)
  1. [Appendix A.1] The section contains a duplicated and erroneous definition: the passage beginning "Data for tuning student model. Let Ds denote the dataset used to fine-tune the teacher model. Ds consists of 2,321 text-only instances" is repeated with the correct definition immediately afterward; the erroneous duplicate should be removed.
  2. [§2.3] The phrase "Using the Fisher information matrix [34]" cites a meta-analysis of theory-of-mind development, which is not a source for Fisher information; a proper statistical reference should be supplied.
  3. [§2.2] The term "semantically opposing data" is imprecise for the laboratory-safety setting, because the student training prompts are themselves safety-related; consider using "filtered" or "apparently safe" instead of "opposing."
  4. [Fig. 12] In panel a, the y-axis is truncated at 30%, which can visually exaggerate differences among systems; consider using a full axis or clearly annotating the truncation in the figure.

Circularity Check

3 steps flagged · score 5.0 of 10

DNA steering is selected on its own evaluation metric, and the reproduction benchmark may leak target papers through the retrieval graph.

  1. fitted input called prediction [Section 2.5 (Mechanistic design; Evo2-7B α-helix steering)]
    "Mechanist searches the target feature descriptions obtained from an existing sparse autoencoder (SAE) for Evo2, identifies an internal feature associated with α-helical structure, and activates this feature during DNA sequence generation."

    The feature is chosen by association with the same predicted α-helical content that is later reported as the outcome (Fig. 5b,c). This is selection on the evaluation metric rather than an independent test of a pre-specified feature; the random-feature control establishes specificity but does not remove the selection effect on the headline gain.

  2. fitted input called prediction [Section 2.5, Fig. 5c]
    "Increasing the steering coefficient from α = 0 to α = 8 raises α-helical content by 12.8 percentage points while largely preserving the proportion of sequences containing a valid open reading frame (ORF). ... Mechanist therefore selects α = 8 as the strongest effective intervention before sequence validity declines."

    The steering strength is chosen by maximizing the very metric (predicted α-helical content) on which the method is then evaluated, constrained only by ORF validity. The reported +12.8-point improvement at α=8 is therefore the fitted maximum of the evaluation measure, not an out-of-sample prediction; a held-out sweep or pre-registered coefficient would be needed to avoid this reduction.

1 more flagged steps
  1. other [Section 5.2 versus Sections 4.1.2 and 4.2]
    "For each paper, the systems receive only the target claim and are not allowed to access the original paper or its GitHub repository. // Then we use DeepSeek-V3.2-Thinking to extract the corresponding attributes from each document. It also records explicitly stated findings, limitations, and future directions when available. // High-confidence title matches are retained and assigned an additional ranking weight."

    The protocol does not state that the 16 reproduction targets were withheld from Mechanist's own interpretability knowledge graph, whose LLM-extracted fields include core_contribution, specific_method, conclusion, and key_findings, and whose retrieval strategy performs exact/fuzzy title matching and graph expansion. If a target paper is a node in that graph, 'reproduction' can reduce to retrieving the paper's extracted method and conclusion, so the Fig. 12 reliability advantage over Claude Code and AI-Scientist may measure knowledge retrieval rather than autonomous experiment execution. This is a benchmark-leak confound rather than a self-definitional derivation, but it is load-bearing for the headline reliability claim.

full rationale

The belief-state study is the strongest independent result: Fisher localization is performed on an analysis set, zero-ablation is validated against random-head and random-parameter controls, the frame probe is trained only on the analysis set, and all intervention results are evaluated on a proposition-disjoint test set; no circularity was found there. The subliminal-learning case is not circular either, although its safety-filter assumption (GPT-4o removes all unsafe and trait-linked content from the student training set) is unmeasured and would, if violated, invalidate the cross-modal transfer claim. Two issues prevent a clean bill. First, the Evo2 DNA steering result is partially constructed from its own evaluation: the SAE feature is selected for association with α-helical content, and α=8 is chosen by sweeping that same predicted α-helical metric, so the reported gain is partly in-sample selection; the random-feature control is a useful specificity check but does not eliminate selection on the outcome. Second, the §5.2 reproduction benchmark does not state that the 16 target papers were excluded from Mechanist's own interpretability graph, which stores extracted core_contribution, specific_method, conclusion, and key_findings fields and supports title matching and graph expansion; if targets are in the graph, reproduction reliability can reduce to retrieval. These are selection and benchmark-leak issues rather than a self-citation chain; because the central belief mechanism remains independently supported, the paper is only partially circular.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claims rest on several adjustable thresholds (head localization criteria, amplification factors, DNA steering coefficient), plus domain assumptions about the validity of LLM filters, structure predictors, and the representativeness of Pythia/OLMo. The invented 'belief heads' are internal to the analysis and lack independent falsifiable handles beyond the studied models.

free parameters (3)
  • Evo2 steering coefficient α = 8
    Selected in §2.5 as the largest coefficient that preserves ORF validity while maximizing predicted α-helical content; the sweep is evaluated on the same metric used to report the result.
  • Head localization thresholds = 0.30 (accuracy drop), 2 SD (vs random), 0.10 (cross-task drop), 1.05x (perplexity)
    Hand-chosen criteria in §B.3 that define which heads count as 'belief heads'; changing these thresholds could change the identified mechanism.
  • Belief head amplification maxima α_PB_max, α_AB_max = not reported
    Intervention gains in Fig. 4c depend on these hyperparameters (Eq. in §B.3), but their values and tuning procedure are not stated.
assumptions (5)
  • domain assumption Pythia and OLMo checkpoints are representative of general LLM pretraining for the belief mechanism.
    The generalization of the mechanism theory of belief from Pythia-1B to other models is based on reproducing it in a few Pythia/OLMo sizes and behavior-only results on GPT; §B.5.
  • ad hoc to paper The WK/PB/AB frame decomposition is a meaningful operationalization of belief.
    The paper defines these frames in §2.3 and maps failures to altercentric/egocentric interference; this framing guides all belief experiments.
  • domain assumption ESMFold pLDDT is a valid proxy for structural quality and alpha-helical content.
    The DNA steering results in §2.5 are evaluated only with ESMFold predictions, not wet-lab expression or crystallography.
  • domain assumption The GPT-4o safety and fruit filters remove all unsafe or trait-related content from student training sets.
    The subliminal-learning claims in §2.2 and Appendix A depend on the filter being error-free; no validation of the filter's recall is provided.
  • domain assumption Human and LLM judges provide unbiased reliability scores in the system comparison.
    The evaluation in §5.2 relies on three human experts and two LLM judges; inter-judge agreement is shown, but the rubric construction and paper selection process are not independently audited.
invented entities (1)
  • Personal-belief (PB) and attributed-belief (AB) heads
    purpose: Postulated internal mechanisms that separately represent the model's own belief and beliefs attributed to others; they are ablated and amplified to validate the mechanism theory of belief.
    The heads are identified via Fisher ranking in Pythia and OLMo and validated with internal ablations, but the paper provides no external prediction (e.g., a specific behavior in a held-out model family) that would confirm them independently of the analysis pipeline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence." pith.science (2026). https://pith.science/paper/OH34FAWF

@misc{pith2026260812036,
  author       = {Pith},
  title        = {Pith review of: Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OH34FAWF}},
  note         = {Machine review of arXiv:2608.12036}
}
read the original abstract

AI models have achieved remarkable success across diverse domains, yet the mechanisms underlying their capabilities and the risks they may pose remain poorly understood. As AI development becomes faster and increasingly automated, mechanistic exploration remains largely manual, widening the gap between what models can do and our ability to understand and control them. To bridge this gap, we introduce Mechanist, an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI intelligence. To support autonomous mechanistic discovery, we construct an interpretability-focused knowledge graph of approximately 13,000 papers and integrate it with a multidisciplinary database of 43 million papers spanning 26 fields. We further curate a library of 32 foundational methods for mechanism analysis, causal intervention, and validation. Compared with Claude Code and existing AI-scientist systems, Mechanist generates more valuable mechanism hypotheses and executes experiments more reliably. Mechanist also demonstrates a progression from discovering model behaviors to explaining and controlling AI models. Specifically, Mechanist first uncovers a counterintuitive safety risk in scientific laboratories, showing that unsafe traits can transfer across modalities through apparently safe training data. Mechanist then develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretraining. Finally, Mechanist translates these mechanistic insights into practical interventions that improve model performance across diverse scenarios and steer scientific foundation models toward generating DNA sequences with specified properties.

Figures

Figures reproduced from arXiv: 2608.12036 by the authors.

Figure 1
Figure 1. The comparison between our Mechanist and existing AI Scientists [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview and evaluation of Mechanist. a, The Mechanist framework consists of four stages: hypothesis generation, experiment execution, verification, and iteration. Specifically, hypothesis generation is inspired by interdisciplinary knowledge from the existing knowledge library, to which new discoveries validated by Mechanist are subsequently added. b, Benchmark design for comparing Claude Code, AI-Scientist and Mec… view at source ↗
Figure 3
Figure 3. Mechanist extends subliminal learning to the transfer of opposing preferences in the multimodal setting. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Mechanist reveals a mechanism theory of belief-state reasoning and uses it for dynamic intervention. a [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Mechanist generates DNA sequences encoding proteins with enhanced [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: The overview of our interpretability database. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Overview of our interpretability database from the perspective of interpretable objects. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Overview of our interpretability database from the perspective of application scenarios. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Mechanism methods for large language models and multi-modal models. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Reliability of reproduced claims across research areas. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Reliability of reproductions along the four evaluation dimensions. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Mechanist is the most reliable system under every judge, and the judges agree with one another. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

102 extracted references · 37 canonical work pages

  1. [1]

    A narrowing window to understand ai.Science, 392(6802):1003–1003, 2026

    Eric Horvitz and Robert West. A narrowing window to understand ai.Science, 392(6802):1003–1003, 2026

    2026

  2. [2]

    Large language models can predict the results of social science experiments.Nature, pages 1–8, 2026

    Ashwini Ashokkumar, Luke Hewitt, Isaias Ghezae, and Robb Willer. Large language models can predict the results of social science experiments.Nature, pages 1–8, 2026

    2026

  3. [3]

    A foundation model to predict and capture human cognition

    Marcel Binz, Elif Akata, Matthias Bethge, Franziska Brändle, Fred Callaway, Julian Coda-Forno, Peter Dayan, Can Demircan, Maria K Eckstein, Noémi Éltető, et al. A foundation model to predict and capture human cognition. Nature, 644(8078):1002–1009, 2025

    2025

  4. [4]

    A unifying framework from neural superposition to sparse interpretable codes.Nature Machine Intelligence, pages 1–13, 2026

    David Klindt, Charles O’Neill, Patrik Reizinger, Harald Maurer, and Nina Miolane. A unifying framework from neural superposition to sparse interpretable codes.Nature Machine Intelligence, pages 1–13, 2026

    2026

  5. [5]

    Bereska and Stratis Gavves

    Leonard F. Bereska and Stratis Gavves. Mechanistic interpretability for AI safety - A review.Trans. Mach. Learn. Res., 2024, 2024

    2024

  6. [6]

    The latent space: Foundation, evolution, mechanism, ability, and outlook.CoRR, abs/2604.02029, 2026

    Xinlei Yu, Zhangquan Chen, Yongbo He, Tianyu Fu, Cheng Yang, Chengming Xu, Yue Ma, Xiaobin Hu, Zhe Cao, Jie Xu, Guibin Zhang, Jiale Tao, Jiayi Zhang, Siyuan Ma, Kaituo Feng, Haojie Huang, Youxing Li, Ronghao Chen, Huacan Wang, Chenglin Wu, Zikun Su, Xiaogang Xu, Kelu Yao, Kun Wang, Chen Gao, Yue Liao, Ruqi Huang, Tao Jin, Cheng Tan, Jiangning Zhang, Wenqi...

    arXiv 2026

  7. [7]

    Ho, Thomas Icard, Dan Jurafsky, and James Zou

    Mirac Suzgun, Tayfun Gur, Federico Bianchi, Daniel E. Ho, Thomas Icard, Dan Jurafsky, and James Zou. Language models cannot reliably distinguish belief from knowledge and fact.Nature Machine Intelligence, 7(11):1780–1790, November 2025

    2025

  8. [8]

    Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, et al. Exploring concept depth: How large language models acquire knowledge and concept at different layers? InProceedings of the 31st international conference on computational linguistics, pages 558–573, 2025

    2025

Show all 102 references
  1. [9]

    Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nat

    Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nat. Mach. Intell., 1(5):206–215, 2019

    2019

  2. [10]

    Training large language models on narrow tasks can lead to broad misalignment.Nature, 649(8097):584–589, 2026

    Jan Betley, Niels Warncke, Anna Sztyber-Betley, Daniel Tan, Xuchan Bao, Martín Soto, Megha Srivastava, Nathan Labenz, and Owain Evans. Training large language models on narrow tasks can lead to broad misalignment.Nature, 649(8097):584–589, 2026

    2026

  3. [11]

    Ziegler, Tim Maxwell, Newton Cheng, Adam S

    Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam S. Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Ksh...

    2024 arXiv

  4. [12]

    Farther the shift, sparser the representation: Analyzing ood mechanisms in llms.arXiv preprint arXiv:2603.03415, 2026

    Mingyu Jin, Yutong Yin, Jingcheng Niu, Qingcheng Zeng, Wujiang Xu, Mengnan Du, Wei Cheng, Zhaoran Wang, Tianlong Chen, and Dimitris N Metaxas. Farther the shift, sparser the representation: Analyzing ood mechanisms in llms.arXiv preprint arXiv:2603.03415, 2026

    2026

  5. [13]

    All circuits lead to rome: rethinking functional anisotropy in circuit and sheaf discovery for llms.arXiv preprint arXiv:2605.12671, 2026

    Xi Chen, Mingyu Jin, Jingcheng Niu, Yutong Yin, Jinman Zhao, Bangwei Guo, Dimitris N Metaxas, Zhaoran Wang, Yutao Yue, and Gerald Penn. All circuits lead to rome: rethinking functional anisotropy in circuit and sheaf discovery for llms.arXiv preprint arXiv:2605.12671, 2026

    2026 arXiv

  6. [14]

    A comprehensive study of knowledge editing for large language models.CoRR, abs/2401.01286, 2024

    Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. ...

    2024 arXiv

  7. [15]

    SAGE: an agentic explainer framework for interpreting SAE features in language models

    Jiaojiao Han, Wujiang Xu, Mingyu Jin, and Mengnan Du. SAGE: an agentic explainer framework for interpreting SAE features in language models. InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026 - Volume 5: Ind...

    2026

  8. [16]

    Neuron to graph: Interpreting language model neurons at scale.arXiv preprint arXiv:2305.19911, 2023

    Alex Foote, Neel Nanda, Esben Kran, Ioannis Konstas, Shay Cohen, and Fazl Barez. Neuron to graph: Interpreting language model neurons at scale.arXiv preprint arXiv:2305.19911, 2023

    2023 arXiv

  9. [17]

    The importance of prompt tuning for automated neuron explanations.arXiv preprint arXiv:2310.06200, 2023

    Justin Lee, Tuomas Oikarinen, Arjun Chatha, Keng-Chi Chang, Yilan Chen, and Tsui-Wei Weng. The importance of prompt tuning for automated neuron explanations.arXiv preprint arXiv:2310.06200, 2023

    2023 arXiv

  10. [19]

    Automated interpretability and feature discovery in language models with agents.CoRR, abs/2605.01555, 2026

    Arnau Marin-Llobet and Javier Ferrando. Automated interpretability and feature discovery in language models with agents.CoRR, abs/2605.01555, 2026

    2026 arXiv

  11. [20]

    Sciatlas: A large-scale knowledge graph for automated scientific research.arXiv preprint arXiv:2605.22878, 2026

    Shuofei Qiao, Yunxiang Wei, Jiazheng Fan, Bin Wu, Busheng Zhang, Mengru Wang, Yuqi Zhu, Ningyu Zhang, Keyan Ding, Qiang Zhang, et al. Sciatlas: A large-scale knowledge graph for automated scientific research.arXiv preprint arXiv:2605.22878, 2026

    2026 arXiv

  12. [21]

    Claude code

    Anthropic. Claude code. https://docs.anthropic.com/en/docs/claude-code/getting-started,

  13. [22]

    Towards end-to-end automation of ai research.Nature, 651(8107):914–919, 2026

    Chris Lu, Cong Lu, Robert Tjarko Lange, Yutaro Yamada, Shengran Hu, Jakob Foerster, David Ha, and Jeff Clune. Towards end-to-end automation of ai research.Nature, 651(8107):914–919, 2026

    2026

  14. [23]

    Language models transmit behavioural traits through hidden signals in data.Nature, 652(8110):615–621, 2026

    Alex Cloud, Minh Le, James Chua, Jan Betley, Anna Sztyber-Betley, Sören Mindermann, Jacob Hilton, Samuel Marks, and Owain Evans. Language models transmit behavioural traits through hidden signals in data.Nature, 652(8110):615–621, 2026

    2026

  15. [24]

    You didn’t have to say it like that: Subliminal learning from faithful paraphrases.CoRR, abs/2603.09517, 2026

    Isaia Gisler, Zhonghao He, and Tianyi Qiu. You didn’t have to say it like that: Subliminal learning from faithful paraphrases.CoRR, abs/2603.09517, 2026

    2026

  16. [25]

    Subliminal learning is a lora artifact.CoRR, abs/2606.00831, 2026

    Todd Nief, Harvey Yiyun Fu, Mark Muchane, and Ari Holtzman. Subliminal learning is a lora artifact.CoRR, abs/2606.00831, 2026

    2026 arXiv

  17. [26]

    Towards understanding subliminal learning: When and how hidden biases transfer.CoRR, abs/2509.23886, 2025

    Simon Schrodi, Elias Kempf, Fazl Barez, and Thomas Brox. Towards understanding subliminal learning: When and how hidden biases transfer.CoRR, abs/2509.23886, 2025

    2025

  18. [27]

    Token entanglement in subliminal learning

    Amir Zur, Zhuofan Ying, Alexander Russell Loftus, Kerem Şahin, Steven Yu, Lucia Quirke, Tamar Rott Shaham, Natalie Shapira, Hadas Orgad, and David Bau. Token entanglement in subliminal learning. InMechanistic Interpretability Workshop at NeurIPS 2025, 2025

    2025

  19. [28]

    Brockers, Roman D

    Vincent C. Brockers, Roman D. Ventzke, Valentin Neuhaus, Belén Hidalgo-Ogalde, and Viola Priesemann. Learning through noise: Why subliminal learning works and when it fails.CoRR, abs/2605.23645, 2026

    2026 arXiv

  20. [29]

    Subliminal learning is steering vector distillation.CoRR, abs/2606.00995, 2026

    Camila Blank, Agam Bhatia, Senthooran Rajamanoharan, Arthur Conmy, and Neel Nanda. Subliminal learning is steering vector distillation.CoRR, abs/2606.00995, 2026

    2026 arXiv

  21. [30]

    Channel location constrains the auditability of subliminal learning.CoRR, abs/2606.22019, 2026

    Tamas Madl. Channel location constrains the auditability of subliminal learning.CoRR, abs/2606.22019, 2026

    2026 arXiv

  22. [31]

    Inoculation prompting: Eliciting traits from llms during training can suppress them at test-time.CoRR, abs/2510.04340, 2025

    Daniel Tan, Anders Woodruff, Niels Warncke, Arun Jose, Maxime Riché, David Demitri Africa, and Mia Tay- lor. Inoculation prompting: Eliciting traits from llms during training can suppress them at test-time.CoRR, abs/2510.04340, 2025

  23. [32]

    Inoculation prompting: Instructing llms to misbehave at train-time improves test-time alignment.CoRR, abs/2510.05024, 2025

    Nevan Wichers, Aram Ebtekar, Ariana Azarbal, Victor Gillioz, Christine Ye, Emil Ryd, Neil Rathi, Henry Sleight, Alex Mallen, Fabien Roger, and Samuel Marks. Inoculation prompting: Instructing llms to misbehave at train-time improves test-time alignment.CoRR, abs/2510.05024, 2025. 27

  24. [33]

    Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers.CoRR, abs/2604.25891, 2026

    Jan Dubinski, Jan Betley, Anna Sztyber-Betley, Daniel Tan, and Owain Evans. Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers.CoRR, abs/2604.25891, 2026

    2026 arXiv

  25. [34]

    Wellman, David Cross, and Julanne Watson

    Henry M. Wellman, David Cross, and Julanne Watson. Meta-analysis of theory-of-mind development: The truth about false belief.Child Development, 72(3):655–684, 2001

    2001

  26. [35]

    Eric Schwitzgebel. Belief. In Edward N. Zalta and Uri Nodelman, editors,The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, 2023

    2023

  27. [36]

    Eric Mandelbaum and Giacomo Ballarini. Belief. Open Encyclopedia of Cognitive Science, 2025

    2025

  28. [37]

    Nucleotide transformer: building and evaluating robust foundation models for human genomics.Nature Methods, 22(2):287–297, 2025

    Hugo Dalla-Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nicolas Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Bernardo P De Almeida, Hassan Sirelkhatim, et al. Nucleotide transformer: building and evaluating robust foundation models...

    2025

  29. [38]

    Evolutionary-scale prediction of atomic-level protein structure with a language model

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123–1130, 2023

    2023

  30. [39]

    Large language models generate functional protein sequences across diverse families.Nature biotechnology, 41(8):1099–1106, 2023

    Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos Jr, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. Large language models generate functional protein sequences across diverse families.Nature biotechnology, 41(8):10...

    2023

  31. [40]

    Bidirectional generation of structure and properties through a single molecular foundation model.Nature Communications, 15(1):2323, 2024

    Jinho Chang and Jong Chul Ye. Bidirectional generation of structure and properties through a single molecular foundation model.Nature Communications, 15(1):2323, 2024

    2024

  32. [41]

    Sequence modeling and design from molecular to genome scale with evo.Science, 386(6723):eado9336, 2024

    Eric Nguyen, Michael Poli, Matthew G Durrant, Brian Kang, Dhruva Katrekar, David B Li, Liam J Bartie, Armin W Thomas, Samuel H King, Garyk Brixi, et al. Sequence modeling and design from molecular to genome scale with evo.Science, 386(6723):eado9336, 2024

    2024

  33. [42]

    Gp-molformer: A foundation model for molecular generation.Digital Discovery, 4(10):2684–2696, 2025

    Jerret Ross, Brian Belgodere, Samuel C Hoffman, Vijil Chenthamarakshan, Jiri Navratil, Youssef Mroueh, and Payel Das. Gp-molformer: A foundation model for molecular generation.Digital Discovery, 4(10):2684–2696, 2025

    2025

  34. [43]

    Genome modelling and design across all domains of life with evo 2.Nature, 652(8112):1349–1361, 2026

    Garyk Brixi, Matthew G Durrant, Jerome Ku, Mohsen Naghipourfar, Michael Poli, Gwanggyu Sun, Greg Brockman, Daniel Chang, Alison Fanton, Gabriel A Gonzalez, et al. Genome modelling and design across all domains of life with evo 2.Nature, 652(8112):1349–1361, 2026

    2026

  35. [44]

    De novo design of protein structure and function with rfdiffusion.Nature, 620(7976):1089–1100, 2023

    Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eisenach, Woody Ahern, Andrew J Borst, Robert J Ragotte, Lukas F Milles, et al. De novo design of protein structure and function with rfdiffusion.Nature, 620(7976):1089–1100, 2023

    2023

  36. [45]

    Efficient evolution of human antibodies from general protein language models

    Brian L Hie, Varun R Shanker, Duo Xu, Theodora UJ Bruun, Payton A Weidenbacher, Shaogeng Tang, Wesley Wu, John E Pak, and Peter S Kim. Efficient evolution of human antibodies from general protein language models. Nature biotechnology, 42(2):275–283, 2024

    2024

  37. [46]

    Controlling gene expression with deep generative design of regulatory dna.Nature communications, 13(1):5099, 2022

    Jan Zrimec, Xiaozhi Fu, Azam Sheikh Muhammad, Christos Skrekas, Vykintas Jauniskis, Nora K Speicher, Christoph S Börlin, Vilhelm Verendel, Morteza Haghir Chehreghani, Devdatt Dubhashi, et al. Controlling gene expression with deep generative design of regulatory dna.Nature comm...

    2022

  38. [47]

    Designing synthetic regulatory elements using the generative ai framework dna-diffusion.Nature Genetics, 58(1):180–194, 2026

    Lucas Ferreira DaSilva, Simon Senan, Judith F Kribelbauer-Swietek, Zain Munir Patel, Lithin Karmel Louis, Aniketh Janardhan Reddy, Sameer Gabbita, Jonathan D Rosen, Zach Nussbaum, César Miguel Valdez Córdova, et al. Designing synthetic regulatory elements using the generative ...

    2026

  39. [48]

    Dragan, Rohin Shah, and Neel Nanda

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca D. Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2.CoRR, abs/2408.05147, 2024. 28

    2024 arXiv

  40. [49]

    Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders.CoRR, abs/2410.20526, 2024

    Zhengfu He, Wentao Shu, Xuyang Ge, Lingjie Chen, Junxuan Wang, Yunhua Zhou, Frances Liu, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang, and Xipeng Qiu. Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders.CoRR, abs/2410.20526, 2024

    arXiv

  41. [50]

    Interplm: Discovering interpretable features in protein language models via sparse autoencoders.CoRR, abs/2412.12101, 2024

    Elana Simon and James Zou. Interplm: Discovering interpretable features in protein language models via sparse autoencoders.CoRR, abs/2412.12101, 2024

    2024 arXiv

  42. [51]

    Mechanistic understanding and validation of large AI models with semanticlens.CoRR, abs/2501.05398, 2025

    Maximilian Dreyer, Jim Berend, Tobias Labarta, Johanna Vielhaben, Thomas Wiegand, Sebastian Lapuschkin, and Wojciech Samek. Mechanistic understanding and validation of large AI models with semanticlens.CoRR, abs/2501.05398, 2025

    2025 arXiv

  43. [52]

    A multi-agent system for automating scientific discovery.Nature, pages 1–3, 2026

    Ali Essam Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J Szostkiewicz, Dmytro Shved, Gavin J Gyimesi, Jon M Laurent, Samantha M Wright, Muhammed T Razzak, et al. A multi-agent system for automating scientific discovery.Nature, pages 1–3, 2026

    2026

  44. [53]

    Accelerating scientific discovery with co-scientist.Nature, pages 1–3, 2026

    Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Petar Sirkovic, Artiom Myaskovsky, Grzegorz Glowaty, Felix Weissenberger, Alessio Orlandi, Dan Popovici, et al. Accelerating scientific discovery with co-scientist.Nature, pages 1–3, 2026

    2026

  45. [54]

    An agentic artificially intelligent x-ray scientist.Nature Machine Intelligence, pages 1–12, 2026

    Zhantao Chen, Alexander N Petsch, Aidan J Israelski, Rajan Plumley, Lingjia Shen, Cong Wang, Cheng Peng, Yuan Ni, Arun Bansil, Sugata Chowdhury, et al. An agentic artificially intelligent x-ray scientist.Nature Machine Intelligence, pages 1–12, 2026

    2026

  46. [55]

    Autonomous biomedical research with an artificial intelligence agent.Science, page eadz4351, 2026

    Kexin Huang, Serena Zhang, Hanchen Wang, Yuanhao Qu, Yingzhou Lu, Ryan Li, Yusuf Roohani, Lin Qiu, Shiyi Cao, Gavin Li, et al. Autonomous biomedical research with an artificial intelligence agent.Science, page eadz4351, 2026

    2026

  47. [56]

    Ml-master: Towards ai-for-ai via integration of exploration and reasoning.CoRR, abs/2506.16499, 2025

    Zexi Liu, Yuzhu Cai, Xinyu Zhu, Yujie Zheng, Runkun Chen, Ying Wen, Yanfeng Wang, Weinan E, and Siheng Chen. Ml-master: Towards ai-for-ai via integration of exploration and reasoning.CoRR, abs/2506.16499, 2025

    2025 arXiv

  48. [57]

    Silico: The platform for ambitious AI research.https://www.goodfire.com/silico, 2026

    Goodfire. Silico: The platform for ambitious AI research.https://www.goodfire.com/silico, 2026

    2026

  49. [58]

    Tenenbaum, Antonio Torralba, Shuang Li, and Igor Mordatch

    Vighnesh Subramaniam, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Shuang Li, and Igor Mordatch. Multiagent finetuning: Self improvement with diverse reasoning chains. InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28,...

    2025

  50. [59]

    Self-discover: Large language models self-compose reasoning structures.Advances in Neural Information Processing Systems, 37:126032–126058, 2024

    Pei Zhou, Jay Pujara, Xiang Ren, Xinyun Chen, Heng-Tze Cheng, Quoc V Le, Denny Zhou, Swaroop Mishra, Huaixiu S Zheng, et al. Self-discover: Large language models self-compose reasoning structures.Advances in Neural Information Processing Systems, 37:126032–126058, 2024

    2024

  51. [60]

    Weston, and Sainbayar Sukhbaatar

    Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason E. Weston, and Sainbayar Sukhbaatar. Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge. In Proceedings of the 2025 Conference on Empirical Methods in Natural La...

    2025

  52. [61]

    A self-improving coding agent.CoRR, abs/2504.15228, 2025

    Maxime Robeyns, Martin Szummer, and Laurence Aitchison. A self-improving coding agent.CoRR, abs/2504.15228, 2025

    2025 arXiv

  53. [62]

    Aflow: Automating agentic workflow generation

    Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. Aflow: Automating agentic workflow generation. InThe Thirteenth International Conference on L...

    2025

  54. [63]

    Mle-bench: Evaluating machine learning agents on machine learning engineering

    Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng. Mle-bench: Evaluating machine learning agents on machine learning engineering. InThe Thirteenth Inte...

    2025

  55. [64]

    Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vulic, Anna Korhonen, and Sercan Ö. Arik. Multi-agent design: Optimizing agents with better prompts and topologies.CoRR, abs/2502.02533, 2025. 29

    2025

  56. [65]

    Bowman, and Evan Hubinger

    Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Samuel Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, ...

    2024 arXiv

  57. [67]

    Closing the loop to discover psychological theories with an automated cognitive scientist.arXiv preprint arXiv:2606.26448, 2026

    Akshay K Jagadish, Younes Strittmatter, Nori Jacoby, George Kachergis, Eric Schulz, Nathaniel Daw, Suyog H Chandramouli, and Thomas L Griffiths. Closing the loop to discover psychological theories with an automated cognitive scientist.arXiv preprint arXiv:2606.26448, 2026

    2026 arXiv

  58. [68]

    Eckstein, Noémi Élteto, Thomas L

    Marcel Binz, Elif Akata, Matthias Bethge, Franziska Brändle, Fred Callaway, Julian Coda-Forno, Peter Dayan, Can Demircan, Maria K. Eckstein, Noémi Élteto, Thomas L. Griffiths, Susanne Haridi, Akshay K. Jagadish, Ji-An Li, Alexander Kipnis, Sreejan Kumar, Tobias Ludwig, Marvin ...

    2025

  59. [69]

    The cognitive revolution in interpretability: From explaining behavior to interpreting representations and algorithms.CoRR, abs/2408.05859, 2024

    Adam Davies and Ashkan Khakzar. The cognitive revolution in interpretability: From explaining behavior to interpreting representations and algorithms.CoRR, abs/2408.05859, 2024

    2024 arXiv

  60. [70]

    Piwowar, and Richard Orr

    Jason Priem, Heather A. Piwowar, and Richard Orr. Openalex: A fully-open index of scholarly works, authors, venues, institutions, and concepts.CoRR, abs/2205.01833, 2022

    2022 arXiv

  61. [71]

    Eliciting latent predictions from transformers with the tuned lens.CoRR, abs/2303.08112, 2023

    Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens.CoRR, abs/2303.08112, 2023

    2023 arXiv

  62. [72]

    Logitlens4llms: Extending logit lens analysis to modern large language models.CoRR, abs/2503.11667, 2025

    Zhenyu Wang. Logitlens4llms: Extending logit lens analysis to modern large language models.CoRR, abs/2503.11667, 2025

    2025 arXiv

  63. [73]

    Transformer feed-forward layers are key-value memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors,Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pr...

    2021

  64. [74]

    Massive values in self-attention modules are the key to contextual knowledge understanding

    Mingyu Jin, Kai Mei, Wujiang Xu, Mingjie Sun, Ruixiang Tang, Mengnan Du, Zirui Liu, and Yongfeng Zhang. Massive values in self-attention modules are the key to contextual knowledge understanding. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp,...

    2025

  65. [75]

    Language-specific neurons: The key to multilingual capabilities in large language models

    Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. Language-specific neurons: The key to multilingual capabilities in large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62...

    2024

  66. [76]

    Truthx: Alleviating hallucinations by editing large language models in truthful space

    Shaolei Zhang, Tian Yu, and Yang Feng. Truthx: Alleviating hallucinations by editing large language models in truthful space. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu...

    2024

  67. [77]

    Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

    Andy Zou, Long Phan, Sarah Li Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Z...

    arXiv

  68. [78]

    Steering llama 2 via contrastive activation addition

    Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computat...

    2024

  69. [79]

    Saes are good for steering - if you select the right features

    Dana Arad, Aaron Mueller, and Yonatan Belinkov. Saes are good for steering - if you select the right features. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language...

    2025

  70. [80]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023

    2023

  71. [81]

    Llms know more than they show: On the intrinsic representation of LLM hallucinations

    Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. Llms know more than they show: On the intrinsic representation of LLM hallucinations. InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Si...

    2025

  72. [82]

    A is for absorption: Studying feature splitting and absorption in sparse autoencoders

    David Chanin, James Wilken-Smith, Tomás Dulka, Hardik Bhatnagar, Satvik Golechha, and Joseph Bloom. A is for absorption: Studying feature splitting and absorption in sparse autoencoders. In Danielle Belgrave, Cheng Zhang, Laura N. Montoya, Hsuan-Tien Lin, Razvan Pascanu, Piotr...

    2025

  73. [83]

    Sparse autoencoders find highly interpretable features in language models

    Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenRevie...

    2024

  74. [84]

    Transcoders find interpretable LLM feature circuits

    Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. Transcoders find interpretable LLM feature circuits. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors,Advances in Neural Information Processing Systems...

    2024

  75. [85]

    Overcoming sparsity artifacts in crosscoders to interpret chat-tuning

    Julian Minder, Clément Dumas, Caden Juang, Bilal Chughtai, and Neel Nanda. Overcoming sparsity artifacts in crosscoders to interpret chat-tuning. In Danielle Belgrave, Cheng Zhang, Laura N. Montoya, Hsuan-Tien Lin, Razvan Pascanu, Piotr Koniusz, Marzyeh Ghassemi, Nancy Chen, I...

    2025

  76. [86]

    ICA lens: Interpreting language models without training another dictionary.CoRR, abs/2606.11722, 2026

    Sida Liu and Feijiang Han. ICA lens: Interpreting language models without training another dictionary.CoRR, abs/2606.11722, 2026

    2026 arXiv

  77. [87]

    What happened in llms layers when trained for fast vs

    Ming Li, Yanhong Li, and Tianyi Zhou. What happened in llms layers when trained for fast vs. slow thinking: A gradient perspective. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Proceedings of the 63rd Annual Meeting of the Association...

    2025

  78. [88]

    Relp: Faithful and efficient circuit discovery via relevance patching.CoRR, abs/2508.21258, 2025

    Farnoush Rezaei Jafari, Oliver Eberle, Ashkan Khakzar, and Neel Nanda. Relp: Faithful and efficient circuit discovery via relevance patching.CoRR, abs/2508.21258, 2025

    2025

  79. [89]

    Unveiling linguistic regions in large language models

    Zhihao Zhang, Jun Zhao, Qi Zhang, Tao Gui, and Xuanjing Huang. Unveiling linguistic regions in large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo...

    2024

  80. [90]

    Locating and editing factual associations in GPT

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors,Advances in Neural Information Processing Systems 35: Annual Conference on Neura...

    2022

  81. [91]

    Dissecting recall of factual associations in auto-regressive language models

    Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. Dissecting recall of factual associations in auto-regressive language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proc...

    2023

  82. [92]

    Attribution patching outperforms automated circuit discovery

    Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. CoRR, abs/2310.10348, 2023

    2023 arXiv

  83. [93]

    Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso

    Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors,Advanc...

    2023

  84. [94]

    Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms.CoRR, abs/2403.17806, 2024

    Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms.CoRR, abs/2403.17806, 2024

    2024 arXiv

  85. [95]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Ann...

    2017

  86. [96]

    Lundberg, Gabriel G

    Scott M. Lundberg, Gabriel G. Erion, and Su-In Lee. Consistent individualized feature attribution for tree ensembles.CoRR, abs/1802.03888, 2018

    2018 arXiv

  87. [97]

    Fastshap: Real-time shapley value estimation

    Neil Jethani, Mukund Sudarshan, Ian Connick Covert, Su-In Lee, and Rajesh Ranganath. Fastshap: Real-time shapley value estimation. InThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022

    2022

  88. [98]

    Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features, 2023

    Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit, and Mikhail Belkin. Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features, 2023

    2023

  89. [99]

    Greg Yang and Edward J. Hu. Feature learning in infinite-width neural networks, 2022

    2022

  90. [100]

    Mechanism of feature learning in convolutional neural networks.CoRR, abs/2309.00570, 2023

    Daniel Beaglehole, Adityanarayanan Radhakrishnan, Parthe Pandit, and Mikhail Belkin. Mechanism of feature learning in convolutional neural networks.CoRR, abs/2309.00570, 2023

    2023 arXiv

  91. [101]

    From attribution maps to human-understandable explanations through concept relevance propagation.Nat

    Reduan Achtibat, Maximilian Dreyer, Ilona Eisenbraun, Sebastian Bosse, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin. From attribution maps to human-understandable explanations through concept relevance propagation.Nat. Mac. Intell., 5(9):1006–1019, 2023

    2023

  92. [102]

    Mechanism for feature learning in neural networks and backpropagation-free machine learning models.Science, 383(6690), 2024

    Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit, and Mikhail Belkin. Mechanism for feature learning in neural networks and backpropagation-free machine learning models.Science, 383(6690), 2024. 32

    2024

  93. [103]

    Benchmarking large language models on safety risks in scientific laboratories

    Yujun Zhou, Jingdong Yang, Yue Huang, Kehan Guo, Zoe Emory, Bikram Ghosh, Amita Bedar, Sujay Shekar, Zhenwen Liang, Pin-Yu Chen, et al. Benchmarking large language models on safety risks in scientific laboratories. Nature Machine Intelligence, 8(1):20–31, 2026. 33

    2026

  94. [2025]

    Accessed: 2026-08-08

    2026

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.