Pith. sign in

REVIEW 3 major objections 5 minor 31 references

Causal dictionary learning reveals and validates transcription-factor binding features in genomic language models

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces a causal dictionary-learning framework to prove that specific intermediate features in genomic language models are used by the model to represent transcription-factor binding, not just to correlate with motifs.

desk verdict Useful framework and a real warning about GC/Alu confounds, but the causal claim needs repeat-matched controls and artifact release. read the letter →

arxiv 2607.19618 v1 pith:BRC5VHGN submitted 2026-07-21 q-bio.GN cs.AIcs.LG

Sarwan Ali This is my paper · ORCID
classification q-bio.GNcs.AIcs.LG
keywords causaldictionarylearningsparseautoencodersgenomiclanguagemodelstranscription-factorbindingablationcompositionconfoundsChIP-seqvalidationinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's aim is to establish that the features a genomic language model uses can be extracted and verified, not just assumed: intermediate activations in DNA transformer models can be decomposed into sparse, nearly monosemantic dictionary directions, and a subset of those directions are causally used by the model to represent cell-type-specific transcription-factor binding. It shows that the standard validation of such features by motif enrichment is badly confounded by GC content and repetitive elements, and replaces it with a composition-matched, binding-resolved test plus a causal ablation that measures how much masking a single dictionary direction shifts the model's own predictions at bound versus unbound motif sites. Across three transcription factors and two architecturally different models, 7–14 of 15 tested binding-sensitive features pass the causal test, while scrambled-label and random-feature controls produce clean nulls. The framework matters because it gives the field a way to tell whether an apparent 'concept' inside a genomic model is actually used by the model, rather than being a statistical artifact of sequence composition.

What carries the argument

The load-bearing mechanism is the causal dictionary: a top-k sparse autoencoder trained on hidden activations with a large overcomplete dictionary (16x hidden width, k=32 active features per token), whose columns d_j are candidate features. Each candidate is first filtered by a composition-matched binding test—bound windows (ChIP-seq peak centers with strong motif) versus unbound-motif windows matched by GC content via rejection sampling, scored by one-sided Mann–Whitney AUC with |corr(f_j, GC)| < 0.2—and then tested by causal ablation, where the feature's contribution s^{-1} f_j(x) d_j is subtracted from the hidden state and the model's output is re-run; the readout is the average KL diverg

What would settle it

Stratify both bound and unbound-motif windows by repeat-element content (e.g., Alu-containing vs. non-Alu) after GC matching and recompute the causal ablation AUC within each stratum: if the bound-vs-unbound effect disappears or reverses inside Alu-matched strata, the features encode repeat composition rather than binding.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that features recovered from a genomic language model's hidden activations by top-k sparse dictionary learning can be causally validated: deleting a single dictionary direction during the forward pass produces a larger shift in the model's masked-token prediction distribution at ChIP-seq-bound windows than at GC-matched unbound-motif windows, so the feature is not merely correlating with a position weight matrix but is part of the model's representation of binding. The evidence is a population-level effect: across CTCF, GATA1, and REST and across both a 6-mer-tokenized and a byte-pair-encoded transformer, 7–14 of 15 tested binding-sensit

Load-bearing premise

The load-bearing premise is that GC-matched unbound-motif windows are compositionally equivalent to bound windows except for the binding label; if unbound windows still differ in repeat-element content or higher-order k-mer statistics, the binding-specific ablation effects could be driven by those differences rather than by transcription-factor binding.

Editorial extensions

If this is right

  • Any genomic-model interpretability claim should now be expected to include a composition-matched control and a causal ablation; motif enrichment alone is demonstrably insufficient.
  • The framework returns clean negatives on scrambled labels, so it can be used as a screening pipeline to test which transcription factors and cell types a model genuinely encodes.
  • Model representations of binding can exist even when no canonical motif is enriched (GATA1), so correlation-based motif scans understate what genomic language models learn.
  • Because the causal effect replicated in two tokenization schemes (6-mer and byte-pair encoding), the result is not an accident of one vocabulary.
  • The set of causally validated dictionary directions gives a concrete, prioritized list of features for downstream hypothesis generation about regulatory grammar.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The causal effect is measured by a single-feature ablation; if several dictionary directions redundantly encode the same binding information, ablating any one of them will understate the feature's true role, so the reported counts likely represent a lower bound on how many directions the model uses.
  • Editorial inference: The fact that GATA1 binding is causally encoded without motif selectivity suggests testable flanking-context experiments: mutating sequence context around GATA1 motifs while preserving the WGATAR core should change the causal effect if the model's binding representation is context-dependent.
  • Editorial inference: The KL-shift readout could be inverted into a variant-prioritization tool; for a candidate SNP inside a feature's receptive window, the bound-versus-unbound difference in ablation shift could flag variants that the model treats as binding-relevant without retraining or cell-type labels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a framework for interpreting genomic language models by training top-k sparse autoencoders on hidden activations of two models (Nucleotide Transformer and DNABERT-2), then validating and causally testing the resulting dictionary features. The validation pipeline first uses a composition-matched test that compares feature activation on ChIP-seq bound windows versus GC-matched unbound motif windows, with a GC-correlation filter and a binding-sensitivity AUC threshold. The causal test ablates individual dictionary directions and measures the KL divergence in masked-token predictions between bound and unbound sites. The authors report that across CTCF, GATA1, and REST, and across both architectures, 7–14 of 15 tested features per condition show significant binding-specific causal effects, while scrambled-label and random-feature controls return nulls. They also provide Proposition 1 to formalize composition confounds and Proposition 2 to argue that the causal test separates general feature importance from binding specificity.

Significance. If the results hold, the paper makes a valuable contribution by importing sparse-dictionary and causal-intervention methodology from NLP interpretability into regulatory genomics, and by explicitly exposing compositional confounds in naive PWM-based feature validation. The experimental design has real strengths: cross-architecture replication, three functionally distinct transcription factors, two negative-control classes, and a theoretical account of why magnitude-based feature importance is insufficient. The observation that GATA1 binding features are causally validated despite weak motif selectivity is biologically interesting and supports the case for intervention-based validation. The paper also benefits from using only public data. However, the central claim that the causally validated features represent cell-type-specific transcription-factor binding, rather than compositionally correlated repeat elements, is not yet fully established because the matching protocol controls only GC content.

major comments (3)
  1. [§3.2, §5.2] The composition-matched test matches only GC content via rejection sampling, yet §5.2 identifies Alu repeats as a major GC-rich confound for CTCF enrichment. The manuscript does not report the repeat-element composition of the bound versus unbound-motif windows after GC matching. If bound ChIP-seq peaks are enriched for Alu or other repeat families beyond what GC matching captures, a feature detecting such repeats will appear binding-sensitive (AUC>0.55) and, because it is more active on bound windows, will also pass the causal ablation test — without representing TF binding. The SCRAMBLE and GATA1-SCRAM controls do not rule this out, since they randomize labels and therefore make the two sets exchangeable; they do not test whether the unbound set matches the bound set in repeat content. Please report repeat composition (e.g., Alu fraction, LINE/SINE content) of bound and unbound sets, a
  2. [§3.2, §3.3] Key thresholds and selection criteria are unspecified: the 'fixed fraction of the matrix maximum' for strong-motif calling, the ranking criterion for the 'top 15 binding-sensitive, GC-robust features' (AUC? p-value? activation?), and the number of windows per class after GC matching. These choices determine the unbound set and which features are causally tested, so without them the population numbers in Table 1 are not reproducible. Specify all constants and include the selection rule.
  3. [§3.3, Proposition 2] Equation (5) subtracts f_j d_j from the hidden state. Since SAE dictionary directions are not orthogonal and reconstruction is approximate, this does not exactly remove feature j's contribution: it also changes the effective contribution of any feature correlated with d_j. Proposition 2 additionally assumes the geometric factor γ(S) is independent of B (or does not anti-correlate with binding), but γ(S) depends on S through the post-layer Jacobian and is not shown to be binding-independent. As stated, the proposition is a heuristic rather than a proof that the test isolates binding-specific use; please either provide conditions/empirical checks (e.g., orthogonality of validated features, distribution of γ across bound/unbound) or soften the claim.
minor comments (5)
  1. [Throughout] No code or data availability statement is provided despite the claim of a 'fully reproducible procedure'; please include a statement or link to an implementation, or specify exactly which public resources and scripts are needed to reproduce the pipeline.
  2. [§3.1, Eq. (5)] Please specify the activation normalization/scale s in Eq. (5) and the exact SAE training details (number of tokens, batch size, layer selection criterion) in the main text rather than in passing.
  3. [Table 1] The '—' entries for random-feature control rows could be confusing; state explicitly that motif-selectivity is undefined for randomly chosen features, and that negative controls have no binding-sensitive features so the causal test is not applicable.
  4. [Figure 3 caption] The caption says (a) shows 'two exemplar binding features' and (b) shows 'three NT features'; please clarify the relationship between the panels, e.g., whether the third NT feature in (b) is one of the two in (a).
  5. [§3.2] Define 'GC-matched' precisely: the bin width used for the GC histogram and the rejection-sampling acceptance criterion should be given, as the result can be sensitive to these choices.

Circularity Check

1 steps flagged · score 6.0 of 10

Causal validation is entailed by the binding-sensitivity selection criterion via Proposition 2.

  1. fitted input called prediction [Section 3.2-3.4, Eqs. (5)-(7), Algorithm 1 lines 9-11, Table 1]
    "for each binding-sensitive feature j (and random controls) do ablate d_j (Eq. (5)); measure Δ_j (Eq. (6)) at bound vs. unbound; mark j causally validated if binding-specific AUC>0.55 (Bonferroni) ... whereas a feature whose activation is elevated on bound inputs, E[f_j |B=1]>E[f_j |B=0], yields E[Δ_j |B=1]>E[Δ_j |B=0] whenever G(S) does not anti-correlate with binding."

    Features are first selected as 'binding-sensitive' by requiring their activation f_j to be higher on bound than on GC-matched unbound windows (AUC>0.55, Sec. 3.2). The 'causal' test then measures the KL shift Δ_j on the same bound-vs-unbound windows. By the paper's own Eq. (7), Δ_j(S) ≈ (1/(2s^2)) f_j(S)^2 γ_j(S) with γ_j≥0. Therefore, if f_j is elevated on bound windows (the selection criterion), Δ_j is also elevated on bound windows whenever γ_j does not anti-correlate with binding, so the binding-specific causal AUC>0.55 is a mathematical consequence of the selection. Proposition 2 states exactly this implication. Thus the reported 7-14/15 'causally validated' counts are not independent confirmations of causal use; they are the selection criterion propagated through the model's quadrati

full rationale

The paper has genuine external grounding: ChIP-seq peaks and JASPAR PWMs define the window labels and motif scores, and the scrambled-label negative controls demonstrate that the pipeline can return nulls. However, the central causal claim is partially circular by construction. The feature set is chosen by the condition E[f_j|B=1] > E[f_j|B=0] (binding-sensitive AUC>0.55), and the causal test measures E[Δ_j|B=1] vs E[Δ_j|B=0] on the same windows. Because Eq. (7) makes Δ_j a monotone (quadratic) function of f_j times a nonnegative model-sensitivity factor, Proposition 2 ensures that any feature passing the selection will also pass the causal test, unless the sensitivity factor anti-correlates with binding. The paper even presents this as a theorem. The motif-only control is a feature that failed the selection, so it does not show that the test can reject a feature with the selected property; the random-feature null is similarly forced by absence of the activation difference. The causal test does add information about whether the feature affects the model's output at all, but the binding-specificity result, which is the paper's headline, reduces to the input selection criterion. This is a partial, not total, circularity: the selected features are still genuinely correlated with external binding data, and the model-output sensitivity is a real measurement. Score 6 reflects that one or more central 'predictions' are entailed by construction.

Assumptions & free parameters 10 free parameters · 7 assumptions · 0 invented entities

No new physical or biological entities are postulated. Dictionary directions are learned from model activations, not invented. The central free parameters are SAE hyperparameters and validation thresholds; the most consequential choice is the GC-only matching, which is insufficient given the paper's own repeat-element findings.

free parameters (10)
  • SAE dictionary expansion factor m=16d = 16x hidden width
    Chosen by hand; sets the number of dictionary features and capacity.
  • SAE sparsity k = 32
    Chosen by hand; fixes the number of active features per token and affects monosemanticity.
  • SAE loss coefficient alpha = 1/16
    Chosen for dead-feature revival; affects dictionary quality.
  • SAE training schedule = 1 epoch, LR 4e-4, batch 4096
    Ad hoc; no tuning analysis is shown.
  • Layer selection = NT layer 14, DNABERT-2 layer 6
    Selected post hoc from reconstruction/live-feature curves; not predetermined.
  • Binding-sensitivity threshold = AUC > 0.55 (Bonferroni)
    Hand-set; determines which features qualify as binding-sensitive or causally validated.
  • GC-robust threshold = |corr(f_j, GC)| < 0.2
    Hand-set; determines which features pass composition control.
  • Motif score threshold = fixed fraction of PWM max (value not stated)
    Critical for defining bound/unbound-motif windows; the exact value is absent.
  • Number of features tested per condition = 15
    Arbitrary; affects power and multiple-testing burden.
  • SAE initialization from random data activations = random activations
    Needed to keep the dictionary alive; an ad hoc choice that affects which features emerge.
assumptions (7)
  • domain assumption ChIP-seq IDR peaks accurately label true TF binding in the tested cell types.
    Used to define bound sets in Section 3.2 and Section 4.
  • domain assumption JASPAR PWMs correctly represent TF sequence specificity, and a fixed log-odds threshold identifies strong motif occurrences.
    Used to build bound, unbound-motif, and background windows in Section 3.2.
  • domain assumption Top-k SAE reconstruction (74-76% variance explained) preserves the activation subspace relevant to TF binding.
    Residual error after reconstruction is ignored; if binding information lives in the residual, features are incomplete (Section 3.1, Section 5.1).
  • ad hoc to paper Equation (5) exactly removes the feature's contribution from the model's computation.
    Assumes linearity of the SAE decoder and that subtracting s^{-1} f_j d_j in normalized SAE space corresponds to causal removal (Section 3.3).
  • domain assumption The KL divergence of masked-token predictions is a valid measure of whether the model uses a feature for binding.
    Causal validation is defined through this readout, and no alternative readout is tested (Section 3.3, Eq. 6).
  • ad hoc to paper GC matching by rejection sampling makes the unbound-motif set compositionally equivalent to the bound set except for binding.
    Only GC is matched; repeat-element composition and k-mer spectra are not, even though the paper itself identifies Alu repeats as a CTCF confound (Section 3.2, Section 5.2).
  • ad hoc to paper Proposition 2's geometric factor gamma(S) does not anti-correlate with binding / is independent of B.
    Explicitly assumed to derive binding-specificity AUC > 1/2; no evidence is provided (Section 3.4).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Causal dictionary learning reveals and validates transcription-factor binding features in genomic language models." pith.science (2026). https://pith.science/paper/BRC5VHGN

@misc{pith2026260719618,
  author       = {Pith},
  title        = {Pith review of: Causal dictionary learning reveals and validates transcription-factor binding features in genomic language models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BRC5VHGN}},
  note         = {Machine review of arXiv:2607.19618}
}
abstract

Genomic language models achieve strong performance across regulatory-genomics tasks, yet what these models internally represent remains opaque, and the field lacks a principled procedure for verifying that an apparent ``concept'' inside a model is real rather than an artifact of sequence composition. We introduce a framework that combines sparse dictionary learning with causal intervention to extract, validate, and causally test interpretable features in genomic foundation models. Training top-$k$ sparse autoencoders on the hidden activations of two architecturally distinct models, Nucleotide Transformer ($6$-mer tokenization) and DNABERT-2 (byte-pair encoding), we recover thousands of monosemantic features that map to transcription-factor (TF) sequence motifs. We show that the naive validation of such features against position weight matrices is severely confounded by GC composition and repetitive elements, producing hundreds of spurious ``TF features'', and we develop a composition-matched, binding-resolved protocol that removes these confounds. Critically, we move beyond correlation: by ablating individual dictionary directions during the model's forward pass and measuring the induced shift in the model's own predictive distribution, we establish that specific features are \emph{causally} used to represent cell-type-specific TF binding, not merely motif presence. Across three transcription factors (CTCF, GATA1, REST) and both architectures, causally validated binding features emerge reproducibly ($7$--$14$ of $15$ tested features per condition), while two classes of negative control, scrambled binding labels and randomly selected features, yield no detectable signal. The framework is purely computational, uses only public data, and provides a reusable standard for interpretability claims in genomic deep learning.

Figures

Figures reproduced from arXiv: 2607.19618 by the authors.

Figure 1
Figure 1. Sparse-autoencoder reconstruction across model depth. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Motif-selective features per transcription factor [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. A causally validated CTCF binding feature. (a) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Causal validation matrix. Number of features (of 15 tested) with a significant binding-specific causal effect (Bonferroni-corrected P < 1.7 × 10−3 and AUC > 0.55), for each transcription factor (rows) and model (columns). Random-feature controls yielded 0/15 in every c…
Figure 5
Figure 5. Figure 5: Causal effect sizes separate binding features from controls. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 4 linked inside Pith

  1. [1]

    Nucleotide transformer: building and evaluating robust foundation models for human genomics.Nature Methods, 22(2):287–297, 2025

    Hugo Dalla-Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nicolas Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Bernardo P De Almeida, Hassan Sirelkhatim, et al. Nucleotide transformer: building and evaluating robust foundation models for human genomics.Nature Methods, 22(2):287–297, 2025

    2025

  2. [2]

    Dnabert-2: Ef- ficient foundation model and benchmark for multi-species genomes

    Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, and Han Liu. Dnabert-2: Ef- ficient foundation model and benchmark for multi-species genomes. InInternational Conference on Learning Representations, volume 2024, pages 41642–41665, 2024

    2024

  3. [3]

    Ca- duceus: Bi-directional equivariant long-range dna sequence modeling.Proceedings of machine learn- ing research, 235:43632, 2024

    Yair Schiff, Chia-Hsiang Kao, Aaron Gokaslan, Tri Dao, Albert Gu, and V olodymyr Kuleshov. Ca- duceus: Bi-directional equivariant long-range dna sequence modeling.Proceedings of machine learn- ing research, 235:43632, 2024

    2024

  4. [4]

    Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution.Advances in neural information processing systems, 36:43177–43201, 2023

    Eric Nguyen, Michael Poli, Marjan Faizi, Armin Thomas, Michael Wornow, Callum Birch-Sykes, Ste- fano Massaroli, Aman Patel, Clayton Rabideau, Yoshua Bengio, et al. Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution.Advances in neural information processing systems, 36:43177–43201, 2023

    2023

  5. [5]

    Sequence modeling and design from molecular to genome scale with evo.Science, 386(6723):eado9336, 2024

    Eric Nguyen, Michael Poli, Matthew G Durrant, Brian Kang, Dhruva Katrekar, David B Li, Liam J Bartie, Armin W Thomas, Samuel H King, Garyk Brixi, et al. Sequence modeling and design from molecular to genome scale with evo.Science, 386(6723):eado9336, 2024

    2024

  6. [6]

    Dna language models are powerful predictors of genome-wide variant effects.Proceedings of the National Academy of Sciences, 120(44):e2311219120, 2023

    Gonzalo Benegas, Sanjit Singh Batra, and Yun S Song. Dna language models are powerful predictors of genome-wide variant effects.Proceedings of the National Academy of Sciences, 120(44):e2311219120, 2023

    2023

  7. [7]

    Predicting effects of noncoding variants with deep learning–based sequence model.Nature methods, 12(10):931–934, 2015

    Jian Zhou and Olga G Troyanskaya. Predicting effects of noncoding variants with deep learning–based sequence model.Nature methods, 12(10):931–934, 2015

    2015

  8. [8]

    Sequential regulatory activity prediction across chromosomes with convolutional neural net- works.Genome research, 28(5):739, 2018

    David R Kelley, Yakir A Reshef, Maxwell Bileschi, David Belanger, Cory Y McLean, and Jasper Snoek. Sequential regulatory activity prediction across chromosomes with convolutional neural net- works.Genome research, 28(5):739, 2018. 11

    2018

Show all 31 references
  1. [9]

    Effective gene expression prediction from sequence by integrating long-range interactions.Nature methods, 18(10): 1196–1203, 2021

    Žiga Avsec, Vikram Agarwal, Daniel Visentin, Joseph R Ledsam, Agnieszka Grabska-Barwinska, Kyle R Taylor, Yannis Assael, John Jumper, Pushmeet Kohli, and David R Kelley. Effective gene expression prediction from sequence by integrating long-range interactions.Nature methods, 1...

    2021

  2. [10]

    Toy models of superposition

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022

    2022 arXiv

  3. [11]

    Obtaining genetics insights from deep learning via explainable artificial intelligence.Nature Reviews Genetics, 24(2):125–137, 2023

    Gherman Novakovsky, Nick Dexter, Maxwell W Libbrecht, Wyeth W Wasserman, and Sara Mostafavi. Obtaining genetics insights from deep learning via explainable artificial intelligence.Nature Reviews Genetics, 24(2):125–137, 2023

    2023

  4. [12]

    Sparse autoencoders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600, 2023

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600, 2023

    2023 arXiv

  5. [13]

    Burke, Tristan Hume, Shan Carter, Tom Henighan, and Chris Olah

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen,...

    2023

  6. [14]

    Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet.Transformer Circuits Thread, 2024

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, et al. Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet.Transformer Circuits Thread, 2024. URLhttps:/...

    2024

  7. [15]

    Scaling and evaluating sparse autoencoders

    Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. InInternational Conference on Learning Representations, volume 2025, pages 26721–26754, 2025

    2025

  8. [16]

    Alu elements: know the sines.Genome biology, 12(12):236, 2011

    Prescott Deininger. Alu elements: know the sines.Genome biology, 12(12):236, 2011

    2011

  9. [17]

    Multiparameter functional diversity of human c2h2 zinc finger proteins.Genome research, 26(12):1742, 2016

    Frank W Schmitges, Ernest Radovani, Hamed S Najafabadi, Marjan Barazandeh, Laura F Campitelli, Yimeng Yin, Arttu Jolma, Guoqing Zhong, Hongbo Guo, Tharsan Kanagalingam, et al. Multiparameter functional diversity of human c2h2 zinc finger proteins.Genome research, 26(12):1742, 2016

    2016

  10. [18]

    Dnabert: pre-trained bidirectional en- coder representations from transformers model for dna-language in genome.Bioinformatics, 37(15): 2112–2120, 2021

    Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. Dnabert: pre-trained bidirectional en- coder representations from transformers model for dna-language in genome.Bioinformatics, 37(15): 2112–2120, 2021

    2021

  11. [19]

    Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  12. [20]

    Evaluating the representational power of pre- trained dna language models for regulatory genomics.Genome Biology, 26(1):203, 2025

    Ziqi Tang, Nirali Somia, Yiyang Yu, and Peter K Koo. Evaluating the representational power of pre- trained dna language models for regulatory genomics.Genome Biology, 26(1):203, 2025

    2025

  13. [21]

    Learning important features through prop- agating activation differences

    Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through prop- agating activation differences. InInternational conference on machine learning, pages 3145–3153. PMlR, 2017. 12

    2017

  14. [22]

    A unified approach to interpreting model predictions.Advances in neural information processing systems, 30, 2017

    Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions.Advances in neural information processing systems, 30, 2017

    2017

  15. [23]

    Investigating gender bias in language models using causal mediation analysis.Advances in neural information processing systems, 33:12388–12401, 2020

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stu- art Shieber. Investigating gender bias in language models using causal mediation analysis.Advances in neural information processing systems, 33:12388–12401, 2020

    2020

  16. [24]

    Locating and editing factual associa- tions in gpt.Advances in neural information processing systems, 35:17359–17372, 2022

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associa- tions in gpt.Advances in neural information processing systems, 35:17359–17372, 2022

    2022

  17. [25]

    Is this the subspace you are looking for? an interpretability illusion for subspace activation patching.arXiv preprint arXiv:2311.17030, 2023

    Aleksandar Makelov, Georg Lange, and Neel Nanda. Is this the subspace you are looking for? an interpretability illusion for subspace activation patching.arXiv preprint arXiv:2311.17030, 2023

    2023 arXiv

  18. [26]

    Jaspar 2024: 20th anniversary of the open-access database of transcription factor binding profiles.Nucleic acids research, 52(D1):D174–D182, 2024

    Ieva Rauluseviciute, Rafael Riudavets-Puig, Romain Blanc-Mathieu, Jaime A Castro-Mondragon, Katalin Ferenc, Vipin Kumar, Roza Berhanu Lemma, Jérémy Lucas, Jeanne Chèneby, Damir Barana- sic, et al. Jaspar 2024: 20th anniversary of the open-access database of transcription facto...

    2024

  19. [27]

    Hocomoco in 2024: a rebuild of the curated collection of binding models for human and mouse transcription factors.Nucleic Acids Research, 52(D1):D154–D163, 2024

    Ilya E V orontsov, Irina A Eliseeva, Arsenii Zinkevich, Mikhail Nikonov, Sergey Abramov, Alexandr Boytsov, Vasily Kamenets, Alexandra Kasianova, Semyon Kolmykov, Ivan S Yevshin, et al. Hocomoco in 2024: a rebuild of the curated collection of binding models for human and mouse ...

    2024

  20. [28]

    An integrated encyclopedia of dna elements in the human genome

    ENCODE Project Consortium et al. An integrated encyclopedia of dna elements in the human genome. Nature, 489(7414):57, 2012

    2012

  21. [29]

    Expanded encyclopaedias of dna elements in the human and mouse genomes.Nature, 583(7818):699–710, 2020

    Jill E Moore, Michael J Purcaro, Henry E Pratt, Charles B Epstein, Noam Shoresh, Jessika Adrian, Trupti Kawli, Carrie A Davis, Alexander Dobin, et al. Expanded encyclopaedias of dna elements in the human and mouse genomes.Nature, 583(7818):699–710, 2020

    2020

  22. [30]

    Ctcf: an architectural protein bridging genome topology and function.Nature Reviews Genetics, 15(4):234–246, 2014

    Chin-Tong Ong and Victor G Corces. Ctcf: an architectural protein bridging genome topology and function.Nature Reviews Genetics, 15(4):234–246, 2014

    2014

  23. [31]

    On a test of whether one of two random variables is stochasti- cally larger than the other.The annals of mathematical statistics, pages 50–60, 1947

    Henry B Mann and Donald R Whitney. On a test of whether one of two random variables is stochasti- cally larger than the other.The annals of mathematical statistics, pages 50–60, 1947. 13

    1947

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.