Pith. sign in

REVIEW 4 major objections 5 minor 42 references

BMFM-DNA: A SNP-aware DNA foundation model to capture variant effects

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A DNA language model pre-trained on SNP-encoded human sequences outperforms its reference-only counterpart on five of six fine-tuning tasks, nearly matching DNABERT-2 trained on 135 species.

desk verdict A useful released pair of DNA models, but the SNP-aware gain is confounded by tokenizer changes and overstated in the abstract. read the letter →

arxiv 2507.05265 v1 pith:XUTUUO4U submitted 2025-06-26 q-bio.GN cs.LG

Hongyang Li , Sanjoy Dey , Bum Chul Kwon , Michael Danziger , Michal Rosen-Tzvi , Jianying Hu , James Kozloski , Ching-Huei Tsou
show 2 more authors
Bharath Dandala Pablo Meyer
This is my paper · ORCID
classification q-bio.GNcs.LG
keywords DNAfoundationmodelSNP-awarepre-trainingvariantencodingBytePairtokenizationgenomicbenchmarkGUEModernBERThumangenomeregulatorygenomics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a DNA language model learns more about biological function when its pre-training corpus includes natural human sequence variation, not just the reference genome. To test this, the authors pre-train two ModernBERT models on the same human genome—one on reference sequences, one on sequences in which each observed variant (SNP, insertion, or deletion) is encoded as a single special character—and fine-tune both on promoter, core-promoter, transcription-factor-binding, splicing, lentiviral MPRA, and SNP-to-disease tasks. The variant-encoded model beats the reference-only model on five of the six tasks, by 0.55 to 1.42 F1 or correlation points, and comes within about half a point of DNABERT-2 on the GUE benchmarks despite being trained on one species for about a third as many steps. If the result holds, explicit variant encoding during pre-training is a cheap route to variant-effect-aware genomics models.

What carries the argument

The load-bearing mechanism is the variant-encoded genome and its BPE tokenizer. From dbSNP the authors build a genome-wide variation frequency matrix; for each genomic position they sample a biallelic representation (two possible nucleotides, insertions, or a deletion) without replacement, then map each allele pair to one of 121 Chinese characters. A SentencePiece BPE tokenizer trained on these sequences learns tokens in which one character can stand for either allele, so a single token implicitly covers multiple DNA sequences—the paper likens it to a degenerate quantum state. The tokenizer analysis shows 6.8% of the variant-tokenizer vocabulary contains variant characters, versus about 1% natural SNP abundance, indicating that BPE captured recurrent variation patterns. This mechanism is doing the work: it is the only systematic difference between REF and SNP pre-training, and it is what the paper credits for the downstream gains.

What would settle it

Train BMFM-DNA-REF on reference sequences using the variant-encoded tokenizer (with each variant character replaced by its reference nucleotide), or train BMFM-DNA-SNP on variant-encoded sequences using the reference tokenizer; if the five-task performance gap disappears or reverses, the advantage is tokenization rather than variant awareness.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the SNP-aware model BMFM-DNA-SNP outperforms the reference-genome model BMFM-DNA-REF on every fine-tuning task except SNP-to-disease, where both score AUC 90.00. Reported numbers: promoter F1 93.5 vs 92.08, core promoter 83.19 vs 81.97, transcription factor binding 82.42 vs 81.87, splicing 90.44 vs 89.29, lenti-MPRA Pearson correlation 75.53 vs 74.70, with DNABERT-2 scoring 94.00, 83.85, 83.91, 90.66, and 75.00 for comparison. The paper further reports that the variant-encoded tokenizer shares 84.3% of its vocabulary with the reference tokenizer, produces shorter variant-containing tokens, and yields a higher k-mer space usage for 2- to 5-mers, which the authors interpret as the model implicitly expanding the DNA sequence space: one special character stands for a biallelic position, so each token containing it represents several possible reference sequences at once.

Load-bearing premise

The comparison that carries the paper treats the variant-encoded corpus and the reference corpus as differing only in variant awareness, but the two models also use different BPE tokenizers, and the paper itself shows those tokenizers produce measurably different tokens; if the performance gains come from token granularity rather than from learning the biology of natural variation, the central claim collapses.

Editorial extensions

If this is right

  • If variant-encoded pre-training is what drives the gains, then adding natural variation to a DNA language model's corpus is a direct way to improve downstream regulatory genomics predictions without expanding to more species.
  • A human-genome-only model trained for 150,000 steps can reach within roughly half an F1 point of DNABERT-2's 500,000-step, 135-species model on promoter, core-promoter, TF-binding, and splicing benchmarks.
  • The tokenizer itself becomes a lens on population variation: the over-representation of variant characters in the vocabulary (6.8% vs about 1% natural abundance) suggests recurrent SNP patterns are learnable and may be useful for variant-effect interpretation.
  • Variant-encoding strategies for negative samples materially change task difficulty and model behavior on promoter detection, with the most biologically faithful negative sampling (Class 3) improving F1 to 94.1 over the 93.5 baseline.
  • SNP-to-disease association does not improve with SNP-aware pre-training (AUC 90.00 for both models), so the benefit of variant encoding is task-dependent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: because REF and SNP models differ in both corpus and tokenizer, the reported gains could be explained by token granularity or by the implicit data augmentation of collapsing allele pairs into one symbol, rather than by the model learning where real variants occur; a matched-tokenizer control would settle this.
  • Editorial extension: the single-character encoding of a biallelic position discards allele-specific phase and frequency information; a testable alternative is to keep both alleles as separate tokens or add allele-frequency features and compare variant-effect predictions.
  • Editorial extension: the Class 4 promoter result (pretrained model F1 64.6 vs from-scratch 57.04 when SNPs are randomly inserted into positive sequences) suggests the model has internalized natural SNP locations; this could be turned into a direct probing benchmark for variant-aware representations.
  • Editorial extension: replicating the comparison with per-seed confidence intervals and on held-out variant-effect benchmarks (e.g., eQTL or saturation mutagenesis) would tell whether the 0.55 to 1.42 point improvements are robust and biologically meaningful.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces two DNA language models based on ModernBERT: BMFM-DNA-REF, pre-trained on reference-genome sequences, and BMFM-DNA-SNP, pre-trained on a variant-encoded representation of the human genome in which biallelic SNPs, insertions, and deletions are mapped to special Chinese characters. The authors compare these models with DNABERT-2 on promoter detection, core promoter detection, TF binding, splicing, lenti-MPRA, and a newly constructed SNP-to-disease association task, and they also explore several strategies for creating variant-encoded promoter-detection datasets. The central claim is that SNP-aware pre-training improves downstream task performance relative to reference-only pre-training, thereby demonstrating that encoding natural sequence variation helps DNA language models learn biological functions. The models and code are released.

Significance. If the central claim were cleanly established, the paper would make a useful contribution: it would show that a single-genome, variant-aware pre-training scheme can approach DNABERT-2's multi-species performance while also offering a concrete way to inject population variation into DNA language model training. The release of models and code, and the use of external benchmarks (GUE, lenti-MPRA, ClinVar/GWAS) rather than fitting the evaluation, are genuine strengths. However, the main REF-versus-SNP comparison is confounded by a simultaneous tokenizer change, and the reported margins are small with no variance information, so the significance is conditional on additional control experiments.

major comments (4)
  1. [Sections 3.1.2, 3.2, Table 2] The central comparison between BMFM-DNA-REF and BMFM-DNA-SNP changes two variables at once: the pre-training corpus (reference versus variant-encoded) and the BPE tokenizer. Section 3.2 and Figure 2C-F show that the two tokenizers share only 84.3% of their vocabulary, that variant-encoded tokens are shorter, and that variant-encoded samples yield more tokens per sequence. Consequently, the Table 2 gains of 0.55-1.42 F1 points cannot be attributed specifically to SNP-aware pre-training; they could arise from token granularity or from collapsing allele pairs into single symbols. A matched-tokenizer ablation (for example, pre-training the reference-only model with the variant tokenizer, or pre-training the variant model with the reference tokenizer) is required to support the abstract and Section 4.3 claim.
  2. [Section 4.2, Table 2] Table 2 reports three-seed averages without any variance or per-seed values. Given the small margins (for example, TF binding 81.87 versus 82.42, splicing 89.29 versus 90.44), the reader cannot determine whether the REF-to-SNP differences are robust. The SNP-to-disease row is 90.00 versus 90.00, which is consistent with the paper's own 'except' qualification but contradicts the abstract's statement of 'improvements on all fine-tuning tasks'; the abstract should be corrected.
  3. [Section 4.2, Table 2] For all Table 2 tasks, the fine-tuning inputs are not variant-encoded, so the model never observes the special variant characters at test time. Any benefit of BMFM-DNA-SNP is therefore mediated only by pre-training distribution and tokenizer differences, not by a direct ability to score variant-encoded inputs. The claim that the model 'captures variant effects' should be scoped accordingly, or the evaluation should include variant-encoded test inputs to demonstrate the mechanism directly.
  4. [Section 2.3, Table 3] The promoter negative-sampling experiments in Table 3 are not yet convincing evidence of SNP-location learning. Class 2 yields F1 = 97.46 because negatives are imputed from all 121 possible variants while positives cover only 20.66%, a class-distribution shortcut that the paper itself acknowledges. Class 4 compares a single pretrained checkpoint with a single from-scratch model, with no seeds or variance reported. These results should be presented as exploratory and should not be used as primary support for the main claim.
minor comments (5)
  1. [Abstract] The phrase 'improvements on all fine-tuning tasks' is inaccurate; Table 2 shows no improvement on SNP-to-disease (AUC 90.00 versus 90.00).
  2. [Section 2.2.5] The text says 'lymphoblasts (K562)', but K562 is a chronic myeloid leukemia cell line, not a lymphoblast line; this should be corrected.
  3. [Section 2.3] The descriptions of Class 2 and Class 3 are nearly identical; the text should clarify that Class 2 samples from all 121 possible variants while Class 3 samples from the variant distribution observed in the positive set.
  4. [Section 3.1.2] The sentence 'For more details, please refer to [35]' is vague; the authors should specify which aspect of the SNP2Vec method (for example, the variation frequency matrix construction) is being reused.
  5. [Section 3.2] The text says a BPE tokenizer with vocabulary size 4,096 was created 'for all three approaches', but only two approaches (reference and variation-encoded) are described; this appears to be a typo.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the SNP-aware performance claims rest on external benchmarks, and the tokenizer discrepancy is a confound, not a derivation that reduces to its inputs.

full rationale

The paper's central claim is that BMFM-DNA-SNP outperforms BMFM-DNA-REF on fine-tuning tasks (Section 4.3, Table 2). The evaluation metrics come from external resources: GUE tasks from DNABERT-2, lenti-MPRA from Zenodo, and SNP-disease associations from ClinVar and GWAS Catalog. No reported number is obtained by fitting the pretraining objective to the test labels, and the models are compared against external baselines (DNABERT-2 and EVE) as well as a no-pretraining control. The SNP encoding procedure is credited to SNP2vec (reference [35]), which is not a self-citation. The author-constructed negative-sample classes in Section 2.3 and Table 3 are explicitly checked against baselines, and the paper acknowledges their limitations rather than presenting them as forced predictions. The main validity threat is that BMFM-DNA-REF and BMFM-DNA-SNP differ in both pretraining data and BPE tokenizer, as shown in Section 3.2 and Figure 2C-F, so the performance gap in Table 2 cannot isolate variant awareness; however, this is an experimental confound, not circularity, because the reported outputs are not equivalent by construction to the inputs. The paper's own limitation statement ('we acknowledge that the current benchmarks are limited in their ability to fully evaluate these models') supports this non-circular reading. Thus no circular step meets the evidentiary bar of the review instructions.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The central claim rests on a small set of hand-chosen representation and training choices rather than on fitted constants: the allele-pair to character mapping, the averaging of population allele frequencies, the 20M variant cutoff, the masking description, and the BPE vocabulary size are all unablated. The domain assumptions (MLM transfer, dbSNP as the variation source, GUE/MPRA as functional proxies) are standard for this literature and partly conceded by the authors' own limitation statement. No physical entities are posited; the invented entries are representational symbols whose falsifiable handle is fine-tuned performance on external benchmarks.

free parameters (5)
  • Biallelic variant-to-character mapping (121 characters from Li Sao) = 121 characters; pairs of 11 alleles sampled without replacement
    Section 3.1.2: the central representation choice. Each variant position is collapsed into one symbol standing for two possible alleles; no ablation tests whether this ambiguity helps or hurts downstream transfer.
  • Variation frequency matrix construction = averaged population-specific frequencies (exact scheme not stated)
    Section 3.1.2: 'we averaged available population-specific variation frequencies'. Which populations and which averaging rule are unspecified; this matrix determines which variants appear in pre-training.
  • Number of variants extracted from dbSNP = 20 million
    Section 3.1.2: sets the coverage of the variation-encoded genome; no sensitivity analysis is reported.
  • Pre-training masking ratio = '90% of the tokens were replaced with [MASK], the remaining 10% unchanged' (ambiguous)
    Section 3.4: as written this is an extreme global mask rate or an underspecified within-masked split; the description is attributed to DNABERT-2 without a checkable citation.
  • BPE vocabulary size and pre-training sample geometry = vocab 4096; sequence lengths 1-10 kb; genome sampled 10x; 9,982,678 samples
    Sections 3.1.1, 3.2: hand-chosen; the tokenizer is trained on 20kb sequences while pre-training samples are 1-10kb, a mismatch not discussed.
assumptions (5)
  • domain assumption Masked language modeling on DNA sequences transfers to supervised genomic tasks
    The whole pre-training pipeline (Section 3.4) assumes MLM representations improve downstream fine-tuning, an assumption this literature is actively debating (references [19,20] cited by the paper itself).
  • domain assumption dbSNP allele frequencies define a representative sample of human variation
    Section 3.1.2 samples variant-encoded sequences from averaged dbSNP frequencies; if the sampling distribution is skewed, the model learns a distorted genome.
  • domain assumption GUE, lenti-MPRA, and ClinVar/GWAS benchmarks are valid proxies for biological function
    Sections 2.2 and 4.3 evaluate the central claim on these tasks; the paper concedes these benchmarks are limited.
  • ad hoc to paper Collapsing the two alleles of a SNP into one character preserves the information downstream tasks need
    Section 3.1.2 and Figure 1: the model cannot distinguish which allele is present at a variant position; no experiment tests information preservation.
  • domain assumption BPE token boundaries are biologically meaningful enough for the task suite
    Section 3.2 adopts BPE on the authority of DNABERT-2/GROVER; tokenization is trained on 20kb sequences and applied to 1-10kb samples without validation of boundary semantics.
invented entities (2)
  • Variant characters (up to 121 Chinese characters, one per allele pair) independent evidence
    purpose: Represent each SNP/indel position as a single token so variation enters pre-training at the same cost as a nucleotide
    Section 3.1.2 and Figure 3. A symbolic, not physical, invention on which the central claim depends. Falsifiable handle: the variant-encoded tokenizer statistics and the model's fine-tuned predictions on external benchmarks can be checked by any re-implementer.
  • The variation-encoded genome (variant symbols overlaid on GRCh38) independent evidence
    purpose: Pre-training corpus that exposes the model to natural variation alongside reference sequence
    Section 3.1.2. Reproducible from GRCh38 + dbSNP plus the stated sampling rules; the claimed downstream gains are falsifiable on GUE/MPRA benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BMFM-DNA: A SNP-aware DNA foundation model to capture variant effects." pith.science (2026). https://pith.science/paper/XUTUUO4U

@misc{pith2026250705265,
  author       = {Pith},
  title        = {Pith review of: BMFM-DNA: A SNP-aware DNA foundation model to capture variant effects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XUTUUO4U}},
  note         = {Machine review of arXiv:2507.05265}
}
read the original abstract

Large language models (LLMs) trained on text demonstrated remarkable results on natural language processing (NLP) tasks. These models have been adapted to decipher the language of DNA, where sequences of nucleotides act as "words" that encode genomic functions. However, the genome differs fundamentally from natural language, as it lacks clearly defined words or a consistent grammar. Although DNA language models (DNALMs) such as DNABERT, GENA-LM have achieved high level of performance on genome-related biological tasks, these models do not encode biological functions in the presence of sequence variations. To address this problem, we pre-train foundation models that effectively integrate sequence variations, in particular Single Nucleotide Polymorphisms (SNPs), as they underlie important biological functions. Specifically, we use ModernBERT to pre-train two different Biomedical Foundation Models (BMFM), namely, BMFM-DNA-REF in which the model is trained with sequences of varying lengths along with their reverse complements derived from the reference genome and BMFM-DNA-SNP in which the model is trained with sequences created using a novel representation scheme that encodes sequence variations. Our findings indicate that integrating sequence variations into DNALMs helps capture the biological functions as seen in improvements on all fine-tuning tasks. To explore the model's practical utility, we experimented with various strategies for SNP imputation on promoter detection task introduced in DNABERT-2. However, we acknowledge that the current benchmarks are limited in their ability to fully evaluate these models. To enable more comprehensive assessment in the future and encourage community contributions, we release our models through HuggingFace and the code to reproduce the results at https://github.com/BiomedSciAI/biomed-multi-omic

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 33 canonical work pages

  1. [35]

    Cahyawijaya, S.et al.Snp2vec: scalable self-supervised pre-training for genome- wide association study.arXiv preprint arXiv:2204.06699(2022)

    work page Pith review arXiv 2022

  2. [1]

    international consortium, T. H. G. P. Initial sequencing and analysis of the human genome.nature409, 860–921 (2001). 15

    work page 2001

  3. [2]

    P.et al.An integrated encyclopedia of dna elements in the human genome.Nature489, 57 (2012)

    Consortium, E. P.et al.An integrated encyclopedia of dna elements in the human genome.Nature489, 57 (2012)

    work page 2012

  4. [3]

    Jaganathan, K.et al.Predicting splicing from primary sequence with deep learning.Cell176, 535–548 (2019)

    work page 2019

  5. [4]

    Poplin, R.et al.A universal snp and small-indel variant caller using deep neural networks.Nature biotechnology36, 983–987 (2018)

    work page 2018

  6. [5]

    & Troyanskaya, O

    Zhou, J. & Troyanskaya, O. G. Predicting effects of noncoding variants with deep learning–based sequence model.Nature methods12, 931–934 (2015)

    work page 2015

  7. [6]

    Avsec, ˇZ.et al.Base-resolution models of transcription-factor binding reveal soft motif syntax.Nature genetics53, 354–366 (2021)

    work page 2021

  8. [7]

    URL http://arxiv.org/abs/2108.07258

    Bommasani, R.et al.On the Opportunities and Risks of Foundation Models (2022). URL http://arxiv.org/abs/2108.07258. ArXiv:2108.07258 [cs]

    arXiv 2022

Show all 42 references
  1. [8]

    URL https://link.springer.com/10.1007/ s13042-024-02443-6

    Zhou, C.et al.A comprehensive survey on pretrained foundation mod- els: a history from BERT to ChatGPT.International Journal of Machine Learning and Cybernetics(2024). URL https://link.springer.com/10.1007/ s13042-024-02443-6

    2024

  2. [9]

    URL https://pmc.ncbi.nlm.nih.gov/articles/PMC11252241/

    Si, Y.et al.Foundation models in molecular biology.Biophysics Reports10, 135 (2024). URL https://pmc.ncbi.nlm.nih.gov/articles/PMC11252241/

    2024

  3. [10]

    Vaswani, A.et al.Attention is all you need.Advances in neural information processing systems30(2017)

    2017

  4. [11]

    & Henikoff, S

    Haig, D. & Henikoff, S. Genomes and evolution: Deciphering the genomic palimpsest.Current Opinion in Genetics & Development14, 599–602 (2004)

    2004

  5. [12]

    T.et al.Systematic localization of common disease-associated variation in regulatory dna.Science337, 1190–1195 (2012)

    Maurano, M. T.et al.Systematic localization of common disease-associated variation in regulatory dna.Science337, 1190–1195 (2012)

    2012

  6. [13]

    Zhou, Z.et al.Dnabert-2: Efficient foundation model and benchmark for multi- species genome.arXiv preprint arXiv:2306.15006(2023)

    2023 arXiv

  7. [14]

    Fishman, V.et al.Gena-lm: a family of open-source foundational dna language models for long sequences.Nucleic Acids Research53, gkae1310 (2025)

    2025

  8. [15]

    Dalla-Torre, H.et al.Nucleotide transformer: building and evaluating robust foundation models for human genomics.Nature Methods1–11 (2024)

    2024

  9. [16]

    Nguyen, E.et al.Sequence modeling and design from molecular to genome scale with evo.Science386, eado9336 (2024). 16

    2024

  10. [17]

    Sanabria, M., Hirsch, J., Joubert, P. M. & Poetsch, A. R. Dna language model grover learns sequence context in the human genome.Nature Machine Intelligence6, 911–923 (2024)

    2024

  11. [18]

    M., Wong, A

    Chen, K. M., Wong, A. K., Troyanskaya, O. G. & Zhou, J. A sequence-based global map of regulatory activity for deciphering human genetics.Nature genetics 54, 940–949 (2022)

    2022

  12. [19]

    & Koo, P

    Tang, Z., Somia, N., Yu, Y. & Koo, P. K. Evaluating the representational power of pre-trained dna language models for regulatory genomics.bioRxiv(2024)

    2024

  13. [20]

    Patel, A.et al.Dart-eval: A comprehensive dna language model evaluation benchmark on regulatory dna.arXiv preprint arXiv:2412.05430(2024)

    2024

  14. [21]

    Agarwal, V.et al.Massively parallel characterization of transcriptional regulatory elements.Nature1–10 (2025)

    2025

  15. [22]

    A.et al.Evaluation of grch38 and de novo haploid genome assem- blies demonstrates the enduring quality of the reference assembly.Genome research27, 849–864 (2017)

    Schneider, V. A.et al.Evaluation of grch38 and de novo haploid genome assem- blies demonstrates the enduring quality of the reference assembly.Genome research27, 849–864 (2017)

    2017

  16. [23]

    M., Sirotkin, K., Ward, M

    Smigielski, E. M., Sirotkin, K., Ward, M. & Sherry, S. T. dbsnp: a database of single nucleotide polymorphisms.Nucleic acids research28, 352–355 (2000)

    2000

  17. [24]

    & Bucher, P

    Dreos, R., Ambrosini, G., Cavin P´ erier, R. & Bucher, P. Epd and epdnew, high- quality promoter resources in the next-generation sequencing era.Nucleic acids research41, D157–D164 (2013)

    2013

  18. [25]

    & Chong, K

    Oubounyt, M., Louadi, Z., Tayara, H. & Chong, K. T. Deepromoter: robust promoter predictor using deep learning.Frontiers in genetics10, 286 (2019)

    2019

  19. [26]

    Wang, R., Wang, Z., Wang, J. & Li, S. Splicefinder: ab initio prediction of splice sites using convolutional neural network.BMC bioinformatics20, 1–13 (2019)

    2019

  20. [27]

    Pi˜ nero, J.et al.The disgenet knowledge platform for disease genomics: 2019 update.Nucleic acids research48, D845–D855 (2020)

    2020

  21. [28]

    Cerezo, M.et al.The nhgri-ebi gwas catalog: standards for reusability, sustainability and diversity.Nucleic acids research53, D998–D1005 (2025)

    2025

  22. [29]

    J.et al.Clinvar: improvements to accessing data.Nucleic Acids Research48, D835–D844 (2019)

    Landrum, M. J.et al.Clinvar: improvements to accessing data.Nucleic Acids Research48, D835–D844 (2019)

    2019

  23. [30]

    Bioinformatics26, 1112–1118 (2010)

    Malone, J.et al.Modeling sample variables with an experimental factor ontology. Bioinformatics26, 1112–1118 (2010)

    2010

  24. [31]

    A.et al.Mondo: unifying diseases for the world, by the world

    Vasilevsky, N. A.et al.Mondo: unifying diseases for the world, by the world. MedRxiv2022–04 (2022). 17

    2022

  25. [32]

    Aligning sequence reads, clone sequences and assembly contigs with bwa- mem.arXiv preprint arXiv:1303.3997(2013)

    Li, H. Aligning sequence reads, clone sequences and assembly contigs with bwa- mem.arXiv preprint arXiv:1303.3997(2013)

    2013 arXiv

  26. [33]

    Attention is all you need.Advances in Neural Information Processing Systems(2017)

    Vaswani, A. Attention is all you need.Advances in Neural Information Processing Systems(2017)

    2017

  27. [34]

    Warner, B.et al.Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference.arXiv preprint arXiv:2412.13663(2024)

    2024 arXiv

  28. [36]

    & Birch, A

    Sennrich, R., Haddow, B. & Birch, A. Neural machine translation of rare words with subword units.arXiv preprint arXiv:1508.07909(2015)

    2015 arXiv

  29. [37]

    & Richardson, J

    Kudo, T. & Richardson, J. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing.arXiv preprint arXiv:1808.06226(2018)

    2018 arXiv

  30. [38]

    Chaton, T. & AI, L. Litdata: Transform datasets at scale. optimize datasets for fast ai model training. https://github.com/Lightning-AI/litdata (2023). Accessed: 2025-04-09

    2023

  31. [39]

    Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805(2018)

    Devlin, J. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805(2018)

    2018 arXiv

  32. [40]

    Frazer, J.et al.Disease variant prediction with deep generative models of evolutionary data.Nature599, 91–95 (2021)

    2021

  33. [41]

    M.et al.A community effort to optimize sequence-based deep learning models of gene regulation.Nature biotechnology1–11 (2024)

    Rafi, A. M.et al.A community effort to optimize sequence-based deep learning models of gene regulation.Nature biotechnology1–11 (2024)

    2024

  34. [42]

    Cheng, J.et al.Accurate proteome-wide missense variant effect prediction with alphamissense.Science381, eadg7492 (2023). 18

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.