BarcodeMamba: State Space Models for Biodiversity Analysis

Graham W. Taylor; Tiancheng Gao

REVIEW 4 major objections 5 minor 24 references

BarcodeMamba: State Space Models for Biodiversity Analysis

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A small Mamba-2 model pretrained on DNA barcodes reaches 99.2% species-level probing accuracy with 8.3% of BarcodeBERT's parameters, and attains 70.2% genus-level accuracy on unseen species.

desk verdict A careful Mamba-2-for-barcodes study with real ablation value, but the unseen-species claim needs a cleaner split description and the abstract oversells fine-tuning. read the letter →

arxiv 2412.11084 v1 pith:AWHZJDCO submitted 2024-12-15 cs.LG q-bio.GNq-bio.QM

Tiancheng Gao , Graham W. Taylor This is my paper · ORCID

classification cs.LGq-bio.GNq-bio.QM

keywords DNAbarcodesbiodiversityanalysisstructuredstatespacemodelsMamba-2self-supervisedpretrainingnext-tokenpredictionk-mertokenizationspeciesidentification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a small structured state space model can learn DNA barcode representations good enough for invertebrate species identification, replacing much larger attention-based models. It introduces BarcodeMamba, a Mamba-2 model pretrained with next-token prediction on 660-base-pair barcodes from 1.5 million Canadian invertebrates. On seen species, a model with roughly 4.3 to 7.4 million parameters reaches 99.2% linear-probe accuracy without fine-tuning, exceeding BarcodeBERT while using about 8.3% of its parameters. On unseen species, the scaled 56.7 million parameter model reaches 70.2% genus-level 1-nearest-neighbor accuracy, indicating transfer to novel taxa. The paper also shows that tokenizer choice matters: character-level tokens help seen-species tasks, while k-mer tokens help unseen-species genus assignment.

What carries the argument

The central object is the Mamba-2 structured state space layer, a sequence-mixing block that unifies a linear recurrence with a kernelized form of attention, giving sub-quadratic time complexity. BarcodeMamba stacks these layers with layer normalization and a multi-layer perceptron, maps d-dimensional states through a p-dimensional head, and pretrains by predicting the next token over 660-base-pair barcodes. This machinery is what lets a small model encode barcode context efficiently; the paper pairs it with two tokenizers, character-level and k-mer with k=4, 5, 6, and shows that the k-mer variant is what carries the unseen-species 1-nearest-neighbor result.

What would settle it

Compute pairwise sequence identity between every held-out unseen-species barcode and every sequence in the 0.9M pretraining and 67.2k fine-tuning sets; if any unseen barcode has a near-identical match (for example, more than 99% identity over the 660 base pairs), the 70.2% 1-NN genus accuracy could be memorization rather than transfer.

Watch

Extended reading notes

Core claim

BarcodeMamba is a decoder-only language model with stacked Mamba-2 state space layers, pretrained on barcode sequences by next-token prediction and then evaluated on species classification. In the authors' comparisons it outperforms BarcodeBERT on linear probing for seen species (99.2% versus 93.0%) at a fraction of the parameters, and after scaling to 56.7 million parameters it achieves 70.2% genus-level accuracy in 1-nearest-neighbor probing for species held out from training. The ablation shows that Mamba-2 generally beats the original Mamba as the mixing layer, that next-token prediction is a stronger pretraining objective than masked language modeling for probing, and that character-level tokenization is best for fine-tuning and linear probing while k-mer tokenization with k=6 is best for generalization to unseen species.

Load-bearing premise

The load-bearing assumption is that the held-out 'unseen' species are genuinely absent from the pretraining and fine-tuning sets; the paper inherits BarcodeBERT's data splits and says only that hidden barcodes did not overlap, with no described deduplication threshold.

Editorial extensions

If this is right

A sub-quadratic backbone can replace attention in barcode foundation models without sacrificing representation quality, and with substantially fewer parameters.
Barcode-specific pretraining remains necessary: off-the-shelf DNA models such as HyenaDNA and Caduceus underperform on unseen-species probing despite similar or larger sizes.
The best configuration differs by endpoint: character-level tokens plus next-token prediction for seen-species classification, and k-mer tokenization with k=6 for unseen-species genus assignment.
Scaling BarcodeMamba from 1.9M to 56.7M parameters raises 1-NN genus accuracy on unseen species, suggesting returns to scale for zero-shot taxonomic transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves open whether longer k-mers or variable-length motifs push the unseen-species 1-NN result higher; the monotone trend from k=4 to k=6 makes this a testable extension.
If the same parameter efficiency carries to the larger BIOSCAN-5M dataset, BarcodeMamba's architecture could make multi-million-specimen pretraining substantially cheaper.
The deduplication check is the key extension: rerunning the unseen-species probe after excluding barcodes with high identity to any training sequence would separate memorization from true taxonomic transfer.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Probably the fairest way to describe this paper is as a solid, careful engineering study that gets slightly oversold in its own abstract. What it does well: it takes Mamba-2, applies it to the Canadian invertebrate COI barcode dataset, and systematically ablates tokenizers (character vs k-mer), pretraining objectives (NTP vs MLM), Mamba vs Mamba-2 layers, and model scale. That is genuinely useful information for anyone building barcode foundation models. The seen-species linear probing result — 99.2% accuracy with about 8% of BarcodeBERT's parameters — is credible and, if reproducible, a real practical win. The public code link helps.

The soft spots are where you'd expect. First, the abstract's blanket 'better performance' is contradicted by their own Table 1: fine-tuning is 97.7 vs BarcodeBERT's 98.1. The efficiency story holds for probing, not for fine-tuning, and the authors should say that plainly. Second, there are no error bars or multiple seeds anywhere; every number is a point estimate from a single run. Given the reported differences between variants are often a few points, that matters.

The bigger issue is the 70.2% genus-level 1-NN accuracy on 'unseen' species. The paper inherits BarcodeBERT's splits and only mentions non-overlap in the context of a perplexity measurement. It never describes how unseen species were separated from pretraining and fine-tuning sequences, whether there was an identity threshold or haplotype-level filtering. COI barcodes are 660 bp, and congeneric species frequently share or nearly share haplotypes. Without a clean species-level split, a 1-NN classifier can label held-out species by sequence memory rather than taxonomic generalization. So the headline 'discovering unseen species' result is not demonstrated as written. This is fixable by documenting the split.

None of this is fatal to the core claim that Mamba-2 is an efficient backbone for barcode identification. The paper deserves a serious referee, but the authors need to add variance estimates, correct the abstract's overstatement, and, most importantly, describe the unseen-species splitting and deduplication. If they do that, it becomes a useful reference point for the biodiversity ML community.

Referee Report

4 major / 5 minor

Summary. The paper introduces BarcodeMamba, a family of Mamba-2-based state space models pretrained on 660-bp COI DNA barcode sequences from the Canadian invertebrate dataset. The authors compare BarcodeMamba with BarcodeBERT, DNABERT, DNABERT-2, HyenaDNA, Caduceus, and a supervised CNN on three tasks: fine-tuned species classification, linear probing of seen species, and genus-level 1-nearest-neighbor probing of unseen species. They also ablate tokenization (character vs. k-mer) and pretraining objectives (NTP vs. MLM), and report a scaling study. The central claims are that BarcodeMamba reaches 99.2% species-level linear-probe accuracy with far fewer parameters than BarcodeBERT, and that a larger BarcodeMamba reaches 70.2% genus-level 1-NN accuracy on unseen species, supporting the biodiversity-discovery narrative.

Significance. If the results hold, the paper makes a useful empirical contribution: it shows that a structured state space architecture with next-token prediction can produce DNA barcode representations competitive with a much larger BERT model, at substantially lower parameter counts and with sub-quadratic scaling. The strong linear-probe result (99.2% at 4.3-7.4M parameters) is a concrete, falsifiable finding. The 70.2% 1-NN genus-level accuracy for unseen species, if the splits are clean, would be the most interesting result because it speaks to generalization to new taxa rather than memorization of seen classes. However, the significance is currently conditional on split integrity and statistical robustness, both of which need to be demonstrated.

major comments (4)

[Section 3.1] The 1-NN unseen-species probe set is inherited from BarcodeBERT, but the paper never describes how 'unseen' is enforced. The only explicit non-overlap statement is for the perplexity set ('unseen data that did not overlap with the pretraining or fine-tuning subsets'), not for the 1-NN probe set, and no deduplication procedure, identity threshold, or species-level filtering is reported. Since congeneric species frequently share COI haplotypes or differ by a few substitutions, the 70.2% genus-level 1-NN accuracy could reflect sequence memory of near-identical barcodes in the 0.9M pretraining or 67.2k fine-tuning sets rather than taxonomic generalization. Please specify how unseen species were selected, report the BOLD record-level deduplication threshold, and verify that no unseen barcode or close variant appears in the training splits.
[Abstract and Table 1] The abstract's blanket claim that 'BarcodeMamba has better performance than BarcodeBERT' is contradicted by Table 1 for fine-tuning: BarcodeMamba-2-large reaches 97.7% while BarcodeBERT reaches 98.1%, and the mini model also reaches 97.7%. The claim should be restricted to linear probing and 1-NN probing, and the parameter-efficiency statement should specify which model size achieves which accuracy, since the 8.3% figure refers to the mini model and the 70.2% figure to the large model.
[Tables 1-5] All experimental results are reported as single numbers with no error bars, standard deviations, or seed replication. Given that the fine-tuning differences are small (e.g., 97.7% vs. 98.1%) and even the probing differences in the scaling study fluctuate by several points, the reported improvements cannot be distinguished from run-to-run noise. Please report mean and standard deviation over at least three independent seeds, and indicate whether the grid search over linear-probe hyperparameters was performed for BarcodeMamba as well as the baselines.
[Section 4.2.3 and Table 5] The text states that 'under optimal model dimensions and number of layers, both linear and 1-NN probing accuracy increase as the parameter count increases,' but the raw numbers in Table 5 are not monotonic. For the k-mer tokenizer, 1-NN accuracy is 63.2% at 7.4M parameters but drops to 49.2% at 9.8M, and 68.5% at 39.9M drops to 60.5% at 90.2M. Even the character-level column shows a dip in fine-tuning accuracy. Please clarify whether the claimed trend is over a selected Pareto frontier rather than the raw configurations, correct the text, or explain the non-monotonicity.

minor comments (5)

[Section 3.1] The text says the dataset 'consisting of 1.5 M samples' but the reported pretraining split (0.9M training + 47.1k validation) sums to approximately 0.95M, not 1.5M. Please clarify the relationship between the 1.5M total, the 0.9M pretraining set, and the 67.2k fine-tuning set.
[Table 1 caption] The notation of the form '(k=6)' inside cells is ambiguous; it is not clear whether the optimal k applies to the column's metric or to the model globally. Please clarify with a sentence such as 'the optimal k-mer length for that column's metric is shown in parentheses.'
[Section 4.2.2] The phrase 'significantly better results' is used without any statistical test; please rephrase to 'better results' or provide confidence intervals.
[Section 4.2.3] The sentence 'the performance of BarcodeMamba with a character-level tokenizer is shown in Table 5 (left), where perplexity, fine-tuning, seen species-level and unseen genus-level probing accuracy improve as BarcodeMamba scales up' is not accurate because linear probing reaches 99.4% at 30.1M and then sits at 99.2-99.3% at larger sizes; please qualify the claim.
[Figure 1] The scaling plots would be easier to interpret if each point were labeled with the corresponding model configuration, and if lines connected only configurations that share architecture choices; currently the reader cannot tell which points correspond to Table 5.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BarcodeMamba is an empirical benchmark with independent held-out evaluations; inherited BarcodeBERT splits do not define the results.

full rationale

This paper contains no derivation chain that could be circular. It is an empirical study that pretrains a Mamba-2 model on DNA barcode sequences and evaluates the resulting representations on species-level fine-tuning, linear probing, and genus-level 1-NN probing for held-out species. The central claims, such as 99.2% species-level linear probing accuracy and 70.2% genus-level 1-NN accuracy for unseen species, are direct measurements on test data and are not defined in terms of BarcodeBERT's outputs or any fitted parameter renamed as a prediction. The paper does inherit data splits and evaluation protocols from BarcodeBERT, but this is a benchmark choice rather than a load-bearing self-citation: the reported accuracies would stand or fall regardless of BarcodeBERT's own results. The skeptical concern about unseen-species contamination is a data-quality or deduplication question, not a circularity one: even if the 1-NN accuracy were inflated by leaked near-identical barcodes, the value would not be equivalent to the training input by construction; it would be an invalid experiment. Similarly, the grid search over linear probe hyperparameters is a standard selection procedure over a validation signal, not a fitted input masquerading as a prediction. No equation in the paper reduces to its own inputs, and no uniqueness theorem or ansatz is imported from the authors' prior work to force the conclusion. The comparison against external baselines and the scaling study provide independent empirical content, so the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on data assumptions (leakage-free splits, meaningful unseen species, valid 1-NN probing) and on standard implementation assumptions for Mamba-2, rather than on mathematical derivations. No new physical or biological entities are introduced, and the only free choices are hyperparameters and tokenizer settings.

free parameters (4)

pretraining learning rate and weight decay = 6e-4 LR, 0.1 weight decay, AdamW
Chosen for all pretraining runs in Section 4.2.2; performance depends on these values, which are standard settings rather than fitted constants.
linear probe hyperparameters = grid: LR 0.01-0.5, momentum 0.2-0.8, weight decay 1e-8 to 1e-11; best per model reported
Grid-searched per model and the best result on the evaluation split is reported in Table 1, which can overstate the headline accuracy.
k-mer length = k=6 for 1-NN and k-mer models; k=1 (character-level) for linear probe
Selected after ablation in Tables 2-3: k=6 maximizes 1-NN accuracy while character tokens maximize linear probe accuracy.
model scale (layers n, dimension d, head dimension p) = ablation: n=2, d=256, p=64; scaling: n=2-10, d=256-768
Architecture sizes are chosen by hand, and the scaling results depend on these configurations; the paper reports no sensitivity analysis over them.

assumptions (4)

domain assumption Fixed 660 bp COI barcode sequences with characters A, T, G, C, N preserve the information needed for species and genus identification.
Assumed throughout Section 3.1 and inherited from BarcodeBERT [1]; if this preprocessing discards taxonomic signal, downstream accuracies are not meaningful.
domain assumption The dataset splits inherited from BarcodeBERT are leakage-free: pretraining (0.9M), fine-tuning (67.2k), and unseen-species evaluation sets do not overlap.
Stated in Section 3.1; if unseen species or close homologs appear in pretraining or fine-tuning, the 1-NN 'unseen' results are inflated.
standard math Mamba-2's selective state-space layer, as implemented in the codebase, correctly models nucleotide sequences under causal NTP and MLM objectives.
Relies on the correctness of the Mamba-2 architecture [6] and its software implementation; not re-derived in this paper.
domain assumption 1-NN probing on pretrained embeddings is a valid measure of generalization to unseen species at the genus level.
Used in Section 4.1; 1-NN can succeed through shallow sequence similarity between unseen species and known genus members, so high accuracy does not necessarily imply taxonomic discovery.

how reviews work

0 comments

Cite this review

Pith. "Pith review of BarcodeMamba: State Space Models for Biodiversity Analysis." pith.science (2026). https://pith.science/paper/AWHZJDCO

@misc{pith2026241211084,
  author       = {Pith},
  title        = {Pith review of: BarcodeMamba: State Space Models for Biodiversity Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AWHZJDCO}},
  note         = {Machine review of arXiv:2412.11084}
}

read the original abstract

DNA barcodes are crucial in biodiversity analysis for building automatic identification systems that recognize known species and discover unseen species. Unlike human genome modeling, barcode-based invertebrate identification poses challenges in the vast diversity of species and taxonomic complexity. Among Transformer-based foundation models, BarcodeBERT excelled in species-level identification of invertebrates, highlighting the effectiveness of self-supervised pretraining on barcode-specific datasets. Recently, structured state space models (SSMs) have emerged, with a time complexity that scales sub-quadratically with the context length. SSMs provide an efficient parameterization of sequence modeling relative to attention-based architectures. Given the success of Mamba and Mamba-2 in natural language, we designed BarcodeMamba, a performant and efficient foundation model for DNA barcodes in biodiversity analysis. We conducted a comprehensive ablation study on the impacts of self-supervised training and tokenization methods, and compared both versions of Mamba layers in terms of expressiveness and their capacity to identify "unseen" species held back from training. Our study shows that BarcodeMamba has better performance than BarcodeBERT even when using only 8.3% as many parameters, and improves accuracy to 99.2% on species-level accuracy in linear probing without fine-tuning for "seen" species. In our scaling study, BarcodeMamba with 63.6% of BarcodeBERT's parameters achieved 70.2% genus-level accuracy in 1-nearest neighbor (1-NN) probing for unseen species. The code repository to reproduce our experiments is available at https://github.com/bioscan-ml/BarcodeMamba.

Figures

Figures reproduced from arXiv: 2412.11084 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 11 canonical work pages

[1]

Wang, Scott C

Pablo Millan Arias, Niousha Sadjadi, Monireh Safari, ZeMing Gong, Austin T. Wang, Scott C. Lowe, Joakim Bruslund Haurum, Iuliia Zarubiieva, Dirk Steinke, Lila Kari, Angel X. Chang, and Graham W. Taylor. BarcodeBERT: Transformers for biodiversity analysis, 2023

work page 2023
[2]

Fine-grained zero-shot learning with dna as side information

Sarkhan Badirli, Zeynep Akata, George Mohler, Christine Picard, and Mehmet M Dundar. Fine-grained zero-shot learning with dna as side information. Advances in Neural Information Processing Systems, 34: 19352–19362, 2021

work page 2021
[3]

Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari S. Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Grégoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann LeCun, and Micah Goldblum. A cookbook of self-supervised learning. ArXiv, a...

arXiv 2023
[4]

Theis, Alan Moses, and Bo Wang

Micaela Elisa Consens, Cameron Dufault, Michael Wainberg, Duncan Forster, Mehran Karimzadeh, Hani Goodarzi, Fabian J. Theis, Alan Moses, and Bo Wang. To transformers and beyond: Large language models for the genome. ArXiv, abs/2311.07621, 2023

arXiv 2023
[5]

The nucleotide transformer: Building and evaluating robust foundation models for human genomics

Hugo Dalla-Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nicolas Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Bernardo P de Almeida, Hassan Sirelkha- tim, et al. The nucleotide transformer: Building and evaluating robust foundation models for human genomics. BioRxiv, pages 2023–01, 2023

work page 2023
[6]

Transformers are ssms: Generalized models and efficient algorithms through structured state space duality

Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060, 2024

arXiv 2024
[7]

A reference library for canadian invertebrates with 1.5 million barcodes, voucher specimens, and dna samples

Jeremy R DeWaard, Sujeevan Ratnasingham, Evgeny V Zakharov, Alex V Borisenko, Dirk Steinke, Angela C Telfer, Kate HJ Perez, Jayme E Sones, Monica R Young, Valerie Levesque-Beaudin, et al. A reference library for canadian invertebrates with 1.5 million barcodes, voucher specimens, and dna samples. Scientific data, 6(1):308, 2019

work page 2019
[8]

Gena-lm: A family of open-source foundational models for long 8 dna sequences

Veniamin Fishman, Yuri Kuratov, Maxim Petrov, Aleksei Shmelev, Denis Shepelin, Nikolay Chekanov, Olga Kardymon, and Mikhail Burtsev. Gena-lm: A family of open-source foundational models for long 8 dna sequences. bioRxiv, 2023. doi: 10.1101/2023.06.12.544594. URL https://www.biorxiv.org/ content/early/2023/06/13/2023.06.12.544594

doi:10.1101/2023.06.12.544594 2023

Show all 24 references

[9]

Lowe, ZeMing Gong, Pablo Millan Arias, Nicholas Pellegrino, Austin T

Zahra Gharaee, Scott C. Lowe, ZeMing Gong, Pablo Millan Arias, Nicholas Pellegrino, Austin T. Wang, Joakim Bruslund Haurum, Iuliia Zarubiieva, Lila Kari, Dirk Steinke, Graham W. Taylor, Paul Fieguth, and Angel X. Chang. BIOSCAN-5M: A multimodal dataset for insect biodiversity, 2024

2024
[10]

Genomic benchmarks: a collection of datasets for genomic sequence classification

Katarína Grešová, Vlastimil Martinek, David ˇCechák, Petr Šimeˇcek, and Panagiotis Alexiou. Genomic benchmarks: a collection of datasets for genomic sequence classification. BMC Genomic Data, 24(1):25, 2023

2023
[11]

Mamba: Linear-time sequence modeling with selective state spaces

Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

2023 arXiv
[12]

Efficiently modeling long sequences with structured state spaces

Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In The International Conference on Learning Representations (ICLR), 2022

2022
[13]

Biological identifications through dna barcodes

Paul DN Hebert, Alina Cywinska, Shelley L Ball, and Jeremy R DeWaard. Biological identifications through dna barcodes. Proceedings of the Royal Society of London. Series B: Biological Sciences, 270 (1512):313–321, 2003

2003
[14]

Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome

Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome. Bioinformatics, 37(15):2112–2120, 2021

2021
[15]

Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei. Scaling laws for neural language models. ArXiv, abs/2001.08361, 2020

2001 arXiv
[16]

The insect cytochrome oxidase i gene: evolutionary patterns and conserved primers for phylogenetic studies

DH Lunt, D-X Zhang, Jacek M Szymura, and OM Hewltt. The insect cytochrome oxidase i gene: evolutionary patterns and conserved primers for phylogenetic studies. Insect molecular biology, 5(3): 153–165, 1996

1996
[17]

BEND: Benchmarking DNA language models on biologically meaningful tasks

Frederikke Isa Marin, Felix Teufel, Marc Horlacher, Dennis Madsen, Dennis Pultz, Ole Winther, and Wouter Boomsma. BEND: Benchmarking DNA language models on biologically meaningful tasks. In The Twelfth International Conference on Learning Representations, 2024. URL https://ope...

2024
[18]

Taxonomic classification of dna sequences beyond sequence similarity using deep neural networks

Florian Mock, Fleming Kretschmer, Anton Kriese, Sebastian Böcker, and Manja Marz. Taxonomic classification of dna sequences beyond sequence similarity using deep neural networks. Proceedings of the National Academy of Sciences of the United States of America, 119, 2022

2022
[19]

Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution

Eric Nguyen, Michael Poli, Marjan Faizi, Armin Thomas, Michael Wornow, Callum Birch-Sykes, Stefano Massaroli, Aman Patel, Clayton Rabideau, Yoshua Bengio, et al. Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution. Advances in neural information proc...

2024
[20]

Bold: The barcode of life data system (http://www

Sujeevan Ratnasingham and Paul DN Hebert. Bold: The barcode of life data system (http://www. barcodinglife. org). Molecular ecology notes, 7(3):355–364, 2007

2007
[21]

Melissa Sanabria, Jonas Hirsch, and Anna R. Poetsch. The human genome’s vocabulary as proposed by the dna language model grover. bioRxiv, 2023

2023
[22]

Caduceus: Bi-directional equivariant long-range dna sequence modeling

Yair Schiff, Chia-Hsiang Kao, Aaron Gokaslan, Tri Dao, Albert Gu, and V olodymyr Kuleshov. Caduceus: Bi-directional equivariant long-range dna sequence modeling. arXiv preprint arXiv:2403.03234, 2024

2024 arXiv
[23]

Attention is all you need

A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

2017
[24]

Dnabert-2: Efficient foundation model and benchmark for multi-species genome

Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, and Han Liu. Dnabert-2: Efficient foundation model and benchmark for multi-species genome. arXiv preprint arXiv:2306.15006, 2023. 9

2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Wang, Scott C

Pablo Millan Arias, Niousha Sadjadi, Monireh Safari, ZeMing Gong, Austin T. Wang, Scott C. Lowe, Joakim Bruslund Haurum, Iuliia Zarubiieva, Dirk Steinke, Lila Kari, Angel X. Chang, and Graham W. Taylor. BarcodeBERT: Transformers for biodiversity analysis, 2023

work page 2023

[2] [2]

Fine-grained zero-shot learning with dna as side information

Sarkhan Badirli, Zeynep Akata, George Mohler, Christine Picard, and Mehmet M Dundar. Fine-grained zero-shot learning with dna as side information. Advances in Neural Information Processing Systems, 34: 19352–19362, 2021

work page 2021

[3] [3]

Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari S. Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Grégoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann LeCun, and Micah Goldblum. A cookbook of self-supervised learning. ArXiv, a...

arXiv 2023

[4] [4]

Theis, Alan Moses, and Bo Wang

Micaela Elisa Consens, Cameron Dufault, Michael Wainberg, Duncan Forster, Mehran Karimzadeh, Hani Goodarzi, Fabian J. Theis, Alan Moses, and Bo Wang. To transformers and beyond: Large language models for the genome. ArXiv, abs/2311.07621, 2023

arXiv 2023

[5] [5]

The nucleotide transformer: Building and evaluating robust foundation models for human genomics

Hugo Dalla-Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nicolas Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Bernardo P de Almeida, Hassan Sirelkha- tim, et al. The nucleotide transformer: Building and evaluating robust foundation models for human genomics. BioRxiv, pages 2023–01, 2023

work page 2023

[6] [6]

Transformers are ssms: Generalized models and efficient algorithms through structured state space duality

Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060, 2024

arXiv 2024

[7] [7]

A reference library for canadian invertebrates with 1.5 million barcodes, voucher specimens, and dna samples

Jeremy R DeWaard, Sujeevan Ratnasingham, Evgeny V Zakharov, Alex V Borisenko, Dirk Steinke, Angela C Telfer, Kate HJ Perez, Jayme E Sones, Monica R Young, Valerie Levesque-Beaudin, et al. A reference library for canadian invertebrates with 1.5 million barcodes, voucher specimens, and dna samples. Scientific data, 6(1):308, 2019

work page 2019

[8] [8]

Gena-lm: A family of open-source foundational models for long 8 dna sequences

Veniamin Fishman, Yuri Kuratov, Maxim Petrov, Aleksei Shmelev, Denis Shepelin, Nikolay Chekanov, Olga Kardymon, and Mikhail Burtsev. Gena-lm: A family of open-source foundational models for long 8 dna sequences. bioRxiv, 2023. doi: 10.1101/2023.06.12.544594. URL https://www.biorxiv.org/ content/early/2023/06/13/2023.06.12.544594

doi:10.1101/2023.06.12.544594 2023