REVIEW 7 cited by
To Transformers and Beyond: Large Language Models for the Genome
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In the rapidly evolving landscape of genomics, deep learning has emerged as a useful tool for tackling complex computational challenges. This review focuses on the transformative role of Large Language Models (LLMs), which are mostly based on the transformer architecture, in genomics. Building on the foundation of traditional convolutional neural networks and recurrent neural networks, we explore both the strengths and limitations of transformers and other LLMs for genomics. Additionally, we contemplate the future of genomic modeling beyond the transformer architecture based on current trends in research. The paper aims to serve as a guide for computational biologists and computer scientists interested in LLMs for genomic data. We hope the paper can also serve as an educational introduction and discussion for biologists to a fundamental shift in how we will be analyzing genomic data in the future.
Forward citations
Cited by 7 Pith papers
-
Deep and Probabilistic Models for Gene Regulatory Network Inference
A dual-stage GRN inference framework combining a sequence-based transformer prior (GLM-Prior) with a probabilistic matrix-factorization model (PMF-GRN) for uncertainty-aware network reconstruction.
-
Hyperbolic Genome Embeddings
Hyperbolic CNNs outperform Euclidean CNNs on 37 of 42 genome classification benchmarks and beat several large DNA language models on 7 GUE tasks using orders of magnitude fewer parameters.
-
METAGENE-1: Metagenomic Foundation Model for Pandemic Monitoring
A 7B transformer pretrained on 1.5T base pairs of wastewater metagenomic reads achieves strong pathogen detection and embedding scores, though some new benchmarks are partly in-distribution.
-
BarcodeMamba: State Space Models for Biodiversity Analysis
A Mamba-2 state space model pretrained on DNA barcodes matches or beats BarcodeBERT on species and genus classification with far fewer parameters, reaching 99.2% seen-species linear probe and 70.2% unseen-species 1-NN...
-
Assessing Foundation Models' Transferability to Physiological Signals in Precision Medicine
A simulation-based evaluation pipeline shows that Moirai embeddings distort physiological signal structure, with feature entanglement, lost temporal dynamics, and reduced scenario discrimination.
-
Computational Protein Science in the Era of Large Language Models (LLMs)
A survey that categorizes protein language models by the knowledge they learn and reviews their applications, with no new experimental results.
-
Deciphering genomic codes using advanced NLP techniques: a scoping review
A PRISMA-based scoping review finds 26 studies (2021-2024) applying NLP and transformers to genomics, with k-mer tokenization and BERT-style models dominating.
Discussion (0). Continue with ORCID to comment.