REVIEW 5 cited by
Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Language models for biological and chemical sequences enable crucial applications such as drug discovery, protein engineering, and precision medicine. Currently, these language models are predominantly based on Transformer architectures. While Transformers have yielded impressive results, their quadratic runtime dependency on the sequence length complicates their use for long genomic sequences and in-context learning on proteins and chemical sequences. Recently, the recurrent xLSTM architecture has been shown to perform favorably compared to Transformers and modern state-space model (SSM) architectures in the natural language domain. Similar to SSMs, xLSTMs have a linear runtime dependency on the sequence length and allow for constant-memory decoding at inference time, which makes them prime candidates for modeling long-range dependencies in biological and chemical sequences. In this work, we tailor xLSTM towards these domains and propose a suite of architectural variants called Bio-xLSTM. Extensive experiments in three large domains, genomics, proteins, and chemistry, were performed to assess xLSTM's ability to model biological and chemical sequences. The results show that models based on Bio-xLSTM a) can serve as proficient generative models for DNA, protein, and chemical sequences, b) learn rich representations for those modalities, and c) can perform in-context learning for proteins and small molecules.
Forward citations
Cited by 5 Pith papers
-
pLSTM: parallelizable Linear Source Transition Mark networks
pLSTM extends linear recurrent networks to general directed acyclic graphs with a parallelizable scheme and two stabilization modes for long-range propagation.
-
KVpop -- Key-Value Cache Compression with Predictive Online Pruning
KVpop supervises fixed-budget KV eviction with an efficiently computed future-attention target and delayed mLSTM scoring, retaining ~97–100% of full-attention math performance at 75–88% compression.
-
BenchECG and xECG: a benchmark and baseline for ECG foundation models
xECG, an xLSTM-based ECG foundation model pretrained with SimDINOv2, achieves the best average performance on BenchECG, a new benchmark spanning eight datasets and ten tasks.
-
Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task Learning
Autoregressive DNA language models fine-tuned jointly on classification, text generation, and image generation achieve strong benchmark results and open-ended cross-modal genomic tasks.
-
BARNN: A Bayesian Autoregressive and Recurrent Neural Network
BARNN converts autoregressive and recurrent networks into Bayesian versions via time-dependent variational dropout and a temporal aggregated-posterior prior, yielding calibrated uncertainty on PDE and molecule-generat...
Discussion (0). Continue with ORCID to comment.