Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper shows that DNA foundation models can be jailbroken with high-homology prompts and pathogenicity-guided beam search, generating sequences over 90% identical to human pathogens.

desk verdict First systematic jailbreak-style benchmark for DNA language models, but the 'jailbreak' label oversells and the training-data exclusion claim is unverified. read the letter →

arxiv 2505.23839 v1 pith:QVHQXRAH submitted 2025-05-28 cs.CR q-bio.GN

Zaixi Zhang , Zhenghong Zhou , Ruofan Jin , Le Cong , Mengdi Wang This is my paper
classification cs.CRq-bio.GN
keywords DNAfoundationmodelsjailbreakattackbiosecuritybeamsearchpathogenicityguidanceEvo2SARS-CoV-2DNABench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GeneBreaker is a framework that tries to prove DNA foundation models can be jailbroken, not just that they have flaws in isolation. It feeds the victim model a prompt made of non-pathogenic DNA sequences that are highly similar to a human pathogen, then runs a beam search in which each candidate chunk is scored by a pathogenicity classifier plus the model's own log-probability, steering generation toward pathogen-like sequence space. On a curated benchmark of six viral categories, the method reports attack success rates up to 60% on Evo2-40B, and the generation of sequences over 90% identical to viruses such as SARS-CoV-2 and HIV-1. Case studies show the generated viral proteins are structurally faithful to their native counterparts when folded with AlphaFold3. If correct, this result implies that simply excluding pathogens from training data is not a sufficient safeguard for genomic AI models, and that model scaling amplifies the risk.

What carries the argument

The central mechanism is the jailbreak-oriented scoring function f = PathoLM(x) + α·log p(x), which combines a pathogenicity prediction from PathoLM, a DNA language model fine-tuned to classify viral and bacterial sequences as pathogenic, with the average log-probability the victim model assigns to a candidate sequence chunk. Higher values of f indicate that a chunk is more likely to be pathogen-like, and the function is used inside a beam search that samples chunks of length 128, keeps the top-K' partial sequences, and repeatedly appends the best chunks to the prompt. The load-bearing design choice is the prompt itself: the non-pathogenic but high-homology sequences serve as in-context examples that shift the victim model's distribution toward the target pathogen region, while the scoring function ensures that each beam step prefers continuations that look pathogenic.

What would settle it

Audit the training corpus of the Evo2 models for the exact coding sequences, or near-exact (>99% identity) copies, of the viruses in JailbreakDNABench: if any target sequence appears in training, the observed outputs could be retrieval rather than generation, and the jailbreak claim loses its force. A cleaner test is to run GeneBreaker against a DNA language model trained on a corpus that provably excludes all human-pathogenic viral sequences and check whether the attack success rate persists.

Watch

Extended reading notes

Core claim

The paper's central claim is that GeneBreaker successfully jailbreaks the latest Evo series DNA language models across six viral categories, with attack success rates reaching 60% for Evo2-40B. The attack works by retrieving non-pathogenic sequences with high homology to the target pathogen (for example bat coronaviruses close to SARS-CoV-2), using them as few-shot prompts, and then guiding an autoregressive beam search with a scoring function f = PathoLM(x) + α·log p(x), where PathoLM is a pathogenicity-classification DNA model and log p(x) is the average log-probability of the candidate under the victim model. Outputs are evaluated with nucleotide and protein BLAST against JailbreakDNABench, a curated database of human-pathogenic viruses built from Select Agents lists, and a jailbreak is scored when identity exceeds 90%. The reported case studies include a SARS-CoV-2 spike protein with 92.77% DNA similarity and 95.29% protein similarity, an HIV-1 envelope protein with 96.09% DNA and 96.47% protein similarity whose AlphaFold3-predicted structure matches the crystal structure at 0.334 RMSD, and an evolutionary experiment in which 201 of 10,000 generated spike sequences match Nextstrain-circulating SARS-CoV-2 variants at over 99.9% identity.

Load-bearing premise

The load-bearing premise is that the target viral sequences in JailbreakDNABench were not part of the Evo series models' training data; if they were, the reported high-similarity outputs could be explained by memorization instead of a generalizable jailbreak capability.

Editorial extensions

If this is right

  • A user with only public model access, no fine-tuning, and no DNA-synthesis access can produce sequences that clear the 90% identity threshold used in biosecurity screening for known human pathogens.
  • Attack success rises monotonically with model size across the tested Evo series, so scaling DNA foundation models without safeguards increases dual-use risk.
  • Excluding pathogenic sequences from training data is not a sufficient defense, because the attack reconstructs pathogen-like sequences from non-pathogenic homologs.
  • The method can also act as an evolutionary modeling tool: Evo2-40B generates SARS-CoV-2 spike variants matching real circulating clades such as Alpha, Delta, and Omicron sublineages.
  • The generated DNA translates into proteins that fold like native viral proteins, so the jailbreak outputs are not just sequence matches but potentially functional sequences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same recipe should transfer to other generative biological sequence models, such as protein language models, whenever a pathogenicity scorer is available, because the guidance signal is model-agnostic.
  • Editorial inference: defenders could invert the attack by using the same scoring function as an output filter, turning GeneBreaker into a red-team benchmark for biosecurity screening rather than only an exploit.
  • Editorial inference: the paper's reported success rates rest on an unverified training-data containment claim, so a third-party audit of Evo2's corpus is the natural next experiment to separate generalization from memorization.
  • Editorial inference: sequences below the 90% identity threshold can still be structurally and functionally dangerous, so the actual biosecurity risk boundary may lie lower than the paper's flag threshold.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper presents GeneBreaker, a three-stage framework for red-teaming DNA foundation models. First, an LLM agent retrieves homologous but purportedly non-pathogenic DNA sequences to construct few-shot prompts. Second, a beam search guided by the PathoLM pathogenicity classifier and average log-probability selects generated chunks. Third, success is measured by nucleotide/protein BLAST identity above 90% against a curated database of human pathogenic viruses (JailbreakDNABench). Experiments on Evo1-7B and Evo2-1B/7B/40B report attack success rates up to 60%, case studies on SARS-CoV-2 spike and HIV-1 envelope with AlphaFold3 structural validation, and an evolutionary analysis of SARS-CoV-2 variants.

Significance. If the results are robust, GeneBreaker would be a useful tool for assessing biosecurity risks of genomic foundation models. The benchmark JailbreakDNABench and the observed scaling trend (larger models are more susceptible) are valuable. However, the significance is currently limited by the lack of evidence that the model outputs reflect a general steering capability rather than memorization of training sequences, and by the overstatement inherent in calling the attack a 'jailbreak' when the target models are not safety-aligned.

major comments (4)
  1. [Section 4 (Benchmark Construction)] The claim that "a thorough validation" ensured the selected sequences do not appear in Evo training data is unsupported; no procedure, thresholds, or results are given. Because the benchmark sequences are the targets of the BLAST evaluation, any contamination would allow the reported success rates in Table 1 to be explained by memorization. Please provide the exact contamination screen (e.g., exact-match or k-mer search against the Evo training corpus), including the similarity threshold and the fraction of target sequences or fragments that were excluded.
  2. [Section 5.1 and Figure 3(c)] The evaluation uses the first half of each target DNA sequence as input, which is a strong memorization cue: if the target or a close relative is present (even partially) in training, the model can produce the continuation from memory. The ablation in Figure 3(c) shows zero success without GeneBreaker, but this only controls for memorization if the no-attack condition uses the identical target prefix and removes only the few-shot prompt and guided beam search. Please specify the exact comparison and add a control with a non-target or shuffled prefix to distinguish generalization from memorization.
  3. [Throughout (title, abstract, Section 1)] The paper calls GeneBreaker a "jailbreak attack" on Evo models, but Evo1 and Evo2 are unaligned base generative models; the text does not identify any safety mechanism (e.g., refusal training, output filtering) that is bypassed. The results are more accurately described as targeted prompt-guided generation. This overstatement affects the claimed significance as a security vulnerability, although the underlying methodology and empirical findings remain of interest.
  4. [Section 5.5 (Evolutionary Modeling)] The evolutionary modeling analysis reports that 201 of 10,000 generated sequences match Nextstrain SARS-CoV-2 entries at >99.9% identity. Since Evo2 was trained on a large corpus that likely contains many SARS-CoV-2 genome sequences, the high hit rate may reflect memorization rather than an emergent evolutionary modeling capability. The paper should provide a contamination analysis against the Nextstrain database, or temper the interpretation accordingly.
minor comments (5)
  1. [Section 2.1] The citation list for gradient-based attacks contains an unresolved placeholder "[?]" and should be completed.
  2. [Figure 3] Two panels are both labeled "(b)" (the correlation with prompt homology and the ablation study); renumber them to avoid confusion.
  3. [Section 2.2 and Table 3] There is a typo: "experineced" should be "experienced", and the model name is inconsistently spelled as "GENERator" and "Generator"; use one consistent spelling.
  4. [Section 4 and Table 2] The category list includes both "Positive-strand RNA viruses" and "Enteric RNA viruses", but Table 2 lists both as "+ssRNA"; clarify whether enteric RNA viruses are a subset or a separate category.
  5. [Table 1] Many entries have very large standard deviations (e.g., 20.0 ± 40.0), so the claim of consistent success across categories should be softened or supported with additional trials.

Circularity Check

3 steps flagged · score 5.0 of 10

GeneBreaker's attack success rate is partly by construction: the prompt includes the target sequence's own prefix and high-homology relatives, the beam search is guided by a pathogenicity proxy calibrated to the same similarity metric, and the 'evolutionary modeling' case study re-labels near-copies of the input Spike gene as novel variants.

  1. self definitional [Sec. 3.1 (prompt construction) and Sec. 5.1 (benchmarking protocol)]
    "In benchmarking, the first half of each DNA sequence is used as input, and the DNA model is asked to generate a subsequent sequence. ... input_prefix corresponds to a short sequence prefix extracted from the genomic region upstream of the target coding sequence."

    The evaluation feeds the model the first half of the target pathogen sequence itself and scores the generated continuation by BLAST identity to the same target database. A model that simply continues the provided prefix—whether by memorization or by exploiting the strong conservation of the few-shot homologous sequences—will mechanically achieve high identity. The reported attack success rate therefore measures the model's ability to continue a target-derived prompt, not an independent 'jailbreak' capability; the target is present in the input by construction.

  2. fitted input called prediction [Sec. 3.2, Eq. (7)]
    "we empirically observe that sequences with higher average log-probabilities tend to exhibit greater similarity to known pathogenic DNA (Figure 3 (a)). Therefore, we define the jailbreak-oriented scoring function as: f = PathoLM(x) + α · log p(x),"

    The log-probability term is adopted precisely because it correlates with the evaluation metric (similarity to the pathogen target), and PathoLM is a pathogenicity classifier aligned with the same construct. The beam search then selects chunks that maximize this proxy, and the paper reports the resulting BLAST identity as attack success. This is optimizing the test metric rather than demonstrating an independent model property; the 'guidance' is calibrated on the same similarity signal used for the verdict.

1 more flagged steps
  1. renaming known result [Sec. 5.5]
    "Our study uses the Wuhan-Hu-1 Spike gene as a few-shot prompt ... Out of 10,000 generated sequences, 201 were found to match this high-similarity criterion. ... Sequences were considered "hits" if they achieved >99.9% nucleotide identity to any entry in the Nextstrain database."

    Because the generation is prompted with the actual Wuhan-Hu-1 Spike CDS and hits are defined as >99.9% identity to a database of known SARS-CoV-2 Spike sequences, the outputs are near-copies of the input rather than evolutionary predictions. The entropy peaks at NTD/RBD are inherited from the known Nextstrain alignments used for comparison, so presenting this as 'modeling the evolution of SARS-CoV-2 variants' renames a sequence-reproduction exercise as a novel result.

full rationale

The paper is a red-team attack, so some degree of optimizing toward the target metric is inherent to the method; that alone need not be circular. The specific reductions above, however, show that the headline attack success rate is partly by construction: the input prefix is taken from the target sequence, the few-shot prompts are selected for high homology to the target, and the beam-search scoring function is fitted to the same similarity signal used for evaluation. The SARS-CoV-2 'evolutionary modeling' case study is essentially reproduction of the input sequence against a reference database of the same sequences. A separate load-bearing gap is the unverified contamination claim in Sec. 4 ('We conducted a thorough validation to ensure that the selected sequences do not appear in the training datasets of the Evo series models'); if that validation is absent or flawed, the first-half-as-input protocol would make the evaluation a memorization test, further reinforcing the first step. This is a correctness risk rather than an independent circular step, because the paper asserts the validation. No self-citation chain or imported uniqueness theorem is load-bearing. Overall, the central vulnerability claim retains some independent content—the model does not produce these sequences under an unguided prompt, and the ablations show the attack components matter—but the quantified success rates and the evolutionary-modeling narrative are inflated by construction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on assumptions about training-data exclusion, the validity of sequence-similarity as a biosecurity proxy, and the reliability of auxiliary models (PathoLM, AlphaFold3, ChatGPT-4o). These are not invented entities but imported tools and assumptions. The only explicit free parameters are the scoring weight alpha and the BLAST threshold, both hand-set.

free parameters (2)
  • alpha (alpha) = 0.5
    Weight for the log-probability term in the scoring function f = PathoLM(x) + alpha * log p(x), chosen by hand. The paper shows robustness across a range, but it is a free parameter.
  • BLAST similarity threshold = 90%
    Attack success is defined as >90% nucleotide identity or >90% translated amino acid similarity. The threshold is hand-set, with the paper stating it reduces false positives.
assumptions (5)
  • domain assumption The JailbreakDNABench target sequences are absent from the training data of the Evo series models.
    Section 4 claims a 'thorough validation' without providing evidence. If false, the high-similarity outputs may be memorization rather than generalization.
  • domain assumption BLAST identity >90% is a valid proxy for biosecurity relevance.
    The evaluator flags jailbreak success based on this threshold. The paper acknowledges the threshold is heuristic and does not demonstrate functional pathogenicity.
  • domain assumption PathoLM pathogenicity scores are sufficiently calibrated to guide generation toward pathogen-like sequences.
    The scoring function relies on PathoLM, a model from prior work. The paper assumes its scores correlate with actual pathogenicity.
  • domain assumption AlphaFold3 predicted structures are accurate enough to assess functional fidelity.
    Section 5.4 uses AlphaFold3 structures to claim structural fidelity. No experimental validation is provided.
  • domain assumption ChatGPT-4o correctly identifies non-pathogenic but homologous sequences.
    The LLM agent's outputs are not verified beyond the authors' curation; errors would affect prompt quality and thus the attack success rates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance." pith.science (2026). https://pith.science/paper/QVHQXRAH

@misc{pith2026250523839,
  author       = {Pith},
  title        = {Pith review of: GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QVHQXRAH}},
  note         = {Machine review of arXiv:2505.23839}
}
read the original abstract

DNA, encoding genetic instructions for almost all living organisms, fuels groundbreaking advances in genomics and synthetic biology. Recently, DNA Foundation Models have achieved success in designing synthetic functional DNA sequences, even whole genomes, but their susceptibility to jailbreaking remains underexplored, leading to potential concern of generating harmful sequences such as pathogens or toxin-producing genes. In this paper, we introduce GeneBreaker, the first framework to systematically evaluate jailbreak vulnerabilities of DNA foundation models. GeneBreaker employs (1) an LLM agent with customized bioinformatic tools to design high-homology, non-pathogenic jailbreaking prompts, (2) beam search guided by PathoLM and log-probability heuristics to steer generation toward pathogen-like sequences, and (3) a BLAST-based evaluation pipeline against a curated Human Pathogen Database (JailbreakDNABench) to detect successful jailbreaks. Evaluated on our JailbreakDNABench, GeneBreaker successfully jailbreaks the latest Evo series models across 6 viral categories consistently (up to 60\% Attack Success Rate for Evo2-40B). Further case studies on SARS-CoV-2 spike protein and HIV-1 envelope protein demonstrate the sequence and structural fidelity of jailbreak output, while evolutionary modeling of SARS-CoV-2 underscores biosecurity risks. Our findings also reveal that scaling DNA foundation models amplifies dual-use risks, motivating enhanced safety alignment and tracing mechanisms. Our code is at https://github.com/zaixizhang/GeneBreaker.

Figures

Figures reproduced from arXiv: 2505.23839 by the authors.

Figure 1
Figure 1. GeneBreaker: Jailbreak DNA Language Models to generate human pathogens. The [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The constructed JailbreakDNABench. (a) show the distribution of virus categories, including [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Further analysis of GeneBreaker with Evo2 7B. (a) correlation between sequence similarity [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: GeneBreaker redesign SARS-CoV-2 Spike Protein (a) and HIV-1 Envolope Protein (b) with [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Modeling the evolution of SARS-CoV-2 Spike Protein with GeneBreaker (Evo2 40B). (a) [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Hyperparameter analysis of GeneBreaker with Evo2 7B. (a) influence of [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. An Early Warning of Emerging Biosecurity Risks in Frontier LLMs

    cs.CL 2026-07 reject novelty 5.0 of 10

    A bio-red-teaming model is reported to jailbreak 14 frontier LLMs into producing dangerous biosecurity outputs, but the claimed wet-lab physical verification was not actually carried out.

Reference graph

Works this paper leans on

82 extracted references · 57 canonical work pages · cited by 1 Pith paper

  1. [1]

    https://ibbis.bio/our-work/common-mechanism/

    Common mechanism - ibbis. https://ibbis.bio/our-work/common-mechanism/ . Ac- cessed: 2025-04-27

    2025

  2. [2]

    Accurate structure prediction of biomolecular interactions with alphafold 3

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, 630(8016):493–500, 2024

    2024

  3. [3]

    Complementary dna sequencing: expressed sequence tags and human genome project

    Mark D Adams, Jenny M Kelley, Jeannine D Gocayne, Mark Dubnick, Mihael H Polymeropou- los, Hong Xiao, Carl R Merril, Andrew Wu, Bjorn Olde, Ruben F Moreno, et al. Complementary dna sequencing: expressed sequence tags and human genome project. Science, 252(5013):1651– 1656, 1991

    work page 1991

  4. [4]

    Jailbreaking leading safety-aligned llms with simple adaptive attacks

    Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety-aligned llms with simple adaptive attacks. arXiv preprint arXiv:2404.02151, 2024

    arXiv 2024

  5. [5]

    Many-shot jailbreaking

    Cem Anil, Esin Durmus, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Nina Rimsky, Meg Tong, Jesse Mu, Daniel Ford, et al. Many-shot jailbreaking

    work page

  6. [6]

    Effective gene expression prediction from sequence by integrating long-range interactions

    Žiga Avsec, Vikram Agarwal, Daniel Visentin, Joseph R Ledsam, Agnieszka Grabska- Barwinska, Kyle R Taylor, Yannis Assael, John Jumper, Pushmeet Kohli, and David R Kelley. Effective gene expression prediction from sequence by integrating long-range interactions. Nature methods, 18(10):1196–1203, 2021

    work page 2021

  7. [7]

    Protein design meets biosecurity, 2024

    David Baker and George Church. Protein design meets biosecurity, 2024

    work page 2024

  8. [8]

    Feline immunod- eficiency virus: an interesting model for aids studies and an important cat pathogen

    Mauro Bendinelli, Mauro Pistello, Stefania Lombardi, Alessandro Poli, Carlo Garzelli, Do- natella Matteucci, Luca Ceccherini-Nelli, Gino Malvaldi, and Franco Tozzini. Feline immunod- eficiency virus: an interesting model for aids studies and an important cat pathogen. Clinical microbiology reviews, 8(1):87–112, 1995

    work page 1995

Show all 82 references
  1. [9]

    Synthetic biology

    Steven A Benner and A Michael Sismour. Synthetic biology. Nature reviews genetics, 6(7):533– 543, 2005

    2005

  2. [10]

    Data poisoning in llms: Jailbreak-tuning and scaling laws

    Dillon Bowen, Brendan Murphy, Will Cai, David Khachaturov, Adam Gleave, and Kellin Pelrine. Data poisoning in llms: Jailbreak-tuning and scaling laws. arXiv preprint arXiv:2408.02946, 2024

    2024 arXiv

  3. [11]

    Genome modeling and design across all domains of life with evo 2

    Garyk Brixi, Matthew G Durrant, Jerome Ku, Michael Poli, Greg Brockman, Daniel Chang, Gabriel A Gonzalez, Samuel H King, David B Li, Aditi T Merchant, et al. Genome modeling and design across all domains of life with evo 2. bioRxiv, pages 2025–02, 2025

    2025

  4. [12]

    Rabies virus

    Kirstyn Brunker and Nardus Mollentze. Rabies virus. Trends in microbiology, 26(10):886–887, 2018

    2018

  5. [13]

    Jailbreakbench: An open robustness benchmark for jailbreaking large language models

    Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tramer, et al. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. arXiv pre...

    2024 arXiv

  6. [14]

    Pappas, and Eric Wong

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries, 2023. 10

    2023

  7. [15]

    Central dogma of molecular biology

    Francis Crick. Central dogma of molecular biology. Nature, 227(5258):561–563, 1970

    1970

  8. [16]

    Nucleotide transformer: building and evaluating robust foundation models for human genomics

    Hugo Dalla-Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nicolas Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Bernardo P de Almeida, Hassan Sirelkhatim, et al. Nucleotide transformer: building and evaluating robust foundation models...

    2025

  9. [17]

    Patholm: Identifying pathogenicity from the dna sequence through the genome foundation model

    Sajib Acharjee Dip, Uddip Acharjee Shuvo, Tran Chau, Haoqiu Song, Petra Choi, Xuan Wang, and Liqing Zhang. Patholm: Identifying pathogenicity from the dna sequence through the genome foundation model. arXiv preprint arXiv:2406.13133, 2024

    2024 arXiv

  10. [18]

    A survey on in-context learning

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.00234, 2022

    2022 arXiv

  11. [19]

    Select agents and toxins list, 2025

    Federal Select Agent Program. Select agents and toxins list, 2025. Accessed: 2025-04-28

    2025

  12. [20]

    Measles virus, immune control, and persistence

    Diane E Griffin, Wen-Hsuan Lin, and Chien-Hsiung Pan. Measles virus, immune control, and persistence. FEMS microbiology reviews, 36(3):649–662, 2012

    2012

  13. [21]

    Cold-attack: Jailbreaking llms with stealthiness and controllability, 2024

    Xingang Guo, Fangxu Yu, Huan Zhang, Lianhui Qin, and Bin Hu. Cold-attack: Jailbreaking llms with stealthiness and controllability, 2024

    2024

  14. [22]

    Maria G Guzman and Eva Harris. Dengue. The Lancet, 385(9966):453–465, 2016

    2016

  15. [23]

    Nextstrain: real-time tracking of pathogen evolution

    James Hadfield, Colin Megill, Sidney M Bell, John Huddleston, Barney Potter, Charlton Callender, Pavel Sagulenko, Trevor Bedford, and Richard A Neher. Nextstrain: real-time tracking of pathogen evolution. Bioinformatics, 34(23):4121–4123, 2018

    2018

  16. [24]

    Sars-cov-2 variants, spike mutations and immune escape

    William T Harvey, Alessandro M Carabelli, Ben Jackson, Ravindra K Gupta, Emma C Thomson, Ewan M Harrison, Catherine Ludden, Richard Reeve, Andrew Rambaut, COVID-19 Genomics UK (COG-UK) Consortium, et al. Sars-cov-2 variants, spike mutations and immune escape. Nature reviews mi...

    2021

  17. [25]

    Sars-cov-2 journey: from alpha variant to omicron and its sub-variants

    Dima Hattab, Mumen FA Amer, Zina M Al-Alami, and Athirah Bakhtiar. Sars-cov-2 journey: from alpha variant to omicron and its sub-variants. Infection, 52(3):767–786, 2024

    2024

  18. [26]

    Beavertails: Towards improved safety alignment of llm via a human-preference dataset

    Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems , 36:24678–24704, 2023

    2023

  19. [27]

    Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome

    Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome. Bioinformatics, 37(15):2112–2120, 2021

    2021

  20. [28]

    Artprompt: Ascii art-based jailbreak attacks against aligned llms, 2024

    Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. Artprompt: Ascii art-based jailbreak attacks against aligned llms, 2024

    2024

  21. [29]

    Guard: Role-playing to generate natural-language jailbreakings to test guideline adherence of large language models, 2024

    Haibo Jin, Ruoxi Chen, Andy Zhou, Jinyin Chen, Yang Zhang, and Haohan Wang. Guard: Role-playing to generate natural-language jailbreakings to test guideline adherence of large language models, 2024

    2024

  22. [30]

    A watermark for large language models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pages 17061–17084. PMLR, 2023

    2023

  23. [31]

    Rapid and parallel adaptive mutations in spike s1 drive clade success in sars-cov-2

    Kathryn E Kistler, John Huddleston, and Trevor Bedford. Rapid and parallel adaptive mutations in spike s1 drive clade success in sars-cov-2. Cell Host & Microbe, 30(4):545–555, 2022

    2022

  24. [32]

    Hepatitis c virus infection

    Georg M Lauer and Bruce D Walker. Hepatitis c virus infection. New England journal of medicine, 345(1):41–52, 2001

    2001

  25. [33]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  26. [34]

    Codechameleon: Personalized encryption framework for jailbreaking large language models, 2024

    Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, Tao Gui, Qi Zhang, and Xuanjing Huang. Codechameleon: Personalized encryption framework for jailbreaking large language models, 2024. 11

    2024

  27. [35]

    The evolution of sars-cov-2

    Peter V Markov, Mahan Ghafari, Martin Beer, Katrina Lythgoe, Peter Simmonds, Nikolaos I Stilianakis, and Aris Katzourakis. The evolution of sars-cov-2. Nature Reviews Microbiology, 21(6):361–379, 2023

    2023

  28. [36]

    Harmbench: A standardized evaluation framework for automated red teaming and robust refusal

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint arXiv:2402.0...

    2024 arXiv

  29. [37]

    Semantic mining of functional de novo genes from a genomic language model

    Aditi T Merchant, Samuel H King, Eric Nguyen, and Brian L Hie. Semantic mining of functional de novo genes from a genomic language model. bioRxiv, pages 2024–12, 2024

    2024

  30. [38]

    Diverse variola virus (smallpox) strains were widespread in northern europe in the viking age

    Barbara Mühlemann, Ashot Margaryan, Peter de Barros Damgaard, Morten E Allentoft, Lasse Vinner, Anders J Hansen, André W Weber, Vladimir I Bazaliiskii, Martyna Molak, Jette Arneborg, et al. Diverse variola virus (smallpox) strains were widespread in northern europe in the viki...

    2020

  31. [39]

    Sequence modeling and design from molecular to genome scale with evo

    Eric Nguyen, Michael Poli, Matthew G Durrant, Brian Kang, Dhruva Katrekar, David B Li, Liam J Bartie, Armin W Thomas, Samuel H King, Garyk Brixi, et al. Sequence modeling and design from molecular to genome scale with evo. Science, 386(6723):eado9336, 2024

    2024

  32. [40]

    Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution

    Eric Nguyen, Michael Poli, Marjan Faizi, Armin Thomas, Michael Wornow, Callum Birch- Sykes, Stefano Massaroli, Aman Patel, Clayton Rabideau, Yoshua Bengio, et al. Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution. Advances in neural information pro...

    2023

  33. [41]

    Reovirus: a new approach to cancer therapy

    Kyle L Norman and Peter W Lee. Reovirus: a new approach to cancer therapy. Journal of Clinical Investigation, 113(7):828–830, 2004

    2004

  34. [42]

    Developing guardrails for ai biodesign tools

    Nuclear Threat Initiative. Developing guardrails for ai biodesign tools. Online report, November

  35. [43]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, a...

    2022

  36. [44]

    Noroviruses: a comprehensive review

    Manish M Patel, Aron J Hall, Jan Vinjé, and Umesh D Parashar. Noroviruses: a comprehensive review. Journal of Clinical Virology, 44(1):1–8, 2009

    2009

  37. [45]

    homology

    William R Pearson. An introduction to sequence similarity (“homology”) searching. Current protocols in bioinformatics, 42(1):3–1, 2013

    2013

  38. [46]

    Increased cyber- biosecurity for dna synthesis

    Rami Puzis, Dor Farbiash, Oleg Brodt, Yuval Elovici, and Dov Greenbaum. Increased cyber- biosecurity for dna synthesis. Nature Biotechnology, 38(12):1379–1381, 2020

    2020

  39. [47]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

    2023

  40. [48]

    The family herpesviridae: an update

    B Roizmann, RC Desrosiers, B Fleckenstein, C Lopez, AC Minson, and MJ Studdert. The family herpesviridae: an update. Archives of virology, 123:425–449, 1992

    1992

  41. [49]

    Molecular biology, pathogenesis and pathology of mumps virus

    Steven Rubin, Michael Eckhaus, Linda J Rennick, Connor GG Bamford, and W Paul Duprex. Molecular biology, pathogenesis and pathology of mumps virus. The Journal of pathology, 235(2):242–252, 2015

    2015

  42. [50]

    Molecular characterization of multidrug-resistant eskapee pathogens from clinical samples in chonburi, thailand (2017–2018)

    Sirigade Ruekit, Apichai Srijan, Oralak Serichantalergs, Katie R Margulieux, Patrick Mc Gann, Emma G Mills, William C Stribling, Theerasak Pimsawat, Rosarin Kormanee, Suthisak Nako- rnchai, et al. Molecular characterization of multidrug-resistant eskapee pathogens from clinica...

    2017

  43. [51]

    Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäschel, and Roberta Raileanu

    Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäschel, and Roberta Raileanu. Rainbow teaming: Open-ended generation of diverse adversarial prompts, 2024. 12

    2024

  44. [52]

    Dna language model grover learns sequence context in the human genome

    Melissa Sanabria, Jonas Hirsch, Pierre M Joubert, and Anna R Poetsch. Dna language model grover learns sequence context in the human genome. Nature Machine Intelligence, 6(8):911– 923, 2024

    2024

  45. [53]

    Ncbi taxonomy: a comprehensive update on curation, resources and tools

    Conrad L Schoch, Stacy Ciufo, Mikhail Domrachev, Carol L Hotton, Sivakumar Kannan, Rogneda Khovanskaya, Detlef Leipe, Richard Mcveigh, Kathleen O’Neill, Barbara Robbertse, et al. Ncbi taxonomy: a comprehensive update on curation, resources and tools. Database, 2020:baaa062, 2020

    2020

  46. [54]

    A long-context language model for deciphering and generating bacteriophage genomes

    Bin Shao and Jiawei Yan. A long-context language model for deciphering and generating bacteriophage genomes. Nature Communications, 15(1):9392, 2024

    2024

  47. [55]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models, 2023

    2023

  48. [56]

    Hiv-1 pathogenesis

    Mario Stevenson. Hiv-1 pathogenesis. Nature medicine, 9(7):853–860, 2003

    2003

  49. [57]

    Built-in safeguards might stop ai from designing bioweapons, April 2025

    Kristel Tjandra. Built-in safeguards might stop ai from designing bioweapons, April 2025. Accessed: 2025-05-05

    2025

  50. [58]

    The sequence of the human genome

    J Craig Venter, Mark D Adams, Eugene W Myers, Peter W Li, Richard J Mural, Granger G Sutton, Hamilton O Smith, Mark Yandell, Cheryl A Evans, Robert A Holt, et al. The sequence of the human genome. science, 291(5507):1304–1351, 2001

    2001

  51. [59]

    https://old.reddit.com/r/ChatGPT/comments/zlcyr9/dan_is_my_ new_friend/, 2022

    walkerspider. https://old.reddit.com/r/ChatGPT/comments/zlcyr9/dan_is_my_ new_friend/, 2022. Accessed: 2023-09-28

    2022

  52. [60]

    A call for built-in biosecurity safeguards for generative ai tools

    Mengdi Wang, Zaixi Zhang, Amrit Singh Bedi, Alvaro Velasquez, Stephanie Guerra, Sheng Lin-Gibson, Le Cong, Yuanhao Qu, Souradip Chakraborty, Megan Blewett, et al. A call for built-in biosecurity safeguards for generative ai tools. Nature Biotechnology, pages 1–3, 2025

    2025

  53. [61]

    Foot in the door: Understanding large language model jailbreaking via cognitive psychology, 2024

    Zhenhua Wang, Wei Xie, Baosheng Wang, Enze Wang, Zhiwen Gui, Shuoyoucheng Ma, and Kai Chen. Foot in the door: Understanding large language model jailbreaking via cognitive psychology, 2024

    2024

  54. [62]

    Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483, 2023

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483, 2023

    2023 arXiv

  55. [63]

    Jailbreak and guard aligned language models with only few in-context demonstrations, 2024

    Zeming Wei, Yifei Wang, and Yisen Wang. Jailbreak and guard aligned language models with only few in-context demonstrations, 2024

    2024

  56. [64]

    Genetics of poliovirus

    Eckard Wimmer, Christopher UT Hellen, and Xuemei Cao. Genetics of poliovirus. Annual review of genetics, 27:353–437, 1993

    1993

  57. [65]

    Sars-cov-2: a new coronavirus and its impact on human health

    Mark Woolhouse and Eleanor Gaunt. Sars-cov-2: a new coronavirus and its impact on human health. Nature Reviews Microbiology, 18(7):401–402, 2020

    2020

  58. [66]

    A new coronavirus associated with human respiratory disease in china

    Fan Wu, Su Zhao, Bin Yu, Yan-Mei Chen, Wen Wang, Zhi-Gang Song, Yi Hu, Zhao-Wu Tao, Jun-Hua Tian, Yuan-Yuan Pei, et al. A new coronavirus associated with human respiratory disease in china. Nature, 579(7798):265–269, 2020

    2020

  59. [68]

    Generator: A long-context generative genomic foundation model

    Wei Wu, Qiuyi Li, Mingyang Li, Kun Fu, Fuli Feng, Jieping Ye, Hui Xiong, and Zheng Wang. Generator: A long-context generative genomic foundation model. arXiv preprint arXiv:2502.07272, 2025

    2025

  60. [69]

    Cognitive overload: Jailbreaking large language models with overloaded logical thinking, 2024

    Nan Xu, Fei Wang, Ben Zhou, Bang Zheng Li, Chaowei Xiao, and Muhao Chen. Cognitive overload: Jailbreaking large language models with overloaded logical thinking, 2024

    2024

  61. [70]

    Blast: improvements for better sequence analysis

    Jian Ye, Scott McGinnis, and Thomas L Madden. Blast: improvements for better sequence analysis. Nucleic acids research, 34(suppl_2):W6–W9, 2006

    2006

  62. [71]

    Jailbreak attacks and defenses against large language models: A survey

    Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. Jailbreak attacks and defenses against large language models: A survey. arXiv preprint arXiv:2407.04295, 2024

    2024 arXiv

  63. [72]

    Zheng-Xin Yong, Cristina Menghini, and Stephen H. Bach. Low-resource languages jailbreak gpt-4, 2024. 13

    2024

  64. [73]

    Human parvovirus b19: an update on its biology, epidemiology, and clinical manifestations

    Neal S Young and Kathryn E Brown. Human parvovirus b19: an update on its biology, epidemiology, and clinical manifestations. The Journal of infectious diseases, 190(10):1466– 1473, 2004

    2004

  65. [74]

    Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher, 2024

    Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher, 2024

    2024

  66. [75]

    How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms, 2024

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms, 2024

    2024

  67. [76]

    Dnagpt: a generalized pre-trained tool for versatile dna sequence analysis tasks

    Daoan Zhang, Weitong Zhang, Yu Zhao, Jianguo Zhang, Bing He, Chenchen Qin, and Jianhua Yao. Dnagpt: a generalized pre-trained tool for versatile dna sequence analysis tasks. arXiv preprint arXiv:2307.05628, 2023

    2023 arXiv

  68. [77]

    {REMARK-LLM}: A robust and efficient watermarking framework for generative large lan- guage models

    Ruisi Zhang, Shehzeen Samarah Hussain, Paarth Neekhara, and Farinaz Koushanfar. {REMARK-LLM}: A robust and efficient watermarking framework for generative large lan- guage models. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1813–1830, 2024

    2024

  69. [78]

    Gonzalez, Eric P

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Hao Zhang, Joseph E. Gonzalez, Eric P. Xing, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685, 2023

    2023 arXiv

  70. [79]

    Autoredteamer: Autonomous red teaming with lifelong attack integration

    Andy Zhou, Kevin Wu, Francesco Pinto, Zhaorun Chen, Yi Zeng, Yu Yang, Shuang Yang, Sanmi Koyejo, James Zou, and Bo Li. Autoredteamer: Autonomous red teaming with lifelong attack integration. arXiv preprint arXiv:2503.15754, 2025

    2025 arXiv

  71. [80]

    How alignment and jailbreak work: Explain llm safety through intermediate hidden states

    Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. How alignment and jailbreak work: Explain llm safety through intermediate hidden states. arXiv preprint arXiv:2406.05644, 2024

    2024 arXiv

  72. [81]

    Dnabert- 2: Efficient foundation model and benchmark for multi-species genome

    Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, and Han Liu. Dnabert- 2: Efficient foundation model and benchmark for multi-species genome. arXiv preprint arXiv:2306.15006, 2023

    2023 arXiv

  73. [82]

    sequences

    Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. Autodan: Interpretable gradient-based adversarial attacks on large language models, 2023. 14 A More Information on JailbreakDNABench Table 2: Categorization of high-pr...

    2023

  74. [2024]

    Accessed: 2025-05-12

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.