REVIEW 4 major objections 5 minor 45 references
Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A knowledge graph of harmful and benign proteins can steer protein language models away from hazardous sequences, and the fine-tuning that does so preserves or improves functional fitness.
desk verdict A timely and reasonable safety-alignment paper whose headline numbers are weakened by a broad 'harmful' label and an incorrectly specified objective, but it deserves a serious referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machine is the Protein Safety Knowledge Graph (PSKG): a tripartite graph with harmful protein nodes, benign protein nodes, and Gene Ontology nodes, plus edges between proteins and GO terms and among GO terms. It supplies the training signal. A weighted metric scores each GO node by how many harmful-benign pairs it bridges and how many benign proteins it touches, then scores each benign protein by its connections to top-scoring GO nodes and its degree centrality; top-ranked nodes are kept. Preference pairs are formed by taking benign proteins within a hop threshold of a harmful protein and similar in embedding space, and the model is fine-tuned by Direct Preference Optimization to raise the probability of the benign sequence relative to the harmful one.
What would settle it
Take the same KPO pipeline but replace the keyword-annotated harmful-protein training set with an independently expert-curated list of validated toxins, retrain, and score generated sequences against a held-out toxin set using sequence-similarity, functional-domain, toxicity-prediction, and predicted structural-distance measures. If the safety gains disappear or generated proteins remain structurally close to the held-out toxins, the claim that KPO reduces genuine hazard rather than merely teaching avoidance of one annotation set would be falsified.
Extended reading notes
Core claim
The central claim is that safety can be treated as a preference-optimization problem in protein sequence space rather than as a post-hoc filter. KPO constructs a Protein Safety Knowledge Graph in which harmful proteins curated from keyword-annotated public sequence databases and benign proteins are connected through shared Gene Ontology terms, prunes that graph by weighted importance scores, and then extracts preference pairs: for each harmful protein, the benign proteins that are structurally and embedding-similar to it are treated as preferred outputs. Fine-tuning the language model with the Direct Preference Optimization objective on these pairs reduces the probability it assigns to harmful-like sequences while keeping model generations in biologically productive sequence regions. The authors report consistent safety gains across three base models and argue that steering away from harmful sequence space is what allows functional fitness to be retained or improved.
Load-bearing premise
The method rests on the curated label that a sequence is harmful: if the keyword-annotated toxin and antigen set mislabels benign proteins or misses real hazards, the preference pairs teach the model to avoid the wrong sequence space, and the reported safety gains measure avoidance of a possibly arbitrary category rather than genuine danger reduction.
Editorial extensions
If this is right
- Existing generative protein models can be made safer by fine-tuning alone, without redesigning their architectures.
- The knowledge-graph preference pairs, not random benign sampling, drive the safety improvement, as the ablations show KPO outperforming DPO with random sampling.
- Safety gains do not come at the cost of function: fitness scores on four standard mutation benchmarks are retained or improved across the three base models.
- Generated proteins after KPO are not only sequence-distant but also structurally distant from harmful proteins, as shown by higher predicted 3D RMSD values.
- The pruning strategy roughly halves the time needed to build preference data, so safety alignment remains feasible for large knowledge graphs.
Reading between the lines
- The same preference-pair machinery could be pointed at other undesirable properties, such as immunogenicity, allergenicity, or off-target enzyme activity, by swapping the harmful-protein annotation set in the graph.
- Because the method operates on sequence-level likelihoods, combining it with structure-aware scoring, such as predicted structural distance to known toxins, could close the gap the authors flag between sequence safety and conformational safety.
- A sharper test of the safety claim would be a prospective wet-lab assay: generate candidates with and without KPO, express them, and measure actual toxicity or channel-binding activity instead of relying only on similarity and classifier proxies.
- The withheld harmful-protein dataset means external replication must reconstruct the training set from public annotations; the framework's reproducibility will depend on how stable the safety gains are across different curations of what counts as harmful.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. KPO fine-tunes generative protein language models to reduce the probability of producing harmful sequences. A Protein Safety Knowledge Graph (PSKG) is built from UniProt/Swiss-Prot proteins labeled harmful or benign based on 'toxin' and 'antigen' keywords and GO annotations; a weighted-metric pruning algorithm selects informative benign nodes; preference pairs are constructed from benign proteins structurally/embedding-similar to harmful proteins; and the PLM is fine-tuned with a DPO-style loss. Experiments on ProtGPT2, ProGen2, and InstructProtein report lower BLAST, MMseqs2, Pfam, and ToxinPred3 scores after KPO, with largely retained or improved functional fitness on GB1, PhoQ, UBC9, and GFP.
Significance. If the effectiveness claim holds, KPO is a useful method for biosafety of protein LMs. The paper introduces a Protein Safety Knowledge Graph, integrates GO priors via a pruning strategy, compares against DPO/random/community ablations, provides code, and responsibly withholds the harmful dataset. The central claim is potentially important, but currently rests on label definitions shared between training and evaluation, an incomplete statement of the DPO objective, and point estimates without significance testing.
major comments (4)
- [§3.2, §4.3, Eq. (9)] Eq. (9) is written as L_KPO = -log σ(φ·[log P_θ(p_j^B|x) - log P_θ(p_i^H|x)]), which omits the reference-model terms π_ref(y_w|x) and π_ref(y_l|x) that define DPO in §3.2, as well as the λ factor. As written it is a margin contrast on the policy alone, not DPO, so the comparison against 'DPO' in the ablation of Fig. 4 is not an apples-to-apples comparison. Moreover, the conditioning variable x is never defined: since p_j^B and p_i^H are entire protein sequences, the reader cannot tell whether x is a text prompt, an N-terminal fragment, or empty. Please correct the objective and specify x precisely.
- [§5.1, Appendix A.1, Appendix B] The harmful-protein test set used for all sequence-similarity and domain metrics is an 8:2 split of the same UniProt keyword query ('toxin' and 'antigen') used to build PH for training (Appendix A.1 vs. Appendix B). Because 'antigen' is a broad functional category that includes many non-toxic immunogenic proteins, the preference pairs and the evaluation metrics encode the same labeling bias, so the reported reductions in BLAST/MMseqs2/Pfam similarity could reflect avoidance of a mislabeled sequence space rather than genuine hazard reduction. The paper's Limitations section concedes that structural-level safety is not directly addressed, leaving sequence-level similarity against this same-source set as the only safety evidence. Please evaluate on an independently curated, expert-validated harmful-protein benchmark and, at minimum, report results after separating 'toxin' and 'antigen' labels.
- [§5.2, Table 1] The claim that KPO preserves or even improves functional capabilities is not uniformly supported by Table 1. For ProGen2, three of four functional metrics decline after KPO (GB1 0.144→0.024, PhoQ 0.027→0.017, GFP 1.683→1.562), with only UBC9 improving. All functional values are point estimates with no standard deviation or significance testing across generative runs. Please report repeated-run statistics and either soften the functional-preservation claim or provide a per-model discussion of the trade-off.
- [§4.2-4.3, Appendix A.3] Several parameters that determine the preference data and training objective are not reported: the hop threshold τ, the number of selected benign proteins M, the weighting μ in Eq. (8), the scaling factor φ in Eq. (9), the TransE embedding dimension d, and the margin η in Eq. (7). Only α, β, γ, δ, Q, and K are specified in Appendix A.3. This prevents reproduction of the method from the manuscript; please provide a full hyperparameter table, even if defaults also appear in the public code.
minor comments (5)
- [Appendix B] ProGen2 is described as using a masked language modeling objective and 'bidirectional contextual representations,' but ProGen2 is an autoregressive language model; this mischaracterization should be corrected.
- [Tables 3 and 4] The column header 'γ:σ' appears to be a typo; the text describes γ:δ for Table 3, and Table 4 shows Q,K values, so the headers should read 'γ:δ' and 'Q,K' respectively.
- [Figure 3] The claim of 'significant divergence' between generated and harmful-protein embeddings is made without a quantitative separation measure; reporting a centroid distance, silhouette score, or similar statistic would strengthen the visualization.
- [Appendix C] The statement that the harmful-protein dataset will be withheld or released under strict guidelines is responsible, but at least the evaluation test set (or hashed versions) should be made accessible to reviewers and readers to enable verification.
- [Eqs. (5)-(6)] The formulas CGO and CDeg are raw counts with no normalization, so the importance score S in Eq. (1) may be dominated by node degree; a brief discussion of normalization would clarify the pruning behavior.
Circularity Check
No significant circularity: KPO's safety gains are evaluated against held-out and external benchmarks rather than by construction.
full rationale
The KPO derivation chain is self-contained and does not reduce to its own inputs by construction. The harmful protein set is curated from UniProt keywords "toxin" and "antigen" (Appendix A.1), and the same keyword-defined database is split into an 8:2 training/testing ratio (Appendix B), with the held-out twenty percent used for BLAST/MMseqs2/Pfam safety evaluation. The DPO objective in Eq. (9) is trained only on the training split, so the safety metrics are not directly optimized on the evaluation set. Independent evidence is also provided: ToxinPred3 is an externally trained toxicity classifier, structural RMSD comparisons use four manually described toxins not drawn from the preference training pairs, and functional preservation is tested on external GB1, PhoQ, UBC9, and GFP datasets. The sole self-citation, "Inspired by Ontoprotein" (Section 4.1), is motivational and not load-bearing; no uniqueness claim or ansatz is imported from it. While the broad "antigen" keyword may introduce label noise, that is a validity concern, not circularity, and the paper does not claim its safety metrics are predictions from the preference objective alone. Therefore the central derivation chain is not circular.
Assumptions & free parameters
free parameters (9)
- alpha =
0.5
- beta =
0.5
- gamma =
1.0
- delta =
0.5
- Q =
50%
- K =
50%
- tau =
not specified
- M =
not specified
- phi =
not specified
assumptions (3)
- domain assumption Proteins annotated with UniProt keywords 'toxin' or 'antigen' are a valid proxy for the class of harmful proteins.
- domain assumption Sequence and embedding similarity to known harmful proteins is a valid proxy for biological harmfulness of generated proteins.
- domain assumption The GO annotation graph adequately captures functional relationships that distinguish harmful from benign proteins.
Cite this review
Pith. "Pith review of Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization." pith.science (2026). https://pith.science/paper/NAK72Q2Y
@misc{pith2026250710923,
author = {Pith},
title = {Pith review of: Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/NAK72Q2Y}},
note = {Machine review of arXiv:2507.10923}
}
read the original abstract
Protein language models have emerged as powerful tools for sequence generation, offering substantial advantages in functional optimization and denovo design. However, these models also present significant risks of generating harmful protein sequences, such as those that enhance viral transmissibility or evade immune responses. These concerns underscore critical biosafety and ethical challenges. To address these issues, we propose a Knowledge-guided Preference Optimization (KPO) framework that integrates prior knowledge via a Protein Safety Knowledge Graph. This framework utilizes an efficient graph pruning strategy to identify preferred sequences and employs reinforcement learning to minimize the risk of generating harmful proteins. Experimental results demonstrate that KPO effectively reduces the likelihood of producing hazardous sequences while maintaining high functionality, offering a robust safety assurance framework for applying generative models in biotechnology.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862
arXiv 2022
-
[2]
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pages 141--159. IEEE
work page 2021
-
[3]
Huajun Chen. 2023. Large knowledge model: Perspectives and challenges. arXiv preprint arXiv:2312.02706
work page Pith review arXiv 2023
-
[4]
Sean R Eddy. 1996. Hidden markov models. Current opinion in structural biology, 6(3):361--365
work page 1996
-
[5]
Noelia Ferruz, Steffen Schmidt, and Birte H \"o cker. 2022. Protgpt2 is a deep unsupervised language model for protein design. Nature communications, 13(1):4348
work page 2022
-
[6]
Robert D Finn, Alex Bateman, Jody Clements, Penelope Coggill, Ruth Y Eberhardt, Sean R Eddy, Andreas Heger, Kirstie Hetherington, Liisa Holm, Jaina Mistry, et al. 2014. Pfam: the protein families database. Nucleic acids research, 42(D1):D222--D230
work page 2014
-
[7]
HaoHuai He, Bing He, Lei Guan, Yu Zhao, Guanxing Chen, Qingge Zhu, Calvin Yu-Chian Chen, Ting Li, and Jianhua Yao. 2023. De novo generation of antibody cdrh3 with a pre-trained generative large language model. bioRxiv, pages 2023--10
work page 2023
-
[8]
Yoichi Ishibashi and Hidetoshi Shimodaira. 2023. Knowledge sanitization of large language models. arXiv preprint arXiv:2309.11852
arXiv 2023
Show all 45 references
-
[9]
Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2022. Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504
2022 arXiv
-
[10]
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Z \' dek, Anna Potapenko, et al. 2021. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583--589
2021
-
[11]
Aly Kassem, Omar Mahmoud, and Sherif Saad. 2023. Preserving privacy through dememorization: An unlearning technique for mitigating memorization risks in language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4360--4379
2023
-
[12]
Puck Knipscheer, Annette Flotho, Helene Klug, Jesper V Olsen, Willem J van Dijk, Alexander Fish, Erica S Johnson, Matthias Mann, Titia K Sixma, and Andrea Pichler. 2008. Ubc9 sumoylation regulates sumo target discrimination. Molecular cell, 31(3):371--382
2008
-
[13]
Mingchen Li, Bingxin Zhou, Yang Tan, and Liang Hong. 2024. Unlearning virus knowledge toward safe and responsible mutation effect predictions. bioRxiv, pages 2024--10
2024
-
[14]
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. 2022. Language models of protein sequences at the scale of evolution enable accurate structure prediction. BioRxiv, 2022:500902
2022
-
[15]
Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022. Quark: Controllable text generation with reinforced unlearning. Advances in neural information processing systems, 35:27591--27609
2022
-
[16]
Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. 2023. Large language models generate functional protein sequences across diverse families. Nature Biotechnology, 41(8...
2023
-
[17]
Thomas Madden. 2013. The blast sequence analysis tool. The NCBI handbook, 2(5):425--436
2013
-
[18]
Milot Mirdita, Konstantin Sch \"u tze, Yoshitaka Moriwaki, Lim Heo, Sergey Ovchinnikov, and Martin Steinegger. 2022. Colabfold: making protein folding accessible to all. Nature methods, 19(6):679--682
2022
-
[19]
Jaina Mistry, Sara Chuguransky, Lowri Williams, Matloob Qureshi, Gustavo A Salazar, Erik LL Sonnhammer, Silvio CE Tosatto, Lisanna Paladin, Shriya Raj, Lorna J Richardson, et al. 2021. Pfam: The protein families database in 2021. Nucleic acids research, 49(D1):D412--D419
2021
-
[20]
Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani. 2023. Progen2: exploring the boundaries of protein language models. Cell systems, 14(11):968--978
2023
-
[21]
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...
2022
-
[22]
Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. 2023. In-context unlearning: Language models as few shot unlearners. arXiv preprint arXiv:2310.07579
2023 arXiv
-
[23]
Anna I Podgornaia and Michael T Laub. 2015. Pervasive degeneracy and epistasis in a protein-protein interface. Science, 347(6222):673--677
2015
-
[24]
Suresh Pokharel, Pawel Pratyush, Michael Heinzinger, Robert H Newman, and Dukka B Kc. 2022. Improving protein succinylation sites prediction using embeddings from protein language model. Scientific reports, 12(1):16933
2022
-
[25]
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36
2024
-
[26]
Roshan M Rao, Jason Liu, Robert Verkuil, Joshua Meier, John Canny, Pieter Abbeel, Tom Sercu, and Alexander Rives. 2021. Msa transformer. In International Conference on Machine Learning, pages 8844--8856. PMLR
2021
-
[27]
Anand Singh Rathore, Shubham Choudhury, Akanksha Arora, Purva Tijare, and Gajendra PS Raghava. 2024. Toxinpred 3.0: An improved method for predicting the toxicity of peptides. Computers in Biology and Medicine, 179:108926
2024
-
[28]
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. 2021. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the Nat...
2021
-
[29]
Martin Steinegger and Johannes S \"o ding. 2017. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology, 35(11):1026--1028
2017
-
[30]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11)
2008
-
[31]
Robert Verkuil, Ori Kabeli, Yilun Du, Basile IM Wicky, Lukas F Milles, Justas Dauparas, David Baker, Sergey Ovchinnikov, Tom Sercu, and Alexander Rives. 2022. Language models generalize beyond natural proteins. bioRxiv, pages 2022--12
2022
-
[32]
Jesse Vig, Ali Madani, Lav R Varshney, Caiming Xiong, Richard Socher, and Nazneen Fatema Rajani. 2020. Bertology meets biology: Interpreting attention in protein language models. arXiv preprint arXiv:2006.15222
2020 arXiv
-
[33]
Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. 2020. Trl: Transformer reinforcement learning. https://github.com/huggingface/trl
2020
-
[34]
Meng Wang, Jonathan Patsenker, Henry Li, Yuval Kluger, and Steven H Kleinstein. 2024. Supervised fine-tuning of pre-trained antibody language models improves antigen specificity prediction. bioRxiv
2024
-
[35]
Zeyuan Wang, Qiang Zhang, Keyan Ding, Ming Qin, Xiang Zhuang, Xiaotong Li, and Huajun Chen. 2023. Instructprotein: Aligning human and protein language via knowledge instruction. arXiv preprint arXiv:2310.03269
2023 arXiv
-
[36]
Zichen Wang, Steven A Combs, Ryan Brand, Miguel Romero Calvo, Panpan Xu, George Price, Nataliya Golovach, Emmanuel O Salawu, Colby J Wise, Sri Priya Ponnapalli, et al. 2022. Lm-gvp: an extensible sequence and structure informed deep learning framework for protein property pred...
2022
-
[37]
Zachary Wu, SB Jennifer Kan, Russell D Lewis, Bruce J Wittmann, and Frances H Arnold. 2019. Machine learning-assisted directed protein evolution with combinatorial libraries. Proceedings of the National Academy of Sciences, 116(18):8852--8858
2019
-
[38]
Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, and Yuandong Tian. 2023. Rlcd: Reinforcement learning from contrast distillation for language model alignment. arXiv preprint arXiv:2307.12950
2023 arXiv
-
[39]
Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. 2023. Rrhf: Rank responses to align language models with human feedback without tears. arXiv preprint arXiv:2304.05302
2023 arXiv
-
[40]
He Zhang, Fusong Ju, Jianwei Zhu, Liang He, Bin Shao, Nanning Zheng, and Tie-Yan Liu. 2021. Co-evolution transformer for protein contact prediction. Advances in Neural Information Processing Systems, 34:14252--14263
2021
-
[41]
Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Jiazhang Lian, Qiang Zhang, and Huajun Chen. 2022. Ontoprotein: Protein pretraining with gene ontology embedding. arXiv preprint arXiv:2201.11147
2022 arXiv
-
[42]
Bingxin Zhou, Lirong Zheng, Banghao Wu, Kai Yi, Bozitao Zhong, Yang Tan, Qian Liu, Pietro Li \`o , and Liang Hong. 2024. A conditional protein diffusion model generates artificial programmable endonuclease sequences with enhanced activity. Cell Discovery, 10(1):95
2024
-
[43]
Marc Zimmer. 2002. Green fluorescent protein (gfp): applications, structure, and related photophysical behavior. Chemical reviews, 102(3):759--782
2002
-
[44]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[45]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.