{"id":"69edf17f-6cae-4b43-88be-e45b3b46c6fb","arxiv_id":"2507.10923","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A knowledge-graph-guided preference optimization framework that fine-tunes protein language models to generate fewer sequences similar to known harmful proteins.","lead":"This paper introduces KPO, a method that fine-tunes protein language models to reduce their chance of generating harmful or dangerous protein sequences. It builds a safety knowledge graph from protein databases and uses preference-based training to steer models toward safer outputs while aiming to preserve their useful functions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Safety gains may largely reflect UniProt 'antigen' keyword over-labeling, since the same noisy labels define both the KPO training signal and the evaluation test set.","rationale":"I agree with the reader's weakest-assumption analysis: the harmful-protein curation is the load-bearing juncture. The reason it is more central than other issues (e.g., Eq. 9 not being the standard DPO loss, or mixed functional results in Table 1) is that it contaminates both sides of the comparison. The PSKG is built from the keyword query, the preference pairs are built from that PSKG, and the safety evaluation is run against a held-out slice of the same keyword query. Any systematic over-labeling of 'antigen' as harmful will therefore appear as a safety improvement even if the model's true hazard profile is unchanged. This is a correctness risk, not merely an external-consensus disagreement: it is internally testable by re-labeling and re-evaluating. The paper has real merits: the graph-pruning idea is concrete, ablations isolate components, and code is promised. But the central claim is currently conditional on the validity of the 'harmful' label set, and that condition is unverified because the data are withheld and no independent toxicity benchmark is used. The proposed test would settle whether the concern lands; if it does, the headline should be softened to 'reduces similarity to a keyword-defined harmful set' rather than 'reduces hazardous sequences'.","tokens_in":16542,"tokens_out":6508,"duration_ms":83404,"concrete_test":"Construct a clean held-out safety benchmark from expert-curated toxin resources (e.g., Tox-Prot, Animal Toxin Database, or UniProt entries whose 'toxin' keyword is manually verified), excluding all antigen-only entries. Re-run every Table 1 safety metric for ProtGPT2, ProGen2, and InstructProtein with and without KPO against this clean set, using the same generation procedure. In parallel, randomly sample 200 UniProt 'antigen' entries and have two independent annotators classify them as hazardous vs. non-hazardous using authoritative references; report the mislabel rate. If KPO's reductions persist on the clean toxin-only set, the label-noise concern is resolved. If gains shrink or vanish, the central safety claim is an artifact of over-inclusive 'antigen' labeling.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that KPO reduces generation of hazardous sequences while preserving function (Abstract; Section 5.2). This rests on the definition of 'harmful proteins' in Appendix A.1: UniProt sequences annotated with keywords 'toxin' (~10k) and 'antigen' (~8k). 'Antigen' is far broader than 'hazardous': many antigens are non-toxic, immunogenic, or vaccine-relevant proteins. If these non-harmful antigens populate PH, then every preference pair (pjB, piH) in Section 4.3 conditions the model to downweight antigen-like sequences, and every safety metric in Section 5.1 (BLAST/MMseqs2/Pfam against the harmful test set, ToxinPred3) rewards that same downweighting. Appendix B splits the same keyword-query dataset 8:2 into training and test, and Appendix C withholds the data; hence training and evaluation share the same labeling bias. Table 1 improvements (e.g., ProtGPT2 BLAST 0.269 to 0.138) could therefore reflect avoidance of a mislabeled sequence space rather than real hazard reduction. The paper's own Limitations section concedes structural-level harm is not addressed, so the sequence-level metric is the only safety evidence. Without an independent, expert-curated toxin benchmark, the safety claim is not yet distinguished from label artifact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"KPO fine-tunes generative protein language models to reduce the probability of producing harmful sequences. A Protein Safety Knowledge Graph (PSKG) is built from UniProt/Swiss-Prot proteins labeled harmful or benign based on 'toxin' and 'antigen' keywords and GO annotations; a weighted-metric pruning algorithm selects informative benign nodes; preference pairs are constructed from benign proteins structurally/embedding-similar to harmful proteins; and the PLM is fine-tuned with a DPO-style loss. Experiments on ProtGPT2, ProGen2, and InstructProtein report lower BLAST, MMseqs2, Pfam, and ToxinPred3 scores after KPO, with largely retained or improved functional fitness on GB1, PhoQ, UBC9, and GFP.","tokens_in":16814,"tokens_out":6395,"duration_ms":68687,"significance":"If the effectiveness claim holds, KPO is a useful method for biosafety of protein LMs. The paper introduces a Protein Safety Knowledge Graph, integrates GO priors via a pruning strategy, compares against DPO/random/community ablations, provides code, and responsibly withholds the harmful dataset. The central claim is potentially important, but currently rests on label definitions shared between training and evaluation, an incomplete statement of the DPO objective, and point estimates without significance testing.","major_comments":[{"comment":"Eq. (9) is written as L_KPO = -log σ(φ·[log P_θ(p_j^B|x) - log P_θ(p_i^H|x)]), which omits the reference-model terms π_ref(y_w|x) and π_ref(y_l|x) that define DPO in §3.2, as well as the λ factor. As written it is a margin contrast on the policy alone, not DPO, so the comparison against 'DPO' in the ablation of Fig. 4 is not an apples-to-apples comparison. Moreover, the conditioning variable x is never defined: since p_j^B and p_i^H are entire protein sequences, the reader cannot tell whether x is a text prompt, an N-terminal fragment, or empty. Please correct the objective and specify x precisely.","section":"§3.2, §4.3, Eq. (9)"},{"comment":"The harmful-protein test set used for all sequence-similarity and domain metrics is an 8:2 split of the same UniProt keyword query ('toxin' and 'antigen') used to build PH for training (Appendix A.1 vs. Appendix B). Because 'antigen' is a broad functional category that includes many non-toxic immunogenic proteins, the preference pairs and the evaluation metrics encode the same labeling bias, so the reported reductions in BLAST/MMseqs2/Pfam similarity could reflect avoidance of a mislabeled sequence space rather than genuine hazard reduction. The paper's Limitations section concedes that structural-level safety is not directly addressed, leaving sequence-level similarity against this same-source set as the only safety evidence. Please evaluate on an independently curated, expert-validated harmful-protein benchmark and, at minimum, report results after separating 'toxin' and 'antigen' labels.","section":"§5.1, Appendix A.1, Appendix B"},{"comment":"The claim that KPO preserves or even improves functional capabilities is not uniformly supported by Table 1. For ProGen2, three of four functional metrics decline after KPO (GB1 0.144→0.024, PhoQ 0.027→0.017, GFP 1.683→1.562), with only UBC9 improving. All functional values are point estimates with no standard deviation or significance testing across generative runs. Please report repeated-run statistics and either soften the functional-preservation claim or provide a per-model discussion of the trade-off.","section":"§5.2, Table 1"},{"comment":"Several parameters that determine the preference data and training objective are not reported: the hop threshold τ, the number of selected benign proteins M, the weighting μ in Eq. (8), the scaling factor φ in Eq. (9), the TransE embedding dimension d, and the margin η in Eq. (7). Only α, β, γ, δ, Q, and K are specified in Appendix A.3. This prevents reproduction of the method from the manuscript; please provide a full hyperparameter table, even if defaults also appear in the public code.","section":"§4.2-4.3, Appendix A.3"}],"minor_comments":[{"comment":"ProGen2 is described as using a masked language modeling objective and 'bidirectional contextual representations,' but ProGen2 is an autoregressive language model; this mischaracterization should be corrected.","section":"Appendix B"},{"comment":"The column header 'γ:σ' appears to be a typo; the text describes γ:δ for Table 3, and Table 4 shows Q,K values, so the headers should read 'γ:δ' and 'Q,K' respectively.","section":"Tables 3 and 4"},{"comment":"The claim of 'significant divergence' between generated and harmful-protein embeddings is made without a quantitative separation measure; reporting a centroid distance, silhouette score, or similar statistic would strengthen the visualization.","section":"Figure 3"},{"comment":"The statement that the harmful-protein dataset will be withheld or released under strict guidelines is responsible, but at least the evaluation test set (or hashed versions) should be made accessible to reviewers and readers to enable verification.","section":"Appendix C"},{"comment":"The formulas CGO and CDeg are raw counts with no normalization, so the importance score S in Eq. (1) may be dominated by node degree; a brief discussion of normalization would clarify the pruning behavior.","section":"Eqs. (5)-(6)"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and important biosafety problem, and the consistent improvements across three base models are encouraging. The main risk is label circularity between training and evaluation, which I believe the authors can address with an independent benchmark. I also noted the ProGen2 description error, which suggests the baseline handling should be checked more carefully. I support major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper: it is the first serious attempt I've seen to align generative protein LMs with external safety knowledge, and the main safety numbers in Table 1 are weaker than they look because 'harmful' is defined by UniProt 'antigen' and 'toxin' keywords, and the same labels build both the training preferences and the test set.\n\nThe genuinely new part is KPO: build a Protein Safety Knowledge Graph from protein-GO associations, prune it with a weighted score, then use preference pairs of benign vs harmful proteins to fine-tune a PLM with what they call DPO. That combination is new. The consistent drop in BLAST/MMseqs2 similarity across ProtGPT2, ProGen2, and InstructProtein is real and worth taking seriously. The ablation against random pruning and community pruning supports the graph's value, and the sensitivity analysis over the pruning hyperparameters is more thorough than most papers in this space. The structural RMSD appendix on four toxins is a nice touch.\n\nThe soft spots are real but not all equal. The big one is the label. Including UniProt 'antigen' (about 8k sequences) alongside 'toxin' (10k) makes the harmful set much broader than 'hazardous.' Many antigens are vaccine components or innocuous proteins. If the model is trained to downweight them and then evaluated against the same label, the safety gain could partly be avoidance of a mislabeled region of sequence space. The ToxinPred3 reduction and the four toxin RMSD results give some independent support, but ToxinPred3 is peptide-focused and the structure study is small. I would want an external, expert-curated toxic protein benchmark before believing the safety claim at face value.\n\nSecond, Eq. (9) is not DPO. It drops the reference model term, and the conditioning variable x is never defined. This looks like a simple contrastive loss, not the DPO described in Section 3.2. That is fixable, but right now the method is mislabeled. Third, 'maintaining high functionality' overstates Table 1: ProGen2's GB1 drops from 0.144 to 0.024, and its GFP drops. Several metrics improve, but the picture is mixed, and there are no error bars or significance tests. Finally, withholding the harmful-protein dataset for dual-use reasons is defensible, but it does limit reproducibility.\n\nWho is this for? Anyone working on safe protein design or PLM alignment. The idea is timely and the execution is decent. I would send it to peer review, with the label question made a central requirement. A serious referee could push the authors to either narrow the harmful definition to confirmed toxins or add an independent evaluation set, fix the objective, and report per-seed variance.","headline":"A timely and reasonable safety-alignment paper whose headline numbers are weakened by a broad 'harmful' label and an incorrectly specified objective, but it deserves a serious referee.","tokens_in":17349,"tokens_out":3289,"would_cite":false,"duration_ms":36610,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A knowledge graph of harmful and benign proteins can steer protein language models away from hazardous sequences, and the fine-tuning that does so preserves or improves functional fitness.","keywords":["protein language models","biosafety","preference optimization","knowledge graph","Direct Preference Optimization","protein generation","harmful sequence avoidance","Gene Ontology"],"falsifier":"Take the same KPO pipeline but replace the keyword-annotated harmful-protein training set with an independently expert-curated list of validated toxins, retrain, and score generated sequences against a held-out toxin set using sequence-similarity, functional-domain, toxicity-prediction, and predicted structural-distance measures. If the safety gains disappear or generated proteins remain structurally close to the held-out toxins, the claim that KPO reduces genuine hazard rather than merely teaching avoidance of one annotation set would be falsified.","tokens_in":16357,"feed_emoji":"🧬","tokens_out":8653,"duration_ms":86380,"temperature":0.7,"pith_summary":"The paper proposes KPO, a fine-tuning method that teaches protein language models to avoid generating harmful sequences by learning preferences built from a Protein Safety Knowledge Graph (PSKG). The graph links harmful and benign proteins through Gene Ontology terms; a weighted pruning step keeps the most informative benign proteins, and preference pairs of benign over harmful proteins are used with Direct Preference Optimization. On three base models, the authors report that KPO lowers sequence-similarity, functional-domain, and toxicity-prediction hazard scores while preserving or improving mutation fitness on four standard benchmarks. The paper itself notes that the constraints are sequence-level only, that structural safety is not directly enforced, and that the harmful-protein dataset is withheld to limit misuse.","feed_headline":"Knowledge-guided tuning steers protein AI away from toxins","feed_subtitle":"A protein-safety knowledge graph guides preference optimization, cutting toxin-like outputs without hurting mutation fitness.","key_machinery":"The machine is the Protein Safety Knowledge Graph (PSKG): a tripartite graph with harmful protein nodes, benign protein nodes, and Gene Ontology nodes, plus edges between proteins and GO terms and among GO terms. It supplies the training signal. A weighted metric scores each GO node by how many harmful-benign pairs it bridges and how many benign proteins it touches, then scores each benign protein by its connections to top-scoring GO nodes and its degree centrality; top-ranked nodes are kept. Preference pairs are formed by taking benign proteins within a hop threshold of a harmful protein and similar in embedding space, and the model is fine-tuned by Direct Preference Optimization to raise the probability of the benign sequence relative to the harmful one.","core_discovery":"The central claim is that safety can be treated as a preference-optimization problem in protein sequence space rather than as a post-hoc filter. KPO constructs a Protein Safety Knowledge Graph in which harmful proteins curated from keyword-annotated public sequence databases and benign proteins are connected through shared Gene Ontology terms, prunes that graph by weighted importance scores, and then extracts preference pairs: for each harmful protein, the benign proteins that are structurally and embedding-similar to it are treated as preferred outputs. Fine-tuning the language model with the Direct Preference Optimization objective on these pairs reduces the probability it assigns to harmful-like sequences while keeping model generations in biologically productive sequence regions. The authors report consistent safety gains across three base models and argue that steering away from harmful sequence space is what allows functional fitness to be retained or improved.","pith_inferences":["The same preference-pair machinery could be pointed at other undesirable properties, such as immunogenicity, allergenicity, or off-target enzyme activity, by swapping the harmful-protein annotation set in the graph.","Because the method operates on sequence-level likelihoods, combining it with structure-aware scoring, such as predicted structural distance to known toxins, could close the gap the authors flag between sequence safety and conformational safety.","A sharper test of the safety claim would be a prospective wet-lab assay: generate candidates with and without KPO, express them, and measure actual toxicity or channel-binding activity instead of relying only on similarity and classifier proxies.","The withheld harmful-protein dataset means external replication must reconstruct the training set from public annotations; the framework's reproducibility will depend on how stable the safety gains are across different curations of what counts as harmful."],"forward_implications":["Existing generative protein models can be made safer by fine-tuning alone, without redesigning their architectures.","The knowledge-graph preference pairs, not random benign sampling, drive the safety improvement, as the ablations show KPO outperforming DPO with random sampling.","Safety gains do not come at the cost of function: fitness scores on four standard mutation benchmarks are retained or improved across the three base models.","Generated proteins after KPO are not only sequence-distant but also structurally distant from harmful proteins, as shown by higher predicted 3D RMSD values.","The pruning strategy roughly halves the time needed to build preference data, so safety alignment remains feasible for large knowledge graphs."],"supporting_citations":[{"why":"Supplies the Direct Preference Optimization objective that KPO uses for fine-tuning.","marker":"[Rafailov et al., 2024]"},{"why":"Provides ProGen2, one of the three base generative protein language models KPO is applied to.","marker":"[Nijkamp et al., 2023]"},{"why":"Provides ProtGPT2, a base model used for KPO fine-tuning and ablation experiments.","marker":"[Ferruz et al., 2022]"},{"why":"Provides InstructProtein, the instruction-tuned base model on which KPO shows large safety gains.","marker":"[Wang et al., 2023]"},{"why":"The Ontoprotein-style graph construction that inspires the Protein Safety Knowledge Graph.","marker":"[Zhang et al., 2022]"},{"why":"ToxinPred3 is the toxicity classifier used as a safety evaluation metric.","marker":"[Rathore et al., 2024]"},{"why":"Pfam domain models are used to evaluate functional-domain similarity to harmful proteins.","marker":"[Finn et al., 2014]"},{"why":"BLAST is used to measure sequence similarity of generated proteins to harmful test proteins.","marker":"[Madden, 2013]"},{"why":"MMseqs2 is used as a second sequence-similarity safety metric.","marker":"[Steinegger and Söding, 2017]"},{"why":"The embedding model used to analyze separation of generated sequences from harmful proteins.","marker":"[Verkuil et al., 2022]"}],"fun_headline_variants":["Knowledge graph steers protein AI from toxins","Safer protein generation via preference tuning","Protein models trained to avoid harmful sequences","AI protein design guided by safety knowledge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method rests on the curated label that a sequence is harmful: if the keyword-annotated toxin and antigen set mislabels benign proteins or misses real hazards, the preference pairs teach the model to avoid the wrong sequence space, and the reported safety gains measure avoidance of a possibly arbitrary category rather than genuine danger reduction.","fun_headline_variants_meta":{"raw":{"variants":["Knowledge graph steers protein AI from toxins","Safer protein generation via preference tuning","Protein models trained to avoid harmful sequences","AI protein design guided by safety knowledge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000277,"raw_usage":{"total_tokens":1585,"prompt_tokens":813,"completion_tokens":772,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":429,"completion_tokens_details":{"reasoning_tokens":720}},"tokens_in":429,"tokens_out":772,"duration_ms":8913,"temperature":1.0,"reasoning_tokens":720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:20:58.132861+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same KPO pipeline but replace the keyword-annotated harmful-protein training set with an independently expert-curated list of validated toxins, retrain, and score generated sequences against a held-out toxin set using sequence-similarity, functional-domain, toxicity-prediction, and predicted structural-distance measures. If the safety gains disappear or generated proteins remain structurally close to the held-out toxins, the claim that KPO reduces genuine hazard rather than merely teaching avoidance of one annotation set would be falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides ProGen2, one of the three base generative protein language models KPO is applied to."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides ProtGPT2, a base model used for KPO fine-tuning and ablation experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ToxinPred3 is the toxicity classifier used as a safety evaluation metric."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Pfam domain models are used to evaluate functional-domain similarity to harmful proteins."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"BLAST is used to measure sequence similarity of generated proteins to harmful test proteins."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MMseqs2 is used as a second sequence-similarity safety metric."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The embedding model used to analyze separation of generated sequences from harmful proteins."}],"review_version":1}