Pith. sign in

REVIEW 3 major objections 4 minor 54 references

Alignment loss lifts protein scaffold score by 20 percent

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 05:00 UTC pith:N7OBDXRD

load-bearing objection Useful protein-domain REPA transfer with a plausible but numerically under-supported headline gain; deserves peer review with a baseline re-run. the 3 major comments →

arxiv 2607.13503 v1 pith:N7OBDXRD submitted 2026-07-15 cs.CE

Exploring the Alignment of Generation and Understanding in Protein Structure Modeling

Junde Xu , Yuansheng Huang , Zijun Gao , Lihang Liu , Xiaoming Fang , Yu Kang , Jiezhong Qiu , Pheng Ann Heng This is my paper
classification cs.CE
keywords protein generationrepresentation alignmentdiffusion modelsmotif scaffoldingprotein understandingstructure encodersCATH distributionfunctional protein design
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Protein generative models trained purely with denoising objectives learn internal representations that transfer poorly to functional classification, the paper shows by benchmarking diffusion models and tokenizers against dedicated sequence and structure encoders. The paper then demonstrates a fix: add a representation-alignment loss that pulls an intermediate layer of a protein diffusion model toward the frozen embeddings of a pretrained structure encoder. In the resulting model, ReaPro-1c, this single addition raises the MotifBench motif-scaffolding score from 39.2 to 47.1, improves coverage of the natural CATH structure distribution, and reaches baseline-level generation quality in roughly half the training steps. The broader claim is that understanding and generation in protein modeling are not separate regimes: semantic guidance can directly improve generative capacity.

Core claim

The central claim is that aligning a diffusion model's internal hidden states with a frozen pretrained structure encoder during training substantially improves functional protein generation. Concretely, at layer 5 of the U-ViT backbone, the hidden state is projected through a three-layer MLP with residual connection, and the squared L2 distance between that projection and the ProteinMPNN embedding of the clean target structure is added to the EDM denoising loss. On MotifBench, the alignment-equipped model scores 47.1 versus 39.2 for the baseline, designs successful scaffolds for 25 of 30 motifs instead of 23, lowers the CATH Frechet Protein Distance from 0.36 to 0.29 at the local-structure l

What carries the argument

The load-bearing object is the alignment loss L_align = ||h_align - e_target||^2. h_align is the layer-5 hidden state of the generative U-ViT mapped by a lightweight MLP (layer norm, two hidden layers with SiLU, residual projection) into the embedding space of a frozen structure encoder; e_target is that encoder's embedding of the clean ground-truth structure. Added to the structure denoising loss with weight lambda=2.0, this term carries semantic, function-relevant signal back into the denoiser. Ablations show mid-layer alignment (layer 5) with a structure encoder (ProteinMPNN) works best; sequence encoders like ESM2 give less improvement, and very strong alignment (lambda>2) starts to hurt

Load-bearing premise

The load-bearing premise is that the published Protpardelle-1c baseline score of 39.2 was obtained under the same evaluation protocol the paper uses for its aligned model—same checkpoint selection among three epochs and same 100 samples with 8 ProteinMPNN sequences per motif—so the +7.9 point difference reflects alignment rather than protocol differences.

What would settle it

Take the unaligned Protpardelle-1c model and run it through the exact protocol used for ReaPro-1c: generate 100 structures per motif, design 8 ProteinMPNN sequences per structure for the best epoch among 399-401, then compute MotifBench. If the baseline score rises to near 47.1, the alignment gain is mostly an artifact of protocol or checkpoint selection; if it stays near 39.2, the gain is real.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Representation alignment with a structure encoder can be treated as a general training-time regularizer for protein diffusion models, not a task-specific design.
  • The gain is achieved with roughly half the training steps, suggesting alignment accelerates convergence and may reduce compute for future protein generative models.
  • The improvement transfers to a second benchmark: the aligned model outperforms the baseline on 22 of 26 RFDiffusion motifs for total success, and the effect persists in all-atom generation.
  • The choice of understanding model matters: structure-based encoders guide structure generation more effectively than sequence-based language models, even when the latter are up to 650M parameters.
  • Better distributional coverage of CATH indicates the aligned model produces a more diverse set of natural-like folds, not just more hits on the scaffold benchmark.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the headline gain holds under a controlled re-run of the baseline, representation alignment could become a standard component in protein diffusion training, complementing or replacing handcrafted auxiliary objectives.
  • A testable extension suggested by the modality result: align to a structure-aware protein language model or to an ensemble of complementarity encoders, which may push the score further while retaining sequence-level semantics.
  • The reported numbers rely on comparing against a published baseline score; the paper's own figure caption shows 45.6 versus 39.2 in one panel and 47.1 in the abstract, so the exact magnitude depends on checkpoint and evaluation choices. Re-running the baseline with the identical protocol would settle the true improvement.
  • The observation that generative representations are poor classifiers is itself an actionable result: it implies that pretrained generative models should not be used as feature extractors for function annotation without alignment or fine-tuning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper addresses the relationship between representation learning for protein understanding and generative protein structure modeling. The authors first benchmark several generative models (La-Proteina, Kanzi, Protpardelle-1c, RFDiffusion) on EC/GO classification and report that they underperform dedicated encoders such as ESM and ProteinMPNN. They then propose ReaPro-1c, which adds a REPA-style alignment loss to Protpardelle-1c: at layer 5 of the U-ViT, a 3-layer MLP projects the hidden state onto frozen embeddings of a pretrained understanding model (ProteinMPNN or ESM2) computed from the clean structure. The total loss is L_struct + λ L_align. The paper reports a MotifBench improvement from 39.2 to 47.1, improved CATH distribution coverage (FPD 0.36 to 0.29 at Layer 1), and faster convergence, plus ablations on the choice of understanding model, alignment weight, and layer.

Significance. The idea of aligning generative diffusion models with pretrained understanding encoders is timely and could be impactful for protein design, where functional generation remains hard. The paper's strengths include a systematic comparison across several generative and understanding models, ablations over understanding-model choice (including a scaling trend with ESM2 and a structural-encoder comparison), and the use of ESMFold verification in the MotifBench/RFDiffusion success metrics, which provides an independent check on structural validity. If the quantitative claims are confirmed, the method would be a simple, general recipe for improving conditional protein generation and convergence. However, the headline gain is not yet numerically secure because the baseline is not re-run under the evaluation protocol and because the FPD coverage metric shares the same encoder used as the alignment target.

major comments (3)
  1. [Sec. 5.1 and Appendix D.1] The headline claim (+7.9 MotifBench points, 20%) compares the ReaPro-1c score 47.1 against a baseline 39.2 quoted from [29]. Appendix D.1 describes how the ReaPro-1c score is obtained: generate 100 structures with 1 sequence per structure, pick the best checkpoint among epochs 399-401, then generate 100 structures x 8 ProteinMPNN sequences with that checkpoint. The baseline is never re-run under this protocol. The selection-over-checkpoints and the larger design-sampling budget alone could account for part or all of the difference. The paper's own numbers are inconsistent: Fig. 4 caption reports 45.6 vs 39.2, while Sec. 5.1 and the abstract report 47.1; Fig. 7 text gives baseline 39.4. A same-protocol re-run of the baseline (including best-of-3 checkpoint selection and 100x8 scoring) is required to support the 20% claim.
  2. [Eq. (3), Sec. 4.1-4.2, Sec. 5.1] Potential circularity: the alignment target in Eq. (3) is ProteinMPNN's structure encoder, and ProteinMPNN is also the embedding extractor for the SHAPES/FPD coverage metric (Sec. 4.1) and the sequence designer in the MotifBench/RFDiffusion pipelines (Sec. 5.1-5.2). The MotifBench success metric relies on ESMFold folding, which is genuinely independent, so the MotifBench result is not fully circular. However, the FPD improvement (0.36 to 0.29 at Layer 1; 0.23 to 0.21 at Layer 3) is computed in the same embedding space the model was trained to match, and could be inflated by construction. The paper should report FPD with an independent structural embedding (e.g., Foldseek, ESM2) or provide a control with an ESM2-aligned model to show that the coverage improvement is not merely an artifact of aligning to the evaluation encoder.
  3. [Fig. 7, Sec. 6.1] The faster-convergence claim is not supported by an equal-epoch comparison. Fig. 7 compares models trained for 200 epochs against a baseline trained for 416 epochs. This demonstrates that the aligned model exceeds the baseline's final score in fewer epochs, but not that it reaches that score in 'half the training steps,' because the baseline's score at 200 epochs is not reported. The abstract and Sec. 1 claim improvement 'obtained by only half training steps.' A learning curve for the baseline (e.g., at 200 and 400 epochs) is needed, or the claim should be softened to 'fewer total epochs than the baseline's full training.'
minor comments (4)
  1. [Fig. 4, Sec. 5.1, Fig. 7] Inconsistent score reporting: Fig. 4 caption says 45.6 vs 39.2, Sec. 5.1 says 47.1 vs 39.2, and Fig. 7 text says baseline 39.4. Unify and explain which number is final.
  2. [Sec. 2, C.1, Table 1] Model version inconsistency: Sec. 2 and C.1 describe cc58/cc91, but Table 1 uses Protpardelle-1c (cc91) while MotifBench backbone design likely uses cc58. State which base model/checkpoint is used in each experiment.
  3. [Fig. 1, throughout] Typo in Fig. 1: 'Sementic Labels' should be 'Semantic Labels'. Also, 'ProtPardelle-1c' and 'Protpardelle-1c' are used inconsistently.
  4. [Appendix B.7] The classification-head setup is underspecified: it is unclear whether the head is trained on frozen embeddings for all models or fine-tuned jointly. State this explicitly for reproducibility.

Circularity Check

0 steps flagged

No formal circularity; central gains depend on external, ESMFold-verified evaluations, though the shared use of ProteinMPNN as alignment target and evaluation metric warrants caution.

full rationale

The paper's formal chain is L_total = L_struct + λ·L_align (Eq. 4), with L_align = ||h_align − e_target||^2 (Eq. 5), where e_target comes from a frozen pretrained encoder (Eq. 3). No equation defines the reported MotifBench or FPD improvements in terms of L_align; the alignment loss is an auxiliary training signal, and the alignment head is discarded at inference. The headline MotifBench gain is an empirical comparison against an externally published baseline [29], not a consequence of the loss definition, and the ESMFold folding-RMSD success criterion is independent of ProteinMPNN, so success is not forced by construction. The use of ProteinMPNN both as an alignment target (Eq. 3) and in the FPD (Sec. 4.1) and sequence-design (Sec. 5.1) evaluations creates a shared-representation confound: improvements in ProteinMPNN-space metrics may be favored by the training objective. This is a methodological caveat rather than a circular reduction, because the FPD is computed on actual generated structures rather than on the projected hidden states that were supervised, and the final structural validity is checked by ESMFold. The baseline was not re-run under the paper's best-of-three checkpoint protocol (Appendix D.1), and there are internal numerical inconsistencies (39.2 vs 39.4 and 45.6 vs 47.1); these are comparison-validity and consistency concerns, not circularity. Self-citations are not load-bearing: the REPA idea is attributed to external work [50], and the baseline and benchmarks are external. No uniqueness theorem or ansatz is smuggled via self-citation.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 1 invented entities

The method imports REPA from vision and adds three tuned hyperparameters (lambda, layer, target model) plus a checkpoint-selection step, all fitted to the benchmark that is then used to report the headline number. The only invented construct is the alignment projector, which has no independent evidence outside the paper's own, partly self-referential, evaluations. No physical entities, forces, or new conserved quantities are introduced.

free parameters (5)
  • alignment loss weight lambda = 2.0
    Chosen by ablation over {0.5, 1.0, 2.0, 5.0, 10.0} on MotifBench (Sec. 6.2, Fig 6); performance declines for lambda > 2, so the main results use the value that maximized the benchmark.
  • alignment layer k = 5
    Chosen by ablation over layers {3, 5, 7} (Sec. 6.3, Fig 6); layer 5 maximized MotifBench score and is used for all main experiments.
  • understanding model choice = ProteinMPNN (structure encoder)
    Selected post hoc as best among ESM2-8M/35M/150M/650M, ProteinMPNN, and FA-MPNN in the Fig 7 ablation; the headline 47.1 is reported with the best-performing target model.
  • checkpoint selection = best of epochs 399/400/401
    Appendix D.1 selects the checkpoint with the best MotifBench score among three successive epochs before the final evaluation; selection over checkpoints can inflate the reported number relative to a fixed protocol.
  • projector dimension dproj = unspecified
    A hyperparameter of the alignment MLP (Eq. 2, Algorithm 1) whose value is never given; the capacity of the alignment head is not reproducible from the text.
axioms (4)
  • standard math The EDM/SGM preconditioned denoising objective (Karras et al.) is a valid training loss for structure diffusion.
    Used as Eq. 1 with the EDM weight w(sigma); accepted background cited as [21].
  • domain assumption Aligning a denoiser's intermediate latents to a frozen encoder's clean-input embeddings improves generation quality (REPA).
    Imported from vision [50]; the paper assumes this transfers to proteins rather than deriving it (introduced in Sec. 2.2 without proof).
  • domain assumption ProteinMPNN structure-encoder embeddings of clean structures are a semantically meaningful alignment target for denoising latents.
    Core design choice (Eq. 3); validated only by the paper's own benchmarks, which use ProteinMPNN in the evaluation loop (Sec. 4.2, 5.1).
  • domain assumption MotifBench success criteria (ProteinMPNN design + ESMFold RMSD < 1A/2A) measure functional designability.
    Shared with the baseline [29] but load-bearing: the 20% improvement is defined entirely in these terms (Sec. 5.1).
invented entities (1)
  • ReaPro-1c alignment head (3-layer MLP projector onto ProteinMPNN embedding space) no independent evidence
    purpose: Bridges generative denoising latents to the understanding model's semantic embedding space during training; discarded at inference (Sec. 2.1-2.2).
    An architectural/training construct with no falsifiable handle outside this paper's own benchmarks; its benefit is measured only by MotifBench/RFDiffusion/FPD scores computed partly with the same encoder family used as the training target.

pith-pipeline@v1.3.0-alltime-deepseek · 14193 in / 18296 out tokens · 171335 ms · 2026-08-02T05:00:03.693834+00:00 · methodology

0 comments
read the original abstract

Understanding and generation are often treated as two separate paradigms in training deep neural networks, despite the fact that both are trained with closely related objectives such as denoising and masked prediction. While prior studies have shown that generative models often learn suboptimal representations for understanding tasks in vision, it is less understood whether a similar gap exists in the protein domain. In this work, we systematically investigate this question by benchmarking state-of-the-art protein generative models on widely-used protein understanding tasks, and observe that these models exhibit consistently poor performance compared to existing protein encoders. Furthermore, inspired by the Representation Alignment (REPA) framework, we propose to explicitly align generative protein diffusion models with pretrained protein understanding models during training. Experiments on the MotifBench demonstrate that representation alignment significantly improves functional protein generation, boosting the MotifBench score of Protpardelle-1c from 39.2 to 47.1, corresponding to a 20% relative improvement. Our results suggest that representation alignment provides a general and effective mechanism for bridging understanding and generation in protein structure modeling.

Figures

Figures reproduced from arXiv: 2607.13503 by Jiezhong Qiu, Junde Xu, Lihang Liu, Pheng Ann Heng, Xiaoming Fang, Yuansheng Huang, Yu Kang, Zijun Gao.

Figure 1
Figure 1. Figure 1: Diagram of ReaPro-1c. The Generation Process (left) employs a denoising diffusion model that maps noisy inputs to protein structures, yielding unclassifiable latent features with ambiguous semantic labels; the Understanding Process (right) utilizes a mask prediction model that reconstructs masked regions, producing classifiable latent representations with distinct semantic categories. To bridge this gap, w… view at source ↗
Figure 2
Figure 2. Figure 2: Functional classification performance under varying noise scales [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: MotifBench results comparing Protpardelle-1c and ReaPro-1c. The bar plot shows the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Top: RFDiffusion backbone generation benchmark results. (Left) Total success count per motif. (Right) Unique success count after clustering. Top annotations indicate the number of motif segments (1–4). Bottom: RFDiffusion all-atom generation results. (Left) Total success count. (Right) Unique success count. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Ablation on alignment loss weight (λ) and alignment layers. (Left) Diffusion loss curves during training. (Middle) Alignment loss curves. (Right) MotifBench scores for different λ and layers. 5.3 All-Atom Generation To evaluate the extension of our approach to all-atom generation, we train a variant following the CC91 setting of protpardelle-1c [29]. This model generates full atomistic structures, includin… view at source ↗
Figure 7
Figure 7. Figure 7: Ablation on understanding model choice. MotifBench scores of models trained with different understanding models for representation alignment. All models are trained for 200 epochs with identical hyperparameters except for the align￾ment target. The baseline (Protpardelle-1c) is trained for 416 epochs. In this section, we conduct ablation studies to in￾vestigate the key design choices in our approach: (1) t… view at source ↗
Figure 8
Figure 8. Figure 8: Successfully designed case of MotifBench with ReaPro-1c [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Successfully designed case of RFDiffusion benchmark with ReaPro-1c [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 15 linked inside Pith

  1. [1]

    Gene ontology: tool for the unification of biology.Nature genetics, 25(1):25–29, 2000

    Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan P Davis, Kara Dolinski, Selina S Dwight, Janan T Eppig, et al. Gene ontology: tool for the unification of biology.Nature genetics, 25(1):25–29, 2000

    2000

  2. [2]

    Self-supervised learning from images with a joint- embedding predictive architecture

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint- embedding predictive architecture. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15619–15629, 2023

    2023

  3. [3]

    The enzyme database in 2000.Nucleic acids research, 28(1):304–305, 2000

    Amos Bairoch. The enzyme database in 2000.Nucleic acids research, 28(1):304–305, 2000

    2000

  4. [4]

    All are worth words: A vit backbone for diffusion models

    Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22669–22679, 2023

    2023

  5. [5]

    Se (3)-stochastic flow matching for protein backbone generation.arXiv preprint arXiv:2310.02391, 2023

    Avishek Joey Bose, Tara Akhound-Sadegh, Guillaume Huguet, Kilian Fatras, Jarrid Rector- Brooks, Cheng-Hao Liu, Andrei Cristian Nica, Maksym Korablyov, Michael Bronstein, and Alexander Tong. Se (3)-stochastic flow matching for protein backbone generation.arXiv preprint arXiv:2310.02391, 2023

    Pith/arXiv arXiv 2023

  6. [6]

    Deconstructing denoising diffusion models for self-supervised learning.arXiv preprint arXiv:2401.14404, 2024

    Xinlei Chen, Zhuang Liu, Saining Xie, and Kaiming He. Deconstructing denoising diffusion models for self-supervised learning.arXiv preprint arXiv:2401.14404, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Robust deep learning–based protein sequence design using proteinmpnn.Science, 378(6615):49–56, 2022

    Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, et al. Robust deep learning–based protein sequence design using proteinmpnn.Science, 378(6615):49–56, 2022

    2022

  8. [8]

    Flow autoencoders are effective protein tokenizers.arXiv preprint arXiv:2510.00351, 2025

    Rohit Dilip, Evan Zhang, Ayush Varshney, and David Van Valen. Flow autoencoders are effective protein tokenizers.arXiv preprint arXiv:2510.00351, 2025

    arXiv 2025

  9. [9]

    Atomica: Learning universal representations of intermolecular interactions.bioRxiv, pages 2025–04, 2025

    Ada Fang, Zaixi Zhang, Andrew Zhou, and Marinka Zitnik. Atomica: Learning universal representations of intermolecular interactions.bioRxiv, pages 2025–04, 2025

    2025

  10. [10]

    Foldtoken: Learning protein language via vector quantization and beyond

    Zhangyang Gao, Cheng Tan, Jue Wang, Yufei Huang, Lirong Wu, and Stan Z Li. Foldtoken: Learning protein language via vector quantization and beyond. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 219–227, 2025

    2025

  11. [11]

    Learning the language of protein structure.arXiv preprint arXiv:2405.15840, 2024

    Benoit Gaujac, Jérémie Donà, Liviu Copoiu, Timothy Atkinson, Thomas Pierrot, and Thomas D Barrett. Learning the language of protein structure.arXiv preprint arXiv:2405.15840, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    La-proteina: Atomistic protein generation via partially latent flow matching.arXiv preprint arXiv:2507.09466, 2025

    Tomas Geffner, Kieran Didi, Zhonglin Cao, Danny Reidenbach, Zuobai Zhang, Christian Dallago, Emine Kucukbenli, Karsten Kreis, and Arash Vahdat. La-proteina: Atomistic protein generation via partially latent flow matching.arXiv preprint arXiv:2507.09466, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    Simulating 500 million years of evolution with a language model.Science, 387(6736):850–858, 2025

    Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with a language model.Science, 387(6736):850–858, 2025

    2025

  14. [14]

    Bilingual language model for protein sequence and structure.NAR Genomics and Bioinformatics, 6(4):lqae150, 2024

    Michael Heinzinger, Konstantin Weissenow, Joaquin Gomez Sanchez, Adrian Henkel, Milot Mirdita, Martin Steinegger, and Burkhard Rost. Bilingual language model for protein sequence and structure.NAR Genomics and Bioinformatics, 6(4):lqae150, 2024

    2024

  15. [15]

    Contrastive representation learning for 3d protein structures.arXiv preprint arXiv:2205.15675, 2022

    Pedro Hermosilla and Timo Ropinski. Contrastive representation learning for 3d protein structures.arXiv preprint arXiv:2205.15675, 2022

    Pith/arXiv arXiv 2022

  16. [16]

    The coming of age of de novo protein design

    Po-Ssu Huang, Scott E Boyken, and David Baker. The coming of age of de novo protein design. Nature, 537(7620):320–327, 2016. 10

    2016

  17. [17]

    Sequence-augmented se (3)-flow matching for conditional protein generation.Advances in neural information processing systems, 37:33007–33036, 2024

    Guillaume Huguet, James Vuckovic, Kilian Fatras, Eric Thibodeau-Laufer, Pablo Lemos, Riashat Islam, Chenghao Liu, Jarrid Rector-Brooks, Tara Akhound-Sadegh, Michael Bronstein, et al. Sequence-augmented se (3)-flow matching for conditional protein generation.Advances in neural information processing systems, 37:33007–33036, 2024

    2024

  18. [18]

    Illumi- nating protein space with a programmable generative model.Nature, 623(7989):1070–1078, 2023

    John B Ingraham, Max Baranov, Zak Costello, Karl W Barber, Wujie Wang, Ahmed Ismail, Vincent Frappier, Dana M Lord, Christopher Ng-Thow-Hing, Erik R Van Vlack, et al. Illumi- nating protein space with a programmable generative model.Nature, 623(7989):1070–1078, 2023

    2023

  19. [19]

    Evaluating representation learning on the protein structure universe.ArXiv, pages arXiv–2406, 2024

    Arian R Jamasb, Alex Morehead, Chaitanya K Joshi, Zuobai Zhang, Kieran Didi, Simon Mathis, Charles Harris, Jian Tang, Jianlin Cheng, Pietro Liò, et al. Evaluating representation learning on the protein structure universe.ArXiv, pages arXiv–2406, 2024

    2024

  20. [20]

    Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021

    2021

  21. [21]

    Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    2022

  22. [22]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Pith/arXiv arXiv 2014

  23. [23]

    Repa-e: Unlocking vae for end-to-end tuning of latent diffusion transformers

    Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning of latent diffusion transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 18262–18272, 2025

    2025

  24. [24]

    Your diffusion model is secretly a zero-shot classifier

    Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your diffusion model is secretly a zero-shot classifier. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2206–2217, 2023

    2023

  25. [25]

    Out of many, one: Designing and scaffolding proteins at the scale of the structural universe with genie 2.arXiv preprint arXiv:2405.15489, 2024

    Yeqing Lin, Minji Lee, Zhao Zhang, and Mohammed AlQuraishi. Out of many, one: Designing and scaffolding proteins at the scale of the structural universe with genie 2.arXiv preprint arXiv:2405.15489, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    2023

  27. [27]

    Structure language models for protein conformation generation.arXiv preprint arXiv:2410.18403, 2024

    Jiarui Lu, Xiaoyin Chen, Stephen Zhewen Lu, Chence Shi, Hongyu Guo, Yoshua Bengio, and Jian Tang. Structure language models for protein conformation generation.arXiv preprint arXiv:2410.18403, 2024

    Pith/arXiv arXiv 2024

  28. [28]

    Assessing generative model coverage of protein structures with shapes.Cell Systems, 16(8), 2025

    Tianyu Lu, Melissa Liu, Yilin Chen, Jinho Kim, and Po-Ssu Huang. Assessing generative model coverage of protein structures with shapes.Cell Systems, 16(8), 2025

    2025

  29. [29]

    Conditional protein structure generation with protpardelle-1c.bioRxiv, 2025

    Tianyu Lu, Richard Shuai, Petr Kouba, Zhaoyang Li, Yilin Chen, Akio Shirali, Jinho Kim, and Po-Ssu Huang. Conditional protein structure generation with protpardelle-1c.bioRxiv, 2025

    2025

  30. [30]

    Bioto- ken and biofm–biologically-informed tokenization enables accurate and efficient genomic foundation models.bioRxiv, pages 2025–03, 2025

    Aleksandr Medvedev, Karthik Viswanathan, Praveenkumar Kanithi, Kirill Vishniakov, Prateek Munjal, Clément Christophe, Marco AF Pimentel, Ronnie Rajan, and Shadab Khan. Bioto- ken and biofm–biologically-informed tokenization enables accurate and efficient genomic foundation models.bioRxiv, pages 2025–03, 2025

    2025

  31. [31]

    Machine learning for functional protein design.Nature biotechnology, 42(2):216–228, 2024

    Pascal Notin, Nathan Rollins, Yarin Gal, Chris Sander, and Debora Marks. Machine learning for functional protein design.Nature biotechnology, 42(2):216–228, 2024

    2024

  32. [32]

    Gcc: Graph contrastive coding for graph neural network pre-training

    Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. Gcc: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1150–1160, 2020. 11

    2020

  33. [33]

    Instructplm: Aligning protein language models to follow protein structure instructions.bioRxiv, pages 2024–04, 2024

    Jiezhong Qiu, Junde Xu, Jie Hu, Hanqun Cao, Liya Hou, Zijun Gao, Xinyi Zhou, Anni Li, Xiujuan Li, Bin Cui, et al. Instructplm: Aligning protein language models to follow protein structure instructions.bioRxiv, pages 2024–04, 2024

    2024

  34. [34]

    Deep learning enables high-quality and high-throughput prediction of enzyme commission numbers.Proceedings of the National Academy of Sciences, 116(28):13996–14001, 2019

    Jae Yong Ryu, Hyun Uk Kim, and Sang Yup Lee. Deep learning enables high-quality and high-throughput prediction of enzyme commission numbers.Proceedings of the National Academy of Sciences, 116(28):13996–14001, 2019

    2019

  35. [35]

    Fine-tuning protein language models boosts predictions across diverse tasks.Nature Communications, 15(1):7407, 2024

    Robert Schmirler, Michael Heinzinger, and Burkhard Rost. Fine-tuning protein language models boosts predictions across diverse tasks.Nature Communications, 15(1):7407, 2024

    2024

  36. [36]

    Cath: increased structural coverage of functional space.Nucleic acids research, 49(D1):D266–D273, 2021

    Ian Sillitoe, Nicola Bordin, Natalie Dawson, Vaishali P Waman, Paul Ashford, Harry M Scholes, Camilla SM Pang, Laurel Woodridge, Clemens Rauer, Neeladri Sen, et al. Cath: increased structural coverage of functional space.Nucleic acids research, 49(D1):D266–D273, 2021

    2021

  37. [37]

    Saprot: Protein language modeling with structure-aware vocabulary.BioRxiv, pages 2023–10, 2023

    Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with structure-aware vocabulary.BioRxiv, pages 2023–10, 2023

    2023

  38. [38]

    Scaling text-to-image diffusion transformers with representation autoencoders.arXiv preprint arXiv:2601.16208, 2026

    Shengbang Tong, Boyang Zheng, Ziteng Wang, Bingda Tang, Nanye Ma, Ellis Brown, Jihan Yang, Rob Fergus, Yann LeCun, and Saining Xie. Scaling text-to-image diffusion transformers with representation autoencoders.arXiv preprint arXiv:2601.16208, 2026

    arXiv 2026

  39. [39]

    Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    2017

  40. [40]

    Foldseek: fast and accurate protein structure search.Biorxiv, pages 2022–02, 2022

    Michel van Kempen, Stephanie S Kim, Charlotte Tumescheit, Milot Mirdita, Cameron LM Gilchrist, Johannes Söding, and Martin Steinegger. Foldseek: fast and accurate protein structure search.Biorxiv, pages 2022–02, 2022

    2022

  41. [41]

    Fast and accurate protein structure search with foldseek.Nature biotechnology, 42(2):243–246, 2024

    Michel Van Kempen, Stephanie S Kim, Charlotte Tumescheit, Milot Mirdita, Jeongjae Lee, Cameron LM Gilchrist, Johannes Söding, and Martin Steinegger. Fast and accurate protein structure search with foldseek.Nature biotechnology, 42(2):243–246, 2024

    2024

  42. [42]

    Dplm-2: A multimodal diffusion protein language model.arXiv preprint arXiv:2410.13782, 2024

    Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, and Quanquan Gu. Dplm-2: A multimodal diffusion protein language model.arXiv preprint arXiv:2410.13782, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    De novo design of protein structure and function with rfdiffusion.Nature, 620(7976):1089–1100, 2023

    Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eisenach, Woody Ahern, Andrew J Borst, Robert J Ragotte, Lukas F Milles, et al. De novo design of protein structure and function with rfdiffusion.Nature, 620(7976):1089–1100, 2023

    2023

  44. [44]

    Sidechain conditioning and modeling for full-atom protein sequence design with fampnn.Proceedings of machine learning research, 267:66746, 2025

    Talal Widatalla, Richard W Shuai, Brian L Hie, and Po-Ssu Huang. Sidechain conditioning and modeling for full-atom protein sequence design with fampnn.Proceedings of machine learning research, 267:66746, 2025

    2025

  45. [45]

    Generative artificial intelligence for de novo protein design.Current Opinion in Structural Biology, 86:102794, 2024

    Adam Winnifrith, Carlos Outeiral, and Brian L Hie. Generative artificial intelligence for de novo protein design.Current Opinion in Structural Biology, 86:102794, 2024

    2024

  46. [46]

    Denoising diffusion autoencoders are unified self-supervised learners

    Weilai Xiang, Hongyu Yang, Di Huang, and Yunhong Wang. Denoising diffusion autoencoders are unified self-supervised learners. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15802–15812, 2023

    2023

  47. [47]

    Fast protein backbone generation with se (3) flow matching.arXiv preprint arXiv:2310.05297, 2023

    Jason Yim, Andrew Campbell, Andrew YK Foong, Michael Gastegger, José Jiménez-Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan S Veeling, Regina Barzilay, Tommi Jaakkola, et al. Fast protein backbone generation with se (3) flow matching.arXiv preprint arXiv:2310.05297, 2023

    Pith/arXiv arXiv 2023

  48. [48]

    Se (3) diffusion model with application to protein backbone generation.arXiv preprint arXiv:2302.02277, 2023

    Jason Yim, Brian L Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. Se (3) diffusion model with application to protein backbone generation.arXiv preprint arXiv:2302.02277, 2023

    Pith/arXiv arXiv 2023

  49. [49]

    Graph contrastive learning with augmentations.Advances in neural information processing systems, 33:5812–5823, 2020

    Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations.Advances in neural information processing systems, 33:5812–5823, 2020. 12

    2020

  50. [50]

    Representation alignment for generation: Training diffusion transformers is easier than you think.arXiv preprint arXiv:2410.06940, 2024

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think.arXiv preprint arXiv:2410.06940, 2024

    Pith/arXiv arXiv 2024

  51. [51]

    Enzyme function prediction using contrastive learning.Science, 379(6639):1358–1363, 2023

    Tianhao Yu, Haiyang Cui, Jianan Canal Li, Yunan Luo, Guangde Jiang, and Huimin Zhao. Enzyme function prediction using contrastive learning.Science, 379(6639):1358–1363, 2023

    2023

  52. [52]

    Protein representation learning by geometric structure pretraining.arXiv preprint arXiv:2203.06125, 2022

    Zuobai Zhang, Minghao Xu, Arian Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. Protein representation learning by geometric structure pretraining.arXiv preprint arXiv:2203.06125, 2022

    Pith/arXiv arXiv 2022

  53. [53]

    Diffusion transformers with representation autoencoders.arXiv preprint arXiv:2510.11690, 2025

    Boyang Zheng, Nanye Ma, Shengbang Tong, and Saining Xie. Diffusion transformers with representation autoencoders.arXiv preprint arXiv:2510.11690, 2025

    Pith/arXiv arXiv 2025

  54. [54]

    Motifbench: A standardized protein design benchmark for motif-scaffolding problems.arXiv preprint arXiv:2502.12479, 2025

    Zhuoqi Zheng, Bo Zhang, Kieran Didi, Kevin K Yang, Jason Yim, Joseph L Watson, Hai- Feng Chen, and Brian L Trippe. Motifbench: A standardized protein design benchmark for motif-scaffolding problems.arXiv preprint arXiv:2502.12479, 2025. 13 A Preliminaries and Related Works A.1 Protein Understanding Protein Structuredetermines how protein performs their bi...

    Pith/arXiv arXiv 2025