Pith. sign in

REVIEW 3 major objections 7 minor 61 references

Aligning Proteins and Language: A Foundation Model for Protein Retrieval

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A CLIP-style protein-language model retrieves functional Gene Ontology captions from protein surface point clouds at about 60% Top-5 on PDB and 36% Top-5 on unseen EMDB structures.

desk verdict PDB 'zero-shot' evaluation looks contaminated; the EMDB result is the only true zero-shot evidence, and it needs baselines and error bars. read the letter →

arxiv 2506.08023 v1 pith:O5Z2JGDZ submitted 2025-05-27 q-bio.BM cs.AIcs.CEcs.CVcs.LG

Qifeng Wu , Zhengzhe Liu , Han Zhu , Yizhou Zhao , Daisuke Kihara , Min Xu This is my paper
classification q-bio.BMcs.AIcs.CEcs.CVcs.LG
keywords proteinretrievalcontrastivelearningCLIPpointcloudsGeneOntologyzero-shotcryo-EMstructure-function
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a protein's molecular function can be read directly from the shape of its 3D surface, in the form of natural-language Gene Ontology captions, with no task-specific fine-tuning. The authors train a CLIP-style dual encoder on roughly 200,000 protein-caption pairs, where each protein is represented as a point cloud sampled from a marching-cubes surface and each caption is a sentence of leaf-level GO terms. They report zero-shot retrieval of the correct caption from a 100-caption pool at about 60% Top-5 accuracy on PDB and about 35.8% Top-5 when the queries are EMDB density-map point clouds never seen in training. If correct, this means surface-level geometry from cryo-EM-like density maps carries enough functional signal to support database-scale functional annotation and retrieval.

What carries the argument

The central object is a dual-encoder contrastive model in the style of CLIP: a point-cloud encoder (initialised from a pretrained 3D checkpoint) and a frozen text encoder (a sentence-transformer model) project protein geometry and GO captions into a shared embedding space, trained with a symmetric InfoNCE cross-entropy loss over protein-caption pairs. Protein structures are converted to point clouds by simulating electron-scattering density maps with EMAN2, extracting an iso-surface with marching cubes at a density threshold of $\mu + 0.5\sigma$, and uniformly sampling 16,384 points. Captions are formed by concatenating only leaf-level GO terms and prepending a fixed prompt. Retrieval is cosine similarity in the shared space, and the same pipeline is applied to EMDB density maps using their author-recommended contour levels.

What would settle it

Re-run the PDB zero-shot test with a test set verifiably excluded from the 200,000 pretraining pairs and with the 99 negative captions selected from the same GO parent terms as the positive; if Top-5 accuracy collapses toward the 1% chance level of a 100-caption pool, the claim that surface point clouds support zero-shot function retrieval would be refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that contrastive alignment between protein point clouds and GO-derived text captions yields a shared embedding in which cosine similarity acts as a zero-shot functional retriever. A query protein structure—whether from PDB or from an EMDB density map—ranks its correct functional caption among 100 candidates without any fine-tuning on the query data. The reported numbers are about 59.7% Top-5 within PDB and between 34.1% and 36.55% Top-5 across databases depending on the map-model quality filter used. The authors interpret this as evidence that the geometric signal in a uniformly sampled surface point cloud is sufficient to align with symbolic functional vocabulary, and that the alignment transfers across resolution regimes.

Load-bearing premise

The load-bearing premise is that a uniformly sampled surface point cloud preserves enough function-relevant geometry, that leaf-level GO captions are a sufficient functional target, and that the PDB test entries were not part of the pretraining pairs; if any of these fails, the reported accuracies overstate zero-shot generalization.

Editorial extensions

If this is right

  • A structure-alone query can retrieve functional descriptions without any supervised classifier or task-specific fine-tuning, which could speed annotation of newly solved structures.
  • The alignment transfers across databases and resolution regimes, so density maps from cryo-EM experiments can be searched directly against PDB-style functional captions.
  • Surface point clouds, rather than full atomic coordinates, carry enough geometric signal for molecular-function retrieval, simplifying the input representation for future structure-function models.
  • Freezing the text encoder and training only the point-cloud side is sufficient to learn the cross-modal alignment, suggesting the text side supplies stable semantic structure.
  • Long or compositionally complex GO captions are a practical bottleneck: qualitative failures show ground-truth captions dropping to ranks 56 and 60 of 100 when many specific terms are concatenated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If point-cloud surfaces suffice, cryo-EM maps could be searched for function before atomic models are built, shortening the functional-annotation pipeline for high-throughput structural biology.
  • The same recipe likely extends to biological-process and cellular-component GO branches, but the paper's captions carry only molecular-function leaf terms, so that extension is untested.
  • A harder probe of true function understanding would draw the 99 negative captions from proteins sharing the same parent GO term; the paper uses random negatives, so part of the reported accuracy may come from coarse semantic separation.
  • The model's failure on long captions suggests gains from hierarchical or compositional caption encoders, or from chunked multiple-choice retrieval, which the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes a CLIP-style dual-encoder model for protein retrieval: an I2P-MAE point cloud encoder and an all-mpnet-base-v2 text encoder are aligned with a symmetric InfoNCE loss on roughly 200,000 protein–caption pairs. Protein structures are converted to density maps, iso-surfaces are extracted with marching cubes, and surfaces are uniformly sampled into 16,384-point clouds; captions are concatenated GO leaf terms. The authors report zero-shot retrieval on PDB (59.70% Top-5) and cross-database zero-shot retrieval from EMDB to PDB (35.76% Top-5 at a CCC+OVR threshold of 1.5), with qualitative examples in the supplementary material.

Significance. If the zero-shot results are valid, the paper would demonstrate that coarse surface point clouds alone can retrieve functional Gene Ontology descriptions without task-specific fine-tuning, which is relevant for cryo-EM structure interpretation. The strengths include a relatively large curated dataset of protein–caption pairs, use of a standard contrastive framework, a cross-database evaluation setting that explicitly excludes EMDB-linked PDB entries from training (Sec. 6), and qualitative retrieval examples that illustrate both successes and failures. However, the central claim hinges on the PDB test set being genuinely held out, and the current wording in Sec. 4.1 suggests it may not be; the quantitative evidence also lacks baselines, sample sizes, and error bars. These issues currently prevent the paper from supporting its central claim at the standard required by a journal.

major comments (3)
  1. [Sec. 4.1, Table 1] The PDB 'zero-shot' evaluation appears to be contaminated. The text states that for each test point cloud the candidate pool contains 'one correct (positive) caption paired with the structure during pretraining,' which implies that the query structure itself was in the pretraining set and that the positive caption was one of its training pairs. If so, the 59.70% Top-5 accuracy measures retrieval of memorized training pairs among 99 distractors, not generalization to unseen proteins. The supplementary material (Sec. 6) explicitly guarantees that PDB entries corresponding to EMDB queries were excluded from training, but no analogous holdout statement is made for the PDB test set. The authors must specify the exact train/test split for PDB entries, report the number of test proteins, and verify that no test PDB structure or its GO caption appeared during pretraining; otherwise the zero-shot claim is unsupported.
  2. [Tables 1 and 2] The quantitative evaluation reports single point estimates with no sample sizes, confidence intervals, or error bars. Table 2 varies the CCC+OVR threshold from 1.5 to 1.8, but the number of EMDB queries at each threshold is not reported, so the reader cannot judge whether the differences between threshold rows are meaningful. Moreover, no baselines are provided: random retrieval from a 100-candidate pool would give an expected Top-5 accuracy of 5%, but no comparison is made to sequence-based or structure-based retrieval methods, to a text-only baseline, or to chance-level performance. Without sample sizes, variance information, and at least one baseline, the reported accuracies cannot be interpreted as evidence of effective retrieval.
  3. [Sec. 2.2 and Sec. 6] The cross-database evaluation depends on a quality filter whose effect on the test distribution is not characterized. EMDB surfaces are extracted at the author-recommended contour level, whereas PDB surfaces use the empirical iso-value of Eq. (1); this domain shift may be intentional, but no sensitivity analysis is given. More importantly, the CCC+OVR>=1.5 filter (Sec. 6) selects EMDB–PDB pairs whose experimental maps agree well with simulated maps from PDB atomic models. This could bias the retained set toward entries whose surfaces are most PDB-like, making the cross-database result easier than it would be on unselected EMDB entries. The authors should report how many EMDB entries were excluded at each threshold, characterize the retained set, and justify that the filter does not remove the hardest generalization cases.
minor comments (7)
  1. [Abstract and Sec. 4.1] The abstract says the model retrieves 'proteins with similar structures and semantics,' but the experiments retrieve captions for a single query structure; the task definition should be stated consistently as caption retrieval for a given protein point cloud.
  2. [Sec. 3.1] The point cloud encoder is described only as I2P-MAE with a checkpoint from Point-bind; details such as the input point cloud size, whether normals are used, and which layers of I2P-MAE are fine-tuned versus frozen are missing. These details are needed for reproducibility.
  3. [Sec. 3.3] The text says both the point cloud encoder and projection layers are optimized, but the sentence 'we freeze the text encoder and optimize only the point cloud encoder and the projection layers' leaves ambiguous whether the I2P-MAE encoder is fully fine-tuned; please state the parameter update status explicitly.
  4. [Sec. 2.2] The sentence 'This ensures consistent surface extraction across varying density distributions' is repeated verbatim twice in the same paragraph; one occurrence should be removed.
  5. [Supplementary Fig. 3] The caption for the bad case says 'PDB 519K' while the main text and figure description refer to PDB 5I9K; the typo should be corrected.
  6. [Sec. 4.2 and Table 2] Table 2 presents results at four CCC+OVR thresholds, but the main text states that the threshold is 1.5; please clarify whether 1.5 is the prespecified threshold and the other rows are sensitivity analyses, and report the corresponding sample sizes.
  7. [General] No code, model weights, or dataset access information is provided; making the 200,000-pair dataset and split definitions available would substantially increase the reproducibility and usefulness of the work.

Circularity Check

1 steps flagged · score 6.0 of 10

PDB 'zero-shot' retrieval is not zero-shot: the positive caption is one of the pretraining pairs, so Table 1 measures memorization, not generalization.

  1. fitted input called prediction [Section 4.1, 'Zero-shot retrieval on PDB' (Table 1)]
    "For each test point cloud sampled from a PDB entry, we construct a candidate pool of 100 GO-derived captions: one correct (positive) caption paired with the structure during pretraining, and 99 randomly selected negative captions. Cosine similarity scores are computed between the point cloud embedding and each caption embedding. We report Top-1 and Top-5 retrieval accuracy, where success is defined as the correct caption appearing among the top-k ranked candidates."

    The PDB 'zero-shot' test uses as its correct answer the very caption that was paired with the query structure during pretraining. The test pair is therefore part of the training data, and the model can retrieve it by memorizing the seen mapping rather than by generalizing from structure to function. The reported 59.70% Top-5 accuracy in Table 1 is a recall-of-seen-pairs number, not a zero-shot generalization number. The paper nowhere states that PDB test structures were excluded from the roughly 200,000 training pairs, in contrast to the EMDB setting whose corresponding PDB entries are explicitly excluded in Section 6. The evaluation is circular by construction: the test label is an input to training.

full rationale

The paper contains no analytic derivation chain, so no fitted equations reduce to themselves. The one substantive circularity is in the PDB zero-shot experiment: Section 4.1 says the positive caption was 'paired with the structure during pretraining,' which implies the test pair appeared in training. Retrieving that caption among 99 random distractors measures memorization of a seen pair, not zero-shot structure-function transfer, so Table 1 does not support the abstract's 'zero-shot' claim for PDB. The EMDB cross-database experiment is not circular: Section 6 explicitly excludes the corresponding PDB entries from training, making Table 2 independent evidence of cross-domain retrieval. There is no load-bearing self-citation chain; the cited backbones and point-cloud encoders are used as pretrained components rather than to force the protein-retrieval conclusion. Overall score 6 reflects one central 'prediction' that reduces to its training input, while a second reported result remains independent.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a chain of representation and evaluation choices: point clouds from iso-surfaces, GO leaf-term captions, frozen text embeddings, and the EMDB quality filter. The model introduces no new physical or mathematical entities. The paper is purely empirical, so the quantitative claims depend on the dataset and evaluation design rather than on free-standing derivations.

free parameters (5)
  • Surface extraction iso-value coefficient = mu + 0.5 sigma
    Set empirically in Eq. 1 to define PDB surface meshes; affects all point clouds and therefore the entire model input.
  • EMDB evaluation quality threshold = CCC + OVR >= 1.5 (swept 1.5 to 1.8 in Table 2)
    Post hoc filter on EMDB-PDB pairs; cross-database accuracy changes with the threshold and no sample counts are given.
  • Point cloud sample count = 16,384
    Uniform sampling count chosen to standardize input; no experiment justifies this number.
  • GO caption specificity filter = leaf-level terms only
    Hand-chosen filtering rule removes parent GO terms, changing the text captions and the retrieval targets.
  • Candidate pool size = 100
    Evaluation protocol fixed at 1 positive and 99 random negatives; random chance is 1% Top-1 and 5% Top-5, which is the implicit baseline.
assumptions (5)
  • domain assumption Surface point clouds extracted at iso-value mu + 0.5 sigma preserve enough structural information for function alignment.
    Central modeling choice in Section 2.2; if surfaces discard functional sites, retrieval accuracy would not reflect true structure-function relationships.
  • domain assumption GO leaf-term concatenation is an adequate functional description.
    The paper filters annotations to leaf-level terms and uses them as captions; this assumes the selected terms are sufficient and non-redundant.
  • standard math Contrastive learning with random in-batch negatives induces meaningful semantic alignment.
    The symmetric InfoNCE objective in Eq. 3 assumes that random negatives are valid negatives, a standard but non-trivial assumption for protein functions with correlated GO terms.
  • domain assumption EMDB density maps at author-recommended contour levels are geometrically comparable to PDB-simulated surfaces.
    The cross-database evaluation relies on this comparability; resolution and contour differences could weaken alignment.
  • domain assumption Frozen pretrained text encoder (all-mpnet-base-v2) provides a stable semantic space for GO captions.
    The paper freezes the text encoder, assuming its sentence-level embeddings are appropriate for concatenated GO terms.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligning Proteins and Language: A Foundation Model for Protein Retrieval." pith.science (2026). https://pith.science/paper/O5Z2JGDZ

@misc{pith2026250608023,
  author       = {Pith},
  title        = {Pith review of: Aligning Proteins and Language: A Foundation Model for Protein Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O5Z2JGDZ}},
  note         = {Machine review of arXiv:2506.08023}
}
read the original abstract

This paper aims to retrieve proteins with similar structures and semantics from large-scale protein dataset, facilitating the functional interpretation of protein structures derived by structural determination methods like cryo-Electron Microscopy (cryo-EM). Motivated by the recent progress of vision-language models (VLMs), we propose a CLIP-style framework for aligning 3D protein structures with functional annotations using contrastive learning. For model training, we propose a large-scale dataset of approximately 200,000 protein-caption pairs with rich functional descriptors. We evaluate our model in both in-domain and more challenging cross-database retrieval on Protein Data Bank (PDB) and Electron Microscopy Data Bank (EMDB) dataset, respectively. In both cases, our approach demonstrates promising zero-shot retrieval performance, highlighting the potential of multimodal foundation models for structure-function understanding in protein biology.

Figures

Figures reproduced from arXiv: 2506.08023 by the authors.

Figure 1
Figure 1. Contrastive pretraining with protein–text pairs. Protein structures are first converted to volumetric density maps synthesized from atomic coordinates (PDB), from which surface meshes are extracted via marching cubes and uniformly sampled into point clouds. Text descriptions are generated by concatenating Gene Ontology (GO) terms filtered for specificity and prepended with a natural language prompt. The fire icon in… view at source ↗
Figure 2
Figure 2. Zero-shot retrieval. Given a query protein point cloud, the model ranks a pool of candidate captions by similarity in the shared embedding space. To encode protein represented by point clouds, we adopt I2P-MAE [19], which employs pretrained 2D transformer based vision model to supervise 3D representation learning, as the point cloud encoder and used the pretrained check￾point released in Point-bind [5]. To encode te… view at source ↗
Figure 3
Figure 3. Bad case and good case examples. (a) A bad case exam [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Retrieval result for query protein 5VJ6 Ground-truth Caption: serine C-palmitoyltransferase activ￾ity, pyridoxal phosphate binding Ground-truth Ranking in Retrieved Result: 4/100 Top-5 Retrieved Captions: 1. oxalyl-CoA decarboxylase activity, thiamine pyrophosphate bin…
Figure 5
Figure 5. Figure 5: Retrieval result for query protein 7K0M Ground-truth Caption: cystathionine beta-synthase activ￾ity, nitric oxide binding, pyridoxal phosphate binding, nitrite reductase (NO-forming) activity, oxygen binding, ubiquitin protein ligase binding, carbon monoxide binding, m…
Figure 10
Figure 10. Figure 10: Retrieval result for query protein EMD-22506 [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Retrieval result for query protein EMD-22206 [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 56 canonical work pages

  1. [1]

    Multimodal large language models in health care: applications, challenges, and future outlook.JMIR, 2024

    Rawan AlSaad, Alaa Abd-Alrazaq, Sabri Boughorbel, Arfan Ahmed, Max-Antoine Renault, Rafat Damseh, and Javaid Sheikh. Multimodal large language models in health care: applications, challenges, and future outlook.JMIR, 2024. 1

    work page 2024

  2. [2]

    CliNER 2.0: Accessible and Accurate Clinical Concept Extraction

    Willie Boag, Elena Sergeeva, Saurabh Kulshreshtha, Peter Szolovits, Anna Rumshisky, and Tristan Naumann. Cliner 2.0: Accessible and accurate clinical concept extraction. arXiv preprint arXiv:1803.02245, 2018. 1

    work page Pith review arXiv 2018

  3. [3]

    Pathologyvlm: a large vision-language model for pathology image under- standing.Artif

    Dawei Dai, Yuanhui Zhang, Qianlan Yang, Long Xu, Xiao- jing Shen, Shuyin Xia, and Guoyin Wang. Pathologyvlm: a large vision-language model for pathology image under- standing.Artif. Intell. Rev., 2025. 1

    work page 2025

  4. [4]

    Structure-based protein function prediction us- ing graph convolutional networks.Nat

    Vladimir Gligorijevi ´c, P Douglas Renfrew, Tomasz Kosci- olek, Julia Koehler Leman, Daniel Berenberg, Tommi Vata- nen, Chris Chandler, Bryn C Taylor, Ian M Fisk, Hera Vla- makis, et al. Structure-based protein function prediction us- ing graph convolutional networks.Nat. Commun., 2021. 1

    work page 2021

  5. [5]

    Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understand- ing, generation, and instruction following.arXiv preprint arXiv:2309.00615, 2023

    Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xi- anzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xi- anzhi Li, Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understand- ing, generation, and instruction following.arXiv preprint arXiv:2309.00615, 2023. 3

    arXiv 2023

  6. [6]

    Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation.arXiv preprint arXiv:2412.13558, 2024

    Changsun Lee, Sangjoon Park, Cheong-Il Shin, Woo Hee Choi, Hyun Jeong Park, Jeong Eun Lee, and Jong Chul Ye. Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation.arXiv preprint arXiv:2412.13558, 2024. 1

    arXiv 2024

  7. [7]

    Su- pervision exists everywhere: A data efficient contrastive language-image pre-training paradigm.arXiv preprint arXiv:2110.05208, 2021

    Yangguang Li, Feng Liang, Lichen Zhao, Yufeng Cui, Wanli Ouyang, Jing Shao, Fengwei Yu, and Junjie Yan. Su- pervision exists everywhere: A data efficient contrastive language-image pre-training paradigm.arXiv preprint arXiv:2110.05208, 2021. 1

    arXiv

  8. [8]

    Marching cubes: A high resolution 3d surface construction algorithm.Seminal graphics: pioneering efforts that shaped the field, 1998

    William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm.Seminal graphics: pioneering efforts that shaped the field, 1998. 2

    work page 1998

Show all 61 references
  1. [9]

    Daq-score database: assessment of map–model com- patibility for protein structure models from cryo-em maps

    Tsukasa Nakamura, Xiao Wang, Genki Terashi, and Daisuke Kihara. Daq-score database: assessment of map–model com- patibility for protein structure models from cryo-em maps. Nat. Methods, 2023. 4, 6

    2023

  2. [10]

    Multi-modality artificial intelligence in digital pathology.Brief

    Yixuan Qiao, Lianhe Zhao, Chunlong Luo, Yufan Luo, Yang Wu, Shengtong Li, Dechao Bu, and Yi Zhao. Multi-modality artificial intelligence in digital pathology.Brief. Bioinform.,

  3. [11]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InICML, 2021. 1, 2

    2021

  4. [12]

    Next-generation virtual hospital: Integrating discriminative and large multi- modal generative ai for personalized healthcare

    Md Abdur Rahman and Selwa Al-Hazzaa. Next-generation virtual hospital: Integrating discriminative and large multi- modal generative ai for personalized healthcare. InGLOBE- COM, 2024. 1

    2024

  5. [13]

    Sentence-bert: Sentence embeddings using siamese bert-networks.arXiv preprint arXiv:1908.10084, 2019

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks.arXiv preprint arXiv:1908.10084, 2019. 3

    1908 arXiv

  6. [14]

    Mpnet: Masked and permuted pre-training for language understanding.NeurIPS, 2020

    Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. Mpnet: Masked and permuted pre-training for language understanding.NeurIPS, 2020. 3

    2020

  7. [15]

    Eman2: an extensible image processing suite for electron microscopy.J

    Guang Tang, Liwei Peng, Philip R Baldwin, Deepinder S Mann, Wen Jiang, Ian Rees, and Steven J Ludtke. Eman2: an extensible image processing suite for electron microscopy.J. Struct. Biol., 2007. 2

    2007

  8. [16]

    Residue-wise local quality estimation for protein mod- els from cryo-em maps.Nat

    Genki Terashi, Xiao Wang, Sai Raghavendra Maddhuri Venkata Subramaniya, John JG Tesmer, and Daisuke Ki- hara. Residue-wise local quality estimation for protein mod- els from cryo-em maps.Nat. Methods, 2022. 4, 6

    2022

  9. [17]

    Multimodal healthcare ai: identifying and designing clinically relevant vision-language applications for radiology

    Nur Yildirim, Hannah Richardson, Maria Teodora Wetscherek, Junaid Bajwa, Joseph Jacob, Mark Ames Pinnock, Stephen Harris, Daniel Coelho De Castro, Shruthi Bannur, Stephanie Hyland, et al. Multimodal healthcare ai: identifying and designing clinically relevant vision-language a...

    2024

  10. [18]

    Transferring the knowledge of vision-language model for pathological image classification

    Ying Zeng and Jialong Zhu. Transferring the knowledge of vision-language model for pathological image classification. InICSECE, 2024. 1

    2024

  11. [19]

    Learning 3d representations from 2d pre-trained models via image-to-point masked autoencoders

    Renrui Zhang, Liuhui Wang, Yu Qiao, Peng Gao, and Hong- sheng Li. Learning 3d representations from 2d pre-trained models via image-to-point masked autoencoders. InCVPR,

  12. [20]

    Generalized radio- graph representation learning via cross-supervision between images and free-text radiology reports.Nat

    Hong-Yu Zhou, Xiaoyu Chen, Yinghao Zhang, Ruibang Luo, Liansheng Wang, and Yizhou Yu. Generalized radio- graph representation learning via cross-supervision between images and free-text radiology reports.Nat. Mach. Intell.,

  13. [22]

    Specifically, PDB entries corresponding to these EMDB en- tries were excluded from the training set, ensuring a true zero-shot setting even under the cross-database condition

    EMDB-PDB Entry Correspondence To evaluate the generalization ability of the model across modalities, we perform zero-shot retrieval using EMDB entries, which are not seen by the model during training. Specifically, PDB entries corresponding to these EMDB en- tries were exclude...

  14. [23]

    Qualitative Retrieval Results To provide a deeper insight into model behavior, we present qualitative retrieval examples from both the PDB and EMDB test sets. Figs. 4–7 show zero-shot results on the PDB test set, while Figs. 8–11 illustrate cross-database zero-shot retrieval o...

  15. [24]

    identical protein binding, structural molecule activity, metal ion binding

  16. [25]

    structural molecule activity, identical protein binding

  17. [26]

    identical protein binding, receptor ligand activity, structural constituent of virion, host cell surface receptor binding

  18. [27]

    identical protein binding

  19. [28]

    structural molecule activity Figure 4. Retrieval result for query protein 5VJ6 Ground-truth Caption:serine C-palmitoyltransferase activ- ity, pyridoxal phosphate binding Ground-truth Ranking in Retrieved Result:4/100 Top-5 Retrieved Captions:

  20. [29]

    oxalyl-CoA decarboxylase activity, thiamine pyrophosphate binding, identical protein binding, magnesium ion binding, ADP binding

  21. [30]

    methylaspartate mutase activity, cobalamin binding, metal ion binding

  22. [31]

    catalase activity, heme binding, nucleotide binding, metal ion binding

  23. [32]

    serine C-palmitoyltransferase activity, pyridoxal phos- phate binding

  24. [33]

    structural molecule activity, metal ion binding Figure 5. Retrieval result for query protein 7K0M Ground-truth Caption:cystathionine beta-synthase activ- ity, nitric oxide binding, pyridoxal phosphate binding, nitrite reductase (NO-forming) activity, oxygen binding, ubiquitin ...

  25. [34]

    carbohydrate binding, glycosyltransferase activity

  26. [35]

    methionine-tRNA ligase activity, ATP binding

  27. [36]

    K63-linked polyubiquitin modification-dependent protein binding, cysteine-type deubiquitinase activity, ubiquitin pro- tein ligase binding, protein tag activity, metal ion binding, structural constituent of ribosome

  28. [37]

    hydrolase activity, hydrolyzing O-glycosyl compounds

  29. [38]

    Retrieval result for query protein 8S5K Ground-truth Caption:symporter activity Ground-truth Ranking in Retrieved Result:1/100 Top-5 Retrieved Captions:

    NAD+-protein-arginine ADP-ribosyltransferase activity, nu- cleotide binding Figure 6. Retrieval result for query protein 8S5K Ground-truth Caption:symporter activity Ground-truth Ranking in Retrieved Result:1/100 Top-5 Retrieved Captions:

  30. [39]

    mitochondrial promoter sequence-specific DNA binding, mitochondrial transcription factor activity

  31. [40]

    monoatomic cation transmembrane transporter activity, metal ion binding

  32. [41]

    RNA polymerase II cis-regulatory region sequence-specific DNA binding, mediator complex binding Figure 7. Retrieval result for query protein 6XWR 7 Ground-truth Caption:endonuclease activity, RNA binding, RNA-dependent RNA polymerase activity, metal ion binding, nucleotide bin...

  33. [42]

    endonuclease activity, RNA binding, RNA-dependent RNA polymerase activity, metal ion binding, nucleotide binding

  34. [43]

    peptidase activity, proteasome binding, protein tag activity, hydrolase activity, acting on carbon-nitrogen (but not peptide) bonds, in linear amides, metal ion binding, ATP binding

  35. [44]

    zinc ion binding, ribonucleoside binding, magnesium ion binding, protein dimerization activity, DNA binding

  36. [45]

    proton-transporting ATP synthase activity, rotational mech- anism, proton-transporting ATPase activity, rotational mech- anism, ATP hydrolysis activity, ATP binding, lipid binding, ADP binding

  37. [46]

    aminopeptidase activity, serine-type peptidase activity, dipeptidyl-peptidase activity Figure 8. Retrieval result for query protein EMD-12348 Ground-truth Caption:structural molecule activity, identical protein binding Corresponding PDB Entry:6E5P Ground-truth Ranking in Retri...

  38. [47]

    RNA endonuclease activity, DNA binding

  39. [48]

    RNA endonuclease activity, RNA binding

  40. [49]

    retinol binding, chemokine activity, complement binding, endopeptidase inhibitor activity, protein-containing complex binding

  41. [50]

    oxalate decarboxylase activity, metal ion binding

  42. [51]

    metal ion binding Figure 9. Retrieval result for query protein EMD-8981 Ground-truth Caption:identical protein binding, receptor ligand activity, structural constituent of virion, host cell sur- face receptor binding Corresponding PDB Entry:7JVC Ground-truth Ranking in Retriev...

  43. [52]

    identical protein binding, receptor ligand activity, struc- tural constituent of virion, host cell surface receptor bind- ing

  44. [53]

    glycine-gated chloride ion channel activity, transmembrane signaling receptor activity, metal ion binding, extracellularly glycine-gated chloride channel activity, glycine binding

  45. [54]

    maltose binding, BH3 domain binding, channel activity, pro- tein heterodimerization activity, carbohydrate transmembrane transporter activity, protein transmembrane transporter activity

  46. [55]

    hydrolase activity, four-way junction helicase activity, DNA binding, ATP binding

  47. [56]

    toxin activity, calcium channel regulator activity Figure 10. Retrieval result for query protein EMD-22506 Ground-truth Caption:molecular condensate scaffold activity, GTP binding, double-stranded DNA binding, phosphatidylinositol-4,5-bisphosphate binding, poly-ADP-D- ribose m...

  48. [57]

    single-stranded DNA binding

  49. [58]

    citrate (Si)-synthase activity

  50. [59]

    phosphoenolpyruvate carboxylase activity

  51. [60]

    transcription factor binding, DNA binding, metal ion bind- ing

  52. [61]

    Retrieval result for query protein EMD-22206 8

    serine-type endopeptidase inhibitor activity Figure 11. Retrieval result for query protein EMD-22206 8

  53. [2022]

    1 5 Aligning Proteins and Language: A Foundation Model for Protein Retrieval Supplementary Material

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.