Pith. sign in

REVIEW 2 major objections 5 minor 44 references

Joint Embedding Predictive Architecture for self-supervised pretraining on polymer molecular graphs

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims JEPA pretraining on polymer graphs improves property prediction when labels are scarce, lifting electron-affinity R2 from 0.46 to 0.67 at 0.4% finetune data.

desk verdict Solid applied JEPA-for-polymers paper with honest comparisons; the headline gain is plausible but the no-pretraining baseline needs a matched training budget before the exact R2 jump can be taken at face value. read the letter →

arxiv 2506.18194 v1 pith:ZDCBPE2I submitted 2025-06-22 cs.LG

Francesco Piccoli , Gabriel Vogel , Jana M. Weber This is my paper · ORCID
classification cs.LG
keywords self-supervisedlearningJEPApolymerpropertypredictionmoleculargraphsstochastictransferlabel-scarcegraphneuralnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a Joint Embedding Predictive Architecture (JEPA) can learn polymer representations from unlabeled molecular graphs that help when labels for downstream properties are scarce. The answer it argues for is yes: pretraining a graph encoder by predicting the embedding of a small target subgraph from a larger context subgraph improves downstream performance on every dataset tested. The gain is concentrated in low-data regimes; with 0.4% of the conjugated-copolymer finetuning data, electron-affinity prediction $R^2$ rises from $0.46\pm0.15$ without pretraining to $0.67\pm0.01$ with the best configuration. On the chemically different diblock phase-classification task, the pretrained model improves AUPRC by 0.02 to 0.1. A handcrafted-fingerprint random forest can still beat the pretrained model in the most extreme low-label cases, which the paper interprets as evidence that the method's value is graph-native transfer learning rather than a universal accuracy advantage.

What carries the argument

The machinery is the subgraph-prediction objective in embedding space. Its central object is the pair (context, target): a larger context subgraph of the polymer graph and one or more smaller disjoint target subgraphs, created dynamically at each epoch by random-walk subgraphing, with motif-based and METIS partitioning as comparators. The context encoder produces a pooled embedding of the context; the target encoder, applied to the full graph, produces pooled embeddings of the target-subgraph nodes; and an MLP predictor conditioned by random-walk positional encodings maps the context embedding to each target embedding. The L2 loss between predicted and true target embeddings is what forces the encoder to keep chemically and structurally useful information in a compact representation, which is why the downstream head can be trained with few labels. An optional pseudolabel branch predicts polymer molecular weight from the full-graph embedding during pretraining. The underlying encoder is a weighted directed message-passing neural network (wD-MPNN) using node-centred message passing, which the paper verifies matches edge-centred performance.

What would settle it

Run the same wD-MPNN architecture from scratch on the electron-affinity task with the same hyperparameter search, epoch budget, early stopping, and repeated random seeds that the pretrained model receives; if the $R^2$ gap at 0.4% and 0.8% labeled data disappears or reverses under matched budgets, the claimed pretraining benefit is not demonstrated.

Watch

Extended reading notes

Core claim

The paper's central claim is that JEPA-style self-supervised pretraining transfers to polymer property prediction: the same encoder architecture fine-tuned after pretraining outperforms the same architecture trained from scratch, with the largest gains exactly where labels are rarest. The mechanism is a prediction task in embedding space. A polymer graph represented as a stochastic graph with weighted edges for monomer connection probabilities is partitioned into a context subgraph and one or more target subgraphs. A context encoder pools node embeddings from the context subgraph; a target encoder pools embeddings of target-subgraph nodes from the full graph; and a predictor MLP, conditioned on random-walk structural positional encodings, reconstructs the target embeddings from the context embedding. Pretraining minimizes the average L2 distance between predicted and true target embeddings, optionally together with a molecular-weight pseudolabel objective. After pretraining, the target encoder plus a prediction head is fine-tuned end-to-end. The paper further reports that random-walk subgraphing with a 60% context, a 10% target, and a single target gives the best downstream results, and that pretraining benefits persist when the finetuning data comes from a different polymer chemical space.

Load-bearing premise

The results assume that the from-scratch model used as the no-pretraining baseline is trained with the same hyperparameters, number of epochs, and early stopping as the pretrained-then-finetuned model; if the baseline is undertuned, part of the reported gain could come from optimization effort rather than from pretraining.

Editorial extensions

If this is right

  • Pretraining helps most at small label fractions: in the conjugated-copolymer electron-affinity task the $R^2$ gain is substantial at 0.4% to 0.8% data and largely plateaus by roughly 8% labeled data.
  • The pretrained encoder transfers across polymer chemical spaces: on diblock phase classification it improves AUPRC by 0.02 to 0.1 across all tested labeled-data fractions, including high data fractions.
  • Embedding-space JEPA pretraining and input-space node/edge masking give comparable downstream accuracy, with JEPA slightly better at very low label counts and input-space SSL slightly better with more labels.
  • The molecular-weight pseudolabel objective helps both pretraining strategies, but its contribution is smaller for JEPA, suggesting the embedding objective already captures some of that signal.
  • Subgraph ablation gives practical guidance: a random-walk subgraphing algorithm, a context covering about 60% of the graph, a target of about 10%, and a single target per sample yielded the best electron-affinity $R^2$ at 0.4% finetune data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: if the benefit survives larger and more diverse unlabeled polymer corpora, JEPA pretraining could become a graph-based polymer foundation model, providing a single encoder that fine-tunes to many properties without task-specific fingerprint engineering.
  • Inference: the finding that deterministic, chemically meaningful motif subgraphs hurt slightly suggests a testable scaling law: downstream gains should track subgraph diversity per epoch, so random-walk sampling on a broad corpus should dominate deterministic partitioning when pretraining data grows.
  • Inference: because JEPA predicts in embedding space, the same context-and-target objective is a natural fit for multimodal polymer data, such as pairing molecular graphs with text or simulation-derived descriptors; the paper hints at this direction but does not test it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes a Joint Embedding Predictive Architecture (JEPA) for self-supervised pretraining on stochastic polymer molecular graphs. The model pretrains a node-centred wD-MPNN encoder on 40% of a conjugated copolymer dataset (17,186 polymers) by predicting the embedding of a target subgraph from a context subgraph, optionally with an auxiliary molecular-weight prediction task. The authors then finetune the target encoder on two downstream tasks: electron affinity (EA) prediction on the same conjugated copolymer space and diblock copolymer phase classification on a different polymer dataset (transfer learning). They compare against a no-pretraining wD-MPNN baseline, an input-space SSL baseline (Gao et al.), and a random forest model, and present ablations of subgraphing algorithm, context/target sizes, and number of targets. The paper reports consistent improvements over no pretraining at low label fractions, and transfer gains across polymer spaces.

Significance. If the reported effect holds, the paper provides a useful demonstration that embedding-space SSL (JEPA) transfers to polymer graphs, a domain with scarce labeled data, and shows that the pretraining benefit is most pronounced in low-data regimes. The strengths include a publicly available code/data repository, an honest comparison showing that the random forest baseline outperforms the pretrained model in several scenarios, and a set of subgraphing guidelines. The significance is moderate: the method is an application of a known architecture to a new domain, the improvement over the no-pretraining baseline is the main result, and the comparison to the input-space SSL baseline is favorable but not dramatic. The transfer-learning result across different polymer datasets is the most novel empirical contribution.

major comments (2)
  1. [Sections 3.1-3.5, Tables 1-4] The 'No pretraining' baseline, which reports R2 = 0.46±0.15 at 0.4% finetune data, is never described: no epoch count, learning-rate schedule, early-stopping rule, or hyperparameter search is reported, and there is no statement that it uses the same optimization budget as the pretrained model. If this baseline is undertrained or tuned less carefully, part of the observed gain from pretraining (e.g., 0.46 to 0.67 in Table 3) may be an optimization gap rather than an effect of JEPA. Please specify the exact training protocol for the no-pretraining baseline and, ideally, tune it with the same budget as the pretrained model or demonstrate that the comparison is matched.
  2. [Section 3.5, Tables 1-4] The subgraphing configuration (random-walk algorithm, 60% context size, 10% target size, one target) is selected by finetuning on 0.4% of the EA data, which is precisely the setting used for the headline result in Figure 5 and Table 3. This means the reported R2 = 0.67±0.01 is a number selected on the evaluation scenario, which can inflate the apparent benefit of pretraining. Please use a separate validation split for hyperparameter selection and report the final model's performance on a test set that was not used in the ablation.
minor comments (5)
  1. [Abstract] The phrase 'achieving improvements across all tested datasets' should be qualified as 'relative to the no-pretraining baseline', because Section 3.4 shows that a random forest model outperforms the pretrained model in very low-data EA scenarios and across all diblock scenarios.
  2. [Section 2.2.2, Eq. (1)] The notation for the positional encoding term π̃ i ṭ is unclear: the text says 'linearly transformed target subgraph positional token', but the formula suggests a product of a token with a matrix, and the dimensions and source of π̃ i are not defined.
  3. [Section 3.3] When comparing with the Gao et al. input-space SSL method, please state whether the no-pretraining baseline and the Gao et al. model are trained with the same number of epochs, early-stopping rule, and learning-rate schedule; the sentence about 'three layers and a hidden dimension of 300' fixes only the architecture size.
  4. [Appendix B] The node-centred versus edge-centred wD-MPNN comparison is only reported for the 80% training scenario; since the node-centred variant is used throughout the paper, an additional check at a low-data scenario (e.g., 4%) would increase confidence that the architectural modification does not interact with the pretraining benefit.
  5. [Tables 1-4] The identical 'No pretraining' row (0.46±0.15) appears in all four ablation tables; please state explicitly whether this is the same baseline run reused across tables or re-evaluated for each table, and report the number of random splits and seeds for all entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the JEPA pretraining evaluation is self-contained and benchmarked against external baselines.

full rationale

The paper's central claim—that JEPA-based self-supervised pretraining on stochastic polymer graphs improves downstream prediction under label scarcity—is an empirical result evaluated on held-out test splits against external benchmarks ([1], [34]) and a from-scratch wD-MPNN baseline. The JEPA objective in Eq. (1) predicts target embeddings from context embeddings and is trained with an MSE loss; the downstream R2 and AUPRC metrics are not defined in terms of that loss or in terms of the pretraining inputs, so the evaluation is not circular by construction. The paper does adopt the molecular-weight pseudolabel idea from [15], which includes co-author J. M. Weber, but that citation is used as a comparative baseline and as a secondary objective, not as the justification for the central claim; the pseudolabel is an externally defined quantity and is not the target of the downstream evaluation. The comparison against input-space SSL [15] and random forest models [1,34] provides independent controls. Two review concerns—(i) the no-pretraining baseline's optimization budget and hyperparameters are never specified, and (ii) subgraphing hyperparameters in Section 3.5 are selected using test R2 on the same 0.4% scenario that underlies the headline gain—could affect the magnitude or attribution of the reported improvements, but neither is a circular derivation, a fitted parameter renamed as a prediction, or a load-bearing self-citation chain. Under the rubric, these are correctness and reporting risks, not circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a handful of model hyperparameters chosen by ablation on the primary task, plus standard domain assumptions about the polymer graph representation and the transferability of self-supervised representations. No new physical entities or exotic free parameters are introduced.

free parameters (6)
  • context subgraph size = 60% of the polymer graph
    Selected via ablation in Section 3.5, Table 1, as the best context size for the EA prediction task.
  • target subgraph size = 10% of the polymer graph
    Selected via ablation in Section 3.5, Table 2, as the best target size for the EA prediction task.
  • number of target subgraphs = 1
    Selected via ablation in Section 3.5, Table 3, as the best number of targets for the EA prediction task.
  • subgraphing algorithm = random walk
    Selected via ablation in Section 3.5, Table 4, because random walk gave the best and most stable performance.
  • hidden dimension = 300
    Set for both the proposed method and the comparison method in Section 3.3 to ensure a fair comparison.
  • number of GNN layers = 3
    Set for both the proposed method and the comparison method in Section 3.3 to ensure a fair comparison.
assumptions (4)
  • domain assumption The stochastic polymer graph representation from reference [1] faithfully captures the polymer structure relevant for property prediction.
    The entire method operates on this graph representation, including weighted stochastic edges, and the downstream tasks use the same representation.
  • domain assumption The molecular weight pseudolabel is computed exactly from monomer stoichiometry and weights, and is a useful auxiliary training signal.
    The pseudolabel objective in Section 2.4 is defined as the weighted sum of monomer molecular weights and is used to regularize pretraining.
  • domain assumption The diblock copolymer phase labels from reference [34] are reliable and representative of the phase behavior prediction task.
    The transfer learning evaluation in Section 3.2 uses this dataset as the downstream benchmark without further validation of the labels.
  • domain assumption Self-supervised pretraining on unlabeled polymer graphs transfers to downstream tasks through fine-tuning.
    This is the central learning-theoretic assumption of the paper, shared with the wider SSL literature, and is not separately justified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Joint Embedding Predictive Architecture for self-supervised pretraining on polymer molecular graphs." pith.science (2026). https://pith.science/paper/ZDCBPE2I

@misc{pith2026250618194,
  author       = {Pith},
  title        = {Pith review of: Joint Embedding Predictive Architecture for self-supervised pretraining on polymer molecular graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZDCBPE2I}},
  note         = {Machine review of arXiv:2506.18194}
}
read the original abstract

Recent advances in machine learning (ML) have shown promise in accelerating the discovery of polymers with desired properties by aiding in tasks such as virtual screening via property prediction. However, progress in polymer ML is hampered by the scarcity of high-quality labeled datasets, which are necessary for training supervised ML models. In this work, we study the use of the very recent 'Joint Embedding Predictive Architecture' (JEPA), a type of architecture for self-supervised learning (SSL), on polymer molecular graphs to understand whether pretraining with the proposed SSL strategy improves downstream performance when labeled data is scarce. Our results indicate that JEPA-based self-supervised pretraining on polymer graphs enhances downstream performance, particularly when labeled data is very scarce, achieving improvements across all tested datasets.

Figures

Figures reproduced from arXiv: 2506.18194 by the authors.

Figure 1
Figure 1. Polymer graph representation as introduced in [ [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) Conjugated copolymer photocatalyst dataset [ [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The polymer-JEPA model architecture. The model learns to reconstruct [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Pretraining with the additional pseudolabel objective. In the left, faded [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Effectiveness of our pretraining strategy for different finetune dataset [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Effectiveness of our pretraining strategy for transfer learning and [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Comparison between our pretraining strategy and the best performing [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: The effect of using an additional pseudolabel objective in input space [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: (a) Comparison between our pretraining-finetuning strategy and a [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 27 canonical work pages

  1. [1]

    Matteo Aldeghi and Connor W. Coley. A graph representation of molecular ensembles for polymer property prediction.Chemical Science, 13(35):10486– 10498, 2022. doi: 10.1039/D2SC02839E

    work page doi:10.1039/d2sc02839e 2022

  2. [2]

    Polymer informatics with multi- task learning.Patterns, 2(4):100238, April 2021

    Christopher Kuenneth, Arunkumar Chitteth Rajan, Huan Tran, Lihua Chen, Chiho Kim, and Rampi Ramprasad. Polymer informatics with multi- task learning.Patterns, 2(4):100238, April 2021. ISSN 26663899. doi: 10.1016/j.patter.2021.100238

    arXiv 2021

  3. [3]

    polybert: A chemical language model to enable fully machine-driven ultrafast polymer informatics

    Christopher Kuenneth and Rampi Ramprasad. polybert: A chemical language model to enable fully machine-driven ultrafast polymer informatics. Nature Communications, 14(1):4099, July 2023. ISSN 2041-1723. doi: 10.1038/s41467-023-39868-6

    doi:10.1038/s41467-023-39868-6 2023

  4. [4]

    Mulder, Shadi Houshyar, and Tu C

    Yuankai Zhao, Roger J. Mulder, Shadi Houshyar, and Tu C. Le. A re- view on the application of molecular descriptors and machine learning in polymer design.Polymer Chemistry, 14(29):3325–3346, 2023. doi: 10.1039/D3PY00395G

    work page doi:10.1039/d3py00395g 2023

  5. [5]

    Data-driven algorithms for inverse design of polymers.Soft Matter, 17(33):7607–7622, August 2021

    Kianoosh Sattari, Yunchao Xie, and Jian Lin. Data-driven algorithms for inverse design of polymers.Soft Matter, 17(33):7607–7622, August 2021. ISSN 1744-6848. doi: 10.1039/D1SM00725D

    doi:10.1039/d1sm00725d 2021

  6. [6]

    Martin and Debra J

    Tyler B. Martin and Debra J. Audus. Emerging trends in machine learning: A polymer perspective.ACS Polymers Au, 3(3):239–258, June 2023. doi: 10.1021/acspolymersau.2c00053

    work page doi:10.1021/acspolymersau.2c00053 2023

  7. [7]

    The rise of machine learning in polymer discovery.Advanced Intelligent Systems, 5(4):2200243, 2023

    Cheng Yan and Guoqiang Li. The rise of machine learning in polymer discovery.Advanced Intelligent Systems, 5(4):2200243, 2023. ISSN 2640-

    work page 2023

  8. [8]

    Polymer informatics: Current status and critical next steps.Materials Science and Engineering: R: Reports, 144:100595, April 2021

    Lihua Chen, Ghanshyam Pilania, Rohit Batra, Tran Doan Huan, Chiho Kim, Christopher Kuenneth, and Rampi Ramprasad. Polymer informatics: Current status and critical next steps.Materials Science and Engineering: R: Reports, 144:100595, April 2021. ISSN 0927-796X. doi: 10.1016/j.mser. 2020.100595

    work page arXiv 2021

Show all 44 references
  1. [9]

    Graph neural networks for materials science and chemistry.Communications Materials, 3(1):1–18, November 2022

    Patrick Reiser, Marlen Neubert, Andr´ e Eberhard, Luca Torresi, Chen Zhou, Chen Shao, Houssam Metni, Clint van Hoesel, Henrik Schopmans, Timo Sommer, and Pascal Friederich. Graph neural networks for materials science and chemistry.Communications Materials, 3(1):1–18, November ...

    2022 doi

  2. [10]

    Patel, Carlos H

    Roshan A. Patel, Carlos H. Borca, and Michael A. Webb. Featurization strategies for polymer sequence or composition design by machine learning. Molecular Systems Design & Engineering, 7(6):661–676, 2022. doi: 10.1039/ D1ME00160D. 19

    2022

  3. [11]

    Predicting materials properties with little data using shotgun transfer learning.ACS Central Science, 5(10):1717–1730, October 2019

    Hironao Yamada, Chang Liu, Stephen Wu, Yukinori Koyama, Shenghong Ju, Junichiro Shiomi, Junko Morikawa, and Ryo Yoshida. Predicting materials properties with little data using shotgun transfer learning.ACS Central Science, 5(10):1717–1730, October 2019. ISSN 2374-7943. doi: 10...

    2019 doi

  4. [12]

    Naskar, and John Gounley

    Pei Zhang, Logan Kearney, Debsindhu Bhowmik, Zachary Fox, Amit K. Naskar, and John Gounley. Transferring a molecular foundation model for polymer property predictions, October 2023

    2023

  5. [13]

    Polymer informatics at scale with multitask graph neural networks

    Rishi Gurnani, Christopher Kuenneth, Aubrey Toland, and Rampi Ram- prasad. Polymer informatics at scale with multitask graph neural networks. Chemistry of Materials, 35(4):1560–1567, February 2023. ISSN 0897-4756. doi: 10.1021/acs.chemmater.2c02991

    2023 doi

  6. [14]

    McCarver, Saitheeraj Thatigotla, Brendan P

    Owen Queen, Gavin A. McCarver, Saitheeraj Thatigotla, Brendan P. Abolins, Cameron L. Brown, Vasileios Maroulas, and Konstantinos D. Vogiatzis. Polymer graph neural networks for multitask property learning. npj Computational Materials, 9(1):1–10, May 2023. ISSN 2057-3960. doi: ...

    2023 doi

  7. [15]

    Self-supervised graph neural networks for polymer property prediction

    Qinghe Gao, Tammo Dukker, Artur M Schweidtmann, and Jana M Weber. Self-supervised graph neural networks for polymer property prediction. Molecular Systems Design & Engineering, 2024

    2024

  8. [16]

    Transpolymer: A transformer-based language model for polymer property predictions.npj Computational Materials, 9(1):1–14, April 2023

    Changwen Xu, Yuyang Wang, and Amir Barati Farimani. Transpolymer: A transformer-based language model for polymer property predictions.npj Computational Materials, 9(1):1–14, April 2023. ISSN 2057-3960. doi: 10.1038/s41524-023-01016-5

    2023 doi

  9. [17]

    Polycl: con- trastive learning for polymer representation learning via explicit and implicit augmentations.Digital Discovery, 2025

    Jiajun Zhou, Yijie Yang, Austin M Mroz, and Kim E Jelfs. Polycl: con- trastive learning for polymer representation learning via explicit and implicit augmentations.Digital Discovery, 2025

    2025

  10. [18]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv´ e J´ egou, Julien Mairal, Pi- otr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. InProceedings of the IEEE/CVF international confer- ence on computer vision, pages 9650–9660, 2021

    2021

  11. [19]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, pages 1597–1607, Atlanta, Georgia, USA, November 2020. PMLR

    2020

  12. [20]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022. 20

    2022

  13. [21]

    A survey on self-supervised representation learning, August 2023

    Tobias Uelwer, Jan Robine, Stefan Sylvius Wagner, Marc H¨ oftmann, Eric Upschulte, Sebastian Konietzny, Maike Behrendt, and Stefan Harmeling. A survey on self-supervised representation learning, August 2023

    2023

  14. [22]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  15. [23]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018

    2018

  16. [24]

    Self-supervised learning on graphs: Deep insights and new direction, June 2020

    Wei Jin, Tyler Derr, Haochen Liu, Yiqi Wang, Suhang Wang, Zitao Liu, and Jiliang Tang. Self-supervised learning on graphs: Deep insights and new direction, June 2020

    2020

  17. [25]

    Yixin Liu, Ming Jin, Shirui Pan, Chuan Zhou, Yu Zheng, Feng Xia, and Philip S. Yu. Graph self-supervised learning: A survey.IEEE Transactions on Knowledge and Data Engineering, 35(6):5879–5900, June 2023. ISSN 1558-2191. doi: 10.1109/TKDE.2022.3172903

    2023

  18. [26]

    Self-supervised learning of graph neural networks: A unified review

    Yaochen Xie, Zhao Xu, Jingtun Zhang, Zhengyang Wang, and Shuiwang Ji. Self-supervised learning of graph neural networks: A unified review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2): 2412–2429, February 2023. ISSN 0162-8828, 2160-9292, 1939-3539. doi...

    2023

  19. [27]

    Mocl: Data-driven molecular fingerprint via knowledge-aware contrastive learning from molecular graph

    Mengying Sun, Jing Xing, Huijun Wang, Bin Chen, and Jiayu Zhou. Mocl: Data-driven molecular fingerprint via knowledge-aware contrastive learning from molecular graph. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 3585–3594, Virtual...

    2021

  20. [28]

    Motif-based graph self-supervised learning for molecular property prediction

    Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Chee-Kong Lee. Motif-based graph self-supervised learning for molecular property prediction. Advances in Neural Information Processing Systems, 34:15870–15882, 2021

    2021

  21. [29]

    Self-supervised graph transformer on large- scale molecular data.Advances in neural information processing systems, 33:12559–12571, 2020

    Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large- scale molecular data.Advances in neural information processing systems, 33:12559–12571, 2020

    2020

  22. [30]

    Strategies for pre-training graph neural networks

    Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. arXiv preprint arXiv:1905.12265, 2019

    1905 arXiv

  23. [31]

    A path towards autonomous machine intelligence version 0.9.2, 2022-06-27

    Yann LeCun. A path towards autonomous machine intelligence version 0.9.2, 2022-06-27. 21

    2022

  24. [32]

    Graph-level repre- sentation learning with joint-embedding predictive architectures, September 2023

    Geri Skenderi, Hang Li, Jiliang Tang, and Marco Cristani. Graph-level repre- sentation learning with joint-embedding predictive architectures, September 2023

    2023

  25. [33]

    Yang Bai, Liam Wilbraham, Benjamin J Slater, Martijn A Zwijnenburg, Reiner Sebastian Sprick, and Andrew I Cooper. Accelerated discovery of organic polymer photocatalysts for hydrogen evolution from water through the integration of experiment and theory.Journal of the American ...

    2019

  26. [34]

    Random forest predictor for diblock copolymer phase behavior.ACS Macro Letters, 10(11):1339–1345, 2021

    Akash Arora, Tzyy-Shyang Lin, Nathan J Rebello, Sarah HM Av-Ron, Hidenobu Mochigase, and Bradley D Olsen. Random forest predictor for diblock copolymer phase behavior.ACS Macro Letters, 10(11):1339–1345, 2021

    2021

  27. [35]

    A fast and high quality multilevel scheme for partitioning irregular graphs.SIAM Journal on scientific Computing, 20(1):359–392, 1998

    George Karypis and Vipin Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs.SIAM Journal on scientific Computing, 20(1):359–392, 1998

    1998

  28. [36]

    On the art of compiling and using ’drug-like’ chemical fragment spaces.ChemMedChem, 3(10):1503–1507, 2008

    J¨ org Degen, Christof Wegscheid-Gerlach, Andrea Zaliani, and Matthias Rarey. On the art of compiling and using ’drug-like’ chemical fragment spaces.ChemMedChem, 3(10):1503–1507, 2008. ISSN 1860-7187. doi: 10.1002/cmdc.200800178

    2008 doi

  29. [37]

    r-brics–a revised brics module that breaks ring structures and carbon chains.ChemMedChem, page e202300202

    Leili Zhang, Vasumitra Rao, and Wendy Cornell. r-brics–a revised brics module that breaks ring structures and carbon chains.ChemMedChem, page e202300202

  30. [38]

    Graph neural networks with learnable structural and positional representations.arXiv preprint arXiv:2110.07875, 2021

    Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Graph neural networks with learnable structural and positional representations.arXiv preprint arXiv:2110.07875, 2021

    2021 arXiv

  31. [39]

    A generalization of vit/mlp-mixer to graphs

    Xiaoxin He, Bryan Hooi, Thomas Laurent, Adam Perold, Yann Lecun, and Xavier Bresson. A generalization of vit/mlp-mixer to graphs. In Proceedings of the 40th International Conference on Machine Learning, pages 12724–12745, Honolulu, Hawaii, USA, July 2023. PMLR

    2023

  32. [40]

    Extended-connectivity fingerprints.Jour- nal of chemical information and modeling, 50(5):742–754, 2010

    David Rogers and Mathew Hahn. Extended-connectivity fingerprints.Jour- nal of chemical information and modeling, 50(5):742–754, 2010

    2010

  33. [41]

    Rdkit: Open-source cheminformatics, 2006

    Greg Landrum et al. Rdkit: Open-source cheminformatics, 2006

    2006

  34. [42]

    Self-supervised learning from images with a joint-embedding predictive architecture, April 2023

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture, April 2023

    2023

  35. [43]

    Analyzing learned molecular representations for property prediction.Journal of Chemical Information and Modeling, 59(8):3370–3388, August 2019

    Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angel Guzman-Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, Andrew Palmer, Volker Settels, Tommi Jaakkola, Klavs Jensen, and 22 Regina Barzilay. Analyzing learned molecular representations for pr...

    2019 doi

  36. [4567]

    doi: 10.1002/aisy.202200243

    doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.