REVIEW 3 major objections 5 minor 36 references
Sparse hierarchical representation learning on molecular graphs
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper introduces edge-aware sparse pooling layers for molecular graphs and reports benchmark improvements on three of four drug-discovery datasets, with best-in-class results on the fourth.
desk verdict Genuinely new edge-feature pooling layers, but the benchmark claims rest on unguarded model selection and uncontrolled baseline comparisons. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the top-k sparse pooling layer extended to edge-featured graphs. Each node is scored by a learned projection vector $p^{(k)}$; the $K$ highest-scoring nodes are kept, and the kept nodes are rewired by contracting dropped nodes into effective edges, so the graph stays sparse and the selection stays differentiable through a gating $a_i^{(k)} = \tilde a_i^{(k)} \tanh(y_i^{(k)})$. The paper's contribution is in how the new effective edge features are computed: SimplePooling sums the edge-feature vectors along all contracted paths, while CoarseGrainPooling passes the dropped node's features, the adjoining edge features, and the kept edges' features through fully connected networks. This preserves bond-type information at every level of the hierarchy, and the accompanying dual-message convolution ensures edge features are updated before each pooling step.
What would settle it
Rerun every baseline model from Table 1 under the exact protocol used here (same scaffold splits, same 80/10/10 fractions, same hyperband search budget, same number of repeats) and re-measure the test metrics; if the reported margins shrink to within one standard deviation, the outperformance claim collapses. As a narrower check, train the no-pooling and pooled variants with identical hyperparameters rather than retuning per variant to see whether the pooling layer, and not a better hyperparameter draw, is the cause of the gain.
Extended reading notes
Core claim
The central claim is that edge-featured molecular graphs can be hierarchically coarsened without discarding bond information, and that doing so improves both accuracy and speed. The paper defines two pooling layers for this purpose: SimplePooling, which sums the edge-feature vectors along every path contracted when a node is dropped (merging overlaps), and CoarseGrainPooling, which uses two fully connected networks to compute effective edge features—one for new edges that pass through dropped nodes and one for surviving edges. Combined with a dual-message graph-convolutional layer that updates node features from neighbour node features plus edge features, and updates edge features from the two incident node features, the models outperform previously reported numbers on ESOL, lipophilicity, and BBBP, and reach the best ROC-AUC the paper lists on HIV (0.840 ± 0.019 with simple pooling at keep ratio 0.6). Pooling improves performance on BBBP, HIV, and lipophilicity, leaves ESOL stable, and cuts HIV training time by up to 70%.
Load-bearing premise
The central comparison assumes the previously published baseline numbers in Table 1 were produced under the same protocol—same 80/10/10 scaffold split, same evaluation repeats, and comparable hyperparameter tuning; if the baselines were tuned less or evaluated differently, the reported margins could be artifacts of the comparison rather than properties of the new pooling layers.
Editorial extensions
If this is right
- Bond information survives coarse-graining: even after several pooling steps, effective edges encode the bonds that were contracted, so predictions remain sensitive to chemistry at all scales.
- On the paper's protocol, the best results are an RMSE of 0.410 ± 0.023 on ESOL, an RMSE of 0.517 ± 0.005 on lipophilicity, a ROC-AUC of 0.869 ± 0.010 on BBBP, and a ROC-AUC of 0.840 ± 0.019 on HIV; these are the values the authors use to claim outperformance and state-of-the-art status.
- Pooling also pays in compute: simple pooling on HIV shortens elapsed training time by 16% at keep ratio 0.9 up to 70% at keep ratio 0.5, while memory requirements drop because the graph shrinks.
- The authors expect the benefit to grow for larger graphs, such as protein structures, where a sequence of pooling steps can produce a deeper hierarchy than small drug-like molecules allow.
Reading between the lines
- Editorial inference: the keep ratio is fixed per task and only the other hyperparameters are tuned, so treating the keep ratio itself as tunable could change which of the two pooling layers wins on a given dataset.
- Editorial inference: the two edge-aggregation schemes are two points on a spectrum; a natural testable extension is attention-based or message-passing aggregation over contracted paths, which the paper does not explore.
- Editorial inference: if the speedup is driven by graph size, the 70% figure on small drug-like molecules likely undersells the benefit for larger graphs; the supplementary CEP-2017 results, while not central, are consistent with pooling compressing molecules with little accuracy loss.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes sparse hierarchical graph pooling layers for graphs with edge features, building on the Graph U-Net idea of selecting top-scoring nodes and adding edge-feature reconstruction when dropped nodes are removed. Two pooling variants are introduced: SimplePooling, which sums edge features along connecting paths, and CoarseGrainPooling, which learns effective edge features with neural networks. The architecture uses a dual-message graph-convolutional layer that updates both node and edge features. Experiments are reported on four MoleculeNet datasets (ESOL, Lipophilicity, BBBP, HIV), comparing pooled and unpooled versions of the proposed model and against literature numbers. The paper claims significant outperformance on three datasets, state-of-the-art on the fourth, and training speedups; a supplementary section applies the model to a photovoltaic materials dataset (CEP-2017).
Significance. If the stated results are taken at face value, the contribution is practically relevant: it extends sparse hierarchical pooling to edge-featured molecular graphs, reports speedups from pooling, and introduces a plausible coarse-graining mechanism for chemical bonds. The authors include a no-pooling baseline within their own architecture, which is a useful internal control, and the supplementary material provides additional results on a new dataset. However, the significance of the headline claims is not yet established because the internal pooling comparison is confounded by architecture changes and the literature comparison is uncontrolled. The core ideas are promising and the conceptual motivation is sound, but the empirical evidence as presented does not support the strongest conclusions without revision.
major comments (3)
- [Section 3, Table 4] The pooling-versus-no-pooling comparison is confounded by differing architectures. For each dataset, the NoPooling row uses one specific stack count and channel configuration, while each pooling ratio uses different node/edge channel dimensions and sometimes a different number of stacks (e.g., ESOL NoPooling uses [128,128] with two stacks, while SimplePooling (0.7) uses [512,512,512] with three stacks; BBBP NoPooling uses [128,128] while SimplePooling (0.9) uses [128,128,128]). Consequently, differences in performance between pooling and no pooling may reflect changes in model capacity or depth rather than the pooling mechanism itself. To support the causal claim that pooling improves or preserves performance, the authors should run a controlled comparison that fixes the number of stacks and channel dimensions and varies only the pooling layer and keep ratio.
- [Section 3, Table 1] The pooling keep ratio appears to have been selected after seeing test results, not on the validation set. The text states that hyperband tunes the number of stacks and channel dimensions while the keep ratio is fixed, but it does not state that the best-performing ratio for each dataset was chosen on the validation split. Table 1 reports ten pooling configurations per dataset, and the narrative highlights the best one per dataset (e.g., SimplePooling (0.8) for BBBP, SimplePooling (0.6) for HIV). Without a pre-specified model-selection rule or a multiple-comparison correction, the apparent pooling gains are inflated. This is not merely a statistical nicety: for HIV, SimplePooling (0.6) gives ROC-AUC 0.840±0.019 versus NoPool 0.825±0.008, and a two-sample t-test gives t≈1.6 (p≈0.14), so the reported pooling benefit on HIV is within sampling noise. The authors should either select the ratio on validation data, apply an appropriate correction, or report the selection procedure explicitly.
- [Section 3, Table 1] The comparison to literature benchmarks is uncontrolled. Baseline numbers from Wu et al. and other cited papers were not re-run under the authors' protocol (80/10/10 split, scaffold split for classification, same hyperparameter tuning method), and the paper only states that classification splits follow DeepChem's scaffold split. If the literature baselines used different train/validation/test fractions, different splits, or different tuning budgets, the reported outperformance could be an evaluation artifact. The claim of "significantly outperform[ing] previous benchmarks" is therefore not supported by the evidence as presented. I recommend either re-running key baselines (e.g., GC, Weave, MPNN, or the strongest cited model per dataset) under the authors' exact protocol, or substantially softening the claims and explicitly labeling the comparison as indicative rather than controlled.
minor comments (5)
- [Abstract and Section 3] The word "significantly" is used repeatedly without any statistical test or definition; please either define the significance criterion or replace the word with descriptive language such as "numerically better."
- [Section 2.2, Figure 1 caption] There is a typo: "drop the the ones linked to the dropped nodes" should read "drop the ones linked to the dropped nodes."
- [Section 3, Table 2] The speed-up table reports a single number per ratio without error bars or details on how many runs were measured; adding variance and the hardware/software context would help interpretability.
- [Appendix A.1, Table 3] Some entries in Table 3 have anomalously large standard deviations (e.g., RMSE 0.179±0.097 for ratio 0.7), which suggests possible outliers or implementation instability; please comment on these entries or explain the source of the variance.
- [General] No code or model configuration files are provided, which prevents reproducibility; releasing the implementation and exact hyperparameter search details would strengthen the paper significantly.
Circularity Check
No circularity: the paper's claims are external benchmark evaluations, not derivations from fitted inputs.
full rationale
The paper makes no first-principles derivation that reduces to its inputs: the central claims are empirical performance comparisons on MoleculeNet benchmarks against external baselines and against an internal NoPool control. The pooling architecture is defined independently of the reported results by equations for the dual-message layer (Eqs. 3-6), the projection/gating scoring rule (Eqs. 7-8), and the edge-feature aggregation schemes in Sec. 2.2. Test metrics are reported on held-out 10% test sets, with the validation set used for hyperparameter tuning, so no fitted parameter is renamed as a prediction. The pooling keep ratio is treated as a hyperparameter and reported across several values, and the narrative highlights whichever ratio performs best per dataset; that is a model-selection/statistical-validity concern, not a circular step of the form 'output equals input by construction.' There are no load-bearing self-citations, imported uniqueness theorems, or ansatze smuggled in via citation, and the proposed pooling layers are not a mere renaming of a known result. Thus the paper is self-contained as an empirical evaluation, and no circularity is identified.
Assumptions & free parameters
free parameters (2)
- Pooling keep ratio rho =
0.5 to 0.9, best value selected per dataset
- Network architecture (GCN layers, node/edge channels) =
See Table 4; varies by dataset and pooling type
assumptions (3)
- domain assumption The scaffold split from MoleculeNet/DeepChem is an appropriate evaluation protocol and is comparable to the protocols used for previously published baselines.
- ad hoc to paper The pooling edge-reconstruction heuristic preserves task-relevant information when dropped nodes are replaced by effective edges.
- domain assumption The dual-message message-passing steps are powerful enough to distribute edge information into adjacent node features before the global gather.
Cite this review
Pith. "Pith review of Sparse hierarchical representation learning on molecular graphs." pith.science (2026). https://pith.science/paper/DIOOUVRS
@misc{pith2026190802065,
author = {Pith},
title = {Pith review of: Sparse hierarchical representation learning on molecular graphs},
year = {2026},
howpublished = {\url{https://pith.science/paper/DIOOUVRS}},
note = {Machine review of arXiv:1908.02065}
}
read the original abstract
Architectures for sparse hierarchical representation learning have recently been proposed for graph-structured data, but so far assume the absence of edge features in the graph. We close this gap and propose a method to pool graphs with edge features, inspired by the hierarchical nature of chemistry. In particular, we introduce two types of pooling layers compatible with an edge-feature graph-convolutional architecture and investigate their performance for molecules relevant to drug discovery on a set of two classification and two regression benchmark datasets of MoleculeNet. We find that our models significantly outperform previous benchmarks on three of the datasets and reach state-of-the-art results on the fourth benchmark, with pooling improving performance for three out of four tasks, keeping performance stable on the fourth task, and generally speeding up the training process.
Figures
Reference graph
Works this paper leans on
-
[1]
Han Altae-Tran, Bharath Ramsundar, Aneesh S. Pappu, and Vijay S. Pande. 2017. Low Data Drug Discovery with One-shot Learning. American Chemical Society 3 (2017). http://arxiv.org/abs/1611.03199
arXiv 2017
-
[2]
Garrett B. Goh, Nathan O. Hodas, Charles Siegel, and Abhinav Vishnu. 2017. SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Pre- dicting Chemical Properties. (12 2017). arXiv:https://arxiv.org/abs/1712.02034 https://arxiv.org/abs/1712.02034
arXiv 2017
-
[3]
Goh, Charles Siegel, Abhinav Vishnu, and Nathan Hodas
Garrett B. Goh, Charles Siegel, Abhinav Vishnu, and Nathan Hodas. 2018. Using Rule-Based Labels for Weak Supervised Learning: A ChemNet for Transferable Chemical Property Prediction. 302–310. https://doi.org/10.1145/3219819.3219838
arXiv 2018
-
[4]
Goh, Charles Siegel, Abhinav Vishnu, and Nathan O
Garrett B. Goh, Charles Siegel, Abhinav Vishnu, and Nathan O. Hodas. 2017. ChemNet: A Transferable and Generalizable Deep Neural Network for Small- Molecule Property Prediction. (12 2017). arXiv:https://arxiv.org/abs/1712.02734 https://arxiv.org/abs/1712.02734
arXiv 2017
-
[5]
Goh, Charles Siegel, Abhinav Vishnu, Nathan O
Garrett B. Goh, Charles Siegel, Abhinav Vishnu, Nathan O. Hodas, and Nathan Baker. 2017. Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSARQSPR Models. (06 2017). arXiv:https://arxiv.org/abs/1706.06689 https://arxiv.org/abs/1706.06689
arXiv 2017
-
[6]
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spec- tral Networks and Locally Connected Networks on Graphs. preprint (2013). arXiv:1312.6203 http://arxiv.org/abs/1312.6203
arXiv 2013
-
[7]
CÄČtÄČlina Cangea, Petar VeliÄŊkoviÄĞ, Nikola JovanoviÄĞ, Thomas Kipf, and Pietro LiÚ. 2018. Towards Sparse Hierarchical Graph Classifiers. Workshop on Relational Representation Learning (R2L) at NIPS (2018). arXiv:1811.01287 http://arxiv.org/abs/1811.01287
arXiv 2018
-
[8]
Hanjun Dai, Bo Dai, and Le Song. 2016. Discriminative Embeddings of Latent Variable Models for Structured Data. Proceedings of the International Conference on Machine Learning (ICML) 48 (2016). arXiv:1603.05629 http://arxiv.org/abs/ 1603.05629
work page Pith review arXiv 2016
Show all 36 references
-
[9]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolu- tional Neural Networks on Graphs with Fast Localized Spectral Filtering. Ad- vances in Neural Information Processing Systems (NIPS) 29 (2016). arXiv:1606.09375 http://arxiv.org/abs/1606.09375
2016 arXiv
-
[10]
David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. 2015. Convolutional Networks on Graphs for Learning Molecular Fingerprints. In Advances in Neural Information Processing Systems 28 , C. Cortes, N. D....
2015 arXiv
-
[11]
Feinberg, Debnil Sur, Zhenqin Wu, Brooke E
Evan N. Feinberg, Debnil Sur, Zhenqin Wu, Brooke E. Husic, Huanghao Mai, Yang Li, Saisai Sun, Jianyi Yang, Bharath Ramsundar, and Vijay S. Pande
-
[12]
Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Heinrich Müller. 2018. SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018). arXiv:1711.08920 http://arxiv.org/abs/1711.08920
2018 arXiv
-
[13]
Hongyang Gao, Yongjun Chen, and Shuiwang Ji. 2019. Learning Graph Pool- ing and Hybrid Convolutional Operations for Text Representations. preprint abs/1901.06965 (2019). arXiv:1901.06965 http://arxiv.org/abs/1901.06965
2019 arXiv
-
[14]
Hongyang Gao and Shuiwang Ji. 2019. Graph U-Net. ICLR 2019 Conference Blind Submission (2019). https://openreview.net/forum?id=HJePRoAct7
2019
-
[15]
Schoenholz, Patrick F
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural Message Passing for Quantum Chemistry, In ICML. arxiv:1704.01212. http://arxiv.org/abs/1704.01212v2
2017 arXiv
-
[16]
G.B. Goh, C. Siegel, A. Vishnu, N. Hodas, and N. Baker. 2018. How Much Chem- istry Does a Deep Neural Network Need to Know to Make Accurate Predictions?. In 2018 IEEE Winter Conference on Applications of Computer Vision (W ACV). 1340–
2018
-
[17]
Sánchez-Carrera, Aryeh Gold-Parker, Leslie Vogt, Anna M
Johannes Hachmann, Roberto Olivares-Amaya, Sule Atahan-Evrenk, Carlos Amador-Bedolla, Roel S. Sánchez-Carrera, Aryeh Gold-Parker, Leslie Vogt, Anna M. Brockway, and Alán Aspuru-Guzik. 2011. The Harvard Clean Energy Project: Large-Scale Computational Screening and Design of Org...
2011 doi
-
[18]
Sabrina Jaeger, Simone Fulle, and Samo Turk. 2018. Mol2vec: Unsupervised Machine Learning Approach with Chemical Intuition. Journal of Chemical Information and Modeling 58, 1 (2018), 27–35. https://doi.org/10.1021/acs.jcim. 7b00616 arXiv:https://doi.org/10.1021/acs.jcim.7b0061...
2018 doi
-
[20]
Junying Li, Deng Cai, and Xiaofei He. 2017. Learning Graph-Level Representation for Drug Discovery. CoRR abs/1709.03741 (2017). arXiv:1709.03741 http://arxiv. org/abs/1709.03741
2017 arXiv
-
[21]
Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. Journal of Machine Learning Research 18, 185 (2018), 1–52. http: //jmlr.org/papers/v18/16-558.html
2018
-
[22]
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. 2016. Gated Graph Sequence Neural Networks. International Conference on Learning Repre- sentations (ICLR) (2016). arXiv:1511.05493 http://arxiv.org/abs/1511.05493
2016 arXiv
-
[23]
Lin, Max Tegmark, and David Rolnick
Henry W. Lin, Max Tegmark, and David Rolnick. 2017. Why Does Deep and Cheap Learning Work So Well? Journal of Statistical Physics 168, 6 (jul 2017), 1223–1247. https://doi.org/10.1007/s10955-017-1836-5
2017 doi
-
[24]
Lopez, Benjamin Sanchez-Lengeling, Julio de Goes Soares, and Alán Aspuru-Guzik
Steven A. Lopez, Benjamin Sanchez-Lengeling, Julio de Goes Soares, and Alán Aspuru-Guzik. 2017. Design Principles and Top Non-Fullerene Acceptor Can- didates for Organic Photovoltaics. Joule 1, 4 (dec 2017), 857–870. https: //doi.org/10.1016/j.joule.2017.10.006
2017 doi
-
[25]
Bronstein
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Jan Svoboda, and Michael M. Bronstein. 2017. Geometric deep learning on graphs and mani- folds using mixture model CNNs. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017). arXiv:1611.08402...
2017 arXiv
-
[26]
Tenenbaum, and Daniel L
Damian Mrowca, Chengxu Zhuang, Elias Wang, Nick Haber, Li Fei-Fei, Joshua B. Tenenbaum, and Daniel L. K. Yamins. 2018. Flexible Neural Representation for Physics Prediction. Advances in Neural Information Processing Systems (NIPS) 31 (2018). arXiv:1806.08047 http://arxiv.org/a...
2018 arXiv
-
[27]
Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. 2016. Learning Convolutional Neural Networks for Graphs. Proceedings of the International Conference on Machine Learning (ICML) 48 (2016). arXiv:1605.05273 http://arxiv. org/abs/1605.05273
2016 arXiv
-
[28]
M. C. Scharber, D. MÃijhlbacher, M. Koppe, P. Denk, C. Waldauf, A. J. Heeger, and C. J. Brabec. 2006. Design Rules for Donors in Bulk-Heterojunction Solar Cells—Towards 10 % Energy-Conversion Efficiency. Advanced Materials 18, 6 (mar 2006), 789–794. https://doi.org/10.1002/adma...
2006 doi
-
[29]
Chao Shang, Qinqing Liu, Ko-Shin Chen, Jiangwen Sun, Jin Lu, Jinfeng Yi, and Jinbo Bi. 2018. Edge Attention-based Multi-Relational Graph Convolutional Net- works. arXiv e-prints, Article arXiv:1802.04944 (Feb 2018), arXiv:1802.04944 pages. arXiv:stat.ML/1802.04944 https://arxi...
2018 arXiv
-
[30]
Martin Simonovsky and Nikos Komodakis. 2017. Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 29–38. http://arxiv.org/ abs/1704.02901v3
2017 arXiv
-
[31]
Gregor Urban, Niranjan Subrahmanya, and Pierre Baldi. 2018. Inner and Outer Recursive Neural Networks for Chemoinformatics Applications. Journal of Chemical Information and Modeling 58, 2 (2018), 207–211. https://doi.org/10.1021/ acs.jcim.7b00384 arXiv:https://doi.org/10.1021/...
2018 doi
-
[32]
Robin Winter, Floriane Montanari, Frank NoÃľ, and Djork-ArnÃľ Clevert. 2019. Learning continuous and data-driven molecular descriptors by translating equiv- alent chemical representations. Chem. Sci. 10 (2019), 1692–1701. Issue 6. https://doi.org/10.1039/C8SC04175J
2019 doi
-
[33]
Feinberg, Joseph Gomes, Caleb Ge- niesse, Aneesh S
Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Ge- niesse, Aneesh S. Pappu, Karl Leswing, and Vijay S. Pande. 2018. MoleculeNet: A Benchmark for Molecular Machine Learning. Chemical Science 2 (2018). arXiv:1703.00564 http://arxiv.org/abs/1703.00564
2018 arXiv
-
[34]
Hamilton, and Jure Leskovec
Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, and Jure Leskovec. 2018. Hierarchical Graph Representation Learning with Differentiable Pooling. Advances in Neural Information Processing Systems (NIPS) 31 (2018). arXiv:1806.08804 http://arxiv.org/abs...
2018 arXiv
-
[35]
Shuangjia Zheng, Xin Yan, Yuedong Yang, and Jun Xu. 2019. Identi- fying StructureâĂŞProperty Relationships through SMILES Syntax Analy- sis with Self-Attention Mechanism. Journal of Chemical Information and Modeling 59, 2 (2019), 914–923. https://doi.org/10.1021/acs.jcim.8b008...
2019 doi
-
[1349]
https://doi.org/10.1109/WACV.2018.00151
2018
-
[2018]
ACS Central Sci- ence 4, 11 (2018), 1520–1530
PotentialNet for Molecular Property Prediction. ACS Central Sci- ence 4, 11 (2018), 1520–1530. https://doi.org/10.1021/acscentsci.8b00507 Sparse hierarchical representation learning on molecular graphs DLG ’19, August 04–08, 2019, Anchorage, AK arXiv:https://doi.org/10.1021/ac...
2018 doi
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.