REVIEW 5 major objections 4 minor 33 references
Comparative analysis of computational approaches for predicting Transthyretin (TTR) transcription activators and human dopamine D1 receptor antagonists
T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that binned 13C NMR chemical-shift counts combined with six molecular descriptors let a gradient-boosting model reach 75.8% accuracy on dopamine D1 antagonism, and project to 67.4% for TTR transcription activators.
desk verdict A reproducible QSAR study whose headline TTR result is a speculative extrapolation, not a measured outcome; the D1 result is a modest, likely real improvement. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the augmented feature matrix: each molecule is represented by counts of predicted $^{13}$C NMR peaks in integer chemical-shift bins from 0 to beyond 200 ppm, concatenated with six molecular descriptors to form roughly 227 features. The classifier that carries the main results is a gradient-boosting decision tree with a shallow depth limit, selected after comparing random forest, decision tree, gradient boosting, and support-vector classifiers. The hypothetical TTR result is carried by a scaling procedure in which the accuracy gain from moving the D1 dataset from reduced to full size (an 8.85% relative increase, from 65.7% to 71.5% accuracy) is applied to the TTR model's reduced-data accuracy. Training balances the classes with random oversampling rather than SMOTE, and the reported models are the ones judged not overfit by the training-versus-testing accuracy gap.
What would settle it
Train the same gradient-boosting pipeline on roughly 57,000 TTR samples drawn without the water-solubility sieve and check whether held-out accuracy lands near the projected 67.4%; if it does not, the transfer assumption fails.
Extended reading notes
Core claim
The core claim is that the information in a molecule's predicted $^{13}$C NMR spectrum, encoded as counts of peaks falling into integer chemical-shift subranges, becomes substantially more useful for bioactivity classification when combined with six conventional molecular descriptors: rotatable-bond count, hydrogen-bond acceptor and donor counts, topological polar surface area, predicted octanol-water partition coefficient, and molecular weight. With those features, a gradient-boosting classifier trained on the full dopamine D1 antagonist set achieves 75.8% accuracy, versus 71.5% without the descriptors on the same full set. The same feature set on the TTR transcription-activator screen yields 67.1% accuracy on the available 3,041-sample dataset; extrapolating the sample-size-driven accuracy gain observed for D1 gives a hypothetical 67.4% accuracy if TTR data reached the same volume. The paper also claims that a classifier using only compound and substance identifiers from a public chemical database predicts TTR transcription activation with 81.5% accuracy, providing a rapid screen for activation potential in compounds developed for other targets.
Load-bearing premise
The TTR accuracy figures rest on two linked premises: that the water-solubility-filtered inactive compounds represent TTR-inactive chemistry, and that the sample-size-driven accuracy gain measured on dopamine D1 transfers to TTR; if either is false, the 67.4% projection collapses.
Editorial extensions
If this is right
- Dopamine D1 antagonist screening with this feature set reaches 75.8% accuracy and 84.2% precision on a held-out set, making the augmented 13C-NMR representation a viable low-cost virtual screen for this receptor.
- For transthyretin transcription activation, the small-data model reaches 67.1% accuracy, with a projected 67.4% if a full-size TTR training set becomes available.
- Principal-component dimensionality reduction generally lowers or leaves unchanged the best metrics, so the recommended pipeline uses the full feature matrix without PCA.
- The compound/substance identifier-based TTR model offers a rapid, high-precision flag (81.5% accuracy, 94.6% precision) for whether a compound originally designed for another target might activate TTR transcription.
- The recipe is case-study dependent, so applying it to new bioassays requires re-deriving the optimal estimator and depth rather than assuming the D1-grade performance transfers directly.
Reading between the lines
- The paper's sample-scaling projection assumes the D1 accuracy gain transfers to TTR; a direct test on a large TTR screen would likely show a different gain because the two targets differ in chemistry and assay noise.
- Because the inactive TTR pool was sieved by an unrelated water-solubility assay and then thinned by keeping every eighth shuffled sample, a hidden distribution shift could bias every TTR metric; random subsampling of the original inactive pool would quantify this.
- The identifier-based model's high precision suggests identifiers encode useful structural similarity, but it may be overfitting to assay provenance; a scaffold split or a split by screening batch would reveal its true generalization.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper applies a scikit-learn machine learning pipeline based on 13C NMR features derived from SMILES strings, augmented with six PubChem molecular descriptors, to two bioassay datasets: human dopamine D1 receptor antagonists (AID 504652) and transthyretin (TTR) transcription activators (AID 1117267). The D1 model trained on 25,532 samples and tested on 5,466 samples achieves 75.8% accuracy. Because the TTR dataset has only 3,177 samples, the authors extrapolate a hypothetical 67.4% TTR accuracy by assuming that TTR would benefit from increased training data exactly as D1 did. They also present a separate CID_SID model that uses PubChem compound/substance identifiers as features to predict TTR activation, reporting 81.5% accuracy.
Significance. If the D1 result is reliable, it is a modest incremental improvement over the authors' prior pipeline, showing that three additional molecular descriptors (RBC, HBDC, HDAC, TPSA, XLogP3, MW) can slightly raise accuracy. However, the comparative TTR claim is not a measured result but a conditional projection based on an unjustified transfer of D1's sample-size scaling. The CID_SID model, which treats arbitrary database identifiers as predictive features, is theoretically problematic and its reported accuracy is unlikely to generalize. The paper's contribution is therefore largely limited to the D1 full-data experiment; the broader comparative claims are not supported by the evidence presented.
major comments (5)
- [Abstract and Results and discussion] The abstract reports a 'hypothetical 67.4% Accuracy' for TTR if trained on 25,532 samples, but the Results section states that the optimal TTR GBC model 'achieving accuracy 67.4%... was trained on 3,216 samples and tested on 680 samples.' These are contradictory: the same accuracy cannot be both a hypothetical large-sample projection and an actual small-sample result. The manuscript must clarify which number is measured and which is extrapolated.
- [Methodology (sample-size extrapolation)] The methodology assumes that increasing the TTR training set to 25,532 samples would improve accuracy by the same percentage as observed for D1 (8.85% relative increase). This assumption is not justified, and the paper's own Conclusion states that 'the big difference of the ML models' results for the small dataset of these two case studies implied that the ML approach is a case study dependent.' The hypothetical 61.82% and 67.4% TTR figures therefore have no empirical basis and should be presented only as speculative illustrations, not as comparative results.
- [Methodology (inactive set construction)] The TTR inactive compounds were selected by intersecting AID 1117267 with AID 1996 (a water-solubility assay) and then retaining only every eighth shuffled inactive sample. No analysis is provided to show that this filtered subset is representative of the inactive population in the original TTR screen. If the filtering introduces any bias—which is plausible given that AID 1996 measures a different biological property—every TTR accuracy figure, including the 67.1% and 67.4% values, is potentially biased.
- [CID_SID model, Results and discussion] The CID_SID model uses PubChem compound identifiers (CID) and substance identifiers (SID) as input features, with the argument that ID generation 'considers the structure and similarity between substances and compounds.' This claim is not substantiated. PubChem IDs are database keys, not chemical descriptors, and the high held-out accuracy (81.5%) most plausibly reflects memorization of ID-to-label associations rather than a generalizable chemical rule. The model cannot be applied to new compounds that do not yet have IDs, so its practical predictive value is unclear.
- [Conclusion] The Conclusion states that 'the addition of atomic features... improved the ML models except the variant with atomic features and a full dataset on human dopamine D1 receptor antagonists. It performed worse than the ML model with a reduced number of features.' This contradicts the Results section, where the full D1 dataset with molecular features achieved 75.8% accuracy (Table ESM21) versus 71.5% without them (Table ESM9). The manuscript contains an internal inconsistency in its summary of the D1 outcome.
minor comments (4)
- [Throughout] The manuscript uses inconsistent numeric formatting, e.g., '25.532 samples' (Results and discussion) versus '25,532 samples' (Abstract), and comma decimal separators ('0,02') mixed with periods. Please standardize to a single convention.
- [Results and discussion] In the paragraph describing the full D1 model, the text says 'the optimal ML model, predicting whether a compound is a TTR transcription activator was GBC... achieving accuracy 75.8%...' but the metrics that follow (75.8% accuracy, 84.2% precision, etc.) are those of the D1 model; the sentence appears to have been copy-pasted and mislabeled.
- [Results and discussion] The typo 'TTP transcription activators' appears once; it should be 'TTR transcription activators'.
- [Scientific Contribution] The Scientific Contribution section states 'Developing a CID_SID ML model for TTR transcription activators that can predict with 80.2% accuracy...' but the Results section reports 81.5% accuracy for this model. The numbers should be reconciled.
Circularity Check
TTR 'hypothetical 67.4%' is constructed by rescaling TTR's small-data accuracy with D1's fitted improvement, so one headline result reduces by construction.
-
fitted input called prediction
[Methodology ('To ensure fairness in comparison...' paragraph), Results (Figure 2 calculation), and Abstract.]
"It was hypothesised that the existence of the same number of samples for the TTR case would increase the accuracy of the ML model by the same percentage. ... The increase of 5.8 in the accuracy of the ML model due to an increase in the number of samples was 8.85%. So, it was hypothesised that the increase the number of samples for TTR case to the level of samples for the D1 case, i.e. 57,038 compounds would increase the accuracy of the ML model that predicts TTR transcription activators to 61.82%."
The headline TTR accuracy is not obtained from TTR data at the stated size; it is computed by fitting D1's reduced-to-full accuracy gain (65.7% to 71.5%, a relative increase of 8.85%) and applying that same percentage to TTR's small-data accuracy. The 'hypothetical' result is therefore the TTR input accuracy rescaled by a D1-fitted multiplier; by construction it cannot fail and carries no independent evidence about TTR. The abstract elevates this projection to 'hypothetical 67.4% Accuracy ... if it could be trained with 25,532 samples,' while the Results section separately reports a 67.4% GBC trained on only 3,216 samples, so the provenance of the headline number is inconsistent.
full rationale
Most of the pipeline is self-contained: the D1 75.8% model is trained on real PubChem labels and evaluated on a held-out test set, so the central D1 claim is not circular. The methodological chain that is circular is the TTR hypothetical: a D1-fitted sample-size improvement percentage is transplanted to TTR under an explicit proportionality hypothesis, producing 61.82% (and later 67.4%) as a constructed number rather than a measurement. Because the paper labels it hypothetical and recommends real-data confirmation, the circularity is partial rather than fully concealed. The CID_SID model leans on the authors' prior work (ref 32) to justify using PubChem identifiers as features, but its reported 81.5% accuracy is an empirical held-out result, so that self-citation is not the load-bearing derivation. The internal discrepancy between 'trained on 3,216 / tested 680' and the abstract's 'if trained on 25,532 / tested 5,466' is a correctness/consistency problem, not circularity per se, but it reinforces that the headline TTR number does not come from the experiment described.
Assumptions & free parameters
free parameters (5)
- Inactive subsampling ratio =
1/8 after merging with AID 1996
- GBC max_depth =
2 or 3 depending on dataset
- PCA component count =
186 or 187
- Sample-size scaling percentage =
8.85% for D1 without features and about 0.93% for D1 with features
- Random oversampling ratio =
Majority class size after reduction
assumptions (3)
- domain assumption NMRDB-predicted 13C chemical shifts are adequate proxies for experimental spectra for training classifiers.
- ad hoc to paper Merging with AID 1996 and taking every eighth inactive does not bias the inactive class.
- ad hoc to paper The percentage accuracy improvement observed for D1 as sample size grows transfers to TTR.
invented entities (1)
-
PubChem CID and SID as predictive features
Cite this review
Pith. "Pith review of Comparative analysis of computational approaches for predicting Transthyretin (TTR) transcription activators and human dopamine D1 receptor antagonists." pith.science (2026). https://pith.science/paper/CJFLAVBV
@misc{pith2026250601137,
author = {Pith},
title = {Pith review of: Comparative analysis of computational approaches for predicting Transthyretin (TTR) transcription activators and human dopamine D1 receptor antagonists},
year = {2026},
howpublished = {\url{https://pith.science/paper/CJFLAVBV}},
note = {Machine review of arXiv:2506.01137}
}
read the original abstract
The study expands the application of scikit-learn-based machine learning (ML) to the prediction of small biomolecule functionalities based on Carbon 13 isotope (13C) NMR spectroscopy data derived from Simplified Molecular Input Line Entry System (SMILES) notations. The methodology previously demonstrated by predicting dopamine D1 receptor antagonists was upgraded with the addition of new molecular features derived from the PubChem database. The enhanced ML model obtained 75.8% Accuracy, 84.2% Precision, 63.6% Recall, 72.5% F1-score and 75.8 % ROC, when is trained on 25,532 samples and tested on 5,466 samples. To evaluate the applicability of the methodology for a variety of case studies, a comparison was conducted between the prediction capabilities of the ML models based on the human dopamine D1 receptor antagonists and on the neuronal Transthyretin (TTR) transcription activators. Since the TTR bioassay did not contain the required number of samples for comparison, the results were obtained hypothetically. Gradient Boosting classifier was the optimal model for TTR transcription activators, achieving hypothetical 67.4% Accuracy, 74.0% Precision, 53.5% Recall, 62.1% F1-score, 67.4 % ROC, if it could be trained with 25,532 samples and tested with 5,466 samples. In addition to the main study, to the attention of those interested in neuronal TTR, the CID_SID ML model has been developed to predict whether a compound, initially designed for another purpose, possesses TTR transcription activation capabilities. This ML model was based solely on its PubChem CID and SID and achieved 81.5% Accuracy, 94.6% Precision, 66.8% Recall, 78.3% F1-score, 81.5 % ROC.
Figures
Reference graph
Works this paper leans on
-
[1]
Jacobsen, N. E. NMR Spectroscopy Explained: Simplified Theory, Applications and Examples for Organic Chemistry and Structural Biology, 2007, John Wiley & Sons, Inc., DOI: 10.1002/9780470173350
-
[2]
An introduction to biological NMR spectroscopy
Marion, D. An introduction to biological NMR spectroscopy. Mol. Cel. Proteomics 2013, 12 (11), 3006-25, DOI: 10.1074/mcp.o113.030239
- [3]
-
[4]
NMR shift prediction from small data quantities
Rull, H.; Fischer, M.; Kuhn, S. NMR shift prediction from small data quantities. Jornal of Cheminform. 2023, 15, 114, DOI: 10.1186/s13321-023-00785-x
-
[5]
Cortés, I.; Cuadrado, C.; Hernández D. A.; Sarotti, A. M. Machine learning in computational NMR-aided structural elucidation. Front. Nat. Prod. 2023, 2, 1122426, DOI: 10.3389/fntpr.2023.1122426
-
[6]
Klukowski, P .; Riek, R.; Güntert, P. Time-optimized protein NMR assignment with an integrative deep learning approach using AlphaFold and chemical shift prediction.Sci. Adv. 2023, 9, eadi9323. DOI:10.1126/sciadv.adi9323
-
[7]
Bret, C. L. (2000). A General 13C NMR Spectrum Predictor Using Data Mining Techniques. SAR QSAR Environ. Res. 2000, 11(3–4), 211–234, DOI: 10.1080/10629360008033232
-
[9]
Jonas, E.; Kuhn. S.; Schlörer, N. Prediction of chemical shift in NMR: A review, Magn. Reson. Chem. 2022, 60(11), 1021, DOI: 10.1002/mrc.5234
doi:10.1002/mrc.5234 2022
Show all 33 references
-
[10]
Xin, D.; Sader, C., A.; Chaudhary, O.; Jones, P .; Wagner,K.; Tautermann, K.,S; et al Development of a 13C NMR Chemical Shift Prediction Procedure Using B3LYP/cc-pVDZ and Empirically Derived Systematic Error Correction Terms: A Computational Small Molecule Structure Elucidatio...
2017 doi
-
[11]
CASPER Reilly, D., Wren, C., Giles, S., Cunningham, L., & Hargreaves, P . (2016). CASPER: Computer Assisted Search Prioritisation and Environmental Response Application
2016
-
[12]
Computer-aided prediction of 125Te and13CNMRchemicalshiftsofdiorgano tellurides
Emerenciano V., P.; Diego, D., G.; Ferreira M., J., P.; Scotti, M., T.; Comasseto, J., V,; Rodrigues, G., V. Computer-aided prediction of 125Te and13CNMRchemicalshiftsofdiorgano tellurides. J. Braz. Chem. Soc. 2007,18(6),11831188. DOI: 10.1590/S0103- 50532007000600012 13 NMRDB...
2007 doi
-
[14]
Ivanova, M., L.; Russo, N.; Nikolic, K. Leveraging 13C NMR spectroscopic data derived from SMILES to predict the functionality of small biomolecules by machine learning: a case study on human Dopamine D1 receptor antagonists, ArXiv, Preprint at DOI: 10.48550/arXiv.2501.14044
-
[15]
PubChem Bioassay Record for AID 504652, Antagonist of Human D 1 Dopamine Receptor: qHTS, Source: National Center for Advancing Translational Sciences (NCATS)
National Center for Biotechnology Information. PubChem Bioassay Record for AID 504652, Antagonist of Human D 1 Dopamine Receptor: qHTS, Source: National Center for Advancing Translational Sciences (NCATS). https://pubchem.ncbi.nlm.nih.gov/bioassay/504652. Accessed May 18, 2025
2025
-
[16]
National Institutes of Health, PubChem https://pubchem.ncbi.nlm.nih.gov/ Accessed May 18, 2025
2025
-
[17]
PubChem Bioassay Record for AID 1117267, Source: The Scripps Research Institute Molecular Screening Center
National Center for Biotechnology Information. PubChem Bioassay Record for AID 1117267, Source: The Scripps Research Institute Molecular Screening Center. https://pubchem.ncbi.nlm.nih.gov/bioassay/1117267. Accessed Apr. 20, 2025
2025
-
[18]
Transthyretin: The Servant of Many Masters
Buxbaum, J.N.; Reixach, N. Transthyretin: The Servant of Many Masters. Cell Mol Life Sci. 2009, 66(19), 3095-101, DOI: 10.1007/s00018-009-0109-0
2009 doi
-
[19]
Transthyretin: Its Function and Amyloid Formation, Neurochem
Ueda, M. Transthyretin: Its Function and Amyloid Formation, Neurochem. Int. 2022, 155, 105313, DOI: 10.1016/j.neuint.2022.105313
2022
-
[20]
A Narrative Review of the Role of Transthyretin in Health and Disease
Liz, M., A.; Coelho, T.; Bellotti, V.; Fernandez-Arias, M., I; Mallaina, P.; Obici, L. A Narrative Review of the Role of Transthyretin in Health and Disease. Neurol Ther. 2020, 9(2), 395-402, DOI: 10.1007/s40120-020-00217-0
2020 doi
-
[21]
Nikitin, D.; Wasfy, J., H; Winn, A., N.; Raymond, F.; Shah, K., K.; Kim, S. et al. The effectiveness and value of disease-modifying therapies for transthyretin amyloid cardiomyopathy: A summary from the Institute for Clinical and Economic Review’s Midwest Comparative Effective...
2025 doi
-
[22]
Transthyretin enhances nerve regeneration
Fleming CE, Saraiva MJ, Sousa MM. Transthyretin enhances nerve regeneration. J Neurochem. 2007,103, 831–839. doi: 10.1111/j.1471-4159.2007.04828.x
2007
-
[23]
Transthyretin internalization by sensory neurons is megalin mediated and necessary for its neuritogenic activity, J Neurosci
Fleming, C.E.; Mar, F.M.; Franquinho, F.; Saraiva, M., J.; Sousa, M., M. Transthyretin internalization by sensory neurons is megalin mediated and necessary for its neuritogenic activity, J Neurosci. 2009, 29, 3220–3232. DOI: 10.1523/JNEUROSCI.6012-08.2009
2009 doi
-
[24]
A.; Alemi, M
Rios, X.; Gómez-Vallejo, V.; Martín, A.; Cossío, U.; Morcillo, M. A.; Alemi, M. et al. Radiochemical examination of transthyretin (TTR) brain penetration assisted by iododiflunisal, a TTR tetramer stabilizer and a new candidate drug for AD. Sci Rep 2019, 9, 13672, DOI: 10.1038...
2019 doi
-
[25]
J.; Sousa M., M
Nunes, A., F.; Saraiva M. J.; Sousa M., M. Transthyretin knockouts are a new mouse model for increased neuropeptide Y, FASEB J 2007, 20(1), 166-168, DOI: 10.1096/fj.05- 4106fje
2007 doi
-
[26]
The role of transthyretin in cell biology: impact on human pathophysiology
Magalhães, J.; Eira, J; Liz, M., A. The role of transthyretin in cell biology: impact on human pathophysiology. Cell Mol Life Sci. 2021, 78(17-18), 6105-6117, DOI: 10.1007/s00018-021-03899-3
2021 doi
-
[27]
Application of machine learning for predicting G9a inhibitors, Digital Discovery, 2024, 3(10), 2010-2018, DOI: 10.1039/D4DD00101J
Ivanova, M., L.; Russo, N.; Djaid, N.; Nikolic, K. Application of machine learning for predicting G9a inhibitors, Digital Discovery, 2024, 3(10), 2010-2018, DOI: 10.1039/D4DD00101J
2024 doi
-
[28]
Targeting Neurodegeneration: Three Machine Learning Methods for Discovering G9a Inhibitors Using PubChem and Scikit-Learn, ArXiv, 2025, Preprint at DOI: 10.48550/arXiv.2503.16214
Ivanova, M., L.; Russo, N.; Djaid, N.; Nikolic, K. Targeting Neurodegeneration: Three Machine Learning Methods for Discovering G9a Inhibitors Using PubChem and Scikit-Learn, ArXiv, 2025, Preprint at DOI: 10.48550/arXiv.2503.16214
-
[29]
Fast calculation of molecular polar surface area as a sum of fragment-based contributions and its application to the prediction of drug transport properties
Ertl, P .; Rohde, B.; Selzer, P. Fast calculation of molecular polar surface area as a sum of fragment-based contributions and its application to the prediction of drug transport properties. J Med Chem. 2000, 43(20), 3714-7, DOI: 10.1021/jm000942e
-
[30]
Computation of octanol-water partition coefficients by guiding an additive model with knowledge, J
Cheng, T.; Zhao, Y.; Li, X.; Lin, F.; Xu, Y .; Zhang, X. Computation of octanol-water partition coefficients by guiding an additive model with knowledge, J. Chem. Inf. Model. 2007, 47(6), 2140-8, DOI: 10.1021/ci700257y
2007 doi
-
[31]
Interpretation and use of standard atomic weights (IUPAC Technical Report), Pure Appl
van der Veen, A., M., H.; Meija, J.; Possolo, A.,A; Hibbert, D., B. Interpretation and use of standard atomic weights (IUPAC Technical Report), Pure Appl. Chem. 2021, 93 (5), 629- 646, DOI: 10.1515/pac-2017-1002
2021 doi
-
[32]
Predicting novel pharmacological activities of compounds using PubChem IDs and machine learning (CID-SID ML model), ArXiv, 2025, Preprint at DOI: 10.48550/arXiv.2501.02154
Ivanova, M., L.; Russo, N.; Nikolic, K. Predicting novel pharmacological activities of compounds using PubChem IDs and machine learning (CID-SID ML model), ArXiv, 2025, Preprint at DOI: 10.48550/arXiv.2501.02154
2025 doi
-
[33]
Gindulyte, A
Kim, S.; Thiessen, P., A; , Bolton, E., E; Chen, J.; Fu, G. Gindulyte, A. et al. PubChem Substance and Compound databases, Nucleic Acids Res. 2016, 44, D1202-13, DOI: 10.1093/nar/gkv951
2016 doi
-
[34]
PubChem Bioassay Record for AID 1996, Aqueous Solubility from MLSMR Stock Solutions, Source: Burnham Center for Chemical Genomics
National Center for Biotechnology Information. PubChem Bioassay Record for AID 1996, Aqueous Solubility from MLSMR Stock Solutions, Source: Burnham Center for Chemical Genomics. https://pubchem.ncbi.nlm.nih.gov/bioassay/1996. Accessed May 18, 2025
1996
-
[35]
scikit-learn. PCA. https://scikit- learn.org/stable/modules/generated/sklearn.decomposition.PCA.html Accessed May 18, 2025. Supplementary materials Tables Table ESM1. Results of the ML metrics regarding the TTR case performed without dimensionality reduction (GitHub file: http...
2025
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.