{"id":"9871764f-08be-4a6b-8330-a825c91f4dbc","arxiv_id":"2506.00936","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"TrustworthyMS reports new best results on two metabolic stability tasks, with MCC 0.622 on HLM classification and RMSE 0.091 on half-life regression, while adding uncertainty estimates.","lead":"This paper presents TrustworthyMS, a graph neural network that predicts how quickly drug-like molecules are broken down in the body and outputs a confidence level for each prediction. It combines bond-level graph remapping, contrastive learning, and Beta-Binomial uncertainty, reporting improved scores on metabolic stability benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Regression branch is internally inconsistent: §4.1 z-score normalization gives targets outside (0,1), but Eq.17 assumes y∈(0,1) and Eq.15's Beta-mean prediction cannot represent negative targets; the reported HL gains need code verification before the SOTA claim is credible.","rationale":"The reader's identified weak spot is the correct one, and it is the single most load-bearing issue because it is an equation-level inconsistency that directly affects the largest claimed improvement. I refine the mechanism: Eq.17 is algebraically E[(Y−y)^2] for any y, so the issue is not the expectation formula but the bounded Beta mean in Eq.15. With z-scored targets, negative values must exist (unless all raw half-lives are identical), and no value in (0,1) can equal a negative target; the model is therefore misspecified for the stated preprocessing. This is a binary, checkable fact: inspect the scaler in the released code. If the code min-max scales to (0,1), the contradiction is a text error and the regression results may stand; if it z-scores, the reported results cannot be produced by the described method. Other concerns (undefined 46.1% robustness number, no statistical significance tests, no UQ baselines) are real but secondary: they affect how strongly the evidence supports the SOTA claim, whereas the normalization mismatch affects whether the described method was evaluated at all. I therefore keep the reader's CONDITIONAL verdict unchanged, with the condition being verification of the target transform and reproduction of Table 2 from the released repository.","tokens_in":11852,"tokens_out":9053,"duration_ms":92117,"concrete_test":"Clone the released repository (https://github.com/trashTian/TrustworthyMS), locate the Half-Life data preprocessing, and record the exact scaler and the min/max of the normalized target used for training and evaluation. If it is StandardScaler and negative targets occur, run the released model with Eq. 17 on the same splits and check whether predictions are bounded in (0,1); the reported RMSE 0.091 cannot be reached in that case, confirming the §4.1/§3.4 inconsistency. If it is MinMaxScaler or equivalent, rerun Table 2 with that scaler to verify the 0.091 RMSE and 0.833 P-score, which would reduce the concern to a documentation error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. 17 is written for 'normalized targets y_i ∈ (0,1)', but the loss formula itself is valid for any fixed y; the real constraint is Eq. 15. Because α and β in Eq. 13 are sigmoid outputs plus ε>0, the point prediction ŷ=α/(α+β) is confined to (0,1). Section 4.1 says the Half-Life dataset is 'standardized via z-score normalization', and any nontrivial z-scored variable contains negative values (mean 0 after centering). A Beta mean cannot equal a negative target, so the described model cannot fit part of the stated data; the reported RMSE 0.091 on that scale would require the actual targets to lie in (0,1) or the implementation to use a different (undocumented) transform. Since the regression table contains the largest margins over MS-BACL, this mismatch is load-bearing for 'outperforms current state-of-the-art.' The released GitHub code can settle whether targets are min-max scaled (text error, results may stand) or truly z-scored (pipeline misspecified). Secondary issues—undefined 46.1% OOD robustness, absent significance tests, no UQ baseline comparison—weaken evidence but do not identify a concrete broken step.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TrustworthyMS, a graph neural network framework for metabolic stability prediction with three claimed contributions: molecular graph topology remapping that augments atom-centric graphs with bond-centric nodes, dual-view contrastive learning between the molecular and bond views, and Beta-Binomial evidential uncertainty quantification. The authors evaluate on HLM classification (5,876 compounds), Half-Life regression (656 compounds), and a composite OOD set, reporting improved MCC/F1 over MS-BACL in classification, improved RMSE/P in regression, and an OOD robustness improvement. The paper also includes ablations, a parameter sensitivity study, an uncertainty-threshold reliability analysis, and a Shapley-value case study.","tokens_in":12199,"tokens_out":7640,"duration_ms":73402,"significance":"If the results hold, the main contribution is a practical integration of bond-aware graph augmentation, contrastive learning, and evidential uncertainty estimation for ADME-relevant property prediction. The release of code and the use of established benchmarks (HLM, Half-Life, OOD sets) are strengths, and the uncertainty-threshold analysis gives a useful direction for reliability-based screening. However, the central 'outperforms current state-of-the-art' claim is currently weakened by an apparent mismatch between the stated z-score normalization of the regression targets and the Beta-mean predictor, which is confined to (0,1), and by several undefined or internally inconsistent numerical claims. The uncertainty quantification contribution is not benchmarked against standard UQ baselines, so the calibration claim is not yet quantitatively supported.","major_comments":[{"comment":"Section 4.1 states that the Half-Life dataset is 'standardized via z-score normalization.' In Eq. 13, α and β are produced by sigmoid outputs plus ε, so both lie in (ε, 1+ε), and the point prediction in Eq. 15, α/(α+β), is confined to the open interval (0,1). A z-scored target necessarily takes negative values (the mean is 0 after centering), so the described model cannot represent part of the target distribution. Equation 17 explicitly assumes 'normalized targets y_i ∈ (0,1),' which is inconsistent with z-score normalization. This is load-bearing because the regression margin over MS-BACL (RMSE 0.091 vs 0.111) is one of the strongest claimed improvements. The authors must clarify whether the actual preprocessing is min-max scaling (a text error) or z-score normalization (a model misspecification), and adjust the text, equations, or results accordingly.","section":"§4.1 and §3.4 (Eqs. 13, 15, 17)"},{"comment":"The claim of a '46.1% improvement in robustness on out-of-distribution (OOD) data' is undefined and does not match any clean comparison in Table 3. The direct comparison to the previous SOTA method MS-BACL is MCC 0.615 vs 0.588, a 4.6% relative improvement, while comparisons to GAT (0.414) or MGCN (0.447) give 48.6% or 37.6%, respectively. No definition of 'robustness' is provided, yet the same 46.1% figure is repeated in the Conclusion. Please define the metric, show the calculation, or remove the claim.","section":"Abstract, §4.2, Conclusion"},{"comment":"The paper asserts that TrustworthyMS 'outperforms current state-of-the-art methods,' but no significance tests are reported and several key differences are within one standard deviation of the baselines (e.g., HLM MCC 0.622±0.034 vs 0.601±0.053; F1 0.866±0.012 vs 0.863±0.018). Moreover, on the OOD set the AUC is lower than MS-BACL (0.862 vs 0.897), and on HLM classification the AUC is tied (0.873). The text acknowledges the lower OOD AUC but still frames the overall result as SOTA. Please report paired significance tests (e.g., Wilcoxon signed-rank or bootstrap) or temper the superiority claim to the specific metrics that are actually improved.","section":"§4.2 (Tables 1–3)"},{"comment":"The uncertainty quantification is evaluated only through thresholding on the model's own uncertainty scores, with no comparison to standard uncertainty quantification baselines such as Monte Carlo dropout, deep ensembles, or other evidential methods, and no calibration curve or calibration metric (e.g., reliability diagram or expected calibration error). Since uncertainty quantification is a central contribution of the paper, the claim of 'calibrated uncertainty estimation' needs a quantitative benchmark against existing UQ approaches, not only an internal consistency check.","section":"§4.2, 'Uncertainty-Aware Prediction Reliability'"}],"minor_comments":[{"comment":"Several percentage changes are computed inconsistently: the '9.8% MCC gain' from 0.564 to 0.622 is actually 10.3%, the '9.0% improvement versus CMMS-GCL' from 0.566 to 0.622 is 9.9%, and the '48.2% improvement over basic GNNs (D-MPNN: 0.218)' should be approximately 182% under the formula used elsewhere. Please recalculate and standardize the reporting of relative improvements.","section":"§4.2"},{"comment":"The abstract and conclusion refer to a dataset of '10,031 compounds,' but the described datasets sum to 7,142 compounds (HLM 5,876 + HL 656 + OOD 610). Please reconcile this discrepancy.","section":"Abstract and §4.2"},{"comment":"The contrastive loss in Eq. (8) is written on graph encoder outputs z, while Section 3.2 introduces projection heads g(·) producing p_G and states that the contrastive loss operates on the encoder outputs. The architecture figure also shows projection heads feeding the contrastive loss. Please clarify whether the loss uses z or p.","section":"§3.2 and Eq. (8)"},{"comment":"There are several typographical issues: 'with to ensure distributional fairness' in §4.1, 'regresion' in §4.2, 'jontly' in the Fig. 1 caption, and '0.825±0.15' in Table 4 should presumably be '0.825±0.015.'","section":"§4.1, §4.2, Fig. 1"},{"comment":"The anti-smoothing scaling factor s is fixed at 1×10⁻⁶, but the sensitivity analysis in Fig. 3 sweeps only the contrastive weight λ. Since s affects the feature magnitudes substantially, a brief discussion or sensitivity check for s would help.","section":"§3.2 and Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The main risk to the paper is the regression preprocessing mismatch: if the released code uses min-max scaling, the paper is close to acceptable after correcting the text; if z-score normalization is actually used, the regression results are difficult to trust. The repeated undefined or inconsistent percentage claims (46.1%, 48.2%, 9.8%) suggest that all quantitative claims should be independently checked against Tables 1–5. For a journal version, the uncertainty evaluation should also be strengthened with standard UQ baselines."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: TrustworthyMS is a reasonable incremental entry in metabolic-stability prediction, combining bond-graph remapping, dual-view contrastive learning, and Beta-Binomial evidential heads. The classification results are credible on their face; the regression branch has an internal scaling inconsistency that makes the headline regression gains unverifiable until the code is checked.\n\nWhat is new: no prior MS model packages these three pieces together. The topology remapping (atom-bond-atom nodes) plus dual-view contrastive loss is a natural extension of MS-BACL, and Beta-Binomial evidential losses are standard, but the specific combination is not in the cited papers. The paper also ships a code link, runs 10-fold CV over 10 runs, and includes ablations, OOD evaluation, parameter sensitivity, and a case study. That is more than many ML submissions provide.\n\nWhere it is soft: the biggest issue is the regression setup. Eq. 17 is written for normalized targets in (0,1), and Eq. 15 predicts a Beta mean that is structurally confined to (0,1). Section 4.1 says the HL dataset is z-score normalized, which produces negative values and values above 1. If that is what the code does, the described model cannot fit the stated data, and the RMSE 0.091 vs. 0.111 for MS-BACL is not explained. If the code uses min-max scaling instead, the text is wrong and the results may hold. The GitHub link should settle this, but as written the mismatch is load-bearing for the regression SOTA claim.\n\nThree smaller issues. The \"46.1% robustness improvement\" is never defined and does not match any clean comparison in Table 3. OOD AUC is actually lower than MS-BACL (0.862 vs. 0.897), so the OOD story needs a more careful telling. And there are no significance tests; a 0.021 MCC gain and a 0.020 RMSE gain could easily be noise. The uncertainty analysis uses only the model's own confidence scores and never compares against standard UQ baselines like MC Dropout or deep ensembles.\n\nBottom line: this paper deserves a serious referee. The architecture is worth publishing if the regression preprocessing is corrected or clarified, the 46.1% number is either defined or removed, and the UQ section adds a baseline comparison. For me, the classification part is fine; I would not cite the regression numbers until the code matches the text.","headline":"Solid incremental paper with a plausible classification result; the regression branch has a target-scaling mismatch that makes the headline HL gains unverified until the code is checked.","tokens_in":12701,"tokens_out":3165,"would_cite":false,"duration_ms":32714,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TrustworthyMS, a graph neural network that pairs atom- and bond-level views through contrastive learning, claims to outperform state-of-the-art metabolic stability predictors on both classification and regression while supplying…","keywords":["metabolic stability prediction","contrastive learning","graph neural network","uncertainty quantification","Beta-Binomial evidence","drug discovery","bond-level topology","out-of-distribution generalization"],"falsifier":"Inspect the actual preprocessing values: if any z-score-normalized half-life target is at or below 0 or at or above 1, then Eq. 17 is not a proper Beta-expectation loss for those targets. A direct check is to rerun the regression experiments with targets min-max scaled into $(0,1)$ and compare RMSE and Spearman correlation; if the reported gains vanish, the evidential loss was doing the work, and if they persist, the representation and contrastive components carry the improvement.","tokens_in":11680,"feed_emoji":"💊","tokens_out":9174,"duration_ms":75751,"temperature":0.7,"pith_summary":"This paper aims to establish a metabolic stability predictor that gets two things at once: higher accuracy than existing graph-based models and a confidence score for each prediction. The proposed architecture, TrustworthyMS, combines a molecular graph whose nodes include atom-bond-atom triplets, a contrastive loss that aligns the atom view with the bond view, and a Beta-Binomial evidential head that outputs both a prediction and an uncertainty estimate. On the HLM classification benchmark it reports a Matthews correlation coefficient of 0.622, and on the half-life regression benchmark a root mean squared error of 0.091, both better than the previous state of the art. If the claims hold, drug-discovery teams could rank candidate molecules by predicted stability while knowing which predictions to trust.","feed_headline":"Bond-aware graph model with uncertainty beats stability baselines","feed_subtitle":"Dual-view GNN with Beta-Binomial uncertainty improves classification and regression for metabolic stability.","key_machinery":"The load-bearing object is the dual-view graph pair. Molecular graph topology remapping first creates bond-centric nodes $v^r_{ij} = f_{\\mathrm{node}}(v_i \\oplus e_{ij} \\oplus v_j)$ from an atom, its bond, and the neighboring atom, then links those nodes into a second graph when two bonds share an atom. A GIN encoder with centering-and-scaling anti-smoothing normalization produces embeddings for both views, and the contrastive loss $L_{\\mathrm{CL}}$ of Eq. 8 aligns the molecular-topology view with the bond-interaction view. The evidential head uses Beta-Binomial conjugacy: classification evidence $(e^+, e^-)$ maps to Beta parameters $\\alpha = e^+ + 1$, $\\beta = e^- + 1$, giving belief, disbelief, and uncertainty masses by Eq. 11, while regression generates $(\\alpha, \\beta)$ through sigmoid networks and predicts the Beta mean while penalizing variance in Eq. 17. The joint objective $L = L_G + L_{G^r} + \\lambda L_{\\mathrm{CL}}$ is trained end to end.","core_discovery":"The central claim is that representing a molecule as two aligned graphs—the original atom graph $G$ and a bond-centric remapped graph $G^r$ built from atom-bond-atom triplets—lets the model capture interactions that atom-only message passing misses, and that attaching a Beta-Binomial evidential head turns each output into a Beta distribution whose mean is the prediction and whose spread is the uncertainty. The paper reports that this architecture beats existing methods on both tasks: on HLM classification the MCC is 0.622 versus 0.601 for MS-BACL, and on half-life regression the RMSE is 0.091 versus 0.111 with Spearman correlation 0.833 versus 0.652. It also reports an out-of-distribution MCC of 0.615 versus 0.588 when the HLM-trained model is tested on structurally dissimilar compounds. The same uncertainty estimates are used as a filter: as the acceptance threshold tightens, reported accuracy on HLM rises from about 83 percent to 94 percent, and the regression correlation rises from about 0.83 to 0.90.","pith_inferences":["If the z-score normalization was not rescaled into $(0,1)$, the evidential regression loss is misspecified, and the reported regression advantage could be carried by the contrastive and remapping components; rerunning with min-max scaling would separate these effects.","The same Beta-Binomial head could be attached to other ADME endpoints such as clearance or permeability, where calibrated confidence matters as much as point accuracy.","The dual-view contrastive objective is not tied to atom-bond duality; aligning the same encoder across 3D conformer views or fingerprint views would test whether the improvement is specific to bond remapping or general to multi-view regularization."],"forward_implications":["Bond-level topology remapping is what lifts classification: removing it drops MCC on HLM from 0.622 to 0.613.","Contrastive alignment is what most helps regression: removing dual-view contrastive learning raises RMSE from 0.091 to 0.097 and lowers the Spearman correlation from 0.833 to 0.818.","Uncertainty thresholds give a deployable reliability filter: accepting only the highest-confidence predictions raises HLM accuracy to about 94 percent and half-life correlation to about 0.90.","The model generalizes to out-of-distribution scaffolds better than the previous state of the art: OOD MCC is 0.615 versus 0.588 for MS-BACL."],"supporting_citations":[{"why":"MS-BACL is the primary state-of-the-art baseline and supplies the benchmark protocol, OOD construction, and the bond-augmentation approach this work directly compares against.","marker":"[20]"},{"why":"CMMS-GCL is the graph-contrastive baseline whose multimodal architecture the dual-view contrastive objective extends.","marker":"[2]"},{"why":"Supplies the HLM classification dataset and the feature-engineering and GNN baseline implementations used in the evaluation.","marker":"[9]"},{"why":"Provides the half-life regression dataset and the quantum-mechanics-plus-ensemble approach that motivates the regression task.","marker":"[1]"},{"why":"MGCN baseline and the GCN-on-SMILES-graph line that the atom-centric view is compared with.","marker":"[17]"},{"why":"Subjective logic formalism underpinning the Beta-Binomial belief, disbelief, and uncertainty masses in Eq. 11.","marker":"[7]"},{"why":"Clinical candidates OOD dataset used to test generalization under structural dissimilarity.","marker":"[19]"}],"fun_headline_variants":["Dual-view GNN with Beta-Binomial uncertainty improves metabolic stability prediction","Bond-aware dual-view GNN with uncertainty beats metabolic stability baselines","Uncertainty-aware dual-view GNN predicts metabolic stability with confidence","Dual-view contrastive learning with Beta-Binomial uncertainty for metabolic stability"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The regression loss in Eq. 17 computes the expectation of squared error under a Beta distribution and therefore assumes the normalized half-life targets lie strictly between 0 and 1, but Section 4.1 says the half-life dataset was standardized via z-score normalization, which can produce negative values and values above 1; if the dataset was not rescaled into $(0,1)$, that loss is not a valid likelihood for those targets.","fun_headline_variants_meta":{"raw":{"variants":["Dual-view GNN with Beta-Binomial uncertainty improves metabolic stability prediction","Bond-aware dual-view GNN with uncertainty beats metabolic stability baselines","Uncertainty-aware dual-view GNN predicts metabolic stability with confidence","Dual-view contrastive learning with Beta-Binomial uncertainty for metabolic stability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000766,"raw_usage":{"total_tokens":3400,"prompt_tokens":949,"completion_tokens":2451,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":2373}},"tokens_in":565,"tokens_out":2451,"duration_ms":15903,"temperature":1.0,"reasoning_tokens":2373,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:54:10.225225+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the actual preprocessing values: if any z-score-normalized half-life target is at or below 0 or at or above 1, then Eq. 17 is not a proper Beta-expectation loss for those targets. A direct check is to rerun the regression experiments with targets min-max scaled into $(0,1)$ and compare RMSE and Spearman correlation; if the reported gains vanish, the evidential loss was doing the work, and if they persist, the representation and contrastive components carry the improvement.","supporting_citations":[{"cited_title":"Briefings in Bioinformatics25(3), bbae127 (2024)","cited_arxiv_id":null,"evidence_quote":"MS-BACL is the primary state-of-the-art baseline and supplies the benchmark protocol, OOD construction, and the bond-augmentation approach this work directly compares against."},{"cited_title":"Bioinformatics39(8), btad503 (2023)","cited_arxiv_id":null,"evidence_quote":"CMMS-GCL is the graph-contrastive baseline whose multimodal architecture the dual-view contrastive objective extends."},{"cited_title":"Chemical Research in Toxicology35(9), 1614–1624 (2022)","cited_arxiv_id":null,"evidence_quote":"Supplies the HLM classification dataset and the feature-engineering and GNN baseline implementations used in the evaluation."},{"cited_title":"Chemometrics and Intelligent Laboratory Systems257, 105292 (2025) 16 Authors Suppressed Due to Excessive Length","cited_arxiv_id":null,"evidence_quote":"Provides the half-life regression dataset and the quantum-mechanics-plus-ensemble approach that motivates the regression task."},{"cited_title":"Wiley interdisciplinary reviews: computational molecular science11(1), e1479 (2021) Title Suppressed Due to Excessive Length 17","cited_arxiv_id":null,"evidence_quote":"MGCN baseline and the GCN-on-SMILES-graph line that the atom-centric view is compared with."},{"cited_title":"Journal of cheminformatics12, 1–14 (2020)","cited_arxiv_id":null,"evidence_quote":"Clinical candidates OOD dataset used to test generalization under structural dissimilarity."}],"review_version":1}