{"id":"0e3825a0-b2fc-40c9-9c4e-c673e120791d","arxiv_id":"2412.14717","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Applying Sinkhorn-Knopp scaling to fingerprint-based Gaussian kernels and kernel PCA produces embeddings that improve average accuracy on an imbalanced drug-subtype classification task but not solubility regression.","lead":"This paper proposes a kernel for SMILES strings by applying Sinkhorn-Knopp scaling to Gaussian kernels of molecular fingerprints, then uses kernel PCA embeddings for drug classification and solubility regression. The reported gains are limited to accuracy on a heavily imbalanced classification task, while regression results are worse than simpler baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The classification win is likely an artifact of kernel PCA fit on the full dataset before the train/test split, so the central outperformance claim is untested.","rationale":"Agree with the reader's weakest assumption. Without a clean train/test split for kPCA, the reported accuracy, weighted F1, and ROC-AUC in Table 2 cannot be interpreted as predictive performance on new molecules, which is what the abstract promises. The issue is especially acute because Eq. (1) builds the kernel on all N molecules and Section 3.5 does not describe an out-of-sample projection; baselines are evaluated without a comparable dataset-level transform. This makes the central outperformance claim untestable as reported. Secondary problems (incoherent Eq. (3), mismatch between regression results and the abstract, duplicated Table 2 entries) reinforce the REJECT but are not needed for this concern. The proposed check is decisive and feasible with the described pipeline.","tokens_in":11653,"tokens_out":6040,"duration_ms":51365,"concrete_test":"Re-run the DrugBank experiment with kernel PCA fit only on the 70% training split. Compute the training Gram matrix, apply Sinkhorn-Knopp and kPCA, then project test molecules out-of-sample using K_test = kernel between test and training molecules times the training eigenvectors (Nyström or equivalent). Retrain all seven classifiers on the training embeddings and recompute Table 2. If the SMILES-kernel rows no longer beat Morgan KNN (0.9129 accuracy), weighted-kmers RF (0.9135), and similar baselines on the same metrics, the central outperformance claim fails; if they still win, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section 5, abstract) that the SMILES-kernel embedding outperforms baselines on DrugBank classification requires a valid out-of-sample evaluation. As written, the protocol does not provide one. Section 3.5 computes an N×N Gram matrix and applies kernel PCA to the whole dataset; Eq. (1) explicitly iterates over all i,j ∈ N. Section 4 only states a 70/30 split, with no description of refitting kPCA on the training portion or of projecting held-out molecules (e.g., via Nyström). Thus test fingerprints enter the unsupervised embedding before classifiers are trained, and no baseline receives an analogous dataset-wide transform. The comparison is therefore confounded: the gains in Table 2 (e.g., SMILES-kernel LR 0.9150 accuracy, RF 0.8801 weighted F1, DT 0.5733 ROC-AUC) may come from transductive information or from the extra kPCA preprocessing, not from the proposed Sinkhorn kernel. If transductive learning was intended, it must be stated and applied equally to baselines; otherwise the headline claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a kernel-based embedding for SMILES strings: convert SMILES to molecular graphs, compute RDKit fingerprints, form a Gaussian similarity matrix, apply Sinkhorn-Knopp scaling to obtain a kernel matrix, and use kernel PCA for dimensionality reduction. The embeddings are evaluated on DrugBank drug-subcategory classification and solubility regression against several fingerprint baselines. The authors claim the proposed kernel outperforms baselines on classification. The manuscript includes tables of results, t-SNE plots, and heatmaps.","tokens_in":11830,"tokens_out":7824,"duration_ms":58353,"significance":"The problem of representing SMILES strings for machine learning is relevant, and the idea of using Sinkhorn-Knopp balancing to turn a similarity matrix into a kernel is not without interest. However, the paper as written does not provide a coherent description of the kernel (the update equation is undefined), the evaluation protocol leaks test-set information through full-data kernel PCA, and the reported results contradict parts of the abstract. No code or data is provided, and no validation procedure for the free hyperparameters is described. If the technical issues were resolved, the approach might merit further study, but the current claims are unsupported.","major_comments":[{"comment":"The Sinkhorn-Knopp update formula is mathematically incoherent. The expression 'BipartiteGraph ← −δP + (ζ × log(− →a1)) + (ζ × log(− →b1 )) / ζ' mixes a matrix P with vectors a1 and b1 and scalars ζ and δ in a way that is undefined (dimension mismatch, log of a vector with negative sign). The initialization and iterative updates of a, b, a1, b1 are not described. As a result, the kernel matrix K in Eq. (4) cannot be computed from the paper's description. This is a load-bearing error because the entire method rests on this kernel.","section":"Section 3.4, Eq. (3)"},{"comment":"The evaluation protocol leaks test-set information. The kernel matrix D in Eq. (1) is computed over all i,j ∈ N, and kernel PCA is applied to the full kernel matrix in Section 3.5. Section 4 then describes a 70/30 train/test split but provides no procedure for refitting kernel PCA on the training portion or for projecting held-out molecules (e.g., via Nyström). Thus, test molecules influence the embedding used to train classifiers, and the comparison against baselines is confounded. This invalidates the headline classification results in Table 2.","section":"Sections 3.5 and 4"},{"comment":"The claim in Section 5 that the proposed approach 'outperforms other embedding methods and classifiers in terms of average accuracy, precision, recall, weighted F1 score, and ROC-AUC' is not supported by the table. For example, on DrugBank, the Morgan fingerprint with LR achieves ROC-AUC 0.6215, while the best SMILES-kernel ROC-AUC is 0.5733 (DT), and several baselines have higher ROC-AUC. The statement should be restricted to specific metrics and classifiers where it actually holds, and the contradiction should be resolved.","section":"Section 5, Table 2"},{"comment":"The fingerprint is mislabeled. The authors state they use 'RDKit fingerprint' generated using the Morgan algorithm and refer to it as 'RDKFingerprint' and 'Morgan Fingerprint' interchangeably. In RDKit, the RDKFingerprint and Morgan fingerprints are distinct algorithms with different substructure definitions and bit generation. Citing reference [16] for the RDKFingerprint is incorrect, and this inconsistency makes the method difficult to reproduce.","section":"Section 3.2"},{"comment":"The abstract claims the proposed method 'outperforms several baseline methods in terms of supervised analysis,' but this is not supported for regression. In Table 3, the MACCS fingerprint with random forest regression achieves R² = 0.7804 on DrugBank, whereas the proposed SMILES kernel with random forest regression achieves R² = 0.7009, and several other baselines also outperform the proposed method. The discussion correctly acknowledges this, but the abstract's broad statement is misleading and should be qualified.","section":"Section 5, Table 3"},{"comment":"Table 2 contains apparent duplicated entries. For the Morgan fingerprint, the logistic regression row reports identical values for DrugBank and ChEMBL (Acc 0.4934, Prec 0.4868, Recall 0.4934, F1(Wt) 0.4870, F1(Mac) 0.2689, ROC-AUC 0.6215, TrainTime 10.2591). Similarly, the k-mers random forest row for ChEMBL is identical to the MACCS fingerprint random forest row for ChEMBL. These duplicates cannot be correct and undermine the reliability of the reported results.","section":"Section 5, Table 2"}],"minor_comments":[{"comment":"The matrix D is called a 'distance matrix' but Eq. (1) defines a Gaussian similarity (kernel) between feature vectors. This terminology is misleading and should be corrected.","section":"Section 3.3, Eq. (1)"},{"comment":"The notation is inconsistent: X1 and X2 are introduced as sets of molecular graphs, but Eq. (1) uses X_i and X_j as feature vectors. The fingerprint length 'n = 2048' is also conflated with the number of molecules.","section":"Section 3.2"},{"comment":"The initialization of vectors a, b, a1, b1 is described as 'zero vectors' and 'unit vectors,' but the standard Sinkhorn algorithm requires positive vectors for row and column scaling. This description is unclear and should be specified precisely.","section":"Section 3.4"},{"comment":"The statement 'smaller kernel value is better' is not justified. If the kernel value measures similarity, a smaller value indicates less similarity, but the paper does not explain why this is beneficial for the proposed method.","section":"Section 5, Figure 4"},{"comment":"The k-mers linear regression results on DrugBank are numerically absurd (MAE 8.3616e+10, R² −2.7267e+19), indicating overflow or a broken implementation. These values should be removed or a stable procedure should be used, and the issue should be discussed.","section":"Section 5, Table 3"},{"comment":"The paper reports averages over 5 runs but provides no standard deviations, confidence intervals, or significance tests, making it difficult to judge whether the observed differences are reliable.","section":"Section 4"}],"recommendation":"reject","confidential_remarks":"The paper has several serious flaws that preclude acceptance in its current form. The Sinkhorn update equation is not mathematically well-defined, so the proposed kernel cannot be computed from the description. The evaluation protocol leaks test information through full-data kernel PCA, invalidating the central classification claim. The results also contain internal contradictions (e.g., ROC-AUC and regression comparisons) and apparent data errors (duplicate rows). While the underlying idea of a Sinkhorn-balanced Gaussian kernel is not without interest, the manuscript would require substantial rewriting and a complete redo of the experiments under a valid train/test protocol to be credible. I recommend rejection, though a revised version with a coherent method and clean evaluation could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this paper should not be published in its current form. The central outperformance claim for the proposed kernel rests on an evaluation that fits kernel PCA on the full dataset before the train/test split, so test-set structure leaks into the embedding. Equation (3), which defines the Sinkhorn update, is mathematically incoherent as written and cannot be implemented by a reader. Those two problems are enough to sink the paper as an empirical contribution.\n\nWhat is genuinely useful here is the breadth of the empirical comparison. The authors run several standard embeddings (Morgan, MACCS, k-mers, weighted k-mers) and seven classifiers across two public datasets, reporting accuracy, precision, recall, F1, ROC-AUC, and runtime. That is a lot of labor, and the tables are the most informative part of the paper. Applying Sinkhorn-Knopp scaling to a Gaussian kernel on molecular fingerprints is not new—it is a standard optimal transport construction, closely related to Mialon et al. and earlier doubly stochastic kernel work—but it is a reasonable baseline to test.\n\nThe soft spots are large. The leakage issue is load-bearing: because kernel PCA is computed on all N molecules before the 70/30 split, the classifier sees test-set information in its features, and no baseline gets the same preprocessing. The comparison is therefore not apples-to-apples even setting aside transductive information. Equation (3) is not a typo; it is the core update step, and it is unreadable. The paper also mislabels the fingerprint: it says RDKFingerprint but describes the Morgan algorithm, and RDKit's two fingerprints are not the same. Table 2 contains at least one impossible row, where the Morgan LR row shows identical values for DrugBank and ChEMBL (0.4934 accuracy), which suggests a copy-paste error. And the regression results directly contradict the abstract's \"outperforms\" claim: the proposed kernel's R² on DrugBank RF is 0.70 versus 0.78 for MACCS RF, and ridge/lasso go negative on both datasets. The claimed ROC-AUC advantage on ChEMBL is also not systematic—it holds only for Naive Bayes.\n\nFor a reader: this is a rough draft of an empirical note, not a finished paper. A serious referee could have caught all of these issues in a day, but there is no salvaged result here without rerunning the evaluation correctly and fixing the method description. I would not cite it, and I would not bring it to reading group.\n\nRecommendation: desk reject, with an invitation to resubmit if the authors fix the evaluation protocol and the equation. No reviewer time should be spent on the present version.","headline":"The classification win is likely an artifact of test-set leakage via pre-split kernel PCA, and Equation (3) is not implementable as written—this needs a full rewrite before it can be taken seriously.","tokens_in":12415,"tokens_out":4587,"would_cite":false,"duration_ms":37914,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that rebalancing a Gaussian similarity matrix of RDKFingerprints with the Sinkhorn-Knopp algorithm, then reducing with kernel PCA, gives SMILES-string embeddings that beat standard fingerprint and k-mer baselines on…","keywords":["SMILES strings","kernel matrix","Sinkhorn-Knopp algorithm","RDKFingerprint","kernel PCA","molecular embeddings","drug subcategory prediction","solubility prediction"],"falsifier":"Run the pipeline with the 70/30 split placed before the Sinkhorn-Knopp kernel and kernel PCA are estimated: fit the scaling vectors and principal components on the training 70%, project the test 30% through them, and compare accuracy, precision, recall, weighted F1, and ROC-AUC against the same baselines under identical split-before-embedding conditions; if the proposed kernel no longer improves any metric, the central claim fails.","tokens_in":11411,"feed_emoji":"💊","tokens_out":10212,"duration_ms":81873,"temperature":0.7,"pith_summary":"The paper tries to establish that a kernel-based embedding for SMILES strings can beat standard molecular fingerprints on classification. The method converts each SMILES string to an RDKFingerprint, builds a Gaussian similarity matrix, rebalances it with the Sinkhorn-Knopp algorithm into a balanced similarity matrix, and then applies kernel PCA to get low-dimensional vectors for any classifier or regressor. On the DrugBank drug-subcategory task, the authors report the best average accuracy, precision, recall, weighted F1, and ROC-AUC across seven classifiers compared with Morgan, MACCS, k-mer, and weighted k-mer embeddings. On the ChEMBL activity-type task the advantage is narrower: the proposed kernel leads only in ROC-AUC, while Morgan fingerprint leads on accuracy, precision, recall, and weighted F1. On solubility regression the method is competitive but not best, so the claimed contribution is specifically a classification-oriented embedding that could be used in drug discovery pipelines.","feed_headline":"Sinkhorn-Knopp kernel beats fingerprints on drug classification","feed_subtitle":"A balanced kernel from RDKFingerprints plus kernel PCA tops four baselines on DrugBank drug subcategories.","key_machinery":"The central object is the Gram matrix produced by Sinkhorn-Knopp rebalancing. The method normalizes an RBF distance matrix $D$ to a joint probability matrix $P$, then iteratively applies Sinkhorn-Knopp scaling until row and column marginals converge to prescribed sums, with tolerance $\\xi = 10^{-6}$, producing scaling vectors $a$ and $b$; the final kernel is $K = \\operatorname{diag}(a)\\, P\\, \\operatorname{diag}(b)$. Kernel PCA on $K$ yields the top principal components used as embeddings. The Sinkhorn-Knopp iteration is the load-bearing mechanism: it converts a generic similarity matrix into a matrix with balanced row and column sums, and the paper's evidence for its value is the improved class separation visible in t-SNE plots and class-similarity heatmaps.","core_discovery":"The central claim, stated in the abstract and Section 5, is that the proposed kernel-based approach outperforms other embedding methods and classifiers on DrugBank drug-subcategory classification in terms of average accuracy, precision, recall, weighted F1 score, and ROC-AUC. The best reported accuracy is 0.915 with logistic regression, and the kernel also gives the highest weighted F1 (0.8801 with random forest) among the compared embeddings. The authors attribute this to the Sinkhorn-Knopp step, which turns a raw Gaussian kernel into a matrix whose row and column marginals are balanced, making similar classes more similar and different classes less similar in the kernel-PCA embedding. For ChEMBL, the same method only beats all baselines on ROC-AUC, and for regression it trails MACCS with random forest, so the paper's central claim is essentially about classification.","pith_inferences":["Beyond the paper, the same Sinkhorn-Knopp rebalancing could be applied to other molecular similarity matrices, such as graph-edit distances or learned fingerprints, to isolate whether the balancing step or the fingerprint features drive the classification gain.","Beyond the paper, the reported comparisons leave open how much of the gain comes from fitting kernel PCA on the full dataset before the split; a deployment-oriented evaluation would fit the embedding on training molecules only and project test molecules.","Beyond the paper, the row and column scaling is formally related to optimal-transport attention, so a trainable version of this kernel might yield a learnable molecular similarity measure."],"forward_implications":["On DrugBank, the proposed embedding reaches a best accuracy of 0.915 and best weighted F1 of 0.8801 with logistic regression and random forest respectively, exceeding the best baseline on those metrics.","Because the embedding is just a vector per molecule, it can be plugged into any classifier or regressor without changing the downstream model.","The method's advantage is classification-specific: on ChEMBL only ROC-AUC improves, and on solubility regression MACCS fingerprints with random forest still lead.","The kernel construction is fingerprint-based, so it inherits the low cost and substructure interpretability of RDKFingerprint features while adding a nonlinear, globally balanced similarity."],"supporting_citations":[{"why":"Supplies the Sinkhorn-Knopp iteration and its convergence criterion used to rebalance the kernel matrix.","marker":"[11]"},{"why":"Supplies the kernel PCA step that turns the Gram matrix into low-dimensional embeddings.","marker":"[7]"},{"why":"Supplies the Morgan/RDKFingerprint features the kernel is built on.","marker":"[16]"},{"why":"Provides the DrugBank dataset used for the main drug-subcategory classification claim.","marker":"[24]"},{"why":"Provides the ChEMBL dataset used for the second classification and regression evaluation.","marker":"[1]"},{"why":"Motivates the optimal-transport view of the kernel matrix and its connection to attention.","marker":"[15]"},{"why":"Defines the MACCS fingerprint baseline used in the comparison.","marker":"[4,10]"},{"why":"Defines the k-mer sequence embedding baseline used in the comparison.","marker":"[9]"},{"why":"Defines the weighted k-mer baseline used in the comparison.","marker":"[17]"}],"fun_headline_variants":["Sinkhorn-Knopp kernel beats fingerprints on drug classification","Balanced kernel from RDKFingerprints tops drug category prediction","New kernel method outperforms baselines on DrugBank classes","Kernel balancing with Sinkhorn-Knopp aids drug classification","Sinkhorn-Knopp kernel improves DrugBank drug category accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation rests on the premise that fitting the kernel matrix and kernel PCA on the full dataset before the 70/30 split does not inflate the reported classification gains.","fun_headline_variants_meta":{"raw":{"variants":["Sinkhorn-Knopp kernel beats fingerprints on drug classification","Balanced kernel from RDKFingerprints tops drug category prediction","New kernel method outperforms baselines on DrugBank classes","Kernel balancing with Sinkhorn-Knopp aids drug classification","Sinkhorn-Knopp kernel improves DrugBank drug category accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000644,"raw_usage":{"total_tokens":2986,"prompt_tokens":996,"completion_tokens":1990,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":1903}},"tokens_in":612,"tokens_out":1990,"duration_ms":10847,"temperature":1.0,"reasoning_tokens":1903,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:58:04.411463+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline with the 70/30 split placed before the Sinkhorn-Knopp kernel and kernel PCA are estimated: fit the scaling vectors and principal components on the training 70%, project the test 30% through them, and compare accuracy, precision, recall, weighted F1, and ROC-AUC against the same baselines under identical split-before-embedding conditions; if the proposed kernel no longer improves any metric, the central claim fails.","supporting_citations":[{"cited_title":"SIAM Journal on Matrix Analysis and App30(1), 261–275 (2008)","cited_arxiv_id":null,"evidence_quote":"Supplies the Sinkhorn-Knopp iteration and its convergence criterion used to rebalance the kernel matrix."},{"cited_title":"Journal of chemometrics25(2), 92–99 (2011)","cited_arxiv_id":null,"evidence_quote":"Supplies the kernel PCA step that turns the Gram matrix into low-dimensional embeddings."},{"cited_title":"Scientific Reports11(1) (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the Morgan/RDKFingerprint features the kernel is built on."},{"cited_title":"Nature materials17(4), 361–368 (2018)","cited_arxiv_id":null,"evidence_quote":"Provides the DrugBank dataset used for the main drug-subcategory classification claim."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ChEMBL dataset used for the second classification and regression evaluation."},{"cited_title":"Digital Chemical Engineering2, 100016 (2022)","cited_arxiv_id":null,"evidence_quote":"Defines the k-mer sequence embedding baseline used in the comparison."},{"cited_title":"Drug Discovery Today25(4), 689–705 (2020)","cited_arxiv_id":null,"evidence_quote":"Defines the weighted k-mer baseline used in the comparison."}],"review_version":1}