{"id":"47c50e35-a826-461f-a554-c2f49680ed94","arxiv_id":"2412.16483","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"MOL-Mamba combines Mamba state-space layers, graph neural networks, and electronic descriptors to improve molecular property prediction on 8 of 11 MoleculeNet benchmarks.","lead":"MOL-Mamba is a machine learning framework that represents a molecule as a graph of atoms and fragments plus a list of 112 electronic descriptors, and trains a Mamba state-space model to fuse both views. The authors report better accuracy than existing molecular pretraining methods on 8 of 11 standard chemistry benchmarks, with a smaller model than graph transformers like Uni-Mol.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparison in Tables 2–3 is not controlled; reported gains may reflect different pretraining data/splits/evaluation rather than MOL-Mamba's design.","rationale":"I read the paper as an engineering contribution whose central assertion is empirical: the proposed architecture and pretraining recipe outperform existing methods on molecular property prediction. The load-bearing condition is therefore that the comparison in Tables 2 and 3 is apples-to-apples. The reader's weakest assumption identifies exactly this condition, and I agree with that assessment. I did not find a more fundamental internal inconsistency in the high-level design: the modules are separately motivated, the ablations in Tables 5 and 6 are internally consistent with the proposed contributions, and the paper provides a code link and external benchmark evaluations. The main correctness risk is that the baseline numbers appear to be taken from prior papers that used different pretraining corpora, different dataset splits, and possibly different evaluation repetitions. This is a standard and serious concern for SOTA claims, but it is resolvable by re-running the baselines under a common protocol. Because the reader already conditioned acceptance on this issue, my stress-test does not move the verdict. I also note a secondary reproducibility issue in the GraphSSM pseudocode (Algorithm 1): the pointwise multiplication of Delta with adjacency and distance matrices is dimensionally inconsistent (Delta is b x l x d while A_G, D_G are l x l), and the sums in lines 9-10 are not specified to produce the stated output shapes. This supports keeping the verdict conditional, but the baseline comparability issue is the more load-bearing threat to the central claim.","tokens_in":14160,"tokens_out":5061,"duration_ms":46233,"concrete_test":"For each pretraining baseline with public code or checkpoints (PretrainGNN, GROVER, GEM, GraphMVP, MolCLR, Uni-Mol, MOLEBLEND, and the supervised GNNs), re-evaluate the original checkpoints on exactly the downstream splits used for MOL-Mamba (8:1:1, 10-fold cross-validation) with the same metrics and, where retraining is feasible, the same GEOM pretraining setup. Then recompute the per-dataset win counts and paired significance tests. If MOL-Mamba's win count drops below the claimed 8/11, the SOTA claim is not supported; if the wins persist under the aligned protocol, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, 'MOL-Mamba outperforms state-of-the-art baselines across eleven chemical-biological molecular datasets,' rests entirely on Tables 2 and 3. The Implementation Details state only that MOL-Mamba is pretrained on GEOM (~50k molecules) and that reported metrics are the averaged results of 10-fold cross-validation. The paper does not state that any baseline was retrained or re-evaluated under this protocol. The table entries closely match values reported in the baselines' original papers, whose pretraining corpora, split strategies, and evaluation procedures differ. For example, GROVER pretrains on a much larger corpus, Uni-Mol uses a large 3D conformer pretraining set, MolCLR uses a different contrastive pretraining corpus, and MOLEBLEND blends 1D/2D/3D modalities. Different downstream splits and single-run versus 10-fold averaging can shift ROC-AUC/RMSE by more than several of the reported gaps. Because Tables 2 and 3 are the only evidence for the headline SOTA claim, that claim is not established unless the baseline numbers were produced under the same protocol. This is a comparability problem, not an accusation of misreporting.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MOL-Mamba, a self-supervised molecular representation learning framework that combines hierarchical structural reasoning (a fragment-level GNN and an atom-level Mamba-Graph with a custom GraphSSM) with an electronic descriptor stream fused through a Mamba-Transformer fuser. Four self-supervised losses are introduced: distribution-collaborative, structure, fragment, and masked e-semantic fusion. The method is evaluated on seven classification and four regression MoleculeNet benchmarks, with ablations over modules, losses, and MG components. The central claim is that MOL-Mamba outperforms state-of-the-art baselines across these datasets.","tokens_in":14418,"tokens_out":11304,"duration_ms":91140,"significance":"If the empirical claims are established, the paper would make a useful contribution to multimodal molecular pretraining: code is released, the internal ablations in Tables 5 and 6 and Figure 3 coherently show that each component and each loss contributes, and Table 4 indicates a favorable parameter/runtime profile relative to graph transformers. The four-loss pretraining recipe and the fusion of electronic descriptors are interesting design choices. However, the headline comparison to state-of-the-art baselines is not currently controlled, and the paper's own tables contradict the strongest form of the SOTA claim. The central contribution therefore remains conditional on a corrected evaluation and a precise specification of the GSSM mechanism.","major_comments":[{"comment":"The headline comparison to state-of-the-art baselines is not controlled. The Implementation Details describe MOL-Mamba's pretraining on GEOM (50k molecules) and report 10-fold cross-validation averages, but no statement is made that any baseline was re-trained or re-evaluated under the same pretraining corpus, downstream splits, and evaluation procedure. The baseline entries appear to match numbers from the original papers, which used different pretraining data (e.g., Uni-Mol's large 3D conformer corpus, MolCLR's contrastive pretraining corpus, GROVER's larger dataset) and potentially different split and fold protocols. Since Tables 2 and 3 are the only evidence for the abstract's SOTA claim, the authors must either run a controlled comparison (same pretraining data, same splits, same number of folds, same evaluation) or explicitly document the source and protocol of every baseline number and justify comparability. Without this, the reported gaps cannot be attributed to MOL-Mamba's design.","section":"Performance Evaluation (Tables 2–3); Implementation Details"},{"comment":"The claim of outperforming state-of-the-art baselines across eleven datasets is not supported by the paper's own results. In Table 2, MOL-Mamba is below SchNet on BBBP (75.0 vs 84.8), below MolCLR on ClinTox (92.7 vs 93.2) and BACE (86.4 vs 89.0); in Table 3, it is below Uni-Mol on QM9 (MAE 0.007 vs 0.005). The statement of 'superior performance on 8 out of 11' is only defensible if the comparison is restricted to self-supervised/pretraining methods (in which case the losses are BBBP, Tox21, HIV, SIDER, MUV, FreeSolv, ESOL, and Lipo). The abstract and conclusion should state the comparison scope precisely and should not claim universal superiority across all eleven benchmarks.","section":"Abstract; Introduction; Tables 2–3"},{"comment":"Algorithm 1 is not a well-defined specification of the proposed GraphSSM. In line 8, Delta has shape b×l×d while A_G and D_G have shape l×l, so the elementwise product Delta ⊙ A_G ⊙ D_G is undefined without an additional assumption. In line 9, the parameter A of shape d×n is reassigned to a tensor of shape b×l×d×n while the summation over i,j is not defined for that tensor. In line 10, B of shape b×l×n and x of shape b×l×d are contracted as B_{jk} x_{ki} without specifying the summation indices or the roles of l and d. Because the GSSM is a central claimed contribution, this pseudocode must be rewritten with explicit shapes and summation conventions, or replaced by precise equations, before the method is reproducible.","section":"Methodology, Algorithm 1 (GraphSSM)"}],"minor_comments":[{"comment":"The notation for dimensions is inconsistent: V_A is R^{l×d_a} but F^G_A is written as R^{h×d_a}, and F^M_A is introduced as R^{H×d} with H never defined. Please unify the symbols for the number of atoms and the number of fragments throughout the paper.","section":"Methodology, Eqs. (2)–(5)"},{"comment":"The loss weights (λ_d, λ_s, λ_f, λ_mask), mask ratio α, and temperature τ are stated without sensitivity analysis. Given that these hyperparameters are used to tune the pretraining objective, a small sensitivity study (or at least a stability discussion) would strengthen the claim that the four-loss recipe is robust.","section":"Implementation Details"},{"comment":"The caption states that both the best and the second-best self-supervised methods are marked in bold, which is ambiguous; please use distinct markers (e.g., bold for the overall best and underline for the best pretraining method) and state the convention clearly.","section":"Tables 2 caption"},{"comment":"The text says each dataset uses 'the recommended splitting method' with an 8:1:1 ratio, but it does not specify whether scaffold splitting or random splitting is used for each of the eleven datasets, nor how QM9 is split. Please state the exact split procedure to allow reproduction.","section":"Experimental Settings, Datasets"},{"comment":"The reference list contains two entries for Luo, Shi, and Thost (2023a and 2023b) with identical titles; one should be corrected or removed.","section":"References"},{"comment":"The model is referred to as 'SE-Mamba' in Figure 3 but as MOL-Mamba elsewhere in the text; please use a single name consistently.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The main issue for the editor is the uncontrolled baseline comparison: Tables 2 and 3 appear to reuse numbers from different papers with different pretraining and evaluation protocols, and the paper's own tables also contradict the abstract's universal SOTA claim. The internal ablations are competent and the code release is a strength, so the manuscript is worth a major revision rather than rejection, provided the authors supply a controlled comparison or explicitly justify the protocol used for each baseline number."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a legitimate engineering contribution with real architectural novelty, but the headline SOTA claim is not supported by the evidence as presented. The paper combines a Mamba state-space model on sorted atom sequences with a fragment-level GNN and a Mamba-Transformer fuser that ingests classical electronic descriptors. That combination is new, and the four-loss pretraining scheme is coherent. The ablations in Tables 5 and 6 and Figure 3 are the strongest part: each module and each loss contributes, the gains are monotonic, and the reported standard deviations are small. The code link is a plus.\n\nThe main problem is the baseline comparison. Tables 2 and 3 report numbers that appear to be taken from the original papers, whose pretraining corpora, splits, and evaluation protocols differ. MOL-Mamba is pretrained on GEOM (~50k molecules) and evaluated with 10-fold cross-validation. GROVER, Uni-Mol, MolCLR, and MOLEBLEND used different pretraining data and often different downstream splits or single-run evaluation. Without retraining baselines under the same protocol, the \"outperforms SOTA across eleven datasets\" claim is not established. This is a comparability issue, not evidence of misreporting, but it is load-bearing: Tables 2 and 3 are the only evidence for the headline.\n\nTwo smaller issues. Algorithm 1 (GraphSSM) is underspecified and the dimensions do not line up: A is introduced as a d×n parameter then used as b×l×d×n; B is computed with sums over indices that do not match its declared shape. The idea is clear but the pseudocode needs work before someone can reproduce it. Also, the electronic descriptors are computed with ChemDes and the paper does not ablate the descriptor choice; it is plausible that the gains come partly from the descriptors rather than from the fusion mechanism itself.\n\nThe paper is not circular: downstream tasks are external benchmarks, and the pretraining losses do not encode target labels. The design choices are hyperparameters, not derivations, but that is normal for this kind of work.\n\nWho is this for? Researchers working on molecular pretraining who want to see a Mamba-based alternative to GNN/Transformer hybrids. It deserves a serious referee: the architecture is worth discussing, the ablations are honest, and the code is public. But the referee should insist on a controlled baseline comparison before the SOTA claim is accepted.","headline":"Mamba+GNN+descriptor fusion is a genuine new combination, but the SOTA claim rests on uncontrolled baseline numbers.","tokens_in":14939,"tokens_out":2142,"would_cite":false,"duration_ms":17806,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MOL-Mamba claims to outperform state-of-the-art baselines on eleven chemical-biological molecular datasets by fusing hierarchical structural reasoning with electronic descriptors.","keywords":["MOL-Mamba","molecular representation learning","property prediction","Mamba","state space models","graph neural networks","electronic descriptors","self-supervised pretraining"],"falsifier":"Retrain Uni-Mol, MOLEBLEND, GEM, and MolCLR on the same GEOM corpus with the same 8:1:1 splits and evaluation procedure described in the paper; if their mean ROC-AUC or RMSE values reach or exceed MOL-Mamba's reported numbers, the 'outperforms state-of-the-art' claim is not supported.","tokens_in":13971,"feed_emoji":"🧪","tokens_out":6919,"duration_ms":50416,"temperature":0.7,"pith_summary":"MOL-Mamba is a self-supervised pretraining framework for molecular property prediction. Its central claim is that combining hierarchical structural reasoning with electronic information produces state-of-the-art results: a fragment-level graph neural network and an atom-level Mamba-Graph module capture structure, while a Mamba-Transformer fuser learns correlations between structure and molecular electronic descriptors. The framework is trained on the GEOM corpus with four joint losses, then fine-tuned on eleven MoleculeNet benchmarks, where the paper reports it outperforms existing GNN and graph-transformer baselines on most of them. If true, the contribution is a parameter-efficient recipe—6.98M parameters—that closes part of the gap between structure-only pretraining and multimodal fusion.","feed_headline":"MOL-Mamba tops 8 of 11 molecular benchmarks","feed_subtitle":"A hybrid Mamba-Transformer fuses fragment-graph structure with electronic descriptors, outperforming larger graph transformers.","key_machinery":"The central mechanism is the Mamba-Graph (MG) module: a GNN-initialized atom sequence, sorted by fragment membership and node degree, that passes through a Mamba selective state-space block whose SSM parameters are modulated by the adjacency and distance matrices (GraphSSM), followed by fragment-wise max-pooling. The Mamba-Transformer (MT) fuser then concatenates the resulting structure embedding with masked electronic descriptor tokens and reconstructs the mask, aligning the two modalities.","core_discovery":"The paper claims that MOL-Mamba outperforms state-of-the-art baselines across eleven chemical-biological molecular datasets. Its discovery is that a Mamba-enhanced graph learner can serve as the structural backbone and that fusing electronic descriptors through a Mamba-Transformer module adds complementary signal. Across seven classification and four regression benchmarks, the paper reports the best mean scores on eight of them, including Tox21 (81.3 ROC-AUC), MUV (89.0 ROC-AUC), BBBP (75.0 ROC-AUC), FreeSolv (1.02 RMSE), ESOL (0.63 RMSE), and Lipo (0.53 RMSE), while using fewer parameters and lower runtime than graph transformers such as Uni-Mol and MOLEBLEND.","pith_inferences":["A direct extension would be to vary the descriptor set (E-state, molecular property, quantum-chemical, charge) and the mask ratio to identify which electronic descriptors carry the predictive signal; the paper does not report such a breakdown.","The GraphSSM idea—modulating selective state-space parameters with graph adjacency and distance—may transfer to non-molecular graph tasks where long-range dependencies matter.","Because the baseline numbers are taken from the original papers, the claimed margins should be confirmed by re-running baselines under the same GEOM pretraining and identical splits before relying on them.","The fragment-based sorting could be replaced by other graph orderings (e.g., breadth-first or centrality) to test whether fragment membership is essential or only a convenient ordering heuristic."],"forward_implications":["Under the paper's protocol, MOL-Mamba reaches state-of-the-art mean scores on 8 of 11 MoleculeNet benchmarks, including Tox21 (81.3 ROC-AUC) and ESOL (0.63 RMSE).","The architecture runs at GNN-level cost (0.0012 s/molecule, 6.98M parameters) while outperforming graph transformers with 47.1M parameters.","Removing any of the four pretraining losses lowers performance, and removing the GraphSSM modulation causes the largest drop in the MG module's ablations.","The masked E-semantic fusion loss can be viewed as a self-supervised denoising task over electronic descriptors, which the paper shows improves feature separation in t-SNE error rates."],"supporting_citations":[{"why":"Supplies the Mamba selective state-space block that the MG module adapts into GraphSSM.","marker":"Gu and Dao 2023"},{"why":"Provides the Principal Subgraph Mining algorithm that constructs the fragment-level graph.","marker":"Kong et al. 2022"},{"why":"Provides the Transformer attention backbone for the MT fuser and the LayerNorm used throughout.","marker":"Vaswani et al. 2017"},{"why":"The SchNet GNN used as the atom-level encoder inside MG and as a supervised baseline.","marker":"Scḧutt et al. 2017"},{"why":"The GIN architecture used for the fragment-level GNN and as a baseline.","marker":"Xu et al. 2019"},{"why":"The Uni-Mol 3D pretraining baseline that MOL-Mamba must beat.","marker":"Zhou et al. 2023"},{"why":"The MOLEBLEND multimodal pretraining baseline that MOL-Mamba must beat.","marker":"Yu et al. 2024"},{"why":"ChemDes tool that computes the electronic descriptors fused in the MT module.","marker":"Dong et al. 2015"},{"why":"GraphMVP, establishes the GEOM pretraining corpus and protocol followed here.","marker":"Liu et al. 2022"}],"fun_headline_variants":["MOL-Mamba wins 8 of 11 with Mamba-Transformer fusion","Structural + electronic fusion gives MOL-Mamba 8 benchmark wins","Mamba-Transformer powers MOL-Mamba to 8 of 11 SOTA","MOL-Mamba: faster, lighter, and tops 8/11 molecular tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that MOL-Mamba outperforms state-of-the-art methods assumes the baseline results in Tables 2 and 3, reported from their original papers under different pretraining corpora and settings, are directly comparable to results from MOL-Mamba's own protocol.","fun_headline_variants_meta":{"raw":{"variants":["MOL-Mamba wins 8 of 11 with Mamba-Transformer fusion","Structural + electronic fusion gives MOL-Mamba 8 benchmark wins","Mamba-Transformer powers MOL-Mamba to 8 of 11 SOTA","MOL-Mamba: faster, lighter, and tops 8/11 molecular tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000292,"raw_usage":{"total_tokens":1675,"prompt_tokens":886,"completion_tokens":789,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":703}},"tokens_in":502,"tokens_out":789,"duration_ms":7199,"temperature":1.0,"reasoning_tokens":703,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:32:19.370144+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain Uni-Mol, MOLEBLEND, GEM, and MolCLR on the same GEOM corpus with the same 8:1:1 splits and evaluation procedure described in the paper; if their mean ROC-AUC or RMSE values reach or exceed MOL-Mamba's reported numbers, the 'outperforms state-of-the-art' claim is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Principal Subgraph Mining algorithm that constructs the fragment-level graph."},{"cited_title":"u tt, K.; Kindermans, P.; Felix, H. E. S.; Chmiela, S.; Tkatchenko, A.; and M \\","cited_arxiv_id":null,"evidence_quote":"The SchNet GNN used as the atom-level encoder inside MG and as a supervised baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Uni-Mol 3D pretraining baseline that MOL-Mamba must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ChemDes tool that computes the electronic descriptors fused in the MT module."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GraphMVP, establishes the GEOM pretraining corpus and protocol followed here."}],"review_version":1}