{"id":"c47734f9-6be7-47ef-8571-4c6bca38ae12","arxiv_id":"2505.05877","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A multi-modal molecular pretraining framework using autoencoders, hypergraph convolution, and a memory bank reports state-of-the-art ROC-AUC and RMSE on MoleculeNet benchmarks.","lead":"This paper presents MMSA, a self-supervised pretraining framework that combines 2D graphs, 2D images, and 3D graphs to learn molecular representations. It reports improved property prediction and retrieval results on MoleculeNet benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The prediction loss in Eq. (13) depends on labels Y_geom and Y_prop that Section IV-B-2 never defines; this makes the supposedly self-supervised pretraining objective underspecified, and the reported gains cannot be reproduced until the labels and their source are specified.","rationale":"The paper's central claim is a reproducible state-of-the-art result on MoleculeNet. For that claim to hold, the pretraining objective in Eq. (15) must be fully specified. Eq. (13) is the only place where supervised labels appear, and those labels are undefined. This is an internal incompleteness, not a disagreement with consensus. The overlap concern raised by the Reader is also real, but it is a distributional concern that affects many pretraining baselines using GEOM; the missing Y_geom/Y_prop blocks even a single run of the method. I therefore regard conditional acceptance as the appropriate verdict: the method may work, but code and label definitions are required before the empirical claims can be verified. There is no formal verification, and the code link is a placeholder, but the method is coherent and the ablations are internally consistent, so I would not reject on the present record.","tokens_in":21225,"tokens_out":4484,"duration_ms":49158,"concrete_test":"Request the exact script that generates Y_geom and Y_prop for the GEOM 100K subset, then rerun pretraining on the same data with α=1 (removing L_pre) and with the reported α, using identical downstream splits. If the α=1 result is within one standard deviation of the Table II numbers, L_pre and its labels are not responsible for the gains; if the gap is larger, the undefined labels are an active ingredient, and their source must be documented and checked for overlap with MoleculeNet targets.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section IV-B-2, L_pre is defined as |Y_geom - Ŷ_geom^z| + |Y_prop - Ŷ_prop^z| + ... with Y_geom and Y_prop described only as \"true labels for molecular geometry and chemical properties.\" No definition, dimensionality, or source is given in Table I or in the pretraining description in Section V-A.1, where the 100K GEOM Drugs subset is introduced. This matters because L_pre is half of L_sa (Eq. 14) and L_sa enters the total loss directly (Eq. 15): the structure-awareness module, the paper's main novelty, is not self-supervised as written unless these targets are generated without downstream labels. The claim in Section V-D that the method \"does not heavily depend on labeled data\" is therefore unsupported by the stated objective. This problem is more load-bearing than the pretraining/test overlap: even with an overlap-free evaluation, no one can rerun the pretraining objective as specified. This is an omission, not evidence of misconduct, but it blocks verification of the central state-of-the-art claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MMSA, a multi-modal molecular representation pre-training framework that combines modality-specific auto-encers over 2D graphs, 2D images, and 3D conformations with a hypergraph-based structure-awareness module and a memory bank of prototype anchors. The model is pre-trained on 100K molecules sampled from GEOM Drugs, then fine-tuned on eight MoleculeNet classification datasets and four regression datasets, reporting an average ROC-AUC of 75.94% and best RMSE on all regression tasks. The paper also presents retrieval experiments, ablations, and a versatility study in which MMSA is plugged into several existing graph-based methods. The central claim is that MMSA achieves state-of-the-art molecular property prediction by exploiting higher-order inter-molecular relations and invariant knowledge.","tokens_in":21506,"tokens_out":4063,"duration_ms":41012,"significance":"If the evaluation is clean, the work is a useful contribution to molecular representation learning: hypergraph modeling of inter-molecular relations and memory-anchor alignment are reasonable and relatively novel extensions of existing multi-modal pre-training, and the plugin-style integration with multiple GNN baselines (Fig. 4) is practically attractive. The paper also provides a fairly broad evaluation across classification, regression, retrieval, ablation, and backbone-architecture analyses. However, two verification blockers prevent me from assessing whether the reported gains are due to the proposed mechanism or to leakage and unspecified supervision: the prediction targets in Eq. (13) are never defined, and the pretraining corpus is not shown to be disjoint from the downstream evaluation sets.","major_comments":[{"comment":"The prediction loss Lpre uses \"true labels for molecular geometry and chemical properties,\" Ygeom and Yprop, but these labels are never defined: no dimensionality, source, or computation method is given for the 100K GEOM Drugs pretraining corpus. Lpre enters Lsa (Eq. 14) and Loverall (Eq. 15), so the pretraining objective is underspecified and cannot be reproduced as written. This also undermines the self-supervised claim and the statement in Section V-D that MMSA \"does not heavily depend on labeled data,\" because the objective includes supervised prediction targets. Please define these targets precisely and, if they are obtained from external labels, clarify how the method remains self-supervised.","section":"§IV-B-2, Eq. (13)"},{"comment":"The paper does not state whether the 100K GEOM Drugs pretraining subset was checked for overlap with the molecules in the MoleculeNet classification and regression datasets used for downstream evaluation. MoleculeNet datasets draw from public chemical libraries, and GEOM Drugs is a large collection of drug-like molecules, so without an overlap filter there is a concrete risk that test molecules or near-duplicates appeared in pretraining. Please report overlap statistics after exact-matching and, ideally, InChI/ECFP-based near-duplicate filtering, and clarify whether any leaked molecules were excluded before the final runs.","section":"§V-A.1, §V-A.2"},{"comment":"There is a direct inconsistency between the hyperparameter settings: Section V-A.3 states λ=0.6 and α=0.5, while Section V-G states that the main experiments use λ=0.7 and α=0.2, and Fig. 6 reports the best BACE ROC-AUC of 82.8 at λ=0.6, whereas Table II lists BACE=82.4 for the final MMSA model. This makes it unclear which configuration produced the headline results. Because hyperparameters appear to have been tuned on BACE while BACE is included in the headline table, please disclose the model-selection procedure, report results for a fixed configuration chosen before seeing test labels, and either exclude BACE from the headline or present it as a validation-tuned result.","section":"§V-G vs. §V-A.3"}],"minor_comments":[{"comment":"The reproducibility statement says the code is \"accessible on here,\" but no URL or repository identifier is provided; please insert a working link so the listed code, datasets, and configurations are actually available.","section":"Reproducibility Statement"},{"comment":"The phrase \"with values ranging from 0 to 4\" is unclear; I assume it refers to the standard deviation values, but as written it seems to describe the ROC-AUC scores themselves. Please reword.","section":"Table II caption"},{"comment":"There are minor formatting inconsistencies: \"Table 6\" should be \"Table VI,\" and \"TABLE IX\" should be formatted consistently with the other table references.","section":"§V-D, §V-G"},{"comment":"The row label \"Graphformor\" is a typo and should read \"Graphformer\" to match the text and the cited model.","section":"Table VIII"},{"comment":"The figure includes GraphCL as a baseline, but GraphCL is not listed in the baselines of Section V-A.2 and no reference is provided; please add the citation and clarify the evaluation protocol for this method.","section":"Fig. 4"},{"comment":"The sentence \"MMSA without Lpre outperforms the model without Lme, suggesting that the proposed method does not heavily depend on labeled data\" is not logically supported: removing Lpre does not remove all label dependence, and the comparison of two ablations does not establish a general claim about label dependence. Please rephrase or remove this claim once the Ygeom/Yprop targets are specified.","section":"§V-D"}],"recommendation":"major_revision","confidential_remarks":"I would not reject the paper: the architecture and evaluation breadth are potentially valuable, and the issues identified are fixable in principle. However, the undefined Ygeom/Yprop in Eq. (13) and the lack of any pretraining/evaluation overlap analysis are verification blockers that must be resolved before acceptance. I also recommend checking the hyperparameter inconsistency between Sections V-A.3 and V-G, since the headline results depend on which configuration was actually used."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, it reports a plausible and reasonably engineered method: MMSA combines multi-modal autoencoders, hypergraph convolution, and a memory bank for molecular pretraining, and it obtains consistent ROC-AUC improvements (roughly 2 to 10 points) over reproduced baselines on eight MoleculeNet classification tasks. Second, the paper has a load-bearing gap that blocks verification: the prediction loss in Eq. (13) uses labels Y_geom and Y_prop that are never defined. These targets enter half of the structure-awareness loss, which is half of the total pretraining objective. Without knowing where they come from, the method cannot be rerun, and the claim that it is self-supervised is unsupported.\n\nWhat is genuinely new is the specific combination—hypergraph construction over multi-modal embeddings plus memory anchors—which I have not seen in the cited literature. The experiments are broad: classification, regression, retrieval examples, ablations, and integration into five GNN backbones. The method description is mostly clear, and the complexity analysis is a nice touch.\n\nThe soft spots are real. The undefined Y_geom/Y_prop is the most serious. The reader also worried about pretraining/test overlap: the 100K GEOM Drugs subset is randomly selected with no stated exclusion of MoleculeNet molecules. That is a legitimate concern, though secondary—even with an overlap filter, the undefined labels block reproduction. I also noticed an internal inconsistency the reader missed: the ablation tables show the full model at 74.5 ROC-AUC on BBBP, while the main result in Table II is 82.4. No explanation is given for the 8-point gap. The hyperparameters also disagree (α=0.5 in Section V-A.3 vs α=0.2 in Section V-G), and the code link is a placeholder.\n\nThe method is not obviously wrong, and the direction is sensible. But the current write-up does not support the state-of-the-art claim. I would send it to peer review—the idea deserves scrutiny—but the revision must define or remove the prediction targets, run an overlap-free evaluation, and reconcile the ablation numbers with the main tables. As is, I would not cite or build on it.","headline":"MMSA is a workmanlike multi-modal pretraining plugin with modest benchmark gains, but the pretraining objective is underspecified and the ablation numbers contradict the main tables—don't trust the SOTA claim until those are fixed.","tokens_in":22020,"tokens_out":4456,"would_cite":false,"duration_ms":40208,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that MMSA, a self-supervised molecular pretraining framework combining multi-modal autoencoders, hypergraph convolution, and a memory bank, produces embeddings that outperform prior methods on MoleculeNet classification…","keywords":["molecular representation learning","self-supervised pretraining","multi-modal learning","hypergraph neural networks","memory bank","MoleculeNet","drug discovery","graph neural networks"],"falsifier":"Compute the overlap between the 100K GEOM Drugs pretraining set and each MoleculeNet dataset using canonical SMILES or InChI keys, then rerun the eight classification and four regression benchmarks with all overlapping molecules removed from pretraining; if MMSA's reported advantages shrink to near-baseline levels, the central claim of representation-level improvement is not supported. The same check applies to scaffold-disjoint splits, since the paper's evaluation uses scaffold splitting.","tokens_in":21033,"feed_emoji":"🧪","tokens_out":7442,"duration_ms":61904,"temperature":0.7,"pith_summary":"The paper proposes MMSA, a self-supervised pretraining framework for molecular representation learning. It tries to establish that combining several views of the same molecule—the 2D graph, a 2D image, and the 3D conformation—with higher-order structure modeling yields embeddings that transfer to downstream drug-discovery tasks better than existing unimodal or simple fusion pretraining methods. The framework aligns modality-specific embeddings with contrastive and reconstruction losses, aggregates them into a unified embedding, and then refines this embedding through hypergraph convolution over molecule neighborhoods and a memory bank of prototypical anchors. On eight MoleculeNet classification datasets the authors report an average ROC-AUC of 75.94%, an improvement of 1.80 to 10.01 points over their reproduced baselines, and the best average RMSE on four regression datasets. This matters because molecular property prediction is a central step in drug discovery, where label scarcity makes pretrained representations the main practical route.","feed_headline":"Multimodal pretraining hits 75.94% ROC-AUC on MoleculeNet","feed_subtitle":"Hypergraph neighborhoods and a memory bank lift molecular property predictions across eight benchmarks.","key_machinery":"The load-bearing object is the structure-awareness module. A hypergraph is constructed by connecting each molecule to its $K-1$ nearest neighbors in embedding space, so each hyperedge links $K$ molecules; hypergraph convolution (HGNNConv) propagates information along these hyperedges, allowing the model to represent interactions that involve more than two molecules. A memory bank holds $L$ learnable anchor embeddings; the model scores how similar a molecule's embedding is to each anchor, takes a softmax-weighted sum of the anchors to form an aligned embedding, and trains with a memory loss that keeps the aligned embedding close to the original plus prediction losses for molecular geometry and property targets. These two mechanisms are what the paper credits with capturing higher-order correlations and invariant knowledge, and they are the components that distinguish MMSA from earlier modality-fusion methods.","core_discovery":"The central claim is that invariant knowledge shared across molecules, rather than information confined to individual molecules, is what makes molecular representations transfer. Concretely, MMSA encodes a molecule with three feature extractors—GIN for the 2D graph, ResNet-18 for the 2D image, and ComENet for the 3D conformation—projects each modality into a shared latent space with autoencoders, and fuses them into one embedding. The structure-awareness module then places every molecule as a node in a hypergraph built by K-nearest-neighbor search, applies hypergraph convolution to capture dependencies among several molecules at once, and aligns the resulting embedding with learnable memory anchors. The authors report that this pipeline achieves 75.94% average ROC-AUC on eight MoleculeNet classification tasks, gains of 1.80 to 10.01 points over reproduced baselines, best average RMSE on four regression tasks, and successful within- and cross-dataset molecular retrieval.","pith_inferences":["Editorial inference: the paper does not report filtering overlaps between the 100K GEOM Drugs pretraining molecules and the MoleculeNet evaluation sets, so some of the reported advantage could reflect memorization of near-duplicate molecules; a scaffold-disjoint pretraining split would test this.","Editorial inference: the memory bank is effectively a soft clustering of embeddings around learnable prototypes, so the same structure-awareness recipe could transfer to other graph-level self-supervised settings, such as protein or materials property prediction, though the paper does not test this.","Editorial inference: because hyperedges are built within each training batch, the higher-order signal depends on batch composition; larger or more diverse batches could strengthen the effect without any architectural change.","Editorial inference: the image branch uses a fixed pretrained ResNet-18, and the reported gains from adding images come with that encoder frozen; fine-tuning the image encoder during pretraining is an untested extension that might push the image modality further."],"forward_implications":["If the central claim holds, pretraining with MMSA yields one encoder whose embeddings serve classification, regression, and retrieval, so a single pretrained model can replace task-specific pretraining pipelines.","Because the paper shows MMSA improving five existing baselines (AttrMask, GraphCL, GraphMVP, GraphMAE, Mole-BERT), the mechanism is claimed to be a plug-in that raises the performance of whatever graph encoder already exists.","The hypergraph component is doing real work: replacing it with a GCN of matched capacity lowers ROC-AUC on BBBP, Tox21, and ClinTox and raises RMSE on ESOL and Lipo.","The memory bank is doing distinct work: ablating the memory loss hurts BBBP and Tox21 ROC-AUC more than ablating the prediction loss, which the paper reads as evidence that invariant knowledge improves generalization.","The gains are not tied to one architecture, since MMSA improves GCN, GAT, GIN, GraphSAGE, and Graphformer backbones."],"supporting_citations":[{"why":"Supplies the eight MoleculeNet classification datasets and the ROC-AUC evaluation protocol that the main results are measured on.","marker":"[48]"},{"why":"Supplies the 100K GEOM Drugs molecules with 2D and 3D structures used for pretraining.","marker":"[47]"},{"why":"Provides the GIN architecture used to encode 2D molecular graphs and as the backbone for downstream fine-tuning.","marker":"[43]"},{"why":"Provides ComENet, the 3D graph encoder that brings spatial and conformational information into the multimodal embedding.","marker":"[45]"},{"why":"Provides the ResNet-18 image encoder that produces the 2D image modality features.","marker":"[44]"},{"why":"Defines hypergraph convolution (HGNNConv), the operation used to propagate information along hyperedges in the structure-awareness module.","marker":"[46]"},{"why":"Defines GraphMVP, the 2D/3D contrastive pretraining baseline whose regression datasets and evaluation setup MMSA adopts and compares against.","marker":"[27]"},{"why":"Defines IME, the image-enhanced graph representation baseline that MMSA extends with hypergraph and memory mechanisms.","marker":"[42]"}],"fun_headline_variants":["Hypergraph and memory bank lift molecular ROC-AUC to 75.94%","MMSA: structure-aware multimodal pretraining beats MoleculeNet baselines","Invariant knowledge from hypergraph neighborhoods improves molecular embeddings","Image + graph + 3D pretraining with structure awareness hits 75.94% AUC","Memory anchors align molecular representations for better generalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that the 100K GEOM Drugs molecules used for pretraining do not overlap with the MoleculeNet molecules used for downstream evaluation, and no overlap filter is reported; if test molecules appeared in pretraining, the reported ROC-AUC gains could reflect memorization rather than representation learning.","fun_headline_variants_meta":{"raw":{"variants":["Hypergraph and memory bank lift molecular ROC-AUC to 75.94%","MMSA: structure-aware multimodal pretraining beats MoleculeNet baselines","Invariant knowledge from hypergraph neighborhoods improves molecular embeddings","Image + graph + 3D pretraining with structure awareness hits 75.94% AUC","Memory anchors align molecular representations for better generalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000725,"raw_usage":{"total_tokens":3276,"prompt_tokens":997,"completion_tokens":2279,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":2186}},"tokens_in":613,"tokens_out":2279,"duration_ms":16257,"temperature":1.0,"reasoning_tokens":2186,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:53:44.968178+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the overlap between the 100K GEOM Drugs pretraining set and each MoleculeNet dataset using canonical SMILES or InChI keys, then rerun the eight classification and four regression benchmarks with all overlapping molecules removed from pretraining; if MMSA's reported advantages shrink to near-baseline levels, the central claim of representation-level improvement is not supported. The same check applies to scaffold-disjoint splits, since the paper's evaluation uses scaffold splitting.","supporting_citations":[{"cited_title":"How powerful are graph neural networks?","cited_arxiv_id":null,"evidence_quote":"Provides the GIN architecture used to encode 2D molecular graphs and as the backbone for downstream fine-tuning."},{"cited_title":"Moleculenet: A benchmark for molecular machine learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the eight MoleculeNet classification datasets and the ROC-AUC evaluation protocol that the main results are measured on."},{"cited_title":"Geom, energy-annotated molec- ular conformations for property prediction and molecular generation,","cited_arxiv_id":null,"evidence_quote":"Supplies the 100K GEOM Drugs molecules with 2D and 3D structures used for pretraining."},{"cited_title":"An image-enhanced molecular graph representation learning framework,","cited_arxiv_id":null,"evidence_quote":"Defines IME, the image-enhanced graph representation baseline that MMSA extends with hypergraph and memory mechanisms."}],"review_version":1}