{"id":"cc9877a8-9377-40ab-9f9c-9199b0adf3fa","arxiv_id":"2501.15799","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Augmenting molecular encoders with a similarity graph of smaller 'ancestor' molecules and their property labels improves regression MAE by a reported 32.3% on QM7 and QM9.","lead":"Molecular property prediction can be improved, the authors claim, by feeding a model the 'evolutionary path' connecting a molecule to smaller, structurally similar molecules, together with the known property values along that path. The proposed MEvoN-MPP method reports large average gains over standard graph and 3D encoders on QM7 and QM9.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Concern: reported 32.3% gain may be label leakage through EvoL, which consumes property labels of full-dataset evolutionary paths; the paper's ablation shows the path-only EvoP component actually hurts, so split-restricted evaluation is needed.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the EvoL module uses property labels of molecules along evolutionary paths, and the network is constructed on the full dataset without stating that paths and labels are restricted to the training split. Our independent reading confirms this is the most likely source of the reported gains, especially because the ablation shows the path-only EvoP component is harmful while the label-only EvoL component helps substantially. The reader's CONDITIONAL verdict is appropriate: the paper should be accepted only if the authors disclose the split handling and demonstrate that the improvement persists when test-time label inputs are eliminated. We therefore keep the verdict UNCHANGED rather than escalating to REJECT, since the manuscript is internally consistent and the leakage can be repaired with a clarified protocol.","tokens_in":13672,"tokens_out":3773,"duration_ms":37686,"concrete_test":"Re-run the QM9 and QM7 experiments with a strict training-only split protocol: construct the MEvoN using only training-set molecules; for each test molecule, prune its evolutionary path to contain only training-set molecules, and if no such path exists, fall back to the MolE-only baseline; in EvoL, mask the labels of any molecule not in the training set. Run the four baselines on the identical set of molecules that have valid paths. If the 32.3% average improvement shrinks to a few percent or becomes statistically insignificant, the reported gain is an artifact of label leakage via full-dataset path labels.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that evolutionary-path features enhance molecular representation, quantified as a 32.3% average MAE improvement over four encoders (Table 2). This claim depends on the improvement being caused by the evolutionary path representation rather than by the label information fed into the EvoL module. The protocol in Section 4.1 constructs the MEvoN over the full dataset, including validation and test molecules, and Section 3 describes EvoL as taking the property labels along each evolutionary path (with only the final label masked) as input. Because the dataset is randomly split 8:1:1 (Section 4.0), a test molecule's path will typically contain other test or validation molecules whose labels are therefore available as inputs at test time. This turns the regression task into label interpolation along a similarity graph, not representation enhancement. The paper never states that paths or labels are restricted to the training split. The ablation in Table 4 strengthens the concern: adding EvoP alone makes performance worse (MAE 0.2916 to 0.9809), while adding EvoL alone improves MAE to 0.4051, and the combined system reaches 0.2270. Thus the observed gain is attributable to the label-sequence branch, not to the evolutionary path features, and the evaluation as described cannot distinguish genuine representation learning from label leakage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Molecular Evolutionary Network (MEvoN), a similarity-based graph over molecules grouped by atom count, from which evolutionary paths are extracted. The proposed MEvoN-MPP method augments standard molecular encoders (GCN, GIN, SchNet, ComENet) with an EvoP module that encodes path features and an EvoL module that encodes property labels along the path, then predicts both the target property and the property difference between the last two molecules on the path. The authors report an average 32.3% MAE improvement over four base encoders on QM9 and competitive results on QM7, and they present ablations, SHAP analyses, and a path-number hyperparameter study.","tokens_in":13842,"tokens_out":4520,"duration_ms":42092,"significance":"If the reported gains were obtained under a leakage-free protocol, a plug-in that improves four diverse encoders by 32.3% average MAE on standard quantum-property benchmarks would be a meaningful contribution, and the public code is a strength. However, as described, the evaluation pipeline feeds known target-property labels of ancestor molecules into the EvoL module, and the MEvoN is constructed over the full dataset. The reported improvements therefore cannot currently be attributed to evolutionary-path representation; they may be due to label interpolation along the similarity graph. The ablations in Table 4 further show that the path-only EvoP module degrades performance, so the central claim is not supported by the evidence as presented.","major_comments":[{"comment":"The evaluation is compromised by target-label information entering the prediction pipeline. In Section 3, the EvoL module receives the label sequence Y_i along each evolutionary path, and only the final label y_{l-1} is masked; y_pred2 is a function of H_EvoL and is trained against y_{l-2} - y_{l-1}. Section 4.1 states that each MEvoN is constructed from the full molecular dataset, and the 8:1:1 random split is applied to the dataset after network construction. The paper does not state that paths are restricted to training molecules or that ancestor labels are available only for training molecules. Consequently, for a test molecule, its evolutionary path can contain other test or validation molecules whose property labels are known inputs at test time. This turns the regression into label interpolation along the similarity graph rather than representation enhancement. The authors must rerun the experiments with a split-restricted protocol: construct the MEvoN using only training molecules, use only training labels in EvoL, and report test performance under that protocol. Without this, Tables 2-4 cannot be interpreted as evidence for the paper's central claim.","section":"Section 3, Eqs. (14)-(15); Section 4.1"},{"comment":"The ablation study directly contradicts the claim that evolutionary-path features enhance molecular representation. Adding the EvoP module alone worsens the GAP MAE from 0.2916 to 0.9809 on the Mol-branch and to 0.5187 on the Evo-branch, while adding EvoL alone improves the Mol-branch to 0.4051 and the combined system reaches 0.2270. Thus, the observed gain relative to the MolE baseline is driven by the label-sequence branch, not by the evolutionary-path representation. The manuscript should either reframe the contribution as a label-sequence augmentation method or provide a variant that obtains improvements from path features alone (e.g., without any property labels as inputs).","section":"Table 4, Section 4.3"},{"comment":"The exclusion of 555 QM9 molecules from the MEvoN, including 446 fluorine-containing molecules and some N-containing cyclic structures, is a potential source of benchmark bias. These molecules are not reported in the evaluation, and the paper does not state whether they are excluded from both the training and test splits or only from the network construction. If the test set is restricted to molecules that happen to have high-similarity evolutionary neighbors, the reported MAE improvements may not generalize to the full QM9 distribution. The authors should report performance on the excluded molecules and on the full test set, or justify the exclusion as part of the method rather than a post hoc filter.","section":"Section 4.1"}],"minor_comments":[{"comment":"The text introduces the module as 'label-aware (EvoA)' but the rest of the paper uses 'EvoL'; the notation should be made consistent.","section":"Section 3, first paragraph"},{"comment":"The sentence 'Figure 1 shows the number of molecules in the datasets...' appears to refer to Table 1, not Figure 1; the cross-reference should be corrected.","section":"Section 4.1"},{"comment":"The table caption and column headers do not define 'Mol-branch' and 'Evo-branch'; these terms should be linked explicitly to the MolE and Evo branches in Section 3, Eqs. (13)-(15).","section":"Table 4"},{"comment":"The reported 'average of 32.3%' is an average of the four model-specific mean improvements; the paper should state this aggregation explicitly and provide per-seed standard deviations, since only seed 42 is reported for QM9.","section":"Section 4.2, Table 2"},{"comment":"The path-number experiment reports no error bars or repeated-seed statistics, and it is unclear whether K was selected on the validation set; a single deterministic run is insufficient to support the stability claim for K between 3 and 5.","section":"Section 4.4, Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the operational circularity of the evaluation: target labels of non-training molecules are consumed by EvoL, and the ablation shows the non-label path branch hurts. If the authors cannot supply a split-restricted protocol with no target-label inputs at test time, the paper should be rejected; with such results, the contribution could be reframed as label-sequence augmentation. The QM7 comparison against pretraining-based baselines also needs clarification of the exact splits and protocols used for the baseline numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you want a clean example of why split design matters in graph-based property prediction. The paper is not a toy: MEvoN builds a similarity network over molecules grouped by atom count, thresholds edges with edit-distance then Weisfeiler-Lehman similarity, extracts evolutionary paths from root to each molecule, and feeds those paths through a Transformer while a second branch encodes the known property labels along the path. That construction is new as a combination and the writing is clear. The tables are internally arithmetically consistent, and the anonymous code link is a plus.\n\nThe problem is that the central claim, a 32.3% average MAE improvement over four encoders, is not supported by the protocol as written. The network is constructed over the full dataset with an 8:1:1 random split. For a test molecule, its evolutionary path will frequently include other test or validation molecules. The EvoL module takes the property labels of all molecules on the path except the final one as input. Since those labels are known for the whole dataset, this is label interpolation along the similarity graph, not representation enhancement. The paper never states that paths or labels are restricted to the training split. That is the key missing sentence.\n\nThe ablation strengthens the concern rather than resolving it. Table 4 shows that adding EvoP alone makes GAP prediction worse (MAE 0.9809 vs 0.2916), adding EvoL alone helps (0.4051), and the combined system reaches 0.2270. So the observed gain comes from the label-encoding branch, not from the evolutionary-path features. The paper interprets this as collaboration, but the evidence says the path module is at best neutral when labels are present. The title claim, that evolutionary history enhances representation, is therefore not demonstrated.\n\nA smaller issue: 555 QM9 molecules are excluded from the network, and the paper does not say whether the baselines were evaluated on the same molecule set. If the baselines were run on the full split, the comparison is unfair.\n\nWho gets value from this? A reader interested in graph-based property prediction who wants to see an interesting idea and a cautionary protocol flaw. It deserves a serious referee, because the flaw is identifiable and fixable: restrict path and label construction to the training split, add a label-propagation baseline (e.g., predicting from nearest-neighbor labels only) and a parameter-matched model without EvoP, and report same-set baselines. If the gain survives that, the 32.3% claim would be meaningful. I would not cite the current version as evidence for the method, but I would send it to peer review with a request for a corrected evaluation.","headline":"The MEvoN idea is worth a look, but the 32.3% gain is not established: EvoL consumes labels along full-dataset paths, and the ablation shows the path-only module actually hurts.","tokens_in":14471,"tokens_out":2991,"would_cite":false,"duration_ms":27189,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tracing a molecule's evolutionary path cuts property-prediction error by 32%.","keywords":["molecular representation","evolutionary network","property prediction","QM9","QM7","graph neural networks","molecular similarity","HOMO-LUMO gap"],"falsifier":"Re-run the QM9 experiments after building the MEvoN exclusively from the training split, so that every test molecule's evolutionary path lies entirely inside the training set; alternatively, remove the EvoL label-input module and keep only structural path features. If the reported 32.3% average MAE gain collapses under either protocol, the improvement depends on transductive label access rather than on evolutionary representation.","tokens_in":13342,"feed_emoji":"🧬","tokens_out":4096,"duration_ms":38162,"temperature":0.7,"pith_summary":"The paper claims that adding a molecule's evolutionary history as auxiliary input to a standard molecular encoder significantly improves property prediction. The authors construct a molecular evolutionary network (MEvoN) by grouping molecules by atom count and linking similar molecules across groups, then extract structural path and property-label sequences from this network. Plugging these sequences into GCN, GIN, SchNet, and ComENet yields a reported average 32.3% reduction in mean absolute error on the QM7 and QM9 benchmark tasks. If this is right, representation learning for molecules gains from relational structure among similar molecules, not just from each molecule's isolated structure. The evaluation, however, assumes that property labels of ancestor molecules along each test molecule's path are available as input.","feed_headline":"Tracing a molecule's evolution cuts prediction error by 32%","feed_subtitle":"Adding ancestor molecules' structures and labels as input improves four standard encoders on QM7 and QM9 benchmarks.","key_machinery":"The load-bearing object is the molecular evolutionary network (MEvoN): a directed network in which molecules are nodes grouped by atom count, and edges point from smaller 'ancestor' molecules to larger 'descendant' molecules when edit-distance or fingerprint Tanimoto similarity exceeds a threshold $\theta_1$ and Weisfeiler-Lehman graph-kernel similarity exceeds a second threshold $\theta_2$. From this network, evolutionary paths are extracted by backtracking from a target molecule to the root, and both the sequence of molecular embeddings and the sequence of property labels along each path are fed through Transformer encoders. The resulting path and label features are concatenated with the base encoder's own features to produce the final prediction, which is what carries the argument that evolutionary context enhances representation.","core_discovery":"The central claim is that a molecule's evolutionary trajectory, not only its static structure, carries predictive signal for quantum-chemical properties. MEvoN-MPP combines three components: a molecular encoder (MolE) that can be any existing model; an EvoP module that encodes the sequence of graph features along the evolutionary path with a Transformer; and an EvoL module that encodes the property labels along that same path. The model simultaneously predicts the target molecule's property and the property difference between the last two molecules on the path, masking the target's own label. On QM9 the method reports an average MAE improvement of 32.3% across the four encoders, with the largest gains on GCN (53.11%) and GIN (35.00%), and on QM7 a GCN-based variant reaches MAE 45.9, competitive with much larger pretrained models.","pith_inferences":["If the ancestor labels are not strictly restricted to the training split, the 32.3% gain may be partly a transductive label-propagation effect; the fair test would rebuild the network using only training molecules and then evaluate on held-out paths.","A scaffold split or time-based split would stress-test whether evolutionary paths capture general structure-property trends or merely connect near-duplicate molecules that random splits place on both sides of the boundary.","The mutation-type interpretability suggests an untapped use: MEvoN could drive counterfactual search, proposing which single-atom or ring mutation most lowers a target property, rather than only improving regression accuracy.","The same evolutionary-path idea could apply to generative modeling, where an ancestral path is a natural sequence of edit operations for molecule optimization, not merely an auxiliary prediction feature."],"forward_implications":["Because MolE can be any molecule encoder, the reported gains, if correct, transfer across GNNs, CNNs, and Transformers without architectural changes to the base model.","The SHAP analysis of mutation types suggests the network captures chemically interpretable regularities: adding rings or oxygen/nitrogen atoms tends to lower the HOMO-LUMO gap, while adding a three-membered ring raises it.","The hyperparameter study indicates that using 3 to 5 evolutionary paths per molecule gives stable predictions; fewer paths lose diversity and more paths add noise and computation.","On QM7, MEvoN-MPP with a GCN encoder reaches MAE 45.9 without pretraining, placing it in the range of methods that use large-scale pretraining and fine-tuning."],"supporting_citations":[{"why":"Supplies the QM9 dataset, the main benchmark whose eight quantum properties are predicted.","marker":"[Ramakrishnan et al., 2014]"},{"why":"Supplies the QM7 dataset, the second benchmark used for property prediction.","marker":"[Rupp et al., 2012]"},{"why":"SchNet is one of the four base molecular encoders that MEvoN-MPP augments.","marker":"[Schütt et al., 2018]"},{"why":"ComENet is one of the four base encoders; its 3D message-passing is used as the MolE backbone.","marker":"[Wang et al., 2022a]"},{"why":"GIN is one of the base encoders and the graph-isomorphism framework that the ablation study on QM9 uses.","marker":"[Xu et al., 2019]"},{"why":"GCN is the base encoder used in the QM7 comparison and in the QM9 experiments.","marker":"[Kipf and Welling, 2017b]"},{"why":"The Weisfeiler-Lehman graph kernel is the second-stage similarity metric in MEvoN construction.","marker":"[Shervashidze et al., 2011]"},{"why":"MolCLR provides the protocol for selecting the eight QM9 target properties.","marker":"[Wang et al., 2022b]"}],"fun_headline_variants":["Evolutionary paths boost molecular property prediction by 32%","Molecule's family tree improves property forecasts","MEvoN: Mining evolution to cut prediction error","Simulated evolution enhances molecular representations","Adding evolutionary history lifts QM9 accuracy 32%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that, for every test molecule, the property labels of the smaller molecules along its evolutionary path are available as input, with only the test molecule's own label masked.","fun_headline_variants_meta":{"raw":{"variants":["Evolutionary paths boost molecular property prediction by 32%","Molecule's family tree improves property forecasts","MEvoN: Mining evolution to cut prediction error","Simulated evolution enhances molecular representations","Adding evolutionary history lifts QM9 accuracy 32%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000151,"raw_usage":{"total_tokens":1187,"prompt_tokens":918,"completion_tokens":269,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":197}},"tokens_in":534,"tokens_out":269,"duration_ms":3231,"temperature":1.0,"reasoning_tokens":197,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:57:09.669221+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the QM9 experiments after building the MEvoN exclusively from the training split, so that every test molecule's evolutionary path lies entirely inside the training set; alternatively, remove the EvoL label-input module and keep only structural path features. If the reported 32.3% average MAE gain collapses under either protocol, the improvement depends on transductive label access rather than on evolutionary representation.","supporting_citations":[{"cited_title":"Quantum chemistry structures and properties of 134 kilo molecules","cited_arxiv_id":null,"evidence_quote":"Supplies the QM9 dataset, the main benchmark whose eight quantum properties are predicted."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the QM7 dataset, the second benchmark used for property prediction."},{"cited_title":"Weisfeiler-lehman graph kernels","cited_arxiv_id":null,"evidence_quote":"The Weisfeiler-Lehman graph kernel is the second-stage similarity metric in MEvoN construction."}],"review_version":1}