{"id":"2ee40774-b55b-4c39-9530-863ad6a1447d","arxiv_id":"2608.10480","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"MR-MoL feeds ranked, direction-tagged GNN substructure attributions to an LLM as textual evidence, improving results on six of eight MoleculeNet tasks over generalist baselines.","lead":"MR-MoL is a molecular AI model that shows a large language model which substructures of a molecule push a target property up or down, alongside the usual SMILES and graph encodings. A generalist reader might care because it attempts to make property prediction both more accurate and more interpretable in one step.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The generalist comparison is not fair: MR-MoL is task-fine-tuned while the five generalist baselines are zero-shot from released checkpoints, and the paper's own w/oG,R ablation suggests most of the generalist advantage comes from that asymmetry rather than the rationale.","rationale":"The reader's weakest assumption, source predictor quality, is real and the paper itself acknowledges it as a limitation. However, the internal evidence on the eight datasets is reasonably strong: the ablation shows the rationale helps on seven of eight tasks, and the diagnostics show the model is sensitive to direction, rank, and substructure content. I therefore do not see source predictor quality as the single most load-bearing issue for the central claim. The more load-bearing issue is the asymmetric evaluation used for the headline generalist comparison. The paper is transparent about this in C.3, but transparency does not by itself make the central comparative claim valid. Since the reader's conditional verdict already flags baseline fairness and asks for it to be addressed, my concern does not move the verdict; it sharpens the reason for the condition. The concrete test above would settle whether the generalist advantage is real or an artifact of fine-tuning asymmetry.","tokens_in":20525,"tokens_out":8037,"duration_ms":82585,"concrete_test":"Make the comparison symmetric. Either (A) fine-tune the five released generalist checkpoints or their base LLMs with the same Stage 2 LoRA instruction-tuning on the same eight-task scaffold split and recompute the generalist rows of Table 1, or (B) evaluate MR-MoL zero-shot from its Stage 1 checkpoint on the same eight tasks. If MR-MoL no longer leads on six of eight tasks in either setting, the headline should be reworded from best overall generalist to best after task-specific fine-tuning; if it still leads, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Implementation Details states that generalist models are evaluated from their publicly available checkpoints, while MR-MoL is trained in Stage 2 on the eight evaluation tasks. Table 18 confirms the generalist baselines have uneven and sometimes absent coverage of these tasks. The abstract's central quantitative claim, best overall results among generalist models, therefore compares a task-adapted model against zero-shot models, so the claimed generalist superiority is not established by Table 1. The Table 2 ablation makes the concern concrete: the w/oG,R variant, which removes both the graph and rationale channels but keeps the same backbone and Stage 2 multi-task fine-tuning, already beats the generalist baselines on most tasks. Consequently most of the apparent advantage over generalists may be attributable to the strong LLM plus task-level supervision rather than to the graph and rationale channels. This does not invalidate the internal rationale comparison (MR-MoL vs w/oR) or the diagnostics, which are convincing evidence that the model uses the rationale; it invalidates the external best-generalist claim as stated. A fair test requires either a zero-shot MR-MoL or generalists fine-tuned under the same protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MR-MoL, a multi-modal molecular LLM that augments a Llama-3.1-8B backbone with a 2D graph embedding path and a multi-granular, ranked, direction-tagged rationale derived from a fine-tuned GNN's substructure masking attributions. The rationale draws on Murcko scaffolds, BRICS fragments, and functional groups, and is serialized as text alongside the SMILES string and graph tokens. Training proceeds in two stages: graph-language alignment (Stage 1) and multi-task rationale-guided instruction tuning on eight MoleculeNet tasks (Stage 2). The central claims are (i) MR-MoL achieves the best overall results among generalist models and narrows the gap to task-tuned specialists, and (ii) the LLM genuinely reads the rationale, supported by five diagnostics: direction flipping, rank-1 vs. random removal, substructure shuffling, chemical validity of attributions, and individual prediction corrections.","tokens_in":20711,"tokens_out":8000,"duration_ms":71197,"significance":"If the claims hold, the paper introduces a useful and interpretable mechanism: exposing GNN-derived per-substructure attributions to an LLM as auxiliary evidence for property prediction. The internal experimental design is strong: the ablations isolate the rationale and graph channels, and the diagnostics are thoughtful and convincing. In particular, direction flipping systematically drives classification below chance (Table 3), rank sensitivity shows a 1.6–6.2× larger effect for the top item (Table 4), and substructure shuffling degrades all four tested tasks (Table 5). The chemical-validity check (Table 6) provides an additional, interpretability-oriented validation. These experiments support the claim that the model uses the rationale's content. However, the external comparison against 'generalist' baselines is not fair as presented: MR-MoL receives Stage 2 supervision on the eight evaluation tasks, whereas the generalist baselines are evaluated zero-shot from released checkpoints.","major_comments":[{"comment":"The generalist comparison is not apples-to-apples. As stated in Implementation Details, 'generalist models are evaluated from their publicly available checkpoints,' while MR-MoL is trained in Stage 2 on the eight evaluation tasks. Table 18 makes the asymmetry explicit: each generalist baseline has seen a different subset of the tasks, and several tasks are completely absent from some models' training data (e.g., ChemDFM is marked absent on BBBP, Tox21, ESOL, and Lipo). The abstract's claim that MR-MoL 'achieves the best overall results among generalist models' is therefore a comparison of a task-fine-tuned model against zero-shot models, which does not support the stated conclusion. To make the claim meaningful, the authors should either evaluate MR-MoL in a zero-shot setting (without Stage 2 supervision on the eight tasks), fine-tune the generalist baselines under an identical Stage 2 protocol, or explicitly reframe the conclusion as 'best among models evaluated under our multi-task instruction tuning' rather than 'best among generalist models.'","section":"Implementation Details and Table 18 (also Appendix C.3)"},{"comment":"The attribution of the generalist gains is internally inconsistent with the ablation evidence. In the 'Comparison with baselines' paragraph, the authors write that the advantage over generalists is due to 'the structural evidence MR-MoL receives alongside the 1D and 2D molecule representations.' However, Table 2 shows that the w/oG,R variant, which removes both the graph and the rationale and retains only the instruction, the SMILES string, and the same Stage 2 multi-task instruction tuning, already surpasses the generalist baselines in Table 1 on most tasks (as the paper itself acknowledges in the Ablation paragraph). This demonstrates that the dominant contributor to the generalist gap is the strong LLM backbone plus task-level supervision, not the rationale or graph channels. The internal comparison of MR-MoL against w/oR (which isolates the rationale under the same training protocol) remains valid and is compelling, but the external generalist claim as stated is unsupported by the presented evidence. This is a load-bearing issue because it affects the abstract's primary quantitative claim.","section":"Results and Analysis, 'Comparison with baselines' and 'Ablation' paragraphs"}],"minor_comments":[{"comment":"The p-values in Table 4 are formatted abnormally: '5.6×2×10−27' and '4×10−22' include an extra '2' that appears spurious. These should be corrected to a standard scientific notation (e.g., 5.6×10−27). The 'ratio' column is clear but the p-values need fixing.","section":"Table 4 (Rank sensitivity)"},{"comment":"In the direction-sensitivity subsection, the statement 'the ROC-AUC falls below chance' would benefit from an explicit clarification that values below 0.5 indicate that the model is now systematically following the inverted direction tag, since readers may need to connect this to the sign-flipped MCC.","section":"Direction flipping (Table 3) text"},{"comment":"The sentence 'We therefore avoid drawing conclusions from any single cell, and rely on the overall comparison rather than task-by-task claims against these baselines' is a reasonable defense against per-cell cherry-picking, but the overall comparison is still confounded by the training-time asymmetry. This caveat should be moved or echoed in the main text, not only the Appendix, because it directly affects the interpretation of the headline result.","section":"Appendix C.3"},{"comment":"The caption states that the overall best across groups is underlined, but in the table as rendered no underlining is visible. Please ensure the formatting is correct in the final version.","section":"Table 1 caption"},{"comment":"When defining the attribution a_{t,j} in Eq. (4), the exact meaning of masking G\\u_j for each view (Murcko, BRICS, functional group) is left to the reader. A one-sentence example or a pointer to Appendix B for the precise masking operations would improve clarity.","section":"Method, Ranked Multi-Granular Rationale"}],"recommendation":"major_revision","confidential_remarks":"The internal experiments and diagnostics are the strength of this paper; they convincingly show that the LLM reads the rationale. The main problem is the external comparison with generalist baselines, which is not fair because MR-MoL receives task-level supervision in Stage 2 while the baselines are evaluated zero-shot. This is correctable by either re-running a fair comparison or substantially softening the 'best overall among generalist models' claim. I would not reject the paper, but the abstract and Results section need revision. The authors should also be asked to ensure the p-value formatting in Table 4 is corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the mechanism is new — feeding a GNN's substructure-masking attributions to an LLM as ranked, direction-tagged text — and the diagnostics are the strongest part of the paper. The internal claim (the model reads the rationale) holds up. The external claim (best generalist) does not, because the comparison is unfair and the paper's own ablation shows it.\n\nWhat's new: prior work supplies external knowledge (MolRAG, KANO) or graph embeddings, but not per-molecule, per-substructure attribution as textual evidence. The three-granularity decomposition (Murcko, BRICS, functional groups) is a sensible way to cover scaffold-to-motif levels, and the serialization with direction tags is well designed.\n\nWhat the paper does well: the five diagnostics are thoughtful. Direction flipping sends classification ROC-AUC below chance and flips the MCC sign; removing the rank-1 item shifts predictions 1.6–6.2 times more than removing a random lower-ranked item; shuffling substructures hurts on every dataset. That is coherent evidence that the LLM responds to the content of the rationale, not just its presence. The chemical validity check (polar groups raise predicted solubility and lower predicted lipophilicity) is a nice external sanity check, and the case study in Figure 2 is illustrative.\n\nThe soft spot is the generalist comparison. Implementation Details says generalists are evaluated from released checkpoints while MR-MoL is trained on the eight tasks. The abstract's 'best overall among generalist models' is then comparing a task-tuned model against zero-shot ones. The ablation makes this concrete: w/oG,R (no graph, no rationale, same LLM and Stage 2 tuning) already beats most generalist baselines on most tasks. So most of the margin over generalists appears to come from multi-task fine-tuning, not from the rationale. The authors do disclose task coverage in Appendix C.3 and warn against reading single cells, but the headline claim still overreaches. A fair test would need a zero-shot MR-MoL or generalists fine-tuned under the same protocol.\n\nThe source-predictor dependence is a second, smaller soft spot. All rationale content comes from a fine-tuned GNN; if those attributions are noisy or split-biased, the rationale becomes misinformation. The authors acknowledge this in Limitations, and the diagnostics show the model follows the text even when it is wrong (direction flip), so the mechanism is real, but the method's ceiling is set by the source predictor. That is an honest limitation, not a flaw in execution.\n\nWho this is for: anyone working on molecular LLMs, explainable AI for chemistry, or using attribution as input to a model. The diagnostics are a useful template for evaluating whether a model genuinely uses an explanation.\n\nRecommendation: send to peer review. The internal claims are well supported, the method is new, and the baseline problem is fixable by reframing the claim or adding a fine-tuned generalist comparison. Not a desk reject.","headline":"Genuinely new mechanism and convincing diagnostics, but the 'best generalist' headline is undercut by an unfair zero-shot vs fine-tuned comparison.","tokens_in":21270,"tokens_out":2916,"would_cite":true,"duration_ms":24904,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a molecular LLM's property predictions improve measurably, and become more interpretable, when the prompt includes a ranked, direction-tagged list of GNN-derived substructure attributions, and that the model reads…","keywords":["molecular property prediction","large language models","graph neural networks","substructure masking","explainable AI","multi-granular rationale","MoleculeNet","rationale-guided LLM"],"falsifier":"Train the source predictor on randomly shuffled labels, so the masking attributions are pure noise, and feed the resulting 'rationale' to MR-MoL; if predictions do not fall back to (or below) the no-rationale baseline, the model is keying on the rationale's form rather than its chemical content — the diagnostics in the paper only perturb content, not source-predictor quality.","tokens_in":20296,"feed_emoji":"🧪","tokens_out":6019,"duration_ms":47590,"temperature":0.7,"pith_summary":"MR-MoL tests a simple proposal: give a molecular LLM the same evidence a chemist would use — the substructures that push a property up or down — and the model's property predictions improve while becoming legible. The paper claims that a fine-tuned GNN's masking attributions, serialized as a ranked, direction-tagged, multi-granular rationale, let a generalist LLM beat all generalist baselines on most of eight MoleculeNet tasks and narrow the gap to per-task specialists. It further claims the LLM genuinely reads this rationale: flipping the direction tag reverses predictions, removing the top-ranked item shifts the output several times more than removing a lower-ranked one, and replacing a substructure with a random one hurts performance. If these claims hold, the route from black-box GNN explanations to LLM evidence is a practical way to make molecular property prediction both stronger and more interpretable.","feed_headline":"Feeding GNN rationales to an LLM improves molecule forecasts","feed_subtitle":"The model reads each clue's direction, rank, and substructure, beating generalists on eight MoleculeNet tasks.","key_machinery":"The load-bearing object is the multi-granular rationale: a ranked, direction-tagged textual list of the top five substructures from three complementary views — Murcko scaffolds with side chains, BRICS fragments, and chemically named functional groups — each scored by the masking attribution $a_{t,j} = f_t(G) - f_t(G \\setminus u_j)$ of a per-task fine-tuned GNN. This rationale is concatenated into the prompt beside the SMILES string and projected graph tokens, so the LLM conditions directly on per-substructure evidence. Everything that distinguishes MR-MoL from a plain graph-aligned LLM passes through this channel.","core_discovery":"The central discovery is that GNN-derived substructure attributions, when serialized as text and placed in the prompt, behave as real evidence for an instruction-tuned molecular LLM, not as inert filler. MR-MoL computes, for each of three decomposition views, the masking attribution $a_{t,j} = f_t(G) - f_t(G \\setminus u_j)$ from a task-fine-tuned Mole-BERT source predictor, keeps the five largest-magnitude items, and tags each as pushing the predicted property toward higher or lower. The paper reports that this rationale is responsible for the model's best-overall-generalist results on eight MoleculeNet tasks, and that five diagnostics show the model's predictions move with the direction tag, the rank, and the substructure identity of each rationale item. On the chemical side, the attributions reproduce known structure-property relationships, such as polar functional groups raising aqueous solubility and lowering lipophilicity, and the rationale can flip an individual wrong prediction to the correct one. The method is presented as the first to expose GNN-derived attributions to an LLM as input evidence for property prediction.","pith_inferences":["Because the rationale is task-conditioned and trained, the same channel could plausibly be extended to tasks beyond classification and regression — such as reaction outcome prediction — if a source predictor can produce per-fragment attributions for those settings.","The diagnostics suggest the model may be using the rationale as a form of soft chain-of-thought; a testable extension would be to prompt the model to verbalize why a given direction tag from the GNN should or should not apply, and to measure whether such self-verification further improves accuracy.","If attribution noise (rather than the LLM) is the bottleneck, ensembling multiple source predictors or using more stable attribution methods than raw masking differences could push MR-MoL past the specialist GNNs on regression, where the gap currently remains.","The fixed top-five cutoff across all three views may be suboptimal; a task-adaptive selection of granularity and number of items could improve both performance and sparsity of the rationale."],"forward_implications":["A single generalist molecular LLM can approach per-task specialist performance when its prompt carries explicit substructure evidence, without per-task fine-tuning of the LLM itself.","The rationale's direction, rank, and substructure are all causally read by the model; any one of these interventions moves the prediction, so the channel is informative rather than decorative.","Removing the rationale degrades performance on seven of eight tasks, with the largest drops on ClinTox and BACE, so the substructure evidence carries information the graph and SMILES channels do not.","The attributions reproduce known chemistry — polar functional groups raise predicted solubility and lower predicted lipophilicity — which suggests the evidence channel is aligned with domain knowledge.","On individual molecules, the rationale can correct a wrong prediction (e.g., nalidixic acid crossing the blood-brain barrier), and the correction depends on the top-attributed item."],"supporting_citations":[{"why":"Supplies the substructure-masking attribution method ($a_{t,j} = f_t(G) - f_t(G\\setminus u_j)$) that scores every candidate substructure for the rationale.","marker":"Wu et al. (2023)"},{"why":"Provides the Mole-BERT GNN that is fine-tuned per task as the source predictor whose masking differences generate the rationales.","marker":"Xia et al. (2023)"},{"why":"Provides the MolCA stage-1 pretrained GNN encoder and Q-Former that initialize the graph embedding path, grounding molecular tokens in the LLM.","marker":"Liu et al. (2023)"},{"why":"Defines the Murcko decomposition that produces the scaffold-and-side-chain view of the rationale.","marker":"Bemis and Murcko (1996)"},{"why":"Defines BRICS retrosynthetic fragments, the second granularity view of the rationale.","marker":"Degen et al. (2008)"},{"why":"Defines the MoleculeNet benchmark and evaluation protocol (scaffold splits, ROC-AUC, RMSE) on which all comparisons are made.","marker":"Wu et al. (2018)"},{"why":"Supplies the SMolInstruct instruction templates that shape the Stage 2 property-prediction prompts.","marker":"Yu et al. (2024)"}],"fun_headline_variants":["GNN rationales as text lift LLM molecule property forecasts","LLM reads GNN's top substructure clues for better predictions","MR-MoL: LLM consumes ranked GNN attributions for molecules","Feeding GNN substructure evidence to LLM sharpens property calls","Direction, rank, and substructure: LLM uses GNN rationale"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole rationale channel inherits its content from a single fine-tuned GNN's masking attributions; if those attributions are noisy, biased by the training split, or wrong for a particular molecule, the rationale becomes misinformation that can hurt predictions, and the paper's limitations section concedes exactly this risk.","fun_headline_variants_meta":{"raw":{"variants":["GNN rationales as text lift LLM molecule property forecasts","LLM reads GNN's top substructure clues for better predictions","MR-MoL: LLM consumes ranked GNN attributions for molecules","Feeding GNN substructure evidence to LLM sharpens property calls","Direction, rank, and substructure: LLM uses GNN rationale"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1446,"prompt_tokens":1041,"completion_tokens":405,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":312}},"tokens_in":657,"tokens_out":405,"duration_ms":4480,"temperature":1.0,"reasoning_tokens":312,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:19:52.670729+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the source predictor on randomly shuffled labels, so the masking attributions are pure noise, and feed the resulting 'rationale' to MR-MoL; if predictions do not fall back to (or below) the no-rationale baseline, the model is keying on the rationale's form rather than its chemical content — the diagnostics in the paper only perturb content, not source-predictor quality.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Mole-BERT GNN that is fine-tuned per task as the source predictor whose masking differences generate the rationales."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MolCA stage-1 pretrained GNN encoder and Q-Former that initialize the graph embedding path, grounding molecular tokens in the LLM."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines BRICS retrosynthetic fragments, the second granularity view of the rationale."}],"review_version":1}