{"id":"fe31db1d-bd25-4156-8bab-f6122b1cf4b2","arxiv_id":"2505.11878","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"AdaptMol, a prototypical network fusing SMILES and graph features with attention, reports state-of-the-art few-shot molecular property prediction on Tox21, SIDER, and MUV benchmarks.","lead":"AdaptMol merges SMILES text and molecular graph structures using attention for few-shot drug discovery and reports top ROC-AUC scores on three benchmarks. The core adaptive fusion module actually uses fixed scaling constants, so its main conceptual novelty is weaker than advertised.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'adaptive' fusion in Eqs. (3)-(4) is a fixed scalar schedule, and the SMILES encoder is an unidentified pretrained LLM; the SOTA gains are not shown to come from the claimed adaptive mechanism.","rationale":"The reader's weakest assumption is correct but incomplete. The fixed-beta problem undermines the 'adaptive' terminology, but the more load-bearing issue is attribution: an unnamed pretrained LLM could be the source of the reported SOTA. The paper has no ablation that removes or freezes the LLM, and no code, so the central mechanism is not falsifiable from the manuscript. Appendix F's stated limitation supports this. Since the empirical results may still hold, the appropriate disposition is unchanged: conditional acceptance pending identification of the LLM, an input-dependent gating ablation, and code/data release.","tokens_in":13756,"tokens_out":5147,"duration_ms":52668,"concrete_test":"Release the exact LLM and code, fix the LLM weights, and rerun the Tox21 5-shot experiment replacing the fixed beta schedule in Eqs. (3)-(4) with a per-molecule learned gating weight sigma(MLP([g_i; a])), keeping all other hyperparameters identical. If the learned gating does not match or exceed the fixed-beta result, the 'adaptive' fusion claim is falsified; if removing the LLM features drops performance substantially, the SOTA is attributable to pretraining rather than to AMA.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The SOTA claim in Table 1 is supposed to validate AdaptMol's central contribution, 'adaptive multimodal fusion.' But the weighting in Eqs. (3)-(4) is not adaptive: beta(g) and beta(s) take only two fixed values, swapped between local and global levels, with k=2, beta_min=0.9, beta_max=1.1 set in Appendix D. No term in Equations (3)-(9) depends on the input molecule or on the extracted features, so the abstract's 'dynamically integrate' and 'assigning higher attention weights to the more informative modality' are unsupported. Table 4's 'Adaptive' row merely adds this fixed scalar schedule on top of the two attention modules; it cannot demonstrate input-dependent fusion. Compounding this, the SMILES encoder is only called 'a large language model' (Sec. 3.3), with no name, version, pretraining corpus, or fine-tuning status reported. Because Tox21, SIDER, and MUV are widely used benchmarks, an unidentified pretrained LLM might have seen these molecules, which would inflate the reported gains regardless of the fusion mechanism. Appendix F acknowledges the fusion is 'relatively simplistic' and defers data-driven fusion to future work. Thus the mechanism claimed to produce SOTA is neither implemented as described nor isolated from the pretrained encoder's contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AdaptMol, a prototypical network for few-shot molecular property prediction that combines graph-based local features with SMILES-derived global features through a dual-level attention module (AMA). The authors claim that this module adaptively fuses the two modalities and achieves state-of-the-art ROC-AUC on Tox21, SIDER, and MUV in 5-shot and 10-shot settings, and they present an MCTS-based rationale extraction method for interpretability.","tokens_in":14224,"tokens_out":6838,"duration_ms":63722,"significance":"If the claims are substantiated, AdaptMol would offer a strong empirical contribution to few-shot drug discovery, and the rationale-extraction approach adds a useful interpretability dimension. The manuscript includes extensive experiments across multiple benchmarks, a cross-domain TDC study, and an ablation analysis, which are valuable. However, the central 'adaptive' mechanism is not implemented as described: the fusion weights in Eqs. (3)-(4) are fixed constants, and the SMILES encoder is not identified, so the reported gains cannot be attributed to input-dependent fusion and are not reproducible. These issues must be resolved before the performance claims can be fully credited.","major_comments":[{"comment":"The so-called adaptive weights beta(g) and beta(s) are constants determined only by the representation level (local vs. global) and by hand-set hyperparameters (k=2, beta_min=0.9, beta_max=1.1 in Appendix D). No term in Eqs. (3)-(9) depends on the input molecule or its extracted features, so the Abstract's claim of 'dynamically integrate' and Section 1's claim of 'assigning higher attention weights to the more informative modality' are not supported by the presented equations. The ablation row labeled 'Adaptive' in Table 4 can only demonstrate the effect of multiplying features by fixed scalars, not of input-dependent fusion. Please either revise the terminology and claims to describe level-dependent weighting, or introduce an input-dependent computation of beta and provide a sensitivity analysis for k, beta_min, and beta_max.","section":"Sec. 3.3, Eqs. (3)-(4)"},{"comment":"The SMILES encoder is described only as 'a large language model' (also in Figure 1(c)); its name, version, pretraining corpus, and fine-tuning status are not reported. This makes the sequence features unreproducible and raises a potential contamination risk: Tox21, SIDER, and MUV are standard benchmarks that may appear in the pretraining corpora of public LLMs, which could inflate the reported gains relative to baselines that do not use such a pretrained encoder. Please specify the exact model and checkpoint, state whether it is frozen or fine-tuned, and discuss or evaluate the overlap between its pretraining data and the evaluation benchmarks.","section":"Sec. 3.3 and Appendix D"},{"comment":"The text states that AdaptMol 'consistently achieved state-of-the-art performance across different datasets,' but Table 1 shows that on MUV 10-shot, UniMatch achieves 79.40% while AdaptMol achieves 77.16%. The abstract's 'in most cases' is accurate, but the main-text claim is contradicted by the paper's own results and should be corrected. Please also clarify how baseline numbers were obtained (reimplemented or taken from prior papers) and why some entries, such as UniMatch 5-shot, are missing.","section":"Sec. 4.2, Table 1"},{"comment":"The multi-head self-attention layer receives an input Fl_input of dimension N x (d_g + d_a) but is claimed to output Attn_local of dimension N x d_g. Without an explicit projection layer or output dimension specification, this dimension reduction is mathematically undefined. Please provide the exact dimensional transformations for the AMA module, or clarify the projection used in the MultiHead operation.","section":"Sec. 3.3, Eq. (6)"}],"minor_comments":[{"comment":"The header 'Moldel' should be 'Model'.","section":"Table 1 header"},{"comment":"The ablation study title refers to 'DMA' but the module is called AMA; this typo should be fixed.","section":"Sec. 4.5, Table 4 title"},{"comment":"The word 'utlizing' should be 'utilizing'.","section":"Sec. 4.1"},{"comment":"The word 'moleculer' should be 'molecular'.","section":"Sec. 2"},{"comment":"The appendix states that four MoleculeNet datasets are used, but Table 1 includes only Tox21, SIDER, and MUV; BACE is used only in the interpretation case study. Please clarify this distinction.","section":"Appendix B.1"},{"comment":"Reference [42] is mislabeled: it points to the TPN paper (same as [38]) instead of the PAR paper (Liu et al., 'Property-aware relation networks for few-shot molecular property prediction,' NeurIPS 2021). The correct citation should be provided.","section":"References"},{"comment":"The PCA dimension d_a and the MCTS constant cpuct are not reported; please list these hyperparameters for reproducibility.","section":"Appendix D"},{"comment":"The threshold delta_i for accepting rationales is not defined; please state how it is set and whether it varies per property.","section":"Sec. 3.4"}],"recommendation":"major_revision","confidential_remarks":"The two central issues -- the fixed 'adaptive' weights and the unidentified SMILES encoder -- are load-bearing. The first undermines the paper's main conceptual contribution, and the second makes the experimental results unreproducible and potentially unfair. If the authors cannot provide an input-dependent fusion mechanism or at least substantially reframe the contribution, and if they cannot disclose the pretrained encoder, the paper would not meet the standards of the journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the experimental work is competent and fairly thorough: three MoleculeNet benchmarks, a cross-domain TDC evaluation, ablations, additional metrics, and ten repeated runs with standard deviations. Second, the central claim in the abstract and introduction that AdaptMol dynamically integrates global and local features and assigns higher attention weights to the more informative modality is not supported by the actual equations. Equations (3) and (4) define beta(g) and beta(s) as fixed scalar schedules that depend only on whether you are at the local or global level. With k=2, beta_min=0.9, beta_max=1.1, the weights are 1.3 and 0.8, swapped between modalities and levels. No term depends on the input molecule or the extracted features. So the adaptive part is really a hand-set, level-dependent scaling. What is genuinely new is the specific combination: a prototypical network with dual-level self-attention over concatenated GIN node embeddings and PCA-reduced SMILES features from an LLM. I do not know of this exact architecture in the cited literature. The ablation in Table 4 does show that adding this fixed scaling on top of the two attention modules helps by 3-4% ROC-AUC on Tox21, so the mechanism has some empirical value, just not as described. The soft spots are real but mostly fixable. The SMILES encoder is only called a large language model with no name, version, or pretraining details. Since Tox21, SIDER, and MUV are widely used benchmarks, the encoder may have seen these molecules; this is a reproducibility and potential leakage concern, not proof of leakage. No code is provided, which compounds the problem. The adaptive weights have no sensitivity analysis; we do not know if the improvement comes from the specific 1.3/0.8 values or from the overall scaling. Appendix F honestly says the fusion is relatively simplistic and defers data-driven fusion to future work, which is good candor but undercuts the abstract's rhetoric. The MCTS interpretability case study is a consistency check, not an independent validation. Is this worth engaging? Yes. The empirical claims are concrete, the benchmarks are standard, and the baselines are the right ones. The mismatch between the claim and the implementation is severe enough that the paper should not be accepted as is, but it is exactly the kind of paper a good referee can fix: name the LLM, release code, add sensitivity analysis, and either reframe the contribution as level-dependent scaling or actually make beta input-dependent. I would send it to peer review, not desk-reject it, with a request for major revision. For a reading group, it is a useful example of how a fashionable word like adaptive can outrun the mathematics. I would not cite it in my own work until the encoder is identified and the fusion claim is reworked.","headline":"The benchmark work is solid, but the paper's headline 'adaptive fusion' is a fixed scalar schedule, not input-dependent weighting; it deserves a serious referee but needs major revision.","tokens_in":798,"tokens_out":1384,"would_cite":false,"duration_ms":33767,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AdaptMol claims that adaptively fusing SMILES-sequence and molecular-graph representations through a two-level attention module gives state-of-the-art few-shot molecular property prediction on three standard benchmarks.","keywords":["few-shot learning","molecular property prediction","multimodal fusion","prototypical networks","graph neural networks","SMILES","attention mechanism","interpretability"],"falsifier":"A direct test is to rerun AdaptMol on the same three benchmarks with the two fusion weights both set to 1 (plain concatenation through the same attention layers) and, separately, with weights learned by a small input-dependent gating network; if the equal-weight version matches the reported ROC-AUCs, or the learned version does not improve on them, the 'adaptive' weighting is not the driver of the results.","tokens_in":13618,"feed_emoji":"🧪","tokens_out":10529,"duration_ms":101155,"temperature":0.7,"pith_summary":"This paper tries to establish that in few-shot molecular property prediction, where each new task has only a handful of labeled molecules, the limiting factor is the molecular representation itself, and that fusing two complementary views—the SMILES sequence as a global semantic summary and the molecular graph as local topological detail—through an adaptive attention module raises that ceiling. Built on a prototypical network, AdaptMol reports average ROC-AUC gains of about 4.18 percentage points over the best baseline in 5-shot settings and 2.44 percentage points in 10-shot settings across Tox21, SIDER, and MUV. If these results hold, drug-discovery teams could screen candidates against a novel endpoint with only a few assay values and still get reliable rankings, along with a chemically interpretable explanation of what the model is keying on. The paper also claims this fused representation is interpretable: a Monte Carlo tree search over bond deletions finds small active substructures that the fused model scores highly, tying the learned features to chemistry.","feed_headline":"Adaptive SMILES+graph fusion tops few-shot drug benchmarks","feed_subtitle":"On Tox21, SIDER and MUV, the model surpasses prior few-shot methods in 5- and 10-shot settings.","key_machinery":"The load-bearing component is the Adaptive Multi-level Attention (AMA) module: a two-stage attention block that first concatenates graph node embeddings weighted by one fusion weight with SMILES features weighted by another, runs multi-head self-attention with a sigmoid activation over the per-node vectors, and then applies a second, fully-connected sigmoid gate over the pooled node representation. The fusion weights in Equations (3)-(4) are meant to balance the two modalities at local and global levels, and in the implementation they are fixed constants (k=2, beta_min=0.9, beta_max=1.1) rather than learned functions of the input. AMA is what converts two encoders into a single refined molecular embedding, and the same embedding feeds both the prototype classifier and the MCTS rationale extractor.","core_discovery":"The paper's central claim is that the Adaptive Multi-level Attention (AMA) module—by weighting and concatenating GIN node embeddings with PCA-reduced SMILES embeddings from a large language model, then applying local- and global-level attention—produces molecular representations that a prototypical network can use to outperform existing few-shot predictors on Tox21, SIDER, and MUV under 5-shot and 10-shot supervision. The ablation study supports this by showing that removing either attention level or the adaptive weighting consistently lowers ROC-AUC, and the cross-domain experiments extend the claim to generalization across tasks from different domains. Interpretability is part of the claim: rationales extracted by MCTS from the fused model retain high predicted activity scores and highlight substructures such as amide and secondary-amine groups, which the paper reads as evidence that the fusion captures pharmacologically meaningful patterns.","pith_inferences":["Because the fusion weights in Equations (3)-(4) are fixed constants in the implementation, the word 'adaptive' currently describes a hand-set balancing scheme, not a per-molecule decision; learning these weights from the input would test whether the mechanism itself, rather than the constants, drives the gains.","The paper's own limitation note (Appendix F) says the fusion is simplistic and can create redundancy for simple molecules; input-dependent fusion or learned gating is the direct next step, and the ablation setup already provides the template for evaluating it.","The same weighted-concatenation-plus-two-stage-attention pattern could transfer to other graph-plus-sequence domains such as proteins or materials, although AdaptMol is demonstrated only on small-molecule benchmarks.","Using the MCTS rationale extractor as a probe, one could compare the substructures highlighted with and without the SMILES modality on identical molecules; rationales that appear only in the fused model would quantify what the global sequence view adds beyond topology."],"forward_implications":["On the three standard benchmarks, AdaptMol records the best ROC-AUC in every reported 5-shot and 10-shot setting, including 83.79 on Tox21 5-shot and 77.16 on MUV 10-shot.","The ablation study shows that removing either the local-level or the global-level attention, or the adaptive weighting, lowers ROC-AUC on every dataset, with the largest drop when the full adaptive module is replaced.","On a cross-domain benchmark built from classification tasks in a different domain, AdaptMol's best 10-shot ROC-AUC of 69.08 beats the two pretrained baselines it is compared with.","Rationales extracted by MCTS from the fused model keep high predicted activity scores and highlight chemical groups such as amide and secondary-amine moieties, which the paper presents as evidence that the fused representation tracks pharmacologically meaningful structure."],"supporting_citations":[{"why":"Supplies the prototypical-network framework that AdaptMol builds on for episodic few-shot classification.","marker":"[24]"},{"why":"Provides the few-shot molecular property prediction task formulation and the benchmark split used in the experiments.","marker":"[27]"},{"why":"Sets the ROC-AUC evaluation protocol and the support-set sizes used for all compared methods.","marker":"[25]"},{"why":"Defines the MoleculeNet datasets (Tox21, SIDER, MUV) that carry the main comparison.","marker":"[34]"},{"why":"Provides the APN pretrained-encoder baseline that AdaptMol is compared against.","marker":"[16]"},{"why":"Provides the MetaGAT baseline and the support/query sampling protocol used during testing.","marker":"[11]"},{"why":"Provides the PAR baseline numbers reported in Table 1 that AdaptMol claims to beat.","marker":"[42]"},{"why":"Supplies the Monte Carlo tree search used to extract active substructure rationales for interpretability.","marker":"[33]"},{"why":"Supplies the cross-domain classification tasks used in the generalization experiment.","marker":"[43]"}],"fun_headline_variants":["AdaptMol: adaptive fusion tops few-shot drug benchmarks","SMILES-graph attention fusion wins few-shot drug tasks","Adaptive dual-modal fusion beats few-shot drug baselines","Few-shot drug AI leaps with adaptive SMILES-graph fusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the hand-set fusion weights in the model, not the attention layers or the choice of encoders, are what produce the reported performance gap; if those weights were tuned on the test benchmarks, or if equal fixed weights reproduce the same scores, the adaptive-fusion claim would lose its support.","fun_headline_variants_meta":{"raw":{"variants":["AdaptMol: adaptive fusion tops few-shot drug benchmarks","SMILES-graph attention fusion wins few-shot drug tasks","Adaptive dual-modal fusion beats few-shot drug baselines","Few-shot drug AI leaps with adaptive SMILES-graph fusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001523,"raw_usage":{"total_tokens":6093,"prompt_tokens":931,"completion_tokens":5162,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":5095}},"tokens_in":547,"tokens_out":5162,"duration_ms":33502,"temperature":1.0,"reasoning_tokens":5095,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:44:45.338616+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to rerun AdaptMol on the same three benchmarks with the two fusion weights both set to 1 (plain concatenation through the same attention layers) and, separately, with weights learned by a small input-dependent gating network; if the equal-weight version matches the reported ROC-AUCs, or the learned version does not improve on them, the 'adaptive' weighting is not the driver of the results.","supporting_citations":[{"cited_title":"Prototypical networks for few-shot learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the prototypical-network framework that AdaptMol builds on for episodic few-shot classification."},{"cited_title":"Low data drug discovery with one-shot learning,","cited_arxiv_id":null,"evidence_quote":"Provides the few-shot molecular property prediction task formulation and the benchmark split used in the experiments."},{"cited_title":"Property-aware relation networks for few-shot molecular property prediction,","cited_arxiv_id":null,"evidence_quote":"Sets the ROC-AUC evaluation protocol and the support-set sizes used for all compared methods."},{"cited_title":"Moleculenet: A benchmark for molecular machine learning,","cited_arxiv_id":null,"evidence_quote":"Defines the MoleculeNet datasets (Tox21, SIDER, MUV) that carry the main comparison."},{"cited_title":"Attribute-guided prototype network for few-shot molecular property prediction,","cited_arxiv_id":null,"evidence_quote":"Provides the APN pretrained-encoder baseline that AdaptMol is compared against."},{"cited_title":"Meta learning with graph attention networks for low-data drug discovery,","cited_arxiv_id":null,"evidence_quote":"Provides the MetaGAT baseline and the support/query sampling protocol used during testing."},{"cited_title":"Learning to propagate labels: Transduc- tive propagation network for few-shot learning,","cited_arxiv_id":null,"evidence_quote":"Provides the PAR baseline numbers reported in Table 1 that AdaptMol claims to beat."},{"cited_title":"Mastering the game of go without human knowledge,","cited_arxiv_id":null,"evidence_quote":"Supplies the Monte Carlo tree search used to extract active substructure rationales for interpretability."},{"cited_title":"Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development,","cited_arxiv_id":null,"evidence_quote":"Supplies the cross-domain classification tasks used in the generalization experiment."}],"review_version":1}