{"id":"b7db205f-385d-4b1f-afa8-885001ce52fe","arxiv_id":"2608.00776","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A frozen ImageNet vision model reading 2D molecular drawings outperforms a quantum-descriptor tabular model for C-N coupling yield prediction, and cross-attention fusion gives 5.27% RMSE.","lead":"This paper combines plain 2D molecular drawings with tabular quantum descriptors in a cross-attention network to predict reaction yields, reporting a 5.27% RMSE on the Doyle C-N coupling benchmark. A generic image network alone beat the quantum-descriptor model, which would make yield screening much cheaper if the result holds up.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Compound overlap between train/test from the unreported random split is the main threat: the vision model's advantage may be inflated by memorizing RDKit drawings that also appear in training.","rationale":"The most load-bearing assumption is that the random split with a locked seed creates test molecules independent of training. The rxnpredict dataset is a combinatorial screen, so shared components between splits are nearly guaranteed. For the vision modality, this is a direct leakage path: the ResNet sees the exact RDKit drawing, so a training image of the same compound can be memorized, inflating the vision advantage over tabular descriptors. The paper offers no grouped split, no overlap statistics, no seed, and no repeated-seed error bars, so the claimed differences cannot be distinguished from split-induced memorization. I agree with the reader's weakest-assumption diagnosis. The grouped-split retraining is a decisive test because it removes the leakage path; if the vision advantage survives, it would substantially support the central claim. Until that test is run, the conditional verdict is appropriate, so I do not adjust it.","tokens_in":10561,"tokens_out":6590,"duration_ms":74484,"concrete_test":"Re-run all four architectures under a grouped split where every unique aryl halide, ligand, additive, and base appears in only one split (e.g., group by unique compound, or by substrate scaffold). Retrain with the same hyperparameters. If the vision-only RMSE no longer beats the tabular MLP (or the gap shrinks below the reported 1.19 percentage points), the published advantage is an artifact of compound leakage rather than generic-vision generalization. Report mean±std over at least 5 such grouped splits to also rule out split noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparison (vision-only 5.60% RMSE vs. tabular MLP 6.79%, and cross-attention 5.27%) is reported on a single 70/30 random split with a locked but unreported seed. The rxnpredict dataset is a combinatorial HTE screen, so the same aryl halide, ligand, additive, and base are reused across many reactions. A random split inevitably places reactions sharing these components in both train and test. For the vision modality, this is not merely a statistical nuisance: the frozen ResNet input is a deterministic rendering of the exact molecule. A near-duplicate image in the training set lets the model output a weighted average of yields for that drawing, which is much more informative than a tabular descriptor vector that only approximates the same compound. Thus the headline comparison (5.60 vs. 6.79) and the cross-attention gain (5.27 vs. 5.60) could be driven by template matching rather than by learning steric topology. The paper provides no group-split result, no overlap statistics, and no seed, so this alternative explanation is not excluded. The lack of error bars compounds the issue: a single-seed difference of 0.33 RMSE (vision vs. cross-attention) could be sampling noise. The secondary concerns (post-hoc PFI explanations, the omitted-descriptor claim in Section 3.2) are real but less central; they would not by themselves overturn the accuracy comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a dual-modal architecture for reaction yield prediction on the Doyle rxnpredict HTE dataset. A frozen ImageNet-pretrained ResNet-18 encodes four RDKit-drawn molecular components (additive, aryl halide, base, ligand) as separate 224x224 images, while an MLP encodes tabular DFT-derived descriptors. The two modalities are fused either by simple concatenation after mean pooling or by an 8-head cross-attention mechanism in which the tabular features act as queries and the visual features as keys/values, with a concatenated bypass of the raw tabular features. The authors report RMSE values of 6.80% (tabular MLP), 5.60% (vision-only), 5.78% (simple concat), and 5.27% (cross-attention, R2=0.961). They also present permutation feature importance and extracted attention weights to argue that cross-attention offloads steric identification to the visual pathway, learns a chemical hierarchy favoring the aryl halide, and requires a residual bypass to protect non-spatial electronic descriptors.","tokens_in":10878,"tokens_out":5197,"duration_ms":63920,"significance":"If the reported accuracy advantage is real, the finding is significant: a generic, frozen computer-vision backbone on simple 2D drawings could outperform an expensive DFT-descriptor tabular baseline, and cross-attention fusion could further improve accuracy while providing interpretable attention maps. The architecture is clearly specified, the equations are explicit, and the use of a public benchmark is a strength. However, the empirical claims currently rest on a single random split with no error bars, no reported seed, no grouped-split analysis, and no direct comparison to published baselines. These omissions are load-bearing for the central claim that generic vision and cross-attention 'significantly outperform' existing methods. The mechanistic analyses are also based on a single fitted model and are interpreted as if the attention weights and feature-importance rankings were stable, which is not established.","major_comments":[{"comment":"The headline comparison (6.80 vs. 5.60 vs. 5.27 RMSE) is reported from a single 70/30 split with a locked but unreported seed and no error bars. The 0.33 percentage-point advantage of cross-attention over vision-only is within the range of seed-to-seed noise typically seen on HTE datasets. Please report the seed, run multiple seeds, and provide confidence intervals or paired significance tests. Without this, the claim of 'superior predictive accuracy' is not established.","section":"Section 3.1, Fig. 2"},{"comment":"A random split is not adequate for this combinatorial HTE dataset. The same aryl halide, ligand, additive, and base are reused across many reactions, so identical or nearly identical RDKit drawings will appear in both training and test sets. For the vision modality, whose input is a deterministic image of the exact molecule, this allows template matching and can inflate the visual advantage over tabular descriptors. Please quantify train/test overlap (e.g., fraction of test reactions whose components appear in training), perform a grouped or scaffold split, and report performance on reactions with no component overlap.","section":"Section 2.1, dataset partitioning"},{"comment":"The paper claims that the proposed models 'significantly outperform' previous state-of-the-art methods, but no published baseline is evaluated on the same split. The original Random Forest benchmark [1] is mentioned but its RMSE is not reported, and no comparison is made to DRFP [7], GNN-based yield predictors, or other recent methods. The abstract and conclusion's claim of superiority over 'traditional methodologies' therefore lacks direct evidence. Please add baseline experiments with the same split and metrics.","section":"Results, Sections 3.1 and 5"},{"comment":"The PFI and attention-weight analyses are based on a single fitted model with no uncertainty quantification. Differences such as the +1.25 vs. +0.40 PFI penalties in Fig. 3C, or the attention means in Fig. 4C, could be within noise. The claim that molecular weight 'completely vanishes' and that the attention hierarchy is robust requires repeated runs or statistical testing. Moreover, interpreting attention weights as causal importance is descriptive; a perturbation test (e.g., occluding the high-attention aryl halide image) would be needed to support the mechanistic conclusions.","section":"Sections 3.2 and 3.3, Figs. 3 and 4"}],"minor_comments":[{"comment":"Typos: 'T abular Descriptors' and '2D Visual T opologies' should read 'Tabular' and 'Topologies'.","section":"Section 2.1"},{"comment":"The operation is described as a 'residual skip connection,' but Eq. (8) is a concatenation, not an additive residual. Consider calling it a 'bypass connection' to avoid confusion with ResNet-style additive residuals.","section":"Eq. (8)"},{"comment":"The notation X_tab is used for both the raw tabular descriptors and the dimensionally aligned tabular embedding. In Eq. (2) and (8), X_tab appears to be raw, while in Eq. (3) it is projected by W_Q,i. Clarify which quantity is used and specify its dimension so that the Q/K/V dimensions in Eq. (6) are unambiguous.","section":"Eqs. (2), (3), and (8)"},{"comment":"The paragraph discussing behavior when explicit geometric descriptors are omitted is speculative and is not backed by an experiment. Either remove it or add an ablation study that drops volume/surface-area descriptors.","section":"Section 3.2, last paragraph"},{"comment":"No code, seed value, or reproducible training setup is provided. Releasing these would strengthen reproducibility, especially because the reported results depend on a particular split and set of hyperparameters.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a straightforward empirical study whose central claim could be of interest to the ML-for-chemistry community, but the evidence as submitted is not sufficient. The lack of multi-seed results, overlap analysis, and comparison to published baselines are not cosmetic; they directly affect whether the reported 5.27% RMSE is meaningful. If the authors can provide a robust evaluation with grouped splits and baseline comparisons, the paper may become suitable. I would also note that the mechanistic interpretation goes beyond what the current experiments support, and the authors should soften those claims or add perturbation tests."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, the paper's headline claim is that a frozen ImageNet ResNet-18 on plain RDKit 2D skeletal images beats a tabular MLP fed with DFT descriptors on the rxnpredict benchmark (5.60 vs 6.80% RMSE), and that cross-attention fusion with a residual skip gets 5.27%. The architecture is a modest extension of the same first author's Chemfusion preprint [25], so the real novelty is in the specific empirical findings: vision outperforming quantum descriptors, the attention hierarchy (aryl halide receives ~53% of visual attention), and the 'residual bottleneck' ablation showing that forcing all tabular features through attention degrades performance (6.17%) unless a residual connection bypasses it. Those observations are plausible and the mechanistic story is coherent.\n\nThe problem is the evaluation. Everything rests on one 70/30 random split with no error bars and no seed value reported. rxnpredict is a combinatorial HTE screen; the same aryl halides, ligands, bases, and additives recur across many reactions. A random split will place identical molecules in both train and test. For a model whose input is the exact RDKit rendering of that molecule, this is near-duplicate contamination: the network can output a training-set average yield rather than learn generalizable steric structure. That could easily explain part of the 1.2-point vision advantage over tabular descriptors, and the 0.33-point gap between vision-only and cross-attention could be seed noise. No grouped or scaffold split, no overlap statistics, and no repeated seeds are provided. The paper also mentions that its MLP beats the original Random Forest baseline but never reports the RF number, and it does not compare against DRFP or GNN baselines. The Section 3.2 passage about dropping geometric descriptors is speculation without an ablation.\n\nThat said, the paper is clearly written, honest about its limitations, and the method details are reproducible given the constraints. These are fixable problems: grouped splits, multiple seeds, external baselines, and code release would turn it into a solid contribution. It deserves a serious referee, not a desk reject. I would engage with it, but I wouldn't cite the numbers in my own work yet.","headline":"Vision-over-quantum claim for yield prediction is plausible but under-validated; the single random split and missing baselines make the headline RMSE numbers provisional.","tokens_in":11403,"tokens_out":3693,"would_cite":false,"duration_ms":37989,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A frozen generic image model reading 2D molecular drawings beats quantum-descriptor baselines for reaction yield prediction, and cross-attention fusion improves it further.","keywords":["reaction yield prediction","cross-attention","2D molecular images","quantum descriptors","ResNet-18","multimodal fusion","C–N cross-coupling","steric effects"],"falsifier":"Re-run the same four architectures with a grouped split that keeps every reaction of a given aryl halide (or ligand) entirely in either training or test; if the vision-only model's RMSE advantage over the tabular MLP shrinks or reverses under this split, the headline comparison is driven by molecule-level leakage rather than visual steric learning.","tokens_in":10405,"feed_emoji":"🧪","tokens_out":2604,"duration_ms":33549,"temperature":0.7,"pith_summary":"The paper claims that a generic, ImageNet-pretrained vision network—never exposed to chemistry—can extract enough steric information from simple 2D skeletal drawings of reactants to outperform a baseline MLP built on expensive quantum-mechanical descriptors (5.60% vs. 6.79% RMSE). It then claims that fusing the visual topology with tabular physical-organic descriptors through a cross-attention mechanism, where the tabular features query the images, reaches 5.27% RMSE and outperforms both single-modality models and naive concatenation. The authors use permutation feature importance and attention-weight extraction to argue that the vision pathway is not a passive feature pool: it actively localizes steric bulk, especially around the aryl halide, while a residual skip connection protects non-spatial electronic descriptors from being attenuated by the attention bottleneck. If correct, the work offers a computationally lighter route to spatially aware yield prediction that does not require domain-specific chemical pretraining or explicit 3D structure.","feed_headline":"A frozen image model beats quantum descriptors for yield prediction","feed_subtitle":"Cross-attention fusing 2D drawings with tabular physics reaches 5.27% RMSE on C–N coupling yields.","key_machinery":"The central object is a multi-head cross-attention fusion block in which the tabular quantum descriptors act as the Query and the four component-level visual embeddings from the frozen ResNet-18 act as Keys and Values, forcing the explicit physics to actively interrogate the 2D topology. A residual skip connection concatenates the raw tabular features past the attention block, protecting non-spatial electronic parameters from what the paper calls the Residual Bottleneck. The argument is carried by comparing this architecture against a mean-pooled concatenation baseline, a vision-only model, and a tabular-only MLP, and by probing attention softmax outputs and permutation feature importance.","core_discovery":"The central claim is that 2D reaction topology alone, processed by an off-the-shelf frozen ResNet-18, carries predictive signal for reaction yield that explicit DFT-derived 1D descriptors miss, and that the optimal way to combine the two modalities is descriptor-guided cross-attention rather than vector concatenation. The champion architecture reaches 5.27% RMSE and R2=0.961 on the held-out test set. Mechanistically, the paper shows that cross-attention shifts feature reliance away from molecular weight as a steric proxy and toward explicit volume, surface area, and ovality queries that the visual pathway grounds in the images, while raw attention scores reveal sparse, binary-like routing th","pith_inferences":["If the split is grouped by reagent identity rather than random molecules, the vision advantage may shrink or vanish; the paper's split choice is testable and the reported seed is not disclosed, so the headline comparison should be re-run under a scaffold or grouped split before relying on it.","The learned attention hierarchy suggests a new diagnostic tool for high-throughput screening: one could use the per-component attention budget as a cheap proxy for which reactant's substitution pattern most destabilizes the transition state, potentially prioritizing experimental design before full DFT optimization.","The framework could be extended to reaction classes beyond C–N coupling by replacing the fixed four-component image sequence with an arbitrary reagent set and letting the cross-attention mechanism learn how many visual slots to attend to.","Because the paper shows volume estimates work better as directional queries than as scalar answers when vision is available, other noisy computed descriptors might similarly be repurposed as search keys in multimodal architectures rather than being discarded for their conformer sensitivity."],"forward_implications":["Reaction yield models that currently pay the computational cost of DFT descriptors could instead derive macroscopic steric information from cheap 2D drawings, reserving quantum descriptors for electronic effects.","Cross-attention fusion, by making tabular features query images, avoids the modality-dominance failure mode of simple concatenation, so multimodal chemical models should adopt guided fusion rather than naive feature stacking.","The attention maps give a per-component chemical hierarchy automatically learned from data, which could be used to identify which reactant's steric shape most controls a reaction family before any mechanistic assumptions are made.","The residual-skip design principle generalizes: any fusion that routes all tabular features through a spatial attention bottleneck will degrade purely non-spatial features, so bypassing the bottleneck is necessary for mixed steric/electronic descriptor sets.","A frozen generic backbone performing at this level challenges the necessity of chemistry-specific pretraining for yield prediction, suggesting that simpler visual encoders can serve as scalable feature extractors in synthesis optimization pipelines."],"supporting_citations":[{"why":"Supplies the benchmark rxnpredict dataset of C–N coupling reactions with tabular physical-organic descriptors and the earlier random-forest baseline this work compares against.","marker":"[1]"},{"why":"Supplies the scaled dot-product attention and multi-head Query-Key-Value formulation used for the cross-attention fusion.","marker":"[23]"},{"why":"Supplies the ResNet-18 architecture used as the frozen vision backbone and the residual skip-connection principle used to protect tabular features.","marker":"[27]"},{"why":"Generates the 2D skeletal structure images of the additive, aryl halide, base, and ligand from SMILES.","marker":"[26]"},{"why":"Provides the precedent that CNNs operating on 2D molecular drawings can extract chemical heuristics without explicit physical calculations.","marker":"[17]"},{"why":"Supplies the permutation feature importance method used to analyze which tabular features carry predictive weight in each architecture.","marker":"[30]"},{"why":"Provides the cross-attention design precedent for querying one modality with another in a general perception setting.","marker":"[24]"}],"fun_headline_variants":["Frozen vision model beats quantum descriptors at yield prediction","Cross-attention fuses 2D drawings with physical data for better yields","Generic ResNet-18 sees steric bottlenecks that DFT descriptors miss","Descriptor-guided attention beats pure quantum methods for yield RMSE","Vision cross-attention hits 5.27% RMSE on coupling yield prediction"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The reported test set comes from a random 70/30 split with a locked but unreported seed, and the paper does not verify that the same aryl halide, ligand, additive, or base never appears in both training and test; if that independence fails, the frozen vision model can partly recognize memorized drawings and its advantage over tabular descriptors is inflated.","fun_headline_variants_meta":{"raw":{"variants":["Frozen vision model beats quantum descriptors at yield prediction","Cross-attention fuses 2D drawings with physical data for better yields","Generic ResNet-18 sees steric bottlenecks that DFT descriptors miss","Descriptor-guided attention beats pure quantum methods for yield RMSE","Vision cross-attention hits 5.27% RMSE on coupling yield prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000704,"raw_usage":{"total_tokens":2993,"prompt_tokens":707,"completion_tokens":2286,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":451,"completion_tokens_details":{"reasoning_tokens":2205}},"tokens_in":451,"tokens_out":2286,"duration_ms":19240,"temperature":1.0,"reasoning_tokens":2205,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T00:17:32.272336+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same four architectures with a grouped split that keeps every reaction of a given aryl halide (or ligand) entirely in either training or test; if the vision-only model's RMSE advantage over the tabular MLP shrinks or reverses under this split, the headline comparison is driven by molecule-level leakage rather than visual steric learning.","supporting_citations":[{"cited_title":"Predicting re- action performance in c–n cross-coupling using machine learning.Science, 360:186– 190, 2018","cited_arxiv_id":null,"evidence_quote":"Supplies the benchmark rxnpredict dataset of C–N coupling reactions with tabular physical-organic descriptors and the earlier random-forest baseline this work compares against."},{"cited_title":"Attention is all you need.Adv","cited_arxiv_id":null,"evidence_quote":"Supplies the scaled dot-product attention and multi-head Query-Key-Value formulation used for the cross-attention fusion."},{"cited_title":"Deep residual learn- ing for image recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the ResNet-18 architecture used as the frozen vision backbone and the residual skip-connection principle used to protect tabular features."},{"cited_title":"Rdkit: Open-source cheminformatics software, 2024","cited_arxiv_id":null,"evidence_quote":"Generates the 2D skeletal structure images of the additive, aryl halide, base, and ligand from SMILES."},{"cited_title":"Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSAR/QSPR Models","cited_arxiv_id":"1706.06689","evidence_quote":"Provides the precedent that CNNs operating on 2D molecular drawings can extract chemical heuristics without explicit physical calculations."},{"cited_title":"All models are wrong, but many are useful: Learning a variable’s importance.J","cited_arxiv_id":null,"evidence_quote":"Supplies the permutation feature importance method used to analyze which tabular features carry predictive weight in each architecture."},{"cited_title":"Perceiver: General perception with iterative attention","cited_arxiv_id":null,"evidence_quote":"Provides the cross-attention design precedent for querying one modality with another in a general perception setting."}],"review_version":1}