{"id":"d40c56c2-8ceb-4d96-8e86-ac1b9ceb87f1","arxiv_id":"2607.03007","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Training LLMs first on bidirectional SMILES–graph conversion plus progressive CoT yields large structure-perception gains and better property prediction and molecular optimization.","lead":"Molecular language models often score well on property tasks while failing basic atom and bond counting from SMILES. MolBasic first forces bidirectional SMILES-to-graph translation, then stacks progressive CoT training, producing large gains on structure tasks and solid downstream property and optimization results.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The structure-first claim is only weakly isolated from the effect of GPT-5 CoT distillation and progressive multi-stage SFT volume.","rationale":"The Reader correctly flags the linear staircase + GPT-5 CoT as the weakest assumption and assigns CONDITIONAL. My stress-test isolates the same soft spot more sharply: the impressive SMILES↔graph numbers (Table 1) are solid, yet the causal link from that structural competence to downstream property/optimization gains is confounded by the external CoT source that is present in every successful path. The paper’s own ablations (Table 4) and Limitations already surface the issue; they simply do not run the decisive control that would disentangle structure grounding from high-quality reasoning distillation. Because the empirical diagnosis and the conversion results remain strong, the verdict stays CONDITIONAL rather than moving to REJECT; the concrete re-training check above would settle whether the structure-first narrative is necessary or merely correlated with the CoT curriculum. No stronger internal inconsistency or data-leakage concern appears in the manuscript.","tokens_in":21726,"tokens_out":664,"duration_ms":6652,"concrete_test":"Re-train the full staircase (MSC→Prop→Opt) on the same Qwen3-8B + LoRA schedule, but replace Stage-2 GPT-5 CoT with either pure answer-only supervision or CoT generated by a model that scores near-zero on SMILES↔graph (e.g., base Qwen3-8B or ChemLLM). If property MAE and ΔLogP remain within ~10% of Table 2/3 MolBasic numbers, the structure-first claim is robust; if they collapse toward the w/o-CoT or Direct-Opt rows of Table 4, the headline gains are largely CoT-driven rather than structure-driven.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that bidirectional SMILES–Graph conversion (plus the structure→property→opt staircase) is what restores reliable molecular reasoning. Table 1’s near-zero→94.65%/85.66% conversion gains are real and impressive. However, the transfer evidence that this structural competence is the causal foundation for downstream gains is less secure than the claim requires. Ablation Table 4(a) shows that skipping structure hurts property MAE and optimization ΔLogP, and Mixed SFT underperforms ordered training, but every positive path still receives Stage-2 CoT trajectories distilled from GPT-5 (Eq. 4–5, §3.3–3.4). There is no control that keeps the identical progressive order and answer-only Stage-1 data while replacing GPT-5 CoT with either (i) no CoT or (ii) CoT from a weaker model that itself cannot do SMILES↔graph. Consequently the paper cannot cleanly separate “structure-first grounding” from “high-quality external reasoning supervision + multi-stage LoRA volume.” The Limitations section already notes that structural influence weakens on farther tasks; that admission, together with the missing CoT-source control, makes the necessity claim for SMILES–Graph translation load-bearing but only partially tested.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper argues that molecular LLMs achieve strong downstream scores without reliable structural grounding, conflicting with the chemistry principle that structure determines function. It introduces MolBasic, a structure-first post-training framework centered on bidirectional SMILES–Graph conversion within a multi-level structure comprehension (MSC) benchmark, then transfers that competence via a staircase curriculum (structure → structural property prediction → objective optimization) with two-stage LoRA training and standardized CoT distilled from GPT-5. Empirically, conversion accuracy rises from near zero to 94.65% (SMILES→Graph) and 85.66% (Graph→SMILES), with large gains on atom/bond counting and formula conversion (Table 1); property prediction becomes competitive with specialist models (Table 2); LogP-oriented optimization is best among LLM baselines (Table 3); and ablations (w/o Struct, Direct Opt, Mixed SFT, w/o CoT) plus external CleanMol/ChemCoTBench checks support the progressive structure-first design.","tokens_in":22095,"tokens_out":1327,"duration_ms":30019,"significance":"If the results hold, the work supplies a concrete, reproducible corrective to a real failure mode of molecular LLMs: near-total inability to recover 2D topology from canonical SMILES. The MSC benchmark, the explicit SMILES↔adjacency-list tasks, and the staged ablations are useful contributions beyond a single model checkpoint. Gains on held-out PubChem/TDC splits and external parsing benchmarks, together with honest Limitations on staircase weakening, make the paper a solid empirical argument that explicit structural alignment should precede high-level molecular reasoning. Strengths include thorough training/data documentation (Appendix B, Table 7), RDKit-verified conversion metrics, and public-dataset reproducibility.","major_comments":[{"comment":"§3.3–3.4, Eqs. (4)–(5), Table 4, Appendix B: The central causal claim is that SMILES–Graph translation is the foundation for downstream gains. Appendix B states that CoT is applied only to atom counting, total bond counting, and formula conversion, while conversion tasks remain answer-only—this is important and under-emphasized in the main text. Even so, every successful property/optimization path still receives GPT-5 Stage-2 CoT. Table 4 separates w/o Struct and w/o CoT but does not fully cross structure training with CoT source quality. Please state explicitly in the main paper that conversion supervision is not GPT-5 CoT, and discuss (even without a new run) how much of the transfer claim rests on high-quality external reasoning trajectories versus the learned SMILES–Graph equivalence.","section":null},{"comment":"Table 11 vs Table 1 and §4.1: GPT-5 reaches 95% atom-counting accuracy and strong formula conversion, yet scores ~0–1.4% on bidirectional SMILES–Graph conversion, while MolBasic is the reverse pattern (weaker counting ACC, 85–95% conversion). The main narrative treats conversion as the missing structural foundation, but does not reconcile why elementary counting remains far below GPT-5 after structure-first training. This comparison should appear in the main results discussion, not only the appendix, because it qualifies what “structure comprehension” MolBasic actually acquires and how that differs from general LLM reasoning.","section":null},{"comment":"§3.2 Eqs. (1)–(3) and Limitations: The linear path structure ≺ property ≺ optimization is presented as the natural chemist DAG, yet Limitations and Table 4(b) (Mixed SFT) already show that structural influence weakens on farther tasks and that joint training interferes. Frame the staircase more carefully as an effective curriculum choice rather than a validated dependency structure; otherwise the “structure determines function” rhetoric overclaims relative to the evidence that each stage plus CoT jointly drive gains.","section":null}],"minor_comments":[{"comment":"Figure 1 and §1: The pilot atom/bond counting failure is persuasive; state the pilot sample size and exact prompt format in the main text or caption so the figure is self-contained.","section":null},{"comment":"Table 2: Chemprop remains stronger on several MAE columns (e.g., MW 1.63 vs 13.98). The text correctly says “comparable,” but a short note on when LLM structure reasoning still loses to graph specialists would avoid over-reading the LLM wins.","section":null},{"comment":"Table 3 / §4.2.2: Optimization is single-objective LogP; multi-objective results are only in Appendix H.2. A one-sentence pointer in the main optimization section would help readers assess side-effect control.","section":null},{"comment":"§3.1 / conversion evaluation: You verify exact RDKit molecular match and give an adjacency-list template to all models—good. Briefly note whether hydrogen handling, aromatic kekulization, or stereochemistry are normalized, since those choices affect “exact match.”","section":null},{"comment":"Presentation: Expand acronyms on first use in figures (MSC, CoT); fix minor inconsistencies such as “MolBasic (BasicStructure Identification andComprehension)” spacing in §1; ensure Table 4 “Skipped” cells are defined in the caption.","section":null},{"comment":"Related Work §2.1: Prior SMILES-parsing work (Jang et al., 2025; Hao et al., 2026) is cited; a clearer one-sentence contrast—local syntax/substructure vs full bidirectional graph equivalence—would sharpen novelty without lengthening the section.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The skeptic’s CoT-confound concern is real for transfer but overstated for the conversion result itself: Appendix B and Table 11 together show that SMILES↔Graph is learned by supervised mapping, not inherited from GPT-5 (which fails conversion). I would not require a new weaker-CoT experiment for acceptance if the authors clarify this in the main text. Fit for a solid ML/AI-for-science venue is good; novelty is empirical and curriculum-centric rather than architectural."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful part of this paper is the diagnosis and the MSC benchmark. Existing molecular LLMs really do score near zero on bidirectional SMILES–graph conversion and basic atom/bond counting (Table 1), which is a clean, embarrassing fact that the field has been skating past. MolBasic fixes that: SMILES→graph jumps to ~95% and graph→SMILES to ~86%, with large gains on the other structure tasks. That alone is worth having.\n\nWhat they do well: they treat structure as the root of a progressive curriculum (structure → property → optimization), ship standardized CoT templates, and run the right ablations (w/o Struct, Direct Opt, Mixed SFT, w/o CoT). Property MAEs become competitive with specialist models; LogP optimization is best among the LLM baselines they try. External checks on CleanMol and ChemCoTBench, plus scaffold-OOD splits, keep the result from looking purely in-distribution. The math is ordinary LoRA next-token loss; the data pipeline is transparent; citations are appropriate.\n\nSoft spots, in proportion. The stress-test is partly right: every strong path still gets Stage-2 CoT distilled from GPT-5, and there is no control that keeps the staircase and answer-only data while swapping in weaker or no CoT. So you cannot fully separate “structure-first grounding” from “high-quality external reasoning supervision + multi-stage volume.” The authors already note in Limitations that structural influence weakens on farther tasks, and the optimization gains are milder than the property gains. Task sampling weights and the linear order are design choices, not proven necessities. No code/data release yet, which is the main practical caveat.\n\nThis is for people building molecular LLMs or post-training chemistry models who care about reliability rather than leaderboard captioning. It is not a theory paper. I would send it to peer review: the core empirical claim is sharp enough and the ablations are honest enough to deserve referee time, even if reviewers will push for a cleaner CoT-source control and code. Worth reading and citing for the benchmark and the structure-first framing; treat the causal necessity claim as provisional.","headline":"Solid empirical diagnosis and a useful structure benchmark; the transfer story is real but still entangled with GPT-5 CoT and multi-stage SFT volume.","tokens_in":22653,"tokens_out":527,"would_cite":true,"duration_ms":6288,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Molecular LLMs fail basic structure tasks; SMILES–graph translation first restores reliable reasoning.","keywords":["molecular LLMs","SMILES-graph translation","structure comprehension","Chain-of-Thought","progressive learning","property prediction","molecular optimization","structure determines function"],"falsifier":"Train the same base model with matched data volume and CoT style but scramble or reverse the staircase order (or train property and optimization without prior SMILES–graph conversion) and check whether structure conversion stays near zero and downstream property and optimization metrics collapse relative to the full MolBasic order.","tokens_in":22616,"feed_emoji":"🧪","tokens_out":585,"duration_ms":5710,"temperature":0.7,"pith_summary":"Current molecular language models can look strong on property prediction and molecule design, yet they often cannot count atoms or bonds or turn a SMILES string into a correct molecular graph. That gap breaks the chemistry principle that structure determines function. The paper argues the right order is structure first: teach models to convert both ways between SMILES and explicit graph adjacency lists, then use progressive training and standardized Chain-of-Thought to climb from structure to properties to optimization. On a multi-level structure benchmark, models that previously scored near zero on graph conversion reach high accuracy, and the same foundation improves property prediction and LogP-directed optimization. A sympathetic reader cares because high-level scores without structural grounding are brittle; fixing the foundation makes molecular reasoning more accurate, interpretable, and transferable.","feed_headline":"LLMs miss molecular structure; SMILES–graph training fixes it","feed_subtitle":"Structure-first conversion and progressive CoT raise graph accuracy and improve property and optimization tasks","key_machinery":"MolBasic: a staircase structure-first framework whose core is bidirectional SMILES–Graph translation on a multi-level structure comprehension benchmark, followed by two-stage answer-then-CoT training along the path structure → property → optimization.","core_discovery":"The paper shows that mainstream molecular LLMs almost universally fail basic structure perception—especially bidirectional SMILES–graph conversion—and that a structure-first pipeline, MolBasic, built on multi-level structure tasks with bidirectional conversion as the core, plus staircase progressive learning and CoT supervision, substantially raises structural accuracy and transfers to property prediction and objective molecular optimization.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Molecular LLMs fail SMILES-graph basics; MolBasic rebuilds structure first","Structure-first SMILES-graph translation restores molecular LLM grounding","MolBasic: bidirectional SMILES-graph tasks fix LLM structure blind spots","Progressive SMILES-graph CoT lifts molecular LLM structure and properties","LLMs miss molecular graphs; SMILES-graph training and CoT recover them"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The claim depends on the idea that a fixed linear order—structure then property then optimization—plus CoT trajectories distilled from a strong external model is the right way to transfer structural skill; if that order or those trajectories are wrong, the transfer story weakens.","fun_headline_variants_meta":{"raw":{"variants":["Molecular LLMs fail SMILES-graph basics; MolBasic rebuilds structure first","Structure-first SMILES-graph translation restores molecular LLM grounding","MolBasic: bidirectional SMILES-graph tasks fix LLM structure blind spots","Progressive SMILES-graph CoT lifts molecular LLM structure and properties","LLMs miss molecular graphs; SMILES-graph training and CoT recover them"]},"model":"grok-4.5","effort":"low","cost_usd":0.003746,"raw_usage":{"total_tokens":1135,"prompt_tokens":718,"num_sources_used":0,"completion_tokens":81,"cost_in_usd_ticks":37460000,"prompt_tokens_details":{"text_tokens":718,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":336,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":718,"tokens_out":81,"duration_ms":3902,"temperature":1.0,"reasoning_tokens":336,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T05:29:20.737901+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the same base model with matched data volume and CoT style but scramble or reverse the staircase order (or train property and optimization without prior SMILES–graph conversion) and check whether structure conversion stays near zero and downstream property and optimization metrics collapse relative to the full MolBasic order.","supporting_citations":[],"review_version":1}