Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad

Jiatong Li; Weida Wang; Xiaoyong Wei; Yatao Bian; Yuxuan Ren

REVIEW 4 major objections 5 minor 16 references

Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad

T0 review · 4 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Chemical chain-of-thought is neither a faithful explanation nor merely post-hoc rationalization, but a hallucination-prone molecular scratchpad.

desk verdict Chemical CoT is convincingly shown to be hallucination-prone and answer-decoupled, but the ER metric's completeness assumption and shared training/eval verifier keep the process-supervision claims on probation. read the letter →

arxiv 2607.20935 v1 pith:R5TKZSWG submitted 2026-07-23 cs.CE cs.CL

Jiatong Li , Yuxuan Ren , Weida Wang , Xiaoyong Wei , Yatao Bian This is my paper

classification cs.CEcs.CL

keywords chain-of-thoughtfaithfulnessmolecularhallucinationfunctional-groupgroundingSMILESscratchpadprocesssupervisionanswer-levelevaluationchemicalreasoningmodelsextrinsicfabrication

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the chain-of-thought traces produced by chemistry reasoning models cannot be trusted as faithful explanations, but are also not useless: they function as a molecular scratchpad in which some structural drafts, especially fragmented SMILES, genuinely influence the final answer. Across four model families and twelve chemistry tasks, the authors show that fabricated functional-group claims are common and largely decoupled from answer correctness—correct molecules frequently come with invented structural claims, and answer-level metrics cannot see this. Through targeted perturbations, they argue that corrupting verbal functional-group claims barely changes the answer, while corrupting drafted SMILES structures does, and that different models externalize the scratchpad in different forms. The paper concludes that chemical chain-of-thought should be evaluated at both the answer and process levels, and offers a verification-grounded training intervention as a proof of concept for reducing fabrication.

What carries the argument

The load-bearing object is the extrinsic reasoning fabrication (ER) score: a rule-based extractor parses each response into a trace and an answer, pulls out chemical claims (functional groups, ring systems, molecular classes), and checks each against the input, the predicted molecule, and the reference using exact substructure matching on molecular graphs. A claim is fabricated only if it is absent from all three. This feeds a 2x2 hallucination taxonomy (reasoning vs output, intrinsic vs extrinsic) and is paired with causal probes—teacher-forced and behavioural corruption of trace claims vs input, draft-SMILES corruption, whole-trace swapping, conditional entropy, and attribution analyses—th

What would settle it

Find a case where corrupting a verified functional-group claim in the trace flips a correct answer to wrong at a rate comparable to corrupting the same fact in the input, rather than the 1–11% versus 16–79% gap reported; or show a retrosynthesis trace whose only 'fabricated' group is a chemically necessary intermediate absent from product and reactants, which would violate the grounding-completeness assumption and require relabeling.

Watch

Extended reading notes

Core claim

The central discovery, stated on the paper's own terms, is that chemical chain-of-thought functions as a hallucination-prone molecular scratchpad: the verbal functional-group claims in a reasoning trace are often unsupported and exert little causal influence on the final answer, while fragmented SMILES drafts written in the same trace can be causally load-bearing. Correct answers frequently coexist with fabricated structural claims—on one caption-to-molecule test set, 13% of responses were exactly correct yet contained at least one fabricated functional-group claim, with near-zero correlation between fabrication and correctness. The paper supports this with targeted interventions: corrupting

Load-bearing premise

The load-bearing premise is that a functional-group claim is 'fabricated' only when it appears in neither the input, the predicted molecule, nor the reference; if legitimate chemical reasoning can refer to structures or intermediates outside that triple, then the measured decoupling between correctness and hallucination would be partly an artifact of labeling.

Editorial extensions

If this is right

Answer-only evaluation is insufficient for chemistry reasoning: a correct molecular output can be produced alongside a fabricated rationale, so benchmark scores that ignore the trace overstate reasoning quality.
Functional-group prose inside a trace should not be read as evidence of the model's computation; it is often inert with respect to the answer.
Structural drafts—especially fragmented SMILES—can be computationally consequential, so process-level supervision and interpretability tools should target structural intermediates, not just verbal explanations.
Verification-grounded process rewards can reduce extrinsic fabrication without a measurable performance loss in at least one model lineage, but they can also push the model to write the full answer early in the trace, a form of reward hacking that warrants monitoring.
The scratchpad form is model-specific, so faithfulness probes must look beyond SMILES to scaffold, positional, and naming cues depending on the model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the scratchpad account generalizes, human review of chemical rationales—for example in retrosynthesis or drug discovery—should treat a fabricated functional-group claim as a potential trigger for downstream decisions even when the model itself did not use that claim; the paper notes this as a plausible risk but does not measure it.
The sharpest test of the paper's labeling scheme is to ask expert chemists whether 'fabricated' claims are ever legitimate references to intermediates or mechanistic objects outside the input/prediction/reference triple; if such cases exist, the headline decoupling numbers would need recalibration.
The same grounding-and-perturbation recipe could be exported to other scientific domains where trace claims refer to formal objects—reaction graphs, proofs, physical constraints—giving a general way to audit whether chain-of-thought is explanation or scratchpad.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is the strongest empirical case I've seen that chemical CoT traces are not faithful explanations, and the scratchpad framing is genuinely useful. The central decoupling claim holds up much better than the stress-test note suggests, but the ER metric has a real construct-validity gap that needs patching.

What's actually new: they turn "unfaithful CoT" from a generic natural-language worry into a checkable claim about molecular graphs. Verifying functional-group statements against input, prediction, and reference via RDKit is a clean idea, and the results are concrete: on 3,300 cap2mol responses, correct and wrong answers have nearly identical ER means (6.4 vs 6.8) and |r|<0.02; 13% of responses are exactly correct yet contain a fabricated claim. The causal interventions are well done — corrupting verified functional-group mentions in the trace barely moves answer log-probabilities, while corrupting the same info in the input flips far more; and replacing a drafted SMILES with a wrong structure flips answers, with a partial-only subset controlling for early-answer leakage. The scratchpad distinction — inert verbal claims, load-bearing structural drafts — is a real step beyond either 'faithful' or 'post-hoc' labels.

Soft spots are real but mostly not fatal. The biggest is the ER definition: a claim counts as fabricated only if absent from input, predicted, and reference. Legitimate retrosynthetic or mechanistic traces often mention intermediates or reagent-derived groups that no final structure contains. The paper's rule-based exclusion of 'precursor groups' is plausible but unvalidated, and the methods explicitly admit no recall audit. That means the 13% right-yet-fabricating figure and the near-zero correlations are probably over-stated, though not by enough to flip the qualitative conclusion — the vast majority of ER claims look like genuine contradictions of the molecule at hand. Second, Chem-R-Faithful reduces ER using the same verifier in the reward loop that defines the metric, so the 73–95% reduction is partly by construction. The paper calls this a proof-of-concept, which is fair, but the process-supervision recommendation leans harder on it than the design supports. Minor: the confidence–faithfulness correlation uses five model points, and the per-task ER scale κ and aggregation weights are not reported — should appear in the supplement.

Who benefits: anyone working on scientific CoT evaluation, LLM reasoning fidelity, or chemistry LLMs. It deserves a serious referee. My recommendation: send it out, ask for the code/data, a validation set for the claim extractor's recall, and a sensitivity analysis of the decoupling results to plausible label noise. Those are patchable.

Referee Report

4 major / 5 minor

Summary. The paper studies chain-of-thought (CoT) faithfulness in chemical reasoning language models. It introduces a claim-grounding framework that parses model traces, extracts functional-group-level structural claims, and verifies them with RDKit against task-relevant structures. A claim is labeled an extrinsic reasoning fabrication (ER) when it is absent from the input, the predicted molecule, and the reference. Applying this framework to four released reasoning models across twelve chemistry tasks, the authors report that ER is widespread and largely uncorrelated with answer correctness (e.g., 13% of Chem-R caption-to-molecule responses are exactly correct yet contain at least one fabricated claim; |Pearson(ER, exact match)| < 0.02). Interventions show that corrupting verified functional-group claims in the trace barely affects the answer, while corrupting the same fact in the input has large effects. Conversely, perturbing drafted SMILES fragments in the trace degrades generation for Chem-R and Chem-R-Faithful, supporting a 'molecular scratchpad' account. Attribution analyses suggest model-specific scratchpad forms. Finally, the authors train Chem-R-Faithful via GRPO with the answer-reward gated on ER=0, reporting reduced fabrication with preserved task performance. The central claim is that chemical CoT is neither a faithful explanation nor a post-hoc rationalization, but a hallucination-prone molecular scratchpad.

Significance. If the main claims hold, this is a significant contribution to the evaluation of scientific CoT. The paper provides a concrete, deterministic method for checking chemical claims against molecular structure, and the large-scale n=3300 decoupling result is striking. The paper is unusually transparent about its limitations, including the absence of a recall audit, the conditional nature of ER, and the descriptive role of attribution analyses. The answer-level unidentifiability argument is clean, and the causal perturbation designs (e.g., scoring all conditions on the same draft-writing subset, partial-only control, metric-free conditional entropy) are careful. The SMILES-draft finding for Chem-R, corroborated by multiple independent probes, is a genuine and falsifiable observation. The Chem-R-Faithful experiment demonstrates a proof of concept for verification-grounded process supervision, even though the metric-gating raises questions about by-construction reduction. The paper's main risk is the construct validity of the ER metric; the acknowledged missing recall audit and the hand-crafted context-exclusion rules could affect the headline decoupling numbers and the training intervent

major comments (4)

[Methods, 'Task-aware grounding'; Discussion, 'Our analysis is scoped...'] The ER definition—a claim is fabricated when absent from the input, predicted molecule, and reference—presupposes that this grounding set exhausts all legitimate referents of chemical reasoning. In retrosynthetic or mechanistic traces, a model may validly mention an intermediate, a leaving group, or a reagent-derived species that is not present in the final product or predicted reactants. The rule-based exclusion of 'precursor groups mentioned only as sources of a transformation' is not quantitatively validated, and the paper states that no recall audit was performed. This directly affects the headline '13% right-yet-fabricating' figure, the near-zero ER-correctness correlations in §2.3, and the Chem-R-Faithful reward in training. I recommend a recall/completeness audit on a stratified sample of traces, a validation of the context-exclusion rules against expert chemists, and a sensitivit
[Methods, 'Hallucination score and derived metrics' (ER equation)] The treatment of responses with no extractable structural claims is unspecified ('responses with no extractable structural claims receive task-specific handling'). If such traces are assigned ER=0 and counted as 'clean,' then the clean-trace rates in Fig. 3d and the joint distribution in Fig. 3b are inflated by vacuous traces that contain no checkable claims. The paper should report, for each model and task, the proportion of traces with m=0, and should re-compute the decoupling statistics and clean-trace rates restricted to traces with at least one extractable claim. Without this, the 'clean trace' concept conflates 'no fabrication' with 'no claims made.'
[Methods, 'Process-supervised training'; Results §2.6] The Chem-R-Faithful ER reduction is partly by construction because the GRPO reward is gated on ER=0 using the same verifier that is later used to evaluate the reduction. The model can lower ER by avoiding claims that the verifier recognizes, by writing claims only after the answer is fixed, or by the early-answer behavior acknowledged as potential reward hacking (draft-copy 0.14→0.22 in Fig. 5d). The paper should report performance on a held-out claim library or a human audit of the trained model's traces, and should quantify the degree to which the ER reduction is accompanied by a shift in claim vocabulary or claim position. Without this, the 'verification-grounded' interpretation of the training result is not fully established.
[Results §2.4, Fig. 4] The conclusion that 'functional-group claims are largely inert' is based on corrupting a claim in the trace and measuring the change in answer log-probability or flip-to-wrong rate. The paper carefully controls for early answering in the SMILES-draft experiment (Fig. 5e), but not for the functional-group claim experiment. If the model writes the answer—or commits to it—before the corrupted claim appears, the corruption will have little effect regardless of whether the claim would otherwise be causally relevant. I recommend a positional control: repeat the FG-claim corruption on traces where the answer SMILES does not appear before the corrupted claim, or condition on the position of the claim relative to the first draftable answer tokens. This would strengthen the 'inert' claim and the scratchpad account derived from it.

minor comments (5)

[Methods, 'Hallucination score and derived metrics'] The per-task scale κ in the ER definition is not reported. Without κ values, the cross-task ER comparisons in Fig. 2a are not interpretable; a score of 7 on cap2mol and 7 on retro may represent different unsupported-claim rates if κ differs. Please report κ for each task or justify a common scale.
[Results §2.2, Fig. 2b] The Pearson correlation r=-0.54 between semantic entropy and aggregate hallucination is computed over only five model points. This is descriptive, but the strength of the claim 'confidence does not reflect faithfulness' should be tempered given the tiny sample size. A per-task or per-prompt correlation would be more informative.
[Results §2.6, Fig. 6] The account of ChemDFM-R's 'scaffold, positional, and naming cues' is based solely on attribution and attention analyses, which the paper itself labels descriptive. The abstract's phrasing 'attribution analyses suggest a shared scratchpad function' is appropriately hedged, but the Results section still presents the ChemDFM-R profile as a positive finding. Please make the asymmetry in evidence strength explicit in the main text of Fig. 6, not only in the Discussion.
[Discussion, 'Some measurements are bounded...'] The statement that S2-Bench 'success does not yet weight edits and optimizations by similarity to the source molecule' is important but appears only in the Discussion. Since S2-Bench contributes nine of the twelve task variants, the aggregate numbers in Figs. 1d and 2a should carry a footnote or main-text caveat.
[Abstract and Introduction] The abstract says 'across four reasoning model families and twelve chemistry tasks, hallucination is widespread.' This is correct for the released models, but the causal scratchpad claim is only firmly established for the Chem-R lineage. The abstract later uses 'suggest' for attribution, which is fine; however, the title phrase 'Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad' may over-generalize. Consider a qualifier in the title or abstract, such as 'in the Chem-R lineage, chemical CoT functions as...' or keep the title but make the scope explicit in the first sentence of the abstract.

Circularity Check

1 steps flagged · score 3.0 of 10

Main benchmark findings are self-contained; the Chem-R-Faithful ER reduction is partly by construction because the same verifier serves as both training reward and evaluation metric.

fitted input called prediction [Methods — Process-supervised training; Results §2.6]
"The same verifier used for evaluation supplies ER, the anti-hallucination score and the grounded-claim counts during training. ... We gate the answer-accuracy term on trace faithfulness, paying it only when the trace is clean: r=λf rformat +1[ER = 0]λa ranswer +λh (1−hallucination/100) +λg rgrounded. ... As a proof of concept, continuing Chem-R training with the same verifier as an online process reward reduces mean ER by 73–95% across the four task families (caption-to-molecule ER 6.7→1.8, retrosynthesis 15.1→0.8)."

Chem-R-Faithful is trained by optimizing a reward whose answer term is paid only when ER=0 and whose anti-hallucination term uses the same aggregate hallucination score. The later reported ER reduction is therefore computed with the identical verifier and metric that constituted the optimization objective. The decrease is a measure of how well the model was fitted to that metric, not an independent estimate of process-level supervision's effect on a separately measured faithfulness construct. The paper explicitly calls this a proof of concept and discusses reward hacking, so the circularity is partial and transparent, but the evaluation is not independent of the training signal.

full rationale

The core observational results—widespread ER, near-zero ER–correctness correlation, and the scratchpad perturbation findings—are not circular. They are external measurements against molecular graphs using RDKit-based exact structure matching, with no training signal tied to those measurements. The functional-group interventions and SMILES-draft perturbations are causal tests on released checkpoints, and the attribution analyses are explicitly descriptive. Self-citations (Chem-R, S2-Bench, Mol-R1) are used as model/data sources or external benchmarks, not as load-bearing justifications for the paper's own claims; no uniqueness theorem or ansatz is imported from the authors' prior work. The one partially circular element is Chem-R-Faithful: the reward is gated on ER=0 and includes an anti-hallucination term, and the same verifier is then used to report the ER reduction. The paper acknowledges this by labeling the intervention a proof of concept and by discussing potential reward hacking (draft-copy 0.14→0.22). Separately, the ER construct has a genuine validity limitation—the grounding-set completeness and missing recall audit are acknowledged in Methods—but that is a measurement-validity concern, not circularity. Overall, the central scratchpad claim is supported by independent evidence, with only the process-supervision demonstration partly reducing to its own optimization metric; hence a score of 3.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central empirical claim (released models fabricate structural claims decoupled from correctness) rests mainly on domain assumptions about grounding completeness and SMARTS matching, not on fitted parameters. The fitted/chosen scaling and reward coefficients affect reported magnitudes but not the qualitative decoupling. No new physical or conceptual entities are introduced.

free parameters (4)

κ (per-task ER scale) = not stated
In ER = min(100, κ u), κ is a fixed per-task scale; the paper does not report how it is set, yet it sets the magnitude of all reported ER values and clean-trace rates.
Hallucination aggregation weights (IR, IO, ER, EO) = (0.15, 0.25, 0.25, 0.35)
Chosen weights define the aggregate hallucination score; other weightings would change model rankings in Fig. 1d/e.
Chem-R-Faithful reward weights (λf, λa, λh, λg) = (0.1, 0.4, 0.4, 0.2)
Training hyperparameters for the process-supervised proof of concept; the ER reduction and preserved performance are conditional on this weighting and the ER=0 gate.
Semantic entropy Tanimoto threshold = 0.85
Clustering threshold for semantic entropy; a different threshold changes the entropy values used to claim confidence is not linked to faithfulness.

assumptions (5)

domain assumption A functional-group claim is fabricated iff it is absent from the input, predicted molecule, and reference.
Defines ER; presumes the grounding set is complete for judging whether a trace statement is an unsupported fabrication (Methods, Task-aware grounding).
domain assumption RDKit SMARTS and exact graph matching faithfully encode chemical structural claims.
All verification rests on RDKit exact matching; if the SMARTS library mis-categorizes phrasings, ER and claim precision inherit the error.
domain assumption The <think>...</think> delimited span is the model's reasoning trace.
Parsing normalizes delimiter variants; any reasoning outside these spans is unobserved and unscored.
domain assumption The four model families and twelve task variants are representative of chemical reasoning LLMs.
General claims about 'chemical CoT' are extrapolated from this model/task sample, with several tasks using 500-prompt subsets.
domain assumption SMILES fragments are approximated as RDKit-parseable substrings of at least six characters.
Token-type saliency and draft-corruption analyses depend on this operational definition of a drafted SMILES.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad." pith.science (2026). https://pith.science/paper/R5TKZSWG

@misc{pith2026260720935,
  author       = {Pith},
  title        = {Pith review of: Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R5TKZSWG}},
  note         = {Machine review of arXiv:2607.20935}
}

read the original abstract

Chemical reasoning language models are expected to derive molecular answers through faithful chain-of-thought (CoT). However, across four reasoning model families and twelve chemistry tasks, hallucination is widespread and largely decoupled from answer correctness: correct answers often coexist with fabricated structural claims absent from the relevant molecules. Yet this does not make the reasoning trace computationally irrelevant. Attribution analyses suggest a shared scratchpad function expressed in model-specific forms: Chem-R and ether-0 rely on fragmented SMILES drafts, whereas ChemDFM-R emphasizes scaffold, positional, and naming cues. Notably, perturbing Chem-R's SMILES sketches degrades generation, showing that structural drafts can be causally load-bearing even when verbal structural claims are largely inert. Together, these results show that chemical CoT is neither a faithful explanation nor merely a post-hoc rationalization, but a hallucination-prone molecular scratchpad. This finding cautions against treating CoT as direct evidence of faithful reasoning and motivates process-level supervision beyond answer-only evaluation.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 9 linked inside Pith

[4]

Translation between molecules and natural language

Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 375–413,

2022
[9]

Mol-r1: Towards explicit long-cot reasoning in molecule discovery

Jiatong Li, Weida Wang, Qinggang Zhang, Junxian Li, Di Zhang, Changmeng Zheng, Shufei Zhang, Xiaoyong Wei, and Qing Li. Mol-r1: Towards explicit long-cot reasoning in molecule discovery. arXiv preprint arXiv:2508.08401,

arXiv
[10]

Let’s verify step by step

Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InInternational Conference on Learning Representations, volume 2024, pp. 39578–39601,

2024
[11]

Narayanan, James D

Siddharth M. Narayanan, James D. Braza, Ryan-Rhys Griffiths, Albert Bou, Geemi Wellawatte, Mayk Caldas Ramos, Ludovico Mitchener, Samuel G. Rodriques, and Andrew D. White. Training a scientific reasoning model for chemistry.arXiv preprint arXiv:2506.17238,

arXiv
[13]

Solving math word problems with process-and outcome-based feedback.arXiv preprint arXiv:2211.14275,

Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback.arXiv preprint arXiv:2211.14275,

arXiv
[14]

Chem-r: Learning to reason as a chemist.arXiv preprint arXiv:2510.16880, 2025a

Weida Wang, Benteng Chen, Di Zhang, Wanhao Liu, Shuchen Pu, Ben Gao, Jin Zeng, Xiaoyong Wei, Tianshu Yu, Shuzhou Sun, et al. Chem-r: Learning to reason as a chemist.arXiv preprint arXiv:2510.16880, 2025a. Zhizheng Wang, Qiao Jin, Chih-Hsuan Wei, Shubo Tian, Po-Ting Lai, Qingqing Zhu, Chi-Ping Day, Christina Ross, Robert Leaman, and Zhiyong Lu. Geneagent: ...

arXiv
[15]

Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373,

Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373,

arXiv
[16]

Molreasoner: Toward effective and interpretable reasoning for molecular llms.arXiv preprint arXiv:2508.02066, 2025a

Guojiang Zhao, Zixiang Lu, Yutang Ge, Sihang Li, Zheng Cheng, Haitao Lin, Lirong Wu, Hanchen Xia, Hengxing Cai, Wentao Guo, et al. Molreasoner: Toward effective and interpretable reasoning for molecular llms.arXiv preprint arXiv:2508.02066, 2025a. Zihan Zhao, Ziping Wan, Lu Chen, Xuanze Lin, Shiyang Yu, Situo Zhang, Da Ma, Zichen Zhu, Danyang Zhang, Huaya...

arXiv

Show all 16 references

[2013]

Measuring faithfulness in chain-of-thought reasoning.arXiv preprint arXiv:2307.13702,

Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernan- dez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. Measuring faithfulness in chain-of-thought reasoning.arXiv preprint arXiv:2307.13702,

arXiv
[2020]

Attention is not explanation

Sarthak Jain and Byron C Wallace. Attention is not explanation. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 3543–3556,

2019
[2021]

Text2mol: Cross-modal molecule retrieval with natural language queries

Carl Edwards, ChengXiang Zhai, and Heng Ji. Text2mol: Cross-modal molecule retrieval with natural language queries. InProceedings of the 2021 conference on empirical methods in natural language processing, pp. 595–607,

2021
[2022]

Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models.arXiv preprint arXiv:2402.03300,

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models.arXiv preprint arXiv:2402.03300,

arXiv
[2023]

Speak-to-structure: Evaluating llms in open-domain natural language-driven molecule generation.arXiv preprint arXiv:2412.14642,

Jiatong Li, Junxian Li, Weida Wang, Yunqing Liu, Changmeng Zheng, Yatao Bian, Dongzhan Zhou, Xiao-yong Wei, and Qing Li. Speak-to-structure: Evaluating llms in open-domain natural language-driven molecule generation.arXiv preprint arXiv:2412.14642,

arXiv
[2024]

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

arXiv
[2025]

Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

arXiv
[2026]

Reasoning models don’t always say what they think.arXiv preprint arXiv:2505.05410,

Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, et al. Reasoning models don’t always say what they think.arXiv preprint arXiv:2505.05410,

arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [4]

Translation between molecules and natural language

Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 375–413,

2022

[2] [9]

Mol-r1: Towards explicit long-cot reasoning in molecule discovery

Jiatong Li, Weida Wang, Qinggang Zhang, Junxian Li, Di Zhang, Changmeng Zheng, Shufei Zhang, Xiaoyong Wei, and Qing Li. Mol-r1: Towards explicit long-cot reasoning in molecule discovery. arXiv preprint arXiv:2508.08401,

arXiv

[3] [10]

Let’s verify step by step

Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InInternational Conference on Learning Representations, volume 2024, pp. 39578–39601,

2024

[4] [11]

Narayanan, James D

Siddharth M. Narayanan, James D. Braza, Ryan-Rhys Griffiths, Albert Bou, Geemi Wellawatte, Mayk Caldas Ramos, Ludovico Mitchener, Samuel G. Rodriques, and Andrew D. White. Training a scientific reasoning model for chemistry.arXiv preprint arXiv:2506.17238,

arXiv

[5] [13]

Solving math word problems with process-and outcome-based feedback.arXiv preprint arXiv:2211.14275,

Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback.arXiv preprint arXiv:2211.14275,

arXiv

[6] [14]

Chem-r: Learning to reason as a chemist.arXiv preprint arXiv:2510.16880, 2025a

Weida Wang, Benteng Chen, Di Zhang, Wanhao Liu, Shuchen Pu, Ben Gao, Jin Zeng, Xiaoyong Wei, Tianshu Yu, Shuzhou Sun, et al. Chem-r: Learning to reason as a chemist.arXiv preprint arXiv:2510.16880, 2025a. Zhizheng Wang, Qiao Jin, Chih-Hsuan Wei, Shubo Tian, Po-Ting Lai, Qingqing Zhu, Chi-Ping Day, Christina Ross, Robert Leaman, and Zhiyong Lu. Geneagent: ...

arXiv

[7] [15]

Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373,

Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373,

arXiv

[8] [16]

Molreasoner: Toward effective and interpretable reasoning for molecular llms.arXiv preprint arXiv:2508.02066, 2025a

Guojiang Zhao, Zixiang Lu, Yutang Ge, Sihang Li, Zheng Cheng, Haitao Lin, Lirong Wu, Hanchen Xia, Hengxing Cai, Wentao Guo, et al. Molreasoner: Toward effective and interpretable reasoning for molecular llms.arXiv preprint arXiv:2508.02066, 2025a. Zihan Zhao, Ziping Wan, Lu Chen, Xuanze Lin, Shiyang Yu, Situo Zhang, Da Ma, Zichen Zhu, Danyang Zhang, Huaya...

arXiv