REVIEW 35 cited by
ChemLLM: A Chemical Large Language Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large language models (LLMs) have made impressive progress in chemistry applications. However, the community lacks an LLM specifically designed for chemistry. The main challenges are two-fold: firstly, most chemical data and scientific knowledge are stored in structured databases, which limits the model's ability to sustain coherent dialogue when used directly. Secondly, there is an absence of objective and fair benchmark that encompass most chemistry tasks. Here, we introduce ChemLLM, a comprehensive framework that features the first LLM dedicated to chemistry. It also includes ChemData, a dataset specifically designed for instruction tuning, and ChemBench, a robust benchmark covering nine essential chemistry tasks. ChemLLM is adept at performing various tasks across chemical disciplines with fluid dialogue interaction. Notably, ChemLLM achieves results comparable to GPT-4 on the core chemical tasks and demonstrates competitive performance with LLMs of similar size in general scenarios. ChemLLM paves a new path for exploration in chemical studies, and our method of incorporating structured chemical knowledge into dialogue systems sets a new standard for developing LLMs in various scientific fields. Codes, Datasets, and Model weights are publicly accessible at https://hf.co/AI4Chem
Forward citations
Cited by 35 Pith papers
-
Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?
A new benchmark called ToxiMol evaluates how well 43 multimodal LLMs can edit toxic molecules into structurally similar, non-toxic, drug-like candidates; the best model succeeds on 43.3% of tasks.
-
Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery
A new expert-curated multimodal benchmark, SEE, shows the strongest AI models answer fewer than half of real-lab science questions correctly, and tool access brings only small gains.
-
Back to Basics: Improving Molecular Understanding in LLMs via SMILES-Graph Translation
Training LLMs first on bidirectional SMILES–graph conversion plus progressive CoT yields large structure-perception gains and better property prediction and molecular optimization.
-
MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understanding
MolSight integrates a Molecular Topology Module and Molecular Grounding Module into VLMs to enhance molecular image understanding and claims to outperform prior models on chemical visual tasks.
-
LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning
LatentChem reasons in continuous latent space for chemistry, achieving a 59.88% non-tie win rate over explicit CoT on ChemCoTBench with a 10.84x average reduction in reasoning overhead.
-
When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs
A new plausibility metric and a 6.4M-reaction dataset are proposed for benchmarking LLM retrosynthesis, and a model trained on that dataset scores highest under the metric.
-
A large-scale nanocrystal database with aligned synthesis and properties enabling generative inverse design
An LLM-built database of ~160,000 nanocrystal synthesis–property pairs enables generative design, experimentally validated by a non-stoichiometric MgF2 route.
-
MatSKRAFT: A framework for large-scale materials knowledge extraction from scientific tables
MatSKRAFT, a graph-neural-network pipeline, extracts material properties and compositions from scientific tables with higher F1 than LLMs and builds a 535k-entry knowledge base from 47k papers.
-
Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning
RetroDFM-R, a ChemDFM-based LLM trained with reasoning distillation and reinforcement learning, reaches 65.0% top-1 retrosynthesis accuracy on USPTO-50K.
-
Large Language Models for Controllable Multi-property Multi-objective Molecule Optimization
Instruction-tuned LLMs trained on C-MuMOInstruct, a new controllable multi-property molecule optimization dataset, outperform strong baselines on in-distribution and out-of-distribution optimization tasks.
-
From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization
EduVisAgent, a five-agent framework, outperforms all baseline AI models at generating pedagogically effective interactive visualizations for STEM problems, according to the new EduVisBench benchmark and its GPT-4o-bas...
-
ChemMLLM: Chemical Multimodal Large Language Model
A chemical multimodal LLM is trained to understand and generate molecule images alongside SMILES and text, with claims of state-of-the-art results on five new tasks.
-
EarthSE: A Benchmark for Evaluating Earth Scientific Exploration Capability of LLMs
EarthSE provides a two-level QA benchmark and an open-ended dialogue benchmark for Earth science and shows current LLMs perform poorly on both.
-
Bridging AI and Carbon Capture: A Dataset for LLMs in Ionic Liquids and CBE Research
Small open-weight language models show factual knowledge of ionic liquids but fail on reasoning-focused entailment tests in a new 5,920-example benchmark for carbon capture.
-
Can Hallucinations Help? Boosting LLMs for Drug Discovery
Adding hallucinated molecule descriptions to prompts improves ROC-AUC for several LLMs on molecular property prediction, with GPT-4o-generated text giving the largest consistent gains.
-
Probing the limitations of multimodal language models for chemistry and materials research
MaCBench, a new multimodal benchmark for chemistry and materials, shows current vision-language models are accurate at basic perception but unreliable for spatial, cross-modal, and multi-step scientific reasoning.
-
ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain
The paper introduces ChemSafetyBench, a 30K-question benchmark across property, usage, and synthesis tasks, and shows current LLMs perform poorly and are vulnerable to jailbreaks in chemistry safety.
-
MolMetaLM: a Physicochemical Knowledge-Guided Molecular Meta Language Model
A molecular language model pretrained on structure-property knowledge triples beats several baselines across generation, prediction, and docking benchmarks.
-
KBAlign: Efficient Self Adaptation on Specific Knowledge Bases
KBAlign is a self-supervised method that generates multi-grained QA pairs from a small text knowledge base and iteratively self-verifies to adapt a RAG model, reaching about 90% of GPT-4-supervised gains on LooGLE F1.
-
CAi Copilot: Reducing Operational Workload in Molecular Design through Intent-Driven Agentic Workflows
CAi Copilot, a three-layer LLM agent, converts broad molecular-design requests into executed, evidence-traceable workflows and outperforms five baseline agents on 45 curated tasks plus external benchmarks.
-
Game Theory Driven Multi-Agent Framework Mitigates Language Model Hallucination
A game-framed multi-agent system synthesizes large chemistry CoT/QA corpora and trains OmniChem-7B to near GPT-4o-mini performance with a large reported drop in hallucinations.
-
VASP Agent: An Agentic Framework for Autonomous First-principles Calculations
An LLM-driven agent with predefined VASP workflows and parameter-checking tools completes DFT simulation tasks more reliably and accurately than standalone LLMs, with a new 80-task benchmark.
-
What do Large Language Models know about materials?
A benchmark-style study shows larger open LLMs reproduce element melting temperatures better, yet all tested models hallucinate values for elements lacking a melting point.
-
CheMatAgent: Enhancing LLMs for Chemistry and Materials Science through Tree-Search Based Tool Learning
CheMatAgent uses hierarchical Monte Carlo tree search with separate policy and execution models, plus trained reward models, to improve tool selection and parameter filling on a new chemistry benchmark, ChemToolBench.
-
Seeing Beyond Words: MatVQA for Challenging Visual-Scientific Reasoning in Materials Science
MatVQA is a new materials-science visual QA benchmark with automated shortcut removal, and current multimodal LLMs score at most about 52% on it.
-
Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems
The paper shows that evaluating prompts by the semantic similarity of repeated LLM outputs, and refining prompts toward higher similarity, improves task success in general-purpose multi-agent systems.
-
El Agente: An Autonomous Agent for Quantum Chemistry
A hierarchical multi-agent LLM system autonomously plans, executes, and debugs quantum chemistry calculations, achieving over 87% success on six benchmark exercise types and two case studies.
-
The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility?
A DPO-based alignment method with a balanced mixture of legal and illegal chemistry prompts improves combined safety and utility scores, but its benchmark shares training compounds and its hyperparameters are tuned on...
-
Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Models
A knowledge-augmented framework using a physics formula set and checklists improves LLM accuracy on physics problem benchmarks.
-
ChemAU: Harness the Reasoning of LLMs in Chemical Research with Adaptive Uncertainty Estimation
ChemAU adds a position penalty to token-level uncertainty estimates so that flagged reasoning steps are corrected by a fine-tuned chemistry model, reporting improved accuracy on GPQA, MMLU-Pro, and SuperGPQA chemistry...
-
AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research
The paper defines a five-level AI4SoS automation hierarchy and demonstrates a preliminary LLM multi-agent society that partially reproduces known correlations between team diversity and citation impact.
-
Automatic Task Detection and Heterogeneous LLM Speculative Decoding
TaskSpec clusters user queries into tasks, fine-tunes a separate small draft model for each task, and routes prompts to the right draft model to raise token acceptance in speculative decoding.
-
BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining
A proposed Persian biomedical LLM, BioPars, is evaluated on medical QA datasets and reported to beat GPT-4 on a self-built Persian QA benchmark, but the training setup is not described.
-
Large Language Models Transform Organic Synthesis From Reaction Prediction to Automation
This survey claims to be the first systematic review of LLMs for organic synthesis, but its central 'evaluation' is never actually performed.
-
Survey on Recent Progress of AI for Chemistry: Methods, Applications, and Opportunities
This review paper organizes recent AI-for-chemistry research by data, representation, and models, and lists open challenges.
Discussion (0). Continue with ORCID to comment.