Mat-Pref benchmark shows GRPO after SFT lets Qwen3-8B reach 65-72% on compositional materials reasoning tasks, exceeding zero-shot 235B models on held-out structure families and cross-property transfer.
arXiv:2510.04704 [cond-mat]
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
abstract
Large language models (LLMs) have shown promising potential in scientific research, enabling tasks ranging from knowledge retrieval to property prediction. Existing science benchmarks mainly focus on perceptual or knowledge-based tasks, largely ignoring the modelling tasks, a fundamental starting point for any real scientific research. For materials science, constructing and manipulating atomic structures is one of the most creative and least automated steps. In this work, we introduce AtomWorld, a benchmark designed to evaluate the abilities of LLMs on structure modifications. The benchmark includes ten fundamental actions under four widely used modelling categories, enabling verifiable evaluation metrics. We find that Claude Opus 4.6 generally performs the best. While the success rate decreases markedly with increasing modelling complexity, with particularly low success rates (below 12\% for rotation) for operations involving complex spatial relations. Our results suggest that contemporary LLMs are better suited as copilots for materials structure modelling rather than fully unsupervised autonomous scientific agents. Beyond evaluation, AtomWorld also serves as a testbed and playground for developing future structure-aware models, including reinforcement learning and agentic approaches.
citation-role summary
background 1
citation-polarity summary
years
2026 3verdicts
UNVERDICTED 3roles
background 1polarities
background 1representative citing papers
El Agente Quntur is a new multi-agent system that uses reasoning over literature and software documentation to autonomously handle the full workflow of quantum chemistry experiments in ORCA.
El Agente Estructural is a new multimodal agent that performs natural-language-driven 3D molecular geometry editing and generation using integrated domain tools and vision-language models.
citing papers explorer
-
Mat-Pref: Verifiable-Reward Training Improves Compositional Reasoning in Inorganic Materials
Mat-Pref benchmark shows GRPO after SFT lets Qwen3-8B reach 65-72% on compositional materials reasoning tasks, exceeding zero-shot 235B models on held-out structure families and cross-property transfer.
-
El Agente Quntur: A research collaborator agent for quantum chemistry
El Agente Quntur is a new multi-agent system that uses reasoning over literature and software documentation to autonomously handle the full workflow of quantum chemistry experiments in ORCA.
-
El Agente Estructural: An Artificially Intelligent Molecular Editor
El Agente Estructural is a new multimodal agent that performs natural-language-driven 3D molecular geometry editing and generation using integrated domain tools and vision-language models.