REVIEW 8 cited by
GeoGalactica: A Scientific Large Language Model in Geoscience
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have achieved huge success for their general knowledge and ability to solve a wide spectrum of tasks in natural language processing (NLP). Due to their impressive abilities, LLMs have shed light on potential inter-discipline applications to foster scientific discoveries of a specific domain by using artificial intelligence (AI for science, AI4S). In the meantime, utilizing NLP techniques in geoscience research and practice is wide and convoluted, contributing from knowledge extraction and document classification to question answering and knowledge discovery. In this work, we take the initial step to leverage LLM for science, through a rather straightforward approach. We try to specialize an LLM into geoscience, by further pre-training the model with a vast amount of texts in geoscience, as well as supervised fine-tuning (SFT) the resulting model with our custom collected instruction tuning dataset. These efforts result in a model GeoGalactica consisting of 30 billion parameters. To our best knowledge, it is the largest language model for the geoscience domain. More specifically, GeoGalactica is from further pre-training of Galactica. We train GeoGalactica over a geoscience-related text corpus containing 65 billion tokens, preserving as the largest geoscience-specific text corpus. Then we fine-tune the model with 1 million pairs of instruction-tuning data consisting of questions that demand professional geoscience knowledge to answer. In this technical report, we will illustrate in detail all aspects of GeoGalactica, including data collection, data cleaning, base model selection, pre-training, SFT, and evaluation. We open-source our data curation tools and the checkpoints of GeoGalactica during the first 3/4 of pre-training.
Forward citations
Cited by 8 Pith papers
-
Geo-Strat-RL: Learning Geological Event Reasoning from Verifiable Tasks
Geo-Strat-RL applies RLVR in a synthetic geological generator-verifier setup to boost VLM stratigraphic reasoning scores and shows transfer to synthetic seismic data without domain-specific training.
-
Auto-Configuring Scientific Simulators with Lightweight Coding-Agent Adapters
SIGA is a coding-agent adapter using retrieval, procedural memory, and validation gates that raises success rate on GEOS from 0.720 to 0.789 while cutting variance 16x and matching expert quality in minutes instead of hours.
-
GeoMind: An Agentic Workflow for Lithology Classification with Reasoned Tool Invocation
GeoMind applies an agentic workflow with tool-augmented modules and process supervision to outperform static models on lithology classification from well logs while producing traceable decisions.
-
PETRA: Transforming Web Text for Petroleum-Engineering Domain Adaptation
PETRA is a curated 1.36M-chunk petroleum-engineering retrieval dataset and pipeline that raises in-domain nDCG from 0.703 to 0.763 via score fusion and delivers 44% relative gain on an Earth Science benchmark through ...
-
Auto-Configuring Scientific Simulators with Lightweight Coding-Agent Adapters
SIGA adapters let off-the-shelf coding agents produce complete, valid configurations for multiphysics simulators like GEOS in minutes rather than hours, with self-evolution further improving performance on held-out cases.
-
HydroAgent: Closing the Gap Between Frontier LLMs and Human Experts in Hydrologic Model Calibration via Simulator-Grounded RL
HydroAgent fine-tunes Qwen3-4B on 2,576 expert calibration trajectories and applies Group-Relative Policy Optimization with NSE reward from live CREST simulations to improve hydrologic model calibration over frontier LLMs.
-
Earth Science Foundation Models: From Perception to Reasoning and Discovery
The paper delivers a unified review and roadmap of Earth science foundation models, structured by capability depth from perception to agentic reasoning and by application breadth across atmosphere, hydrosphere, lithos...
-
Earth Science Foundation Models: From Perception to Reasoning and Discovery
A review of Earth science foundation models covering capability evolution from perception to discovery, applications across atmosphere/hydrosphere/lithosphere/biosphere/anthroposphere/cryosphere, over 200 datasets, an...
Discussion (0). Sign in to comment.