REVIEW 14 cited by
ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
This paper introduces ClimateGPT, a model family of domain-specific large language models that synthesize interdisciplinary research on climate change. We trained two 7B models from scratch on a science-oriented dataset of 300B tokens. For the first model, the 4.2B domain-specific tokens were included during pre-training and the second was adapted to the climate domain after pre-training. Additionally, ClimateGPT-7B, 13B and 70B are continuously pre-trained from Llama~2 on a domain-specific dataset of 4.2B tokens. Each model is instruction fine-tuned on a high-quality and human-generated domain-specific dataset that has been created in close cooperation with climate scientists. To reduce the number of hallucinations, we optimize the model for retrieval augmentation and propose a hierarchical retrieval strategy. To increase the accessibility of our model to non-English speakers, we propose to make use of cascaded machine translation and show that this approach can perform comparably to natively multilingual models while being easier to scale to a large number of languages. Further, to address the intrinsic interdisciplinary aspect of climate change we consider different research perspectives. Therefore, the model can produce in-depth answers focusing on different perspectives in addition to an overall answer. We propose a suite of automatic climate-specific benchmarks to evaluate LLMs. On these benchmarks, ClimateGPT-7B performs on par with the ten times larger Llama-2-70B Chat model while not degrading results on general domain benchmarks. Our human evaluation confirms the trends we saw in our benchmarks. All models were trained and evaluated using renewable energy and are released publicly.
Forward citations
Cited by 14 Pith papers
-
A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)
Only gradient-based (LESS) representations make subset-to-query distance a reliable predictor of instruction-tuning performance; greedy round-robin helps most at small budgets, and random selection is surprisingly com...
-
MMESGBench: Pioneering Multimodal Understanding and Complex Reasoning Benchmark for ESG Tasks
MMESGBench provides 933 validated multimodal QA pairs from 45 ESG documents and shows that multimodal and retrieval-augmented models outperform text-only baselines on ESG document reasoning.
-
The Curious Language Model: Strategic Test-Time Information Acquisition
CuriosiTree is a greedy tree-search policy that lets LLMs select cost-effective information-gathering actions at test time, outperforming baselines in a simulated clinical diagnosis environment.
-
GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data?
GeoGrid-Bench evaluates 11 foundation models on 3,200 expert-curated questions about gridded climate data across 16 variables, finding vision-language models strongest and code generation weakest.
-
A RAG-Based Multi-Agent LLM System for Natural Hazard Resilience and Adaptation
WildfireGPT, a multi-agent RAG system with user profiling, outperforms ChatClimate and Perplexity AI in location-specific wildfire data analysis and evidence-based recommendations across ten expert case studies.
-
Enhancing LLMs for Governance with Human Oversight: Evaluating and Aligning LLMs on Expert Classification of Climate Misinformation for Detecting False or Misleading Claims about Climate Change
Fine-tuned GPT-3.5-turbo agrees with expert climate coders on social media claims as often as the experts agree with each other (alpha=0.89), but the study's open-source benchmark is weakened by a flawed prompt and ra...
-
Domain Adaptation of Foundation LLMs for e-Commerce
Continued pretraining Llama 3.1 on 1 trillion e-commerce tokens produces e-Llama 8B/70B models that improve in-house e-commerce benchmarks by roughly 25-30% while retaining most general-domain accuracy.
-
ClimateChat: Designing Data and Methods for Instruction Tuning LLMs to Answer Climate Change Queries
An automated pipeline creates climate instruction data, and fine-tuning a geoscience LLM on it improves climate question-answering accuracy over general instruction tuning.
-
Climate-Eval: A Comprehensive Benchmark for NLP Tasks Related to Climate Change
ClimateEval unifies 25 climate-related NLP tasks into one benchmark and shows that open-source LLMs gain from few-shot examples but lag on misinformation and fine-grained entity recognition.
-
Multi-Agent Visual-Language Reasoning for Comprehensive Highway Scene Understanding
A two-agent vision-language system with GPT-4o-generated chain-of-thought prompts improves highway weather, wetness, and congestion classification on small curated video datasets, with the biggest gains when sensor da...
-
Large Language Model Powered Intelligent Urban Agents: Concepts, Capabilities, and Applications
The paper defines urban LLM agents, surveys their sensing, memory, reasoning, execution, and learning workflows, and organizes their applications across planning, transportation, environment, safety, and society.
-
ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics
The paper introduces EWRA, a two-stage reasoning-aware fine-tuning method for small language models, together with the ExtremeWeatherNews dataset of over 127,000 extreme-weather news sentences, and evaluates both on t...
-
Integrating Expert Labels into LLM-based Emission Goal Detection: Example Selection vs Automatic Prompt Design
On a public dataset of 769 report passages, automatic instruction refinement by GPT-4o mini outperformed dynamic few-shot example selection for emission goal detection, but adding few-shot examples to optimized prompt...
-
Uncertainty-Aware Adaptation of Large Language Models for Protein-Protein Interaction Analysis
LoRA ensembles and Bayesian LoRA improve accuracy and calibration of LLM-based protein-protein interaction prediction on disease-specific datasets, though gains are small and baselines are limited.
Discussion (0). Continue with ORCID to comment.