Pith. sign in

REVIEW 7 cited by

The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.07361 v2 pith:6NLLA23U submitted 2023-11-13 cs.CL cs.AI

Microsoft Research AI4Science , Microsoft Azure Quantum This is my paper
classification cs.CLcs.AI
keywords scientificlanguagegpt-4modeldiscoverytasksacrosscapabilities
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In recent years, groundbreaking advancements in natural language processing have culminated in the emergence of powerful large language models (LLMs), which have showcased remarkable capabilities across a vast array of domains, including the understanding, generation, and translation of natural language, and even tasks that extend beyond language processing. In this report, we delve into the performance of LLMs within the context of scientific discovery, focusing on GPT-4, the state-of-the-art language model. Our investigation spans a diverse range of scientific areas encompassing drug discovery, biology, computational chemistry (density functional theory (DFT) and molecular dynamics (MD)), materials design, and partial differential equations (PDE). Evaluating GPT-4 on scientific tasks is crucial for uncovering its potential across various research domains, validating its domain-specific expertise, accelerating scientific progress, optimizing resource allocation, guiding future model development, and fostering interdisciplinary research. Our exploration methodology primarily consists of expert-driven case assessments, which offer qualitative insights into the model's comprehension of intricate scientific concepts and relationships, and occasionally benchmark testing, which quantitatively evaluates the model's capacity to solve well-defined domain-specific problems. Our preliminary exploration indicates that GPT-4 exhibits promising potential for a variety of scientific applications, demonstrating its aptitude for handling complex problem-solving and knowledge integration tasks. Broadly speaking, we evaluate GPT-4's knowledge base, scientific understanding, scientific numerical calculation abilities, and various scientific prediction capabilities.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Hidden-State Privacy Has an Empty Middle

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    No Gaussian release fills the moderate utility-moderate privacy middle for hidden states; a Fisher lower bound rules out uniform safety in the class, with only edge diagonal mechanisms succeeding and new split-memory ...

  2. LLM-PDESR: Robust PDE Discovery via Subdomain Weighted Residuals and LLM-Guided Symbolic Hypothesis Generation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    LLM-PDESR pairs LLM structural hypotheses with C4 quintic splines and subdomain weighted residuals to recover PDEs from noisy data more robustly than prior symbolic methods.

  3. Accept or Deny? Evaluating LLM Fairness and Performance in Loan Approval across Table-to-Text Serialization Approaches

    cs.LG 2025-08 conditional novelty 6.0 of 10

    Serialization format and in-context examples change both accuracy and gender fairness of LLM loan approvals, with finance-tuned models often showing larger disparities.

  4. ScienceMeter: Tracking Scientific Knowledge Updates in Language Models

    cs.CL 2025-05 reject novelty 6.0 of 10

    ScienceMeter evaluates language model knowledge updates across three axes, preservation of old scientific claims, acquisition of new claims, and projection to future findings, and finds all current methods fall short.

  5. FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

    cs.AI 2026-08 conditional novelty 5.0 of 10

    FRAQ recovers the rank-truncated exact aggregate of client LoRA updates using one reduced QR and a small Gram eigenproblem, avoiding any full-weight-space SVD.

  6. VASP Agent: An Agentic Framework for Autonomous First-principles Calculations

    cs.AI 2025-12 conditional novelty 5.0 of 10

    An LLM-driven agent with predefined VASP workflows and parameter-checking tools completes DFT simulation tasks more reliably and accurately than standalone LLMs, with a new 80-task benchmark.

  7. Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning

    cs.AI 2025-05 conditional novelty 4.0 of 10

    Hyperparameter tuning of Cognee's knowledge graph pipeline yields consistent but uneven gains across three multi-hop QA benchmarks, with best configurations varying by dataset and metric.

Pith tools