This paper introduces Astro-HEP-BERT, a BERT model adapted to astrophysics and high-energy physics text, plus a large arXiv-based corpus, as a low-cost tool for studying conceptual change in science.
Improving astroBERT using Semantic Textual Similarity
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
The NASA Astrophysics Data System (ADS) is an essential tool for researchers that allows them to explore the astronomy and astrophysics scientific literature, but it has yet to exploit recent advances in natural language processing. At ADASS 2021, we introduced astroBERT, a machine learning language model tailored to the text used in astronomy papers in ADS. In this work we: - announce the first public release of the astroBERT language model; - show how astroBERT improves over existing public language models on astrophysics specific tasks; - and detail how ADS plans to harness the unique structure of scientific papers, the citation graph and citation context, to further improve astroBERT.
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Astro-HEP-BERT: A bidirectional language model for studying the meanings of concepts in astrophysics and high energy physics
This paper introduces Astro-HEP-BERT, a BERT model adapted to astrophysics and high-energy physics text, plus a large arXiv-based corpus, as a low-cost tool for studying conceptual change in science.