Pith. sign in

REVIEW 3 major objections 5 minor 86 references

Symbol-based entity marker highlighting for enhanced text mining in materials science with generative AI

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Symbol-based entity tags in a two-stage pipeline lift text-mining F1 by up to 83%.

desk verdict A genuinely useful NER marker idea and a sensible hybrid pipeline, but the central superiority claim is currently confounded by unequal training budgets (2,000 vs 7,500 steps), so the paper needs revision, not rejection. read the letter →

arxiv 2505.05864 v1 pith:5FNFXKPC submitted 2025-05-09 cs.CL

Junhyeong Lee , Jong Min Yuk , Chan-Woo Lee This is my paper · ORCID
classification cs.CL
keywords Text-miningNaturallanguageprocessingLargemodelGenerativeAINamedentityrecognitionmarkerKnowledgegraphextractionMaterialsscience
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a two-stage text-mining pipeline for materials science: first a generative language model highlights named entities in raw sentences by wrapping them in symbol-based tags, then a second generative model converts the highlighted text into structured knowledge graphs. The central claim is that inserting these entity markers before structuring improves both stages. Across three benchmark NER datasets, the entity-marker approach reports the highest F1 among the compared methods. On a manually annotated set of abstracts, the hybrid pipeline reports up to a 58% improvement in entity-level F1 and up to an 83% improvement in relation-level F1 over the direct approach that goes straight from raw text to structured output.

What carries the argument

The central object is the entity marker, a symbolic tag inserted before and after each entity span, with the entity type encoded in the tag name. It carries the argument by combining two mechanisms: in-context learning of entity descriptions supplied in the prompt, and multi-type extraction in a single prompt, which makes fine-tuning sentences densely highlighted and prevents the model from becoming overly conservative. The downstream part of the pipeline uses the same highlighted text as an intermediate representation that simplifies the final conversion to structured data.

What would settle it

Retrain the special-marker and entity-marker generative models with identical training step counts, identical numbers of highlighted entities per sentence, and identical prompts except for the tag syntax; if entity-level F1 on MatScholar, SOFC, and SOFC Slot does not remain higher under the entity marker, the marker format is not the causal driver.

Watch

Extended reading notes

Core claim

The paper's core claim is that symbolic entity markers, pairs of tags such as <MAT>...</MAT> that encode the entity type, allow a fine-tuned generative language model to detect multiple entity types in a single pass while still learning entity definitions in context. This combination addresses two failure modes of prior NER approaches: encoder-only token classifiers that mislabel out-of-pattern tokens, and generative special-marker methods that miss valid entities because each entity type requires a separate prompt. The paper further claims that producing entity-recognized text before the unstructured-to-structured conversion reduces error propagation, so relation-level F1 improves as a consequence of node-level F1 improving.

Load-bearing premise

The central claim assumes the only meaningful difference between the special-marker and entity-marker NER conditions is the marker format, but the reported fine-tuning used 2000 steps for the special marker and 7500 steps for the entity marker, and the two conditions also differ in how many highlighted entities appear per sentence.

Editorial extensions

If this is right

  • Knowledge-graph construction from scientific abstracts becomes more accurate when an entity-highlighting step is interposed before final structuring.
  • Multi-type named entity recognition can be run with a single prompt per sentence, cutting inference cost relative to methods that need one prompt per entity type.
  • New entity types can be introduced without retraining the NER model, since entity definitions are supplied in the prompt and the marker format is symbolic.
  • Relation-level improvements follow from node-level improvements, meaning error propagation through the pipeline is reduced when entities are recognized first.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the entity-marker benefit extends beyond materials science, the same two-stage highlighting design could help in biomedical or chemical literature, where oppositional entity pairs are also common.
  • The reported NER gains may be partly confounded by unequal training steps and supervision density between the marker conditions; an ablation that equalizes these would clarify how much of the effect is due to the marker format itself.
  • Because the structured-data evaluation used only 49 manually annotated abstracts, the magnitude of the 58% and 83% improvements may shift on a larger or more diverse corpus.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes a hybrid text-mining framework for materials science literature. In the first step, a fine-tuned LLaMA model recognizes and highlights entities in raw sentences using symbol-based 'entity markers' (e.g., <MAT> ... </MAT>); in the second step, a one-shot LLaMA-3.3-70B converts the entity-recognized text into structured knowledge graphs. The authors claim two levels of contribution: (1) the entity-marker NER format outperforms encoder-only models and a 'special marker' (GPT-NER style @@ ##) baseline on MatScholar, SOFC, and SOFC-Slot NER datasets; and (2) the resulting hybrid pipeline improves node-level F1 by up to 58% and relation-level F1 by up to 83% over the direct approach of Dagdelen et al. The paper includes fine-tuning details, evaluation procedures, and performance figures.

Significance. If the empirical claims hold, the entity-marker technique is a simple and practical contribution to NLP-based materials text mining, and the hybrid framework offers a plausible way to reduce error propagation between entity recognition and relation extraction. The NER experiments use standard benchmark datasets and report mean and standard deviation over three repeated runs, which is a solid experimental practice. The idea of using descriptive symbols that simultaneously support multi-type extraction and in-context learning is reasonable and potentially transferable to other scientific domains. However, the central causal claim that the marker format itself drives the improvement is not cleanly established because the two generative-model conditions are trained with different optimization budgets and different supervision densities. The structured-data evaluation also lacks inter-annotator agreement and variance information. These issues are fixable but currently limit the reliability of the headline improvement percentages.

major comments (3)
  1. [Methods, 'Fine tuning hyperparameters' paragraph; Fig. 6] The comparison between the special-marker and entity-marker conditions is confounded by training budget. The Methods state that the special-marker LLaMA was fine-tuned for 2000 steps while the entity-marker LLaMA was fine-tuned for 7500 steps, a 3.75x difference in the maximum step budget. The actual number of steps at the saved checkpoint is not reported, so the effective optimization length difference is unknown. Since Fig. 6 F1 scores are the basis for the claim that entity markers outperform special markers, and since the structured-data results in Fig. 8 depend on entity-recognized text produced by the entity-marker model, both the component-level and the overall-level claims inherit this uncertainty. The authors should retrain the special-marker condition with the same step budget (or demonstrate comparable convergence through training curves or per-step checkpoints) before attributing the observed gains to the marker design.
  2. [Results, 'Comparison of NER performance' and Methods, 'Fine tuning language models on NER'] A second uncontrolled variable is the number of highlighted entities per training sentence. In the special-marker condition, each sentence is duplicated per entity type with only that type highlighted, whereas in the entity-marker condition all entity types are highlighted in a single sentence. The paper itself notes in the Results that this leads to 'fewer highlighted entities' in special-marker fine-tuning sentences, making the model 'more conservative.' Because this supervision-density difference is entangled with the marker format, the improved recall and F1 of the entity-marker approach cannot be attributed solely to the symbolic marker design. An ablation that varies supervision density while holding the marker format fixed, or a control that equals the number of highlighted spans per sentence across conditions, would be needed to isolate the causal factor.
  3. [Results, 'Comparison of structured data construction performance'; Fig. 8] The structured-data evaluation relies on manual annotation, but the paper reports no inter-annotator agreement (e.g., Cohen's kappa) and no variance across repeated runs of the knowledge-graph construction. The abstract's headline '58% improvement in entity-level F1 score and up to 83% improvement in relation-level F1 score' rests entirely on this unquantified single evaluation. The authors should report agreement statistics for the manual annotation and, ideally, run the entire hybrid and direct pipelines multiple times to provide error bars or confidence intervals for the node and relation F1 scores.
minor comments (5)
  1. [Methods, 'Performance evaluation' paragraph] The sentence 'The predictions are evaluated using exact match criteria: a prediction was counted as a true positive if it exactly matched a ground-truth entity, and as a false positive otherwise.' appears twice verbatim near the end of the NER evaluation description. Please delete the duplicate.
  2. [Fig. 7 caption] The caption contains a typo: 'the number of prompts increasing proportionally to the number of entit' should read 'the number of entities.'
  3. [Abstract and Conclusion] The '58%' and '83%' improvement figures are not tied to a specific dataset in the abstract or conclusion; the percentages should be attributed to the dataset (or datasets) where the maximum improvement occurs, and the underlying F1 values for Fig. 8 should be reported in a table or in the text so that readers can verify the computation.
  4. [Methods, 'Fine tuning language models on NER'] The fine-tuned LLaMA model is trained jointly on MatScholar, SOFC, SOFC-Slot, and CoNLL-2003, but the paper does not state whether the special-marker and entity-marker conditions use identical multi-dataset training schedules and loss weighting. If they differ, that should be stated; if they are identical, a brief clarification would help.
  5. [General] The paper does not include a code availability statement or links to the fine-tuned model weights, which limits reproducibility. Providing the fine-tuning scripts, evaluation code, and the manually annotated knowledge-graph dataset would strengthen the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central NER and structured-data claims are empirical comparisons against external ground-truth benchmarks, with no fitted parameter renamed as a prediction.

full rationale

The paper's two central claims are (1) that symbol-based entity markers improve NER F1 over encoder-only and special-marker baselines, and (2) that the hybrid entity-recognized-text pipeline improves knowledge-graph node and relation F1 over the direct approach. Both claims are evaluated with held-out test sets (MatScholar, SOFC, SOFC Slot) and manually annotated knowledge graphs whose ground-truth nodes and relations are external to the model outputs. NER F1 is computed from exact-match true positives, false positives, and false negatives against dataset labels, and knowledge-graph F1 is computed from manual annotation against predefined nodes and relations, so no metric is defined in terms of the method's own outputs. The entity-marker technique is not defined in terms of the special-marker or encoder-only baselines, and no equation in the paper reduces one reported result to another by construction. The closest concern is methodological rather than circular: the Methods 'Fine tuning hyperparameters' paragraph states that the special-marker LLaMA was trained for 2000 steps while the entity-marker LLaMA was trained for 7500 steps, which is an uncontrolled training-budget difference that may confound the causal attribution of the Fig. 6 gains. However, this is a validity or fairness issue, not a definitional or self-referential reduction: the entity-marker F1 is not a fitted parameter that is then relabeled as a prediction, and the benchmark labels do not encode the proposed method. The cited HuggingFace repositories (refs 58-61) are data hosting by the first author, not a load-bearing uniqueness theorem or ansatz, and the entity descriptions generated by ChatGPT are auxiliary prompt inputs rather than the target results. The reported quantities are therefore self-contained empirical measurements, and the paper's derivation chain is not circular.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

No physical or conceptual entities are invented; the entity marker is an input formatting convention, not a postulated scientific object. The central claim rests mainly on data labeling quality, manual annotation quality, and hyperparameter choices rather than on fitted constants.

free parameters (2)
  • Fine-tuning steps per marker condition = special marker: 2000 steps; entity marker: 7500 steps
    The central NER comparison uses different training budgets for the two marker approaches. The entity marker advantage could partly reflect extra training rather than the marker format, so this hand-chosen condition directly affects the headline F1 comparison.
  • Number and wording of entity descriptions = approximately 10 ChatGPT-generated descriptions per entity type, manually curated
    In-context learning behavior depends entirely on these prompt definitions. No ablation measures sensitivity to description count or wording, so part of the measured gain may come from prompt engineering rather than the marker scheme itself.
assumptions (6)
  • domain assumption Ground-truth entity labels in MatScholar, SOFC, SOFC Slot, and CoNLL-2003 are correct and representative.
    All NER F1 scores are computed against these external labels. Incorrect or noisy labels would directly change the reported precision and recall.
  • domain assumption The manually built knowledge graphs for the 49 selected abstracts are accurate and unbiased.
    The structured-data F1 scores in Figure 8 rely on these manual annotations, and no inter-annotator agreement or independent adjudication is reported.
  • domain assumption Exact-match and manual-annotation evaluation protocols capture meaningful entity and relation quality.
    Exact match may over-penalize valid paraphrases, while manual annotation may be lenient. The paper provides no calibration between the two evaluation modes.
  • domain assumption 4-bit quantization and Q4_K_M conversion preserve the base LLaMA models' extraction capability.
    Both NER fine-tuning and structured-data construction use quantized models. Any degradation caused by quantization is not measured.
  • ad hoc to paper The training budget difference between baseline and proposed methods does not drive the reported improvement.
    The entity marker model received 7500 training steps versus 2000 for the special marker model. The comparison assumes marker format, not training length, is the causal factor.
  • domain assumption Entity descriptions generated by ChatGPT and manually curated match the intended semantics of the entity types.
    In-context learning success depends on the descriptions. If a description is off-target, the model could underperform for reasons unrelated to marker design.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Symbol-based entity marker highlighting for enhanced text mining in materials science with generative AI." pith.science (2026). https://pith.science/paper/5FNFXKPC

@misc{pith2026250505864,
  author       = {Pith},
  title        = {Pith review of: Symbol-based entity marker highlighting for enhanced text mining in materials science with generative AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5FNFXKPC}},
  note         = {Machine review of arXiv:2505.05864}
}
read the original abstract

The construction of experimental datasets is essential for expanding the scope of data-driven scientific discovery. Recent advances in natural language processing (NLP) have facilitated automatic extraction of structured data from unstructured scientific literature. While existing approaches-multi-step and direct methods-offer valuable capabilities, they also come with limitations when applied independently. Here, we propose a novel hybrid text-mining framework that integrates the advantages of both methods to convert unstructured scientific text into structured data. Our approach first transforms raw text into entity-recognized text, and subsequently into structured form. Furthermore, beyond the overall data structuring framework, we also enhance entity recognition performance by introducing an entity marker-a simple yet effective technique that uses symbolic annotations to highlight target entities. Specifically, our entity marker-based hybrid approach not only consistently outperforms previous entity recognition approaches across three benchmark datasets (MatScholar, SOFC, and SOFC slot NER) but also improve the quality of final structured data-yielding up to a 58% improvement in entity-level F1 score and up to 83% improvement in relation-level F1 score compared to direct approach.

Figures

Figures reproduced from arXiv: 2505.05864 by the authors.

Figure 1
Figure 1. Fig.1: Schematic comparison of the conv [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Fig.2: Schematic comparison of the conven [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Fig.3: Workflow for transforming unstruc [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Fig.4: Example for interaction between re [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Fig.5: Example for structured data constructi [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Fig.6: Exact match (a) precision, (b) recall, an [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Relative inference time on the test set for MatScholar, SOFC, and SOFC-Slot datasets. Comparison of structured data construction performance For the evaluation of structured data construction, knowledge graphs are chosen as the representative format due to their well-d…
Figure 8
Figure 8. Figure 8: Manually annotated F1 scores for their nodes and relations [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

86 extracted references · 55 canonical work pages

  1. [1]

    Energy Storage Research Department, Korea Institute of Energy Research, Daejeon 34129, Republic of Korea

  2. [2]

    Department of Energy Engineering, Hanyang University, Seoul, 04763, Republic of Korea

  3. [3]

    Department of Materials Science and Engineering, Korea Advanced Institute of Science and Technology, Daejeon 34141, Republic of Korea

  4. [4]

    cathode” and “LiMnPO4

    Energy AI & Computational Science Labora tory, Korea Institute of Energy Research, Daejeon 34129, Republic of Korea *Corresponding author. E-mail: cwandtj@kier.re.kr Keywords: Text-mining, Natural language processing, Large language model, Generative AI, Named entity recognition 2 Abstract The construction of experimental datasets is essential for expandi...

  5. [5]

    The NER phase (as in the multi-step approach): Each entity is recognized and highlighted within the text, producing an en tity-recognized text that enhances entity recognition performance in the following step

  6. [6]

    5 Beyond the overall data structur ing framework, we also contribute at the component level— specifically, in the NER phase

    Unstructured-to-structured phase (as in the direct approach): Generative AI is leveraged to directly conver t the entity-recognized text into structured data, thereby capturing complex relationships. 5 Beyond the overall data structur ing framework, we also contribute at the component level— specifically, in the NER phase. We propose a simple yet effectiv...

  7. [7]

    It contains 4401, 511, and 546 sentences in training, validation and test sets, respectively

    MatScholar NER dataset25: This dataset focuses on material science literature and includes seven entity types: material (MAT), structure of phase (SPL), material descriptions (DSC), application of materi al (APL), property of material (PRO), synthesis method (SMT), and characterization method (CMT). It contains 4401, 511, and 546 sentences in training, va...

    work page

  8. [8]

    It includes four entity types: EXPERIMENT, VALUE, MATERIAL and DEVICE

    SOFC NER dataset 55: This dataset is derived from materials text related to solid oxide fuel cells (SOFCs). It includes four entity types: EXPERIMENT, VALUE, MATERIAL and DEVICE. The training, validation and test sets consist of 568, 135, and 173 sentences, respectively

    work page

Show all 86 references
  1. [9]

    SOFC Slot NER dataset 55: This dataset contains same sentences with SOFC NER dataset, but with a more fine-grained entity labeling scheme. It includes 18 entity types: anode material, cathode material, conductivity, current density, degradation rate, device, electrolyte mate r...

  2. [10]

    CoNLL 2003 dataset 56: This dataset involves genera l-purpose sentences which is composed of four entities: person (PER), organization (ORG), location (LOC) and miscellaneous entity (MISC). To enha nce generative AI model’s ability to comprehend a broader range of entity types...

    2003

  3. [11]

    B-X, which indicates the beginning of an entity of type X

  4. [12]

    I-X, which denotes a token inside an entity of type X

  5. [13]

    Nanostructured

    O, which is used for tokens that do not belong to any named entity For example, in the tokenized sentence [ Nanostructured, Al2O3, …, such, solar, cells], the corresponding labels are [ B-DSC, B-MAT, …, O, B-APL, I-APL] where “Nanostructured” is labeled as a material descripti...

  6. [14]

    Zakutayev, A. et al. An open experimental database for exploring inorganic materials. Sci Data 5, 180053 (2018)

    2018

  7. [15]

    Choudhary, K. et al. Recent advances and applications of deep learning methods in materials science. npj Comput Mater 8, 1–26 (2022)

    2022

  8. [16]

    Schmidt, J., Marques, M. R. G., Botti, S. & Marques, M. A. L. Recent advances and applications of machine learning in solid-state materials science. npj Comput Mater 5, 1–36 (2019)

    2019

  9. [17]

    Guo, K., Yang, Z., Yu, C.-H. & J. Buehler, M. Artificial intelligence and machine learning in design of mechanical materials. Materials Horizons 8, 1153–1172 (2021)

    2021

  10. [18]

    & Aspuru-Guzik, A

    Sanchez-Lengeling, B. & Aspuru-Guzik, A. Inverse molecular design using machine learning: Generative models for matter engineering. Science 361, 360–365 (2018)

    2018

  11. [19]

    & Wata nabe, I

    Zheng, X., Zhang, X., Chen, T.-T. & Wata nabe, I. Deep Learning in Mechanical Metamaterials: From Prediction and Generation to Inverse Design. Advanced Materials 35, 2302530 (2023)

    2023

  12. [20]

    Lee, J. et al. Machine learning-based inverse design methods considering data characteristics and design space size in mate rials design and manufacturing: a review. Materials Horizons 10, 5436–5456 (2023)

    2023

  13. [21]

    Zhong, M. et al. Accelerated discovery of CO2 elect rocatalysts using active machine learning. Nature 581, 178–183 (2020)

    2020

  14. [22]

    Merchant, A. et al. Scaling deep learning for materials discovery. Nature 624, 80–85 (2023)

    2023

  15. [23]

    Zeni, C. et al. A generative model for i norganic materials design. Nature 639, 624– 632 (2025)

    2025

  16. [24]

    Jain, A. et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials 1, 011002 (2013)

    2013

  17. [25]

    Kirklin, S. et al. The Open Quantum Materials Database (OQMD): assessing the accuracy of DFT formation energies. npj Comput Mater 1, 1–15 (2015)

    2015

  18. [26]

    Curtarolo, S. et al. AFLOW: An automatic framewor k for high-throughput materials discovery. Computational Materials Science 58, 218–226 (2012)

    2012

  19. [27]

    Choudhary, K. et al. The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design. npj Comput Mater 6, 1–13 (2020)

    2020

  20. [28]

    & Cole, J

    Huang, S. & Cole, J. M. BatteryBERT: A Pretrained Language Model for Battery Database Enhancement. J. Chem. Inf. Model. 62, 6365–6377 (2022)

    2022

  21. [29]

    Hargreaves, C. J. et al. A database of experimentally measured lithium solid electrolyte conductivities evaluated with machine learning. npj Comput Mater 9, 1–14 (2023)

    2023

  22. [30]

    & Olivetti, E

    Venugopal, V. & Olivetti, E. MatKG: An autonomously generated knowledge graph in Material Science. Sci Data 11, 217 (2024)

    2024

  23. [31]

    Bradford, G. et al. Chemistry-Informed Machine Learning for Polymer Electrolyte Discovery. ACS Cent. Sci. 9, 206–216 (2023)

    2023

  24. [32]

    Kononova, O. et al. Opportunities and challenges of text mining in materials research. iScience 24, (2021)

    2021

  25. [33]

    Olivetti, E. A. et al. Data-driven materials resear ch enabled by natural language processing and information extraction. Applied Physics Reviews 7, 041317 (2020)

    2020

  26. [34]

    & Cole, J

    Huang, S. & Cole, J. M. A database of battery materials au to-generated using ChemDataExtractor. Sci Data 7, 260 (2020)

    2020

  27. [35]

    Pan, S. et al. Unifying Large Language Models and Knowledge Graphs: A Roadmap. Preprint at http://arxiv.org/abs/2306.08302 (2024)

    2024 arXiv

  28. [36]

    Ji, S., Pan, S., Cambria, E., Marttinen, P. & Yu, P. S. A Survey on Knowledge Graphs: Representation, Acquisition, and Applications. IEEE Transactions on Neural Networks and Learning Systems 33, 494–514 (2022)

    2022

  29. [37]

    & Navigli, R

    Huguet Cabot, P.-L. & Navigli, R. RE BEL: Relation Extraction By End-to-end 27 Language generation. in Findings of the Association for Computational Linguistics: EMNLP 2021 (eds. Moens, M.-F., Huang, X., Specia, L. & Yih, S. W.) 2370–2381 (Association for Computational Linguis...

    2021 doi

  30. [38]

    Yao, Y. et al. DocRED: A Large-Scale Document-Level Relation Extraction Dataset. in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (eds. Korhonen, A., Traum, D. & Màrqu ez, L.) 764–777 (Association for Computational Linguistics, Florenc...

    2019 doi

  31. [39]

    Weston, L. et al. Named Entity Recognition and Normalization Applied to Large-Scale Information Extraction from the Materials Science Literature. J. Chem. Inf. Model. 59, 3692– 3702 (2019)

    2019

  32. [40]

    Gupta, T., Zaki, M., Krishnan, N. M. A. & Mausam. MatSciBERT: A materials domain language model for text mining and information extraction. npj Comput Mater 8, 1–11 (2022)

    2022

  33. [41]

    Trewartha, A. et al. Quantifying the advantage of do main-specific pre-training on named entity recognition tasks in materials science. Patterns 3, 100488 (2022)

    2022

  34. [42]

    & Kim, J

    Kang, Y. & Kim, J. ChatMOF: an artificial intelligen ce system for predicting and generating metal-organic framewor ks using large language models. Nat Commun 15, 4705 (2024)

    2024

  35. [43]

    Swain, M. C. & Cole, J. M. ChemDataEx tractor: A Toolkit for Automated Extraction of Chemical Information from the Scientific Literature. J. Chem. Inf. Model. 56, 1894–1904 (2016)

    2016

  36. [44]

    Kononova, O. et al. Text-mined dataset of inorganic materials synthesis recipes. Sci Data 6, 203 (2019)

    2019

  37. [45]

    Wang, Z. et al. Dataset of solution-based inorgani c materials synthesis procedures extracted from the scientific literature. Sci Data 9, 231 (2022)

    2022

  38. [46]

    Brown, T. B. et al. Language Models are Few-S hot Learners. Preprint at https://doi.org/10.48550/arXiv.2005.14165 (2020)

    2020 doi arXiv

  39. [47]

    GPT-4 Technical Report

    OpenAI et al. GPT-4 Technical Report. Preprint at https://doi.org/10.48550/arXiv.2303.08774 (2024)

    2024 doi arXiv

  40. [48]

    Touvron, H. et al. LLaMA: Open and Efficient Foundation Language Models. Preprint at https://doi.org/10.48550/arXiv.2302.13971 (2023)

    2023 doi arXiv

  41. [49]

    Touvron, H. et al. Llama 2: Open Foundation and Fine -Tuned Chat Models. Preprint at https://doi.org/10.48550/arXiv.2307.09288 (2023)

    2023 doi arXiv

  42. [50]

    Grattafiori, A. et al. The Llama 3 Herd of Models. Preprint at https://doi.org/10.48550/arXiv.2407.21783 (2024)

    2024 doi arXiv

  43. [51]

    Team, G. et al. Gemini: A Family of Highly Capable Multimodal Models. Preprint at https://doi.org/10.48550/arXiv.2312.11805 (2024)

    2024 doi arXiv

  44. [52]

    Team, G. et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. Preprint at https://doi.org/10.48550/arXiv.2403.05530 (2024)

    2024 doi arXiv

  45. [53]

    DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

    DeepSeek-AI et al. DeepSeek LLM: Scaling Open-Source Language Models with Longtermism. Preprint at https://doi.org/10.48550/arXiv.2401.02954 (2024)

    2024 doi arXiv

  46. [54]

    DeepSeek-R1: Incentivizing R easoning Capability in LLMs via Reinforcement Learning

    DeepSeek-AI et al. DeepSeek-R1: Incentivizing R easoning Capability in LLMs via Reinforcement Learning. Preprint at https://doi.org/10.48550/arXiv.2501.12948 (2025)

    2025 doi arXiv

  47. [55]

    Raffel, C. et al. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Preprint at https://doi.org/10.48550/arXiv.1910.10683 (2023)

    2023 doi arXiv

  48. [56]

    Sang, E. F. T. K. & Meulder, F. D. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. Preprint at https://doi.org/10.48550/arXiv.cs/0306050 (2003)

    2003 doi

  49. [57]

    A linear learning rate scheduler with 100 warmup steps and a weight decay of 0.01 was used. To account for the random initialization of non-BERT layers, different learning rates were applied to the BERT and non-BERT components: the peak learning rate was set to 5e-4 for the BE...

    2000

  50. [58]

    & Lee, B

    Choi, J. & Lee, B. Accelerating materi als language processing with large language 28 models. Commun Mater 5, 1–11 (2024)

    2024

  51. [59]

    Zhang, W. et al. Fine-tuning large language mode ls for chemical text mining. Chem. Sci. 15, 10600–10611 (2024)

    2024

  52. [60]

    Keloth, V. K. et al. Advancing entity recognition in biomedicine via instruction tuning of large language models. Bioinformatics 40, btae163 (2024)

    2024

  53. [61]

    Dagdelen, J. et al. Structured information extraction from scientific text with large language models. Nat Commun 15, 1418 (2024)

    2024

  54. [62]

    Lee, S. et al. Data-driven analysis of text-mined seed-mediated syntheses of gold nanoparticles. Digital Discovery (2025) doi:10.1039/D4DD00158C

    2025 doi

  55. [63]

    & Tout anova, K

    Devlin, J., Chang, M.-W., Lee, K. & Tout anova, K. BERT: Pre-training of Deep Bidirectional Transformers for Lang uage Understanding. Preprint at https://doi.org/10.48550/arXiv.1810.04805 (2019)

    2019 doi arXiv

  56. [64]

    Wang, S. et al. GPT-NER: Named Entity Recognition via Large Language Models. Preprint at https://doi.org/10.48550/arXiv.2304.10428 (2023)

    2023 doi arXiv

  57. [65]

    A., Fuh, J

    Liu, X., Erkoyuncu, J. A., Fuh, J. Y. H., L u, W. F. & Li, B. Knowledge extraction for additive manufacturing process via named entity recognition with LLMs. Robotics and Computer-Integrated Manufacturing 93, 102900 (2025)

    2025

  58. [66]

    D., McCallum, A

    Lafferty, J. D., McCallum, A. & Pereira, F. C. N. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. in Proceedings of the Eighteenth Interna tional Conference on Machine Learning 282–289 (Morgan Kaufmann Publishers Inc., San Francisco, ...

    2001

  59. [67]

    & Dyer, C

    Lample, G., Ballesteros, M., Subramania n, S., Kawakami, K. & Dyer, C. Neural Architectures for Named Entity Recognition. Preprint at https://doi.org/10.48550/arXiv.1603.01360 (2016)

    2016 doi arXiv

  60. [68]

    & Lotufo, R

    Souza, F., Nogueira, R. & Lotufo, R. Portuguese Named Entity Recognition using BERT-CRF. Preprint at https://doi.org/10.48550/arXiv.1909.10649 (2020)

    2020 doi arXiv

  61. [69]

    https://pytorch- crf.readthedocs.io/en/stable/

    pytorch-crf — pytorch-crf 0.7.2 documentation. https://pytorch- crf.readthedocs.io/en/stable/

  62. [70]

    Friedrich, A. et al. The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domain. in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (eds. Jurafsky, D., Chai, J., Schluter, N. & Tetreault, J.) 1255...

    2020 doi

  63. [71]

    GitHub https://github.com/huggingface

    Hugging Face. GitHub https://github.com/huggingface

  64. [72]

    https://huggingface.co/datasets/Junhyeong86/matscholar_ner

    Junhyeong86/matscholar_ner · Da tasets at Hugging Face. https://huggingface.co/datasets/Junhyeong86/matscholar_ner

  65. [73]

    https://huggingface.co/datasets/Junhyeong86/sofc_ner

    Junhyeong86/sofc_ner · Data sets at Hugging Face. https://huggingface.co/datasets/Junhyeong86/sofc_ner

  66. [74]

    https://huggingface.co/datasets/Junhyeong86/NER-special-tagging

    Junhyeong86/NER-special-tagging · Datasets at Hugging Face. https://huggingface.co/datasets/Junhyeong86/NER-special-tagging

  67. [75]

    https://huggingface.co/datasets/Junhyeong86/NER-html-like-tagging

    Junhyeong86/NER-html-like-tagging · Datasets at Hugging Face. https://huggingface.co/datasets/Junhyeong86/NER-html-like-tagging

  68. [76]

    https://huggingface.co/datasets/eriktks/conll2003 (2024)

    eriktks/conll2003 · Datasets at Hugging Face. https://huggingface.co/datasets/eriktks/conll2003 (2024)

    2024

  69. [77]

    https://ollama.com/llama3.3

    llama3.3. https://ollama.com/llama3.3

  70. [78]

    Unsloth AI (2025)

    unslothai/unsloth. Unsloth AI (2025). 29

    2025

  71. [79]

    https://huggingface.co/docs/bitsandbytes/index

    bitsandbytes. https://huggingface.co/docs/bitsandbytes/index

  72. [80]

    https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-bnb-4bit (2025)

    unsloth/Llama-3.2-3B-Instruct-bnb-4bit · Hugging Face. https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-bnb-4bit (2025)

    2025

  73. [81]

    Hu, E. J. et al. LoRA: Low-Rank Adaptation of Larg e Language Models. Preprint at https://doi.org/10.48550/arXiv.2106.09685 (2021)

    2021 doi arXiv

  74. [82]

    A Rank Stabilization S caling Factor for Fine-Tuning with LoRA

    Kalajdzievski, D. A Rank Stabilization S caling Factor for Fine-Tuning with LoRA. Preprint at https://doi.org/10.48550/arXiv.2312.03732 (2023)

    2023 doi arXiv

  75. [83]

    & Zettlemoyer, L

    Dettmers, T., Lewis, M., Shleifer, S. & Zettlemoyer, L. 8-bit Optimizers via Block- wise Quantization. Preprint at https://doi.org/10.48550/arXiv.2110.02861 (2022)

    2022 doi arXiv

  76. [84]

    ggml (2025)

    ggml-org/llama.cpp. ggml (2025)

    2025

  77. [85]

    https://gith ub.com/langchain-ai

    LangChain. https://gith ub.com/langchain-ai

  78. [86]

    https://github.com/ollama

    Ollama. https://github.com/ollama

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.