Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Variable Extraction for Model Recovery in Scientific Literature

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Large language models are the most effective tested method for extracting variable names, descriptions, and values from scientific papers, and feeding them rule-based candidates as suggestions improves the best results.

desk verdict A useful new benchmark for variable extraction with a sound but incomplete evaluation; the missing inter-annotator agreement is the main thing to fix before relying on it. read the letter →

arxiv 2411.14569 v1 pith:6K5HWWDZ submitted 2024-11-21 cs.IR cs.LG

Chunwei Liu , Enrique Noriega-Atala , Adarsh Pyarelal , Clayton T Morrison , Mike Cafarella This is my paper · ORCID
classification cs.IRcs.LG
keywords variableextractionscientificliteraturelargelanguagemodelsrule-basedinformationmodelrecoverybenchmarkdatasetepidemiologicalmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that the first step of automatic model recovery from scientific literature—extracting a model's variable names, their descriptions, and their numerical values from running text—can be done most effectively with large language models, and that LLMs get a further boost when rule-based extractions are handed to them as reference candidates. To test this, the authors built a manually annotated benchmark from 22 epidemiological papers, producing 2,083 ground-truth annotations over 556 text passages. Across rule-based extraction, several LLMs, and an automated AI pipeline, the LLM-based approaches lead, with the best tool-enhanced configuration reaching F1 0.525 under GPT-4 similarity evaluation and 0.640 under token-level evaluation, versus 0.320 and 0.437 for the rule-based system alone. The paper's own conclusion is that the task remains unsolved and that, because no method used supervised learning, there is clear room for improvement.

What carries the argument

The central object is the variable extraction task: converting a passage of scientific prose into structured triples of variable name, description, and numerical value. The argument is carried by the benchmark—22 manually annotated papers yielding 2,083 annotations across 556 text passages—used as ground truth; a rule-based extractor that matches syntactic patterns for definitions and values; an LLM prompt template that asks for structured output and can optionally receive rule-based candidates as reference suggestions; and two match evaluators, a GPT-4 similarity judge and token-level span scoring, that decide whether a prediction counts as correct.

What would settle it

Re-annotate a random sample of the 22 papers with a second independent annotator and compute inter-annotator agreement, then re-run the evaluation with exact span matching or human adjudication instead of the GPT-4 similarity judge; if agreement is low or exact-match F1 is much lower than reported, the headline comparisons would not reproduce.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is an empirical one: for identifying variable descriptions and values in scientific text, LLM-based extraction is currently more effective than conventional rule-based extraction, and the best configuration combines both, giving the LLM the paper text plus rule-based candidate extractions as suggestions while insisting the original text decides conflicts. The evidence is the benchmark evaluation, in which the rule-based system alone scores F1 0.320 under GPT-4 similarity evaluation and 0.437 under token-level scoring, the best pure LLM scores 0.491 and 0.600, and the best tool-enhanced LLM scores 0.525 and 0.640. The paper also finds that few-shot prompting improves only some models, that an automated declarative pipeline reaches about the same quality as a pure LLM while enforcing structured output, and that on full-paper input, where recall drops, tool integration partially restores performance.

Load-bearing premise

The results stand or fall on whether the human-annotated benchmark is accurate and whether the GPT-4 similarity judge reliably recognizes a correct extraction; if the annotations miss or mislabel variables, or the judge favors certain phrasings, every reported F1 score moves.

Editorial extensions

If this is right

  • Downstream model-recovery systems can treat variable extraction as a viable first stage, since the extracted name–description–value triples supply the basic units needed to assemble symbolic models and run simulations.
  • Adding rule-based candidates to an LLM prompt is a low-cost way to improve extraction without any supervised training, since tool-enhanced versions beat pure LLMs in most configurations.
  • Because the best F1 is only about 0.5–0.64, any fully automatic model recovery pipeline will inherit many missed or misaligned variables, so human review or higher-recall methods are still needed.
  • Since none of the tested methods used supervised learning, fine-tuning on the annotated benchmark is a direct next step that could raise performance substantially.
  • Evaluation protocol matters: GPT-4 similarity and token-level scores are close but not interchangeable, and full-paper extraction is harder than passage-level extraction, so future comparisons should report both.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the paper leaves implicit is that the GPT-4 similarity judge may accept loose paraphrases, so true extractive accuracy could be lower than reported; adjudicating a random sample of matches with human judges would quantify that gap.
  • The same prompt-plus-tool recipe probably transfers to other quantitative fields, such as climate or economics, where model parameters are introduced in prose; the benchmark would need to be rebuilt for those domains.
  • A stronger end-to-end test would assemble the extracted variables into executable models and check whether they reproduce the published simulation curves, testing variable extraction by its real downstream purpose rather than by span matching.
  • The finding that few-shot prompting sometimes hurts performance hints that prompt sensitivity, not model scale, is a binding constraint for scientific extraction; measuring variance across many prompt templates would make that explicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces SciVar, a manually annotated benchmark for variable extraction from scientific literature, comprising 2,083 variable-related annotations (descriptions and values) from 22 epidemiological papers. It evaluates a rule-based Odin extractor, several vanilla, few-shot, and tool-enhanced LLM configurations (GPT-3.5T, GPT-4T, GPT-4o, GPT-4o-mini, Llama-3-8B, Mistral-7B), and the Palimpzest declarative AI pipeline. Performance is measured both by a GPT-4 similarity judge and by token-level span F1. The central claim is that LLM-based extractors outperform the conventional rule-based system, and that tool-enhanced LLMs improve over pure LLM baselines, with the best configuration (tool GPT4o-mini) reaching F1 0.525 under the GPT-4 similarity metric and 0.640 under token-level evaluation. The paper also reports a full-paper-context experiment on selected models.

Significance. If the empirical claims hold, the paper makes a useful contribution: it defines a concrete information-extraction task for model recovery, releases a public benchmark with annotation guidelines, and provides reproducible baseline code (the Odin rules, prompts, and Palimpzest configuration are all described). The token-level evaluation is a valuable external anchor because it does not depend on an LLM judge, and the two metric families largely agree on the overall ordering of LLM-based versus rule-based methods. The main value is as a resource and a baseline comparison for a task that has received little direct attention. However, the strength of the headline comparison depends on the stability and fairness of the manually annotated ground truth, which is not yet demonstrated, and on the reliability of the GPT-4 similarity evaluator.

major comments (3)
  1. [Human Annotation / Quality Control] The benchmark's validity is load-bearing for the central LLM-versus-rules comparison, but no inter-annotator agreement is reported. The annotation guidelines explicitly allow implied variables, as in the example 'the estimated reproduction rate in the United States was around 2.5' where the variable name 'reproduction rate' is not explicitly paired with a symbol in the highlighted span. Because the rule-based baseline is designed around explicit syntactic patterns (symbols plus definitional phrases), a ground truth enriched with semantics-only, implied variables may systematically disadvantage it. Please report agreement statistics (e.g., Cohen's kappa or a span-level agreement measure) and, if possible, a breakdown of how many annotations rely on implied variables versus explicit symbol-description pairs, so that the fairness of the comparison can be assessed.
  2. [Evaluation / GPT-4 as a Similarity Evaluator] The claim that the best integrated result represents 'a 20% improvement over the setups using only LLMs' is not supported by Table 1. The best pure LLM baseline is pureGPT4T at F1 0.491 and the best tool-enhanced result is toolGPT4o-mini at F1 0.525, which is a relative improvement of about 6.9%; even comparing against pureGPT4o (0.480), the improvement is about 9.4%. In addition, no confidence intervals or statistical significance tests are provided, and many differences between configurations in Table 1 are small relative to the size of the corpus (22 papers). Please correct the arithmetic and add uncertainty quantification or significance testing, or temper the corresponding claims in the text.
  3. [GPT-4 as a Similarity Evaluator] The primary similarity-based metric uses GPT-4 as the judge, while most of the systems being judged are also GPT variants, which introduces a possible same-model preference. The token-level evaluation in Table 2 provides an independent ordering and therefore partially mitigates this concern, but it still inherits the same human ground truth and uses a coarse 'generous alignment' span-matching rule. To make the headline F1 values robust, please report a small human evaluation of the GPT-4 judge's agreements with human judgments on a sample of candidate pairs, and ideally a sensitivity check with an alternative judge or with stricter matching criteria. Without this, the exact superiority margins in Table 1 remain tied to a single unvalidated LLM evaluator.
minor comments (5)
  1. [Abstract / Introduction] There is a typo in the Introduction: 'Subsequently, We then conduct' has an incorrectly capitalized 'We' after a comma; it should read 'Subsequently, we then conduct'.
  2. [Table 1 caption] The caption reads 'similarity evolution with ground-truth'; this should be 'similarity evaluation with ground truth'.
  3. [Post-Processing with Structured Format] The sentence 'After the annotation and quality review process, each paper will have a unified color code mapping' uses the future tense inconsistently; the paper will also want to ensure the JSON example in Figure 2 does not contain the stray arrow characters that appear in the current rendering.
  4. [Experimental Setup] The table includes rows labeled 'structuredGPT4o' and 'structuredGPT4o-mini', but these configurations are not defined in the Experimental Setup or in Section 'Variable Extraction Approaches'; please add a short description of what 'structured' means and how it differs from the other LLM settings.
  5. [Full Paper Context Extraction Evaluation] In Table 3 the configuration 'pureGPT3.5T C' is not defined; if 'C' denotes chunking, state this explicitly in the caption or text, and specify the chunk size used.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: central empirical claim rests on manual ground truth, with token-level evaluation independent of the LLM judge.

full rationale

The paper's central claim is an empirical comparison of extraction methods against a manually annotated benchmark, not a result derived from a fitted parameter or a self-citation chain. The headline F1 scores in Tables 1 and 2 are computed by aligning predicted extractions with human-annotated ground truth; the token-level evaluation in Table 2 is explicitly independent of the GPT-4 similarity judge, and the paper states that 'token-level scores rely solely on manual annotations.' The use of GPT-4 Turbo as a similarity evaluator for outputs that include GPT-4 Turbo predictions is methodologically self-referential, but it does not make the result definitional: the judge is not fitted to the data, the matching criteria are not derived from the predicted systems, and the same qualitative ordering is reproduced by token-level scores. Citations to the authors' prior work (e.g., Pyarelal et al. 2020; Liu et al. 2024) motivate the task or describe a baseline system, but they are not invoked as evidence for the comparative result. Quality-control concerns raised by the paper's annotation procedure, such as the absence of inter-annotator agreement and the inclusion of 'implied' variables in the guidelines, are threats to benchmark validity rather than instances of circular reasoning. No equation or construction is shown to reduce the headline comparison to its own inputs, so no circular step is flagged.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted; the paper is an empirical evaluation. The central assumptions are corpus representativeness, annotation reliability without agreement statistics, the validity of GPT-4 as a similarity judge, and standard span-extraction evaluation conventions.

assumptions (4)
  • domain assumption The 22-paper COVID-19 corpus is representative enough to support conclusions about variable extraction in scientific literature.
    The benchmark covers only pandemic modeling papers, but the abstract frames the task for all scientific literature; generalization is assumed.
  • domain assumption Human annotations are correct ground truth despite no reported inter-annotator agreement.
    Annotation quality control is described qualitatively; without agreement statistics, the ground truth is assumed reliable.
  • domain assumption GPT-4 similarity judgments approximate human equivalence judgments for variable descriptions and values.
    The headline F1 metric uses GPT-4 Turbo as evaluator; the paper argues token-level metrics corroborate this but still relies on this assumption for Table 1.
  • standard math Token-level F1 is an appropriate evaluation metric for span extraction.
    Conventional in NLP; not questioned by the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Variable Extraction for Model Recovery in Scientific Literature." pith.science (2026). https://pith.science/paper/6K5HWWDZ

@misc{pith2026241114569,
  author       = {Pith},
  title        = {Pith review of: Variable Extraction for Model Recovery in Scientific Literature},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6K5HWWDZ}},
  note         = {Machine review of arXiv:2411.14569}
}
abstract

The global output of academic publications exceeds 5 million articles per year, making it difficult for humans to keep up with even a tiny fraction of scientific output. We need methods to navigate and interpret the artifacts -- texts, graphs, charts, code, models, and datasets -- that make up the literature. This paper evaluates various methods for extracting mathematical model variables from epidemiological studies, such as ``infection rate ($\alpha$),'' ``recovery rate ($\gamma$),'' and ``mortality rate ($\mu$).'' Variable extraction appears to be a basic task, but plays a pivotal role in recovering models from scientific literature. Once extracted, we can use these variables for automatic mathematical modeling, simulation, and replication of published results. We introduce a benchmark dataset comprising manually-annotated variable descriptions and variable values extracted from scientific papers. Based on this dataset, we present several baseline methods for variable extraction based on Large Language Models (LLMs) and rule-based information extraction systems. Our analysis shows that LLM-based solutions perform the best. Despite the incremental benefits of combining rule-based extraction outputs with LLMs, the leap in performance attributed to the transfer-learning and instruction-tuning capabilities of LLMs themselves is far more significant. This investigation demonstrates the potential of LLMs to enhance automatic comprehension of scientific artifacts and for automatic model recovery and simulation.

Figures

Figures reproduced from arXiv: 2411.14569 by the authors.

Figure 1
Figure 1. Example of variable extraction from a scientific [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example of SciVar JSON output extracted and for [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Example of a pattern-matching rule sys￾tem designed to detect variable descriptions. The word interpreted will anchor the pattern (line 8). Outgoing syntactic dependencies of types nmod as and nsubjpass to entities of types Phrase and Identifier link the rule’s trigger to its description and variable arguments, re￾spectively. Vanilla LLM Extraction LLMs have demonstrated exceptional performance on a va￾riety of sema… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Prompt templates for variable extraction using var [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Palimpzest Code for Variable Extraction from Sci [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: To ensure conciseness, we limit the output token [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 6
Figure 6. Figure 6: GPT4 Turbo prompt templates for evaluating the [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 19 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Chen, Q.; Zhuo, Z.; and Wang, W. 2019. BERT for Joint Intent Classification and Slot Filling. ArXiv, abs/1902.10909

    arXiv 2019

  4. [4]

    Chen, Z.; Cao, L.; Madden, S.; Kraska, T.; Shang, Z.; Fan, J.; Tang, N.; Gu, Z.; Liu, C.; and Cafarella, M. 2023. SEED: Domain-Specific Data Curation With Large Language Models. arXiv e-prints, arXiv--2310

    work page 2023

  5. [5]

    Collard, J.; de Paiva, V.; Fong, B.; and Subrahmanian, E. 2022. Extracting Mathematical Concepts from Text. In Proceedings of the Eighth Workshop on Noisy User-generated Text (W-NUT 2022), 15--23. Gyeongju, Republic of Korea: Association for Computational Linguistics

    work page 2022

  6. [6]

    M.; and Pfeiffer, O

    Jeschke, S.; Wilke, M.; Blanke, M.; Natho, N. M.; and Pfeiffer, O. F. 2007. Information extraction from mathematical texts by means of natural language processing techniques. In Proceedings of the International Workshop on Educational Multimedia and Multimedia Education, Emme '07, 109–114. New York, NY, USA: Association for Computing Machinery. ISBN 9781595937834

    work page 2007

  7. [7]

    Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D

    Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; de las Casas, D.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; Lavaud, L. R.; Lachaux, M.-A.; Stock, P.; Scao, T. L.; Lavril, T.; Wang, T.; Lacroix, T.; and Sayed, W. E. 2023. Mistral 7B. arXiv:2310.06825

    arXiv 2023

  8. [8]

    Kim, J.-D.; Wang, Y.; and Yasunori, Y. 2013. The G enia Event Extraction Shared Task, 2013 Edition - Overview. In N \'e dellec, C.; Bossy, R.; Kim, J.-D.; Kim, J.-j.; Ohta, T.; Pyysalo, S.; and Zweigenbaum, P., eds., Proceedings of the B io NLP Shared Task 2013 Workshop , 8--15. Sofia, Bulgaria: Association for Computational Linguistics

    work page 2013

Show all 33 references
  1. [9]

    Lao, N.; Mitchell, T.; and Cohen, W. W. 2011. Random Walk Inference and Learning in A Large Scale Knowledge Base. In Barzilay, R.; and Johnson, M., eds., Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, 529--539. Edinburgh, Scotland, UK.:...

    2011

  2. [10]

    G.; and Wu, E

    Lin, Y.; Hulsebos, M.; Ma, R.; Shankar, S.; Zeigham, S.; Parameswaran, A. G.; and Wu, E. 2024. Towards Accurate and Efficient Document Analytics with Large Language Models. arXiv preprint arXiv:2405.04674

    2024 arXiv

  3. [11]

    B.; Chen, Z.; Franklin, M.; Kraska, T.; Madden, S.; and Vitagliano, G

    Liu, C.; Russo, M.; Cafarella, M.; Cao, L.; Chen, P. B.; Chen, Z.; Franklin, M.; Kraska, T.; Madden, S.; and Vitagliano, G. 2024. A Declarative System for Optimizing AI Workloads. arXiv:2405.14696

    2024 arXiv

  4. [12]

    Mohan, S.; and Li, D. 2019. MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts. CoRR, abs/1902.09476

    2019 arXiv

  5. [13]

    Neumann, M.; King, D.; Beltagy, I.; and Ammar, W. 2019. S cispa C y: F ast and R obust M odels for B iomedical N atural L anguage P rocessing. In Proceedings of the 18th BioNLP Workshop and Shared Task, 319--327. Florence, Italy: Association for Computational Linguistics

    2019

  6. [14]

    G.; and Nikolich-Zugich, J

    Noriega-Atala, E.; Rahat-Uz-Zaman, M.; Bhat, R.; Jergovic, M.; Kobourov, S. G.; and Nikolich-Zugich, J. 2023. Visualizing Interaction Networks and Evidence in Biomedical Corpora. In 2023 IEEE 16th Pacific Visualization Symposium (PacificVis), 41--50

    2023

  7. [15]

    Ohta, T.; Pyysalo, S.; Rak, R.; Rowley, A.; Chun, H.-W.; Jung, S.-J.; Choi, S.-P.; Ananiadou, S.; and Tsujii, J. 2013. Overview of the Pathway Curation ( PC ) task of B io NLP Shared Task 2013. In N \'e dellec, C.; Bossy, R.; Kim, J.-D.; Kim, J.-j.; Ohta, T.; Pyysalo, S.; and ...

    2013

  8. [16]

    OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774

    2024 arXiv

  9. [17]

    Patel, L.; Jha, S.; Guestrin, C.; and Zaharia, M. 2024. LOTUS: Enabling Semantic Queries with LLMs Over Tables of Unstructured and Structured Data. arXiv preprint arXiv:2407.11418

    2024 arXiv

  10. [18]

    L., Enrique Noriega

    Patty Gahan, C. L., Enrique Noriega. 2024. ASKEM scientific paper annotation guidelines. https://people.csail.mit.edu/chunwei/data/scivar_pub/ASKEM (Accessed on 11/19/2024)

    2024

  11. [19]

    A.; Sharp, R.; Hein, P

    Pyarelal, A.; Valenzuela - Esc \' a rcega, M. A.; Sharp, R.; Hein, P. D.; Stephens, J.; Bhandari, P.; Lim, H.; Debray, S.; and Morrison, C. T. 2020. AutoMATES: Automated Model Assembly from Text, Equations, and Software. CoRR, abs/2001.07295

    2020 arXiv

  12. [20]

    Rajpurkar, P.; Zhang, J.; Lopyrev, K.; and Liang, P. 2016. SQ u AD : 100,000+ Questions for Machine Comprehension of Text. In Su, J.; Duh, K.; and Carreras, X., eds., Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2383--2392. Austin, Te...

    2016

  13. [21]

    Schaffhauser, T.; Garijo, D.; Osorio, M.; Bittner, D.; Pierce, S.; Vargas, H.; Disse, M.; and Gil, Y. 2023. A framework for the broad dissemination of hydrological models for non-expert users. Environmental Modelling & Software, 164: 105695

    2023

  14. [22]

    A.; Nagesh, A.; Yadav, V.; Bachman, J.; Tang, Z.; Lent, H.; Luo, F.; Paul, M.; Bethard, S.; Barnard, K.; Morrison, C.; and Surdeanu, M

    Sharp, R.; Pyarelal, A.; Gyori, B.; Alcock, K.; Laparra, E.; Valenzuela-Esc \'a rcega, M. A.; Nagesh, A.; Yadav, V.; Bachman, J.; Tang, Z.; Lent, H.; Luo, F.; Paul, M.; Bethard, S.; Barnard, K.; Morrison, C.; and Surdeanu, M. 2019. Eidos, INDRA , & Delphi: From Free Text to Ex...

    2019

  15. [23]

    Shimorina, A.; Heinecke, J.; and Herledan, F. 2022. Knowledge Extraction From Texts Based on W ikidata. In Loukina, A.; Gangadharaiah, R.; and Min, B., eds., Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Huma...

    2022

  16. [24]

    F.; and De Meulder, F

    Tjong Kim Sang, E. F.; and De Meulder, F. 2003. Introduction to the C o NLL -2003 Shared Task: Language-Independent Named Entity Recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT - NAACL 2003 , 142--147

    2003

  17. [25]

    Topsakal, O.; and Akinci, T. C. 2023. Creating large language model applications utilizing langchain: A primer on developing llm apps fast. In International Conference on Applied Engineering and Natural Sciences, volume 1, 1050--1056

    2023

  18. [26]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; Rodriguez, A.; Joulin, A.; Grave, E.; and Lample, G. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971

    2023 arXiv

  19. [27]

    A.; Babur, \"O .; Hahn-Powell, G.; Bell, D.; Hicks, T.; Noriega-Atala, E.; Wang, X.; Surdeanu, M.; Demir, E.; and Morrison, C

    Valenzuela-Esc \'a rcega, M. A.; Babur, \"O .; Hahn-Powell, G.; Bell, D.; Hicks, T.; Noriega-Atala, E.; Wang, X.; Surdeanu, M.; Demir, E.; and Morrison, C. T. 2018. Large-scale automated machine reading discovers new cancer-driving mechanisms. Database, 2018: bay098

    2018

  20. [28]

    A.; Hahn-Powell, G.; and Surdeanu, M

    Valenzuela-Esc \'a rcega, M. A.; Hahn-Powell, G.; and Surdeanu, M. 2016. Odin ' s Runes: A Rule Language for Information Extraction. In Calzolari, N.; Choukri, K.; Declerck, T.; Goggi, S.; Grobelnik, M.; Maegaard, B.; Mariani, J.; Mazo, H.; Moreno, A.; Odijk, J.; and Piperidis...

    2016

  21. [29]

    Wang, Y.; Wang, L.; Rastegar-Mojarad, M.; Moon, S.; Shen, F.; Afzal, N.; Liu, S.; Zeng, Y.; Mehrabi, S.; Sohn, S.; and Liu, H. 2018. Clinical information extraction applications: A literature review. Journal of Biomedical Informatics, 77: 34--49

    2018

  22. [30]

    Xu, D.; Chen, W.; Peng, W.; Zhang, C.; Xu, T.; Zhao, X.; Wu, X.; Zheng, Y.; Wang, Y.; and Chen, E. 2024. Large Language Models for Generative Information Extraction: A Survey. arXiv:2312.17617

    2024 arXiv

  23. [31]

    G.; Nghiem, M.-Q.; Matsubayashi, Y.; and AIZAWA, A

    Yoko, K. G.; Nghiem, M.-Q.; Matsubayashi, Y.; and AIZAWA, A. 2012. Extracting Definitions of Mathematical Expressions in Scientific Papers. The 26th Annual Conference of the Japanese Society for Artificial Intelligence, JSAI2012: 3P1IOS2a3--3P1IOS2a3

    2012

  24. [32]

    Zhang, Y.; Zhong, V.; Chen, D.; Angeli, G.; and Manning, C. D. 2017. Position-aware Attention and Supervised Data Improve Slot Filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), 35--45

    2017

  25. [33]

    H.; Cao, S.; Kozyrakis, C.; Stoica, I.; Gonzalez, J

    Zheng, L.; Yin, L.; Xie, Z.; Huang, J.; Sun, C.; Yu, C. H.; Cao, S.; Kozyrakis, C.; Stoica, I.; Gonzalez, J. E.; et al. 2023. Efficiently programming large language models using sglang. arXiv preprint arXiv:2312.07104

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.