Pith. sign in

REVIEW 3 major objections 8 minor 1 cited by

Summarization for Generative Relation Extraction in the Microbiome Domain

T0 review · 3 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that summarizing a passage around a target entity pair before instruction-tuned classification lifts generative relation extraction on MicrobioRel from 3.78% to 59.7% weighted F1.

desk verdict A clearly written empirical recipe that likely helps, but the claimed benefit of summarization is not isolated from instruction tuning, so the headline attribution is under-supported. read the letter →

arxiv 2506.08647 v1 pith:ZHRDLU4L submitted 2025-06-10 cs.CL cs.AI

Oumaima El Khettari , Solen Quiniou , Samuel Chaffron This is my paper
classification cs.CLcs.AI
keywords generativerelationextractionsummarizationinstructiontuningmicrobiomelow-resourceNLPclassificationlargelanguagemodelsbiomedicaltextmining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that generative large language models can do multi-class relation extraction in a specialized low-resource biomedical domain if the input is first summarized. The authors build a two-step pipeline—summarize the passage around the two entities, then instruction-tune a smaller LLM to classify the relation—and test it on MicrobioRel, a manually annotated gut-microbiome corpus with 22 relation types plus a None class. The headline result is a jump from 3.78% to 59.7% weighted F1 for Llama 3.2-3B-Instruct, attributed to noise reduction and model guidance. A sympathetic reader would care because the result suggests generative extraction can be viable where annotated data is scarce, even though domain-tuned BERT encoders still hold the higher ground.

What carries the argument

The central mechanism is a two-stage pipeline: a few-shot summarization pass in which an LLM compresses the passage around a target entity pair into a concise relation-focused summary, followed by instruction tuning of a smaller LLM that learns the relation label set from these summaries and outputs exactly one label. The summary step is designed to strip contextual noise and curb hallucinations before classification; the tuning step transfers the class vocabulary into the model so the prompt no longer needs to enumerate candidate relations.

What would settle it

Run the summarization-based pipeline again with the summarizer removed but with the same instruction-tuned model, the same prompt without a class list, and the same MicrobioRel test set; if weighted F1 stays near 59.7%, summarization is not the source of the gain, and if it drops toward 3.78%, summarization is doing the heavy lifting.

Watch

Extended reading notes

Core claim

The paper's central claim is that inserting an LLM summarization step before instruction-tuned generative relation extraction substantially improves performance on a low-resource biomedical relation extraction corpus. On MicrobioRel, the summarization-based pipeline raises Llama 3.2-3B-Instruct from 3.78% weighted F1 in zero-shot to 59.7%, with fewer hallucinations and better adherence to the predefined label set; the paper attributes this gain to reduced textual noise and focused context. It also reports that BERT-based encoders, led by PubMedBERT at 71.3%, still outperform all generative models on the same test set.

Load-bearing premise

The paper attributes the gain to summarization, but the direct and summarization-based conditions differ in instruction tuning, in whether the relation class list is shown, and in which model weights are used, so the summary effect is not isolated.

Editorial extensions

If this is right

  • The summarization-plus-tuning recipe takes small generative models from unusable to competitive, landing within roughly twelve F1 points of the best encoder on MicrobioRel.
  • Zero-shot prompting with an enumerated class list is the weak mode: every direct model scored below 15% weighted F1 and showed label bias or out-of-set hallucinations.
  • Error analysis points to the classifier, not the summarizer, as the main failure source, so further gains are more likely from better classification than from better summaries.
  • Smaller instruction-tuned models adapt better than large ones to a corpus of 1,994 annotated relations, suggesting data scale, not model scale, limits generative relation extraction in low-resource settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An ablation that instruction-tunes the same model on original passages, with the same prompt and learned label set, would isolate how much of the 55.9-point gain comes from summarization rather than from tuning or prompt changes.
  • The summarizer could be made self-critical by generating several candidate summaries and selecting the one whose predicted relation is most stable across runs, rather than picking by semantic similarity to the original text.
  • The same recipe is likely transferable to other low-resource biomedical subdomains with multi-class relation schemas, but that transfer needs a second corpus to confirm, since MicrobioRel alone may not represent the difficulty of other relation sets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper proposes a generative relation extraction (RE) pipeline for the low-resource microbiome domain. Two strategies are compared: a zero-shot direct prompting approach and a two-step summarization-based approach in which an LLM first generates a concise summary of the entity-pair context and a second, instruction-tuned model then predicts the relation. Experiments on the MicrobioRel corpus show that the direct zero-shot approach performs poorly (e.g., Llama 3.2-3B-Instruct reaches 3.78% weighted F1), while the summarization-based approach with instruction tuning reaches 59.7% for the same model, still below fine-tuned BERT encoders such as PubMedBERT at 71.3%. The paper also reports summary-quality metrics (cosine similarity and BERTScore) and a small manual error analysis of 30 misclassified examples.

Significance. If the central causal claim were established, this would be a useful contribution to low-resource multi-class generative RE: the two-step pipeline is simple, the target domain is genuinely low-resource, and the comparison against strong BERT baselines is informative. The error analysis is a strength, as it distinguishes summarization-stage errors from extraction-stage errors and acknowledges the role of automatic 'None' labels. The manuscript is honest about its limitations and does not overclaim parity with BERT. However, the headline result is not currently attributable to the summarization step because the compared conditions differ in multiple ways, and all results come from single runs without variance or significance testing. The paper ships no code and does not report hyperparameters for the LoRA tuning, which limits reproducibility. The contribution is therefore promising but not yet established.

major comments (3)
  1. [§3.1–3.2, §5.1, Table 1] The central claim that 'summarization improves generative RE performance' is not isolated by the experimental design. The direct approach (Section 3.1) uses zero-shot prompting and includes the full list of 23 relation classes in the prompt, while the summarization-based approach (Section 3.2) adds a LoRA instruction-tuning step, removes the class list, changes the system role and task instruction, and uses a different input (the generated summary). The 3.78% to 59.7% jump in Table 1 therefore conflates summarization with instruction tuning and prompt changes. No ablation instruction-tunes the same model on the original passages without summaries. The authors should add such an ablation, ideally with matched prompts and matched tuning budgets, to support the abstract's attribution. Without this, the paper's own more cautious phrasing in Section 5.1, 'combining instruction tuning with summarization,' is the only defensible statement.
  2. [§5.1, Table 1; §4.2] All reported metrics are single runs with no variance, confidence intervals, or significance tests. The difference between 59.7% and 44.6% or between 71.3% and 70.8% may be within noise, particularly given the small test set and the stochastic nature of LLM decoding. The paper should report multiple seeds or decoding runs with means and standard deviations, especially for the generative models. In addition, the weighted F1 metric can be dominated by the majority classes; reporting macro F1 or per-class breakdowns would strengthen the comparison, since the 'None' class and 'Associated_with' are very frequent in MicrobioRel (Table 2 in Appendix A).
  3. [§4.2, §5.2, §5.3] The summarization step itself is performed by Llama 3.1 in a few-shot setting, but the prompt, the demonstration examples, and the decoding parameters are not described, and no details are given for the LoRA configuration (rank, alpha, batch size, learning rate, epochs) beyond 'configurations adapted to each model's architecture.' This makes the experiments difficult to reproduce. Moreover, the error analysis in Section 5.3 shows that the generated summaries sometimes contain explicit relation statements (e.g., the Class 3 example contains 'MPTP-induced neurotoxicity can damage nigral dopaminergic neurons'), so the summarizer may be performing part of the relational reasoning itself. The paper should acknowledge and ideally measure this possibility, because it bears directly on the claimed mechanism of 'reducing noise and guiding the model.'
minor comments (8)
  1. [Abstract and §2] There are several typos and grammatical issues: 'low-resources setting' in the abstract, 'as been treated' in Section 2, 'You role' in the prompt in Section 3.2, and 'Parkison’s disease' in Section 5.3. A careful proofreading pass is needed.
  2. [§3.1, §4.1, Appendix A] The paper alternates between '22 relation types' and '23 relation classes'; clarify that the 23rd class is the automatically introduced 'None' class. Also, the prompt in Section 3.1 lists 'Marker/Mechanism' while Appendix A uses 'Marker-Mechanism'; the label string should be consistent because exact-match evaluation depends on it.
  3. [§4.3, Equation (1)] Equation (1) is fine as a definition, but the notation 'support_i' and 'total support' should be defined explicitly, and the statement that weighted F1 is derived from per-class F1 scores (not from weighted precision and recall) should be justified or removed, since it is an unusual choice and affects comparability with other work.
  4. [§5.2, Figure 1] Figure 1 is unclear: it does not specify which textual pairs are plotted (summaries vs. original passages? train, validation, or test split?), and the text mentions a 'moderate to strong positive Pearson correlation' but reports no correlation coefficient. Provide the r value, the number of points, and a description of the pairs.
  5. [§5.2] The observation that 'longer original passages tend to slightly reduce similarity scores' and 'longer summaries are weakly associated with higher similarity' is made without quantitative support. Report the correlation coefficients or regression slopes, or remove the claim.
  6. [§4.2, References] The Unsloth framework is mentioned only via a URL; it should be cited formally or at least described with a version. The paper also does not state whether code or model checkpoints will be released, which is relevant for the reproducibility claims.
  7. [Table 1 caption] The caption says 'other rows correspond to generative models used with or without instruction tuning,' but the 'Summarization-Based' rows all include instruction tuning, so the caption is misleading. Clarify that the direct rows are zero-shot and the summarization rows are instruction-tuned.
  8. [§7 Limitations] The limitations section appropriately notes the absence of comparisons with other generative RE approaches, but given the paper's claim of 'comprehensive comparison' in the contributions, this limitation should be reflected in the contribution statement as well, not only in the limitations section.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: evaluation targets a public corpus and independent BERT baselines; the summarization benefit is confounded by instruction tuning, but that is not a circular reduction.

full rationale

This paper does not contain a mathematical derivation chain whose outputs are equivalent to its inputs by construction. The central claim is empirical: summarization plus instruction tuning improves generative relation extraction on MicrobioRel. The comparison in Table 1 contrasts a zero-shot direct approach against a summarization-based approach that also adds LoRA instruction tuning and removes the class list from the prompt, so the specific contribution of summarization is not isolated. This is an experimental confound and a correctness risk, not circularity, because the summarization condition is not defined in terms of the outcome and no fitted parameter is renamed as a prediction. The dataset MicrobioRel is a public, manually annotated corpus, and the BERT-based baselines are independent external models. The only self-citation, El Khettari et al. (2023), supports the choice of PubTator for entity pre-annotation and is not load-bearing for the main summarization claim. No uniqueness theorem is imported, no ansatz is smuggled in via citation, and no known result is merely renamed. The evaluation metric is standard weighted F1, and the error analysis is manual inspection rather than a circular justification. Therefore, the paper shows no significant circularity, though its attribution of the performance gain specifically to summarization would require an ablation that is not present.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the MicrobioRel corpus and its annotations, on the assumption that summary-quality metrics indicate usable relational content, and on the unstated choice of instruction-tuning hyperparameters. No invented entities are introduced.

free parameters (2)
  • LoRA instruction-tuning configuration (rank, alpha, batch size, learning rate, epochs) = not reported; described as 'adapted to each model's architecture' (Section 4.2)
    These choices affect the measured F1 and are a potential source of the improvement attributed to summarization; without reporting them, the comparison is not fully specified.
  • Llama 3.1 summarizer variant and decoding parameters = not reported; only 'Llama 3.1 in a few-shot setting' (Section 4.2)
    Summary quality depends on the summarizer; this choice is a black box in the pipeline and could influence downstream RE.
assumptions (3)
  • domain assumption MicrobioRel's manually annotated relations are correct and serve as gold standard.
    The entire evaluation treats corpus labels as ground truth; the only human evaluation in this paper is the 30-example error analysis, not a re-annotation of the test set.
  • domain assumption BERTScore and cosine similarity are valid proxies for whether summaries preserve relation-relevant information.
    Section 5.2 uses these metrics to argue summaries 'preserve the core semantic content,' but no task-level validation links these similarities to RE accuracy.
  • domain assumption The test set is large enough for weighted F1 differences to be meaningful.
    No test set size, variance, or significance test is reported; this is a standard statistical assumption in NLP evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Summarization for Generative Relation Extraction in the Microbiome Domain." pith.science (2026). https://pith.science/paper/ZHRDLU4L

@misc{pith2026250608647,
  author       = {Pith},
  title        = {Pith review of: Summarization for Generative Relation Extraction in the Microbiome Domain},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZHRDLU4L}},
  note         = {Machine review of arXiv:2506.08647}
}
read the original abstract

We explore a generative relation extraction (RE) pipeline tailored to the study of interactions in the intestinal microbiome, a complex and low-resource biomedical domain. Our method leverages summarization with large language models (LLMs) to refine context before extracting relations via instruction-tuned generation. Preliminary results on a dedicated corpus show that summarization improves generative RE performance by reducing noise and guiding the model. However, BERT-based RE approaches still outperform generative models. This ongoing work demonstrates the potential of generative methods to support the study of specialized domains in low-resources setting.

Figures

Figures reproduced from arXiv: 2506.08647 by the authors.

Figure 1
Figure 1. Relationship between Cosine Similarity and F1-BERTScore, for the evaluated textual pairs. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fine-Tuning and Prompt Engineering of LLMs, for the Creation of Multi-Agent AI for Addressing Sustainable Protein Production Challenges

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A proof-of-concept multi-agent GPT system for microbial protein literature extraction shows both fine-tuning and prompt engineering improve cosine-similarity scores, with fine-tuning slightly ahead but more variable.

Reference graph

Works this paper leans on

53 extracted references · 46 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    & Fukuda K

    Asada M. & Fukuda K. (2024). Enhancing relation extraction from biomedical texts by large language models. In Proc. of the HCI International Conference , p.\ 3--14

    work page 2024

  3. [3]

    & Cohan A

    Beltagy I., Lo K. & Cohan A. (2019). S ci BERT : A pretrained language model for scientific text. In K. Inui, J. Jiang, V. Ng & X. Wan , \'Eds., Proc. of EMNLP-IJCNLP , p.\ 3615--3620

    work page 2019

  4. [4]

    & Borrego J

    Borrego-Ruiz A. & Borrego J. J. (2025). Human gut microbiome, diet, and mental disorders. International Microbiology , 28 (1), 1--15

    work page 2025

  5. [5]

    & Furlong L

    Bravo \`A ., Pi \ n ero J., Queralt-Rosinach N., Rautschka M. & Furlong L. I. (2015). Extraction of relations between genes and diseases from text and large-scale data analysis: implications for translational research. BMC bioinformatics , 16 , 1--17

    work page 2015

  6. [6]

    & Kavuluru R

    Brokman A., Ai X., Jiang Y., Gupta S. & Kavuluru R. (2025). A benchmark for end-to-end zero-shot biomedical relation extraction with llms: Experiments with openai models. arXiv preprint arXiv:2504.04083

    work page arXiv 2025

  7. [7]

    & Lin Y.-H

    Chang Y.-C., Huang M.-S., Huang Y.-H. & Lin Y.-H. (2025). The influence of prompt engineering on large language models for protein–protein interaction identification in biomedical literature. Scientific Reports , 15 (1), 15493

    work page 2025

  8. [8]

    & Yang D

    Chen J., Dodda M. & Yang D. (2023). Human-in-the-loop abstractive dialogue summarization. In A. Rogers, J. Boyd-Graber & N. Okazaki , \'Eds., Proc. of ACL , p.\ 9176--9190

    work page 2023

Show all 53 references
  1. [9]

    Dubey A., Jauhri A., Pandey A., Kadian A., Al-Dahle A., Letman A., Mathur A., Schelten A., Yang A., Fan A. et al. (2024). The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  2. [10]

    & Chaffron S

    El Khettari O., Quiniou S. & Chaffron S. (2023). Building a corpus for biomedical relation extraction of species mentions. In D. Demner-fushman, S. Ananiadou & K. Cohen , \'Eds., Proc. of BioNLP

    2023

  3. [11]

    & Radev D

    Erkan G. & Radev D. R. (2004). Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of artificial intelligence research , 22 , 457--479

    2004

  4. [12]

    & Herzig J

    Gekhman Z., Yona G., Aharoni R., Eyal M., Feder A., Reichart R. & Herzig J. (2024). Does fine-tuning LLM s on new knowledge encourage hallucinations? In Y. Al-Onaizan, M. Bansal & Y.-N. Chen , \'Eds., Proc. of EMNLP , p.\ 7765--7784

    2024

  5. [13]

    & Koz owska E

    G \'o ralczyk-Bi \'n kowska A., Szmajda-Krygier D. & Koz owska E. (2022). The microbiota--gut--brain axis in psychiatric disorders. International journal of molecular sciences , 23 (19), 11245

    2022

  6. [14]

    Goyal T., Li J. J. & Durrett G. (2022). News summarization and evaluation in the era of gpt-3. arXiv preprint arXiv:2209.12356

    2022 arXiv

  7. [15]

    & Zhou G

    Gu J., Sun F., Qian L. & Zhou G. (2017). Chemical-induced disease relation extraction via convolutional neural network. Database , 2017 , bax024

    2017

  8. [16]

    & Poon H

    Gu Y., Tinn R., Cheng H., Lucas M., Usuyama N., Liu X., Naumann T., Gao J. & Poon H. (2021). Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH) , 3 (1), 1--23

    2021

  9. [17]

    & Gupta S

    Gupta S. & Gupta S. K. (2019). Abstractive summarization: An overview of the state of the art. Expert Systems with Applications , 121 , 49--65

    2019

  10. [18]

    & Hsu C.-N

    He Z., Wang Y., Yan A., Liu Y., Chang E., Gentili A., McAuley J. & Hsu C.-N. (2023). M ed E val: A multi-level, multi-task, and multi-domain medical benchmark for language model evaluation. In H. Bouamor, J. Pino & K. Bali , \'Eds., Proc. of EMNLP

    2023

  11. [19]

    & Wilmes P

    Heintz-Buschart A. & Wilmes P. (2018). Human gut microbiome: function matters. Trends in microbiology , 26 (7), 563--574

    2018

  12. [20]

    & Hsu C.-n

    Hogan W., Bartko A., Shang J. & Hsu C.-n. (2024). Midred: An annotated corpus for microbiome knowledge base construction. In Proc. of BioNLP , p.\ 398--408

    2024

  13. [21]

    C., Antunes R., Matos S., Chen Q., Elangovan A., Panyam N

    Islamaj Do g an R., Kim S., Chatr-Aryamontri A., Wei C.-H., Comeau D. C., Antunes R., Matos S., Chen Q., Elangovan A., Panyam N. C. et al. (2019). Overview of the biocreative vi precision medicine track: mining protein interactions and mutations for precision medicine. Databas...

    2019

  14. [22]

    & Vateekul P

    Jettakul A., Wichadakul D. & Vateekul P. (2019). Relation extraction between bacteria and biotopes from biomedical texts with attention mechanisms and domain-specific contextual representations. BMC bioinformatics , 20 , 1--17

    2019

  15. [23]

    Q., Sablayrolles A., Mensch A., Bamford C., Chaplot D

    Jiang A. Q., Sablayrolles A., Mensch A., Bamford C., Chaplot D. S., de las Casas D., Bressand F., Lengyel G., Lample G., Saulnier L., Lavaud L. R., Lachaux M.-A., Stock P., Scao T. L., Lavril T., Wang T., Lacroix T. & Sayed W. E. (2023). Mistral 7b

    2023

  16. [24]

    & Dufour R

    Labrak Y., Bazoge A., Morin E., Gourraud P.-A., Rouvier M. & Dufour R. (2024). B io M istral: A collection of open-source pretrained large language models for medical domains. In L.-W. Ku, A. Martins & V. Srikumar , \'Eds., Proc. of ACL , p.\ 5848--5864

    2024

  17. [25]

    Lee J., Yoon W., Kim S., Kim D., Kim S., So C. H. & Kang J. (2020). Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics , 36 (4), 1234--1240

    2020

  18. [26]

    & Zettlemoyer L

    Lewis M., Liu Y., Goyal N., Ghazvininejad M., Mohamed A., Levy O., Stoyanov V. & Zettlemoyer L. (2020). BART : Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In D. Jurafsky, J. Chai, N. Schluter & J. Tetreault , \'E...

    2020

  19. [27]

    Li G., Wang P. & Ke W. (2023). Revisiting large language models as zero-shot relation extractors. In H. Bouamor, J. Pino & K. Bali , \'Eds., Proc. of EMNLP

    2023

  20. [28]

    & Wang X

    Liu S., Tang B., Chen Q. & Wang X. (2016). Drug-drug interaction extraction via convolutional neural networks. Computational and mathematical methods in medicine , 2016 (1), 6918381

    2016

  21. [29]

    Luo L., Lai P.-T., Wei C.-H., Arighi C. N. & Lu Z. (2022). Biored: a rich biomedical relation extraction dataset. Briefings in Bioinformatics , 23 (5), bbac282

    2022

  22. [30]

    Ma Y., Cao Y., Hong Y. & Sun A. (2023). Large language model is not a good few-shot information extractor, but a good reranker for hard samples! In H. Bouamor, J. Pino & K. Bali , \'Eds., Proc. of EMNLP

    2023

  23. [31]

    & Atkinson K

    Mandya A., Bollegala D., Coenen F. & Atkinson K. (2018). Combining long short term memory and convolutional neural network for cross-sentence n-ary relation extraction. arXiv preprint arXiv:1811.00845

    2018 arXiv

  24. [32]

    & Tarau P

    Mihalcea R. & Tarau P. (2004). Textrank: Bringing order into text. In Proc. of EMNLP , p.\ 404--411

    2004

  25. [33]

    & Neveol A

    Naguib M., Tannier X. & Neveol A. (2024). Few-shot clinical entity recognition in english, french and spanish: masked language models outperform generative model prompting. In Proc. of EMNLP , p.\ 6829--6852

    2024

  26. [34]

    & Zhou B

    Nallapati R., Zhai F. & Zhou B. (2017). Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Proc. of AAAI , p.\ 3075–3081

    2017

  27. [35]

    Pu X., Gao M. & Wan X. (2023). Summarization is (almost) dead. arXiv preprint arXiv:2309.09558

    2023 arXiv

  28. [36]

    B., Basmaci M., Zheng J., Jemiyo C., He Y., \"O zg \"u r A

    Rehana H., C am N. B., Basmaci M., Zheng J., Jemiyo C., He Y., \"O zg \"u r A. & Hur J. (2024). Evaluating gpt and bert models for protein--protein interaction identification in biomedical text. Bioinformatics Advances , 4 (1), vbae133

    2024

  29. [37]

    Shaib C., Li M., Joseph S., Marshall I., Li J. J. & Wallace B. (2023). Summarizing, simplifying, and synthesizing medical evidence using GPT -3 (with varying success). In A. Rogers, J. Boyd-Graber & N. Okazaki , \'Eds., Proc. of ACL , p.\ 1387--1407

    2023

  30. [38]

    B., Kao J

    Shreiner A. B., Kao J. Y. & Young V. B. (2015). The gut microbiome in health and in disease. Current opinion in gastroenterology , 31 (1), 69--75

    2015

  31. [39]

    G., Soroush A., Elias P

    Tang L., Sun Z., Idnay B., Nestor J. G., Soroush A., Elias P. A., Xu Z., Ding Y., Durrett G., Rousseau J. F. et al. (2023). Evaluating large language models on medical evidence summarization. NPJ digital medicine , 6 (1), 158

    2023

  32. [40]

    Touvron H., Martin L., Stone K., Albert P., Almahairi A., Babaei Y., Bashlykov N., Batra S., Bhargava P., Bhosale S. et al. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  33. [41]

    M., Fourrier-Reglat A., Gurwitz D., Molokhia M., Nieto A., Trifiro G., Kors J

    Van Mulligen E. M., Fourrier-Reglat A., Gurwitz D., Molokhia M., Nieto A., Trifiro G., Kors J. A. & Furlong L. I. (2012). The eu-adr corpus: annotated drugs, diseases, targets, and their relationships. Journal of biomedical informatics , 45 (5), 879--884

    2012

  34. [42]

    P., Seehofnerova A

    Van Veen D., Van Uden C., Blankemeier L., Delbrouck J.-B., Aali A., Bluethgen C., Pareek A., Polacin M., Reis E. P., Seehofnerova A. et al. (2023). Clinical text summarization: adapting large language models can outperform human experts. Research Square

    2023

  35. [43]

    & Wallace B

    Wadhwa S., Amir S. & Wallace B. (2023). Revisiting relation extraction in the era of large language models. In A. Rogers, J. Boyd-Graber & N. Okazaki , \'Eds., Proc. of ACL

    2023

  36. [44]

    Wei C.-H., Allot A., Leaman R. & Lu Z. (2019). Pubtator central: automated concept annotation for biomedical full text articles. Nucleic acids research , 47 (W1), W587--W593

    2019

  37. [45]

    & Chen E

    Xu D., Chen W., Peng W., Zhang C., Xu T., Zhao X., Wu X., Zheng Y., Wang Y. & Chen E. (2024). Large language models for generative information extraction: A survey. Frontiers of Computer Science , 18 (6), 186357

    2024

  38. [46]

    & Liang P

    Yasunaga M., Leskovec J. & Liang P. (2022). L ink BERT : Pretraining language models with document links. In S. Muresan, P. Nakov & A. Villavicencio , \'Eds., Proc. of ACL , p.\ 8003--8016

    2022

  39. [47]

    K., Hu Y., Zhang R., Xu H

    Zhang J., Wibert M., Zhou H., Peng X., Chen Q., Keloth V. K., Hu Y., Zhang R., Xu H. & Raja K. (2024a). A study of biomedical relation extraction using gpt models. AMIA Summits on Translational Science Proceedings , 2024 , 391

    2024

  40. [48]

    Zhang J., Zhang D., Xu Y., Zhang J., Liu R., Gao Y., Shi Y., Cai P., Zhong Z., He B. et al. (2025). Large-scale biosynthetic analysis of human microbiomes reveals diverse protective ribosomal peptides. Nature Communications , 16 (1), 3054

    2025

  41. [49]

    Zhang T., Kishore V., Wu F., Weinberger K. Q. & Artzi Y. (2019). Bertscore: Evaluating text generation with BERT . arXiv preprint arXiv:1904.09675

    2019 arXiv

  42. [50]

    & Hashimoto T

    Zhang T., Ladhak F., Durmus E., Liang P., McKeown K. & Hashimoto T. B. (2024b). Benchmarking large language models for news summarization. Transactions of the Association for Computational Linguistics , 12 , 39--57

    2024

  43. [51]

    & Ruan T

    Zhang W., Lu W., Wang J., Wang Y., Chen L., Jiang H., Liu J. & Ruan T. (2024c). Unexpected phenomenon: LLM s ' spurious associations in information extraction. In L.-W. Ku, A. Martins & V. Srikumar , \'Eds., Proc. of ACL

    2024

  44. [52]

    & Bansal M

    Zhang Z., Elfardy H., Dreyer M., Small K., Ji H. & Bansal M. (2023). Enhancing multi-document summarization with cross-document graph-based information extraction. In A. Vlachos & I. Augenstein , \'Eds., Proc. of EACL

    2023

  45. [53]

    & Wen J.-R

    Zhu Y., Yuan H., Wang S., Liu J., Liu W., Deng C., Chen H., Liu Z., Dou Z. & Wen J.-R. (2023). Large language models for information retrieval: A survey. arXiv preprint arXiv:2308.07107

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.