Pith. sign in

REVIEW 3 major objections 4 minor 63 references

RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplements

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single instruction-tuned LLM whose prompts are augmented with retrieved training examples can extract named entities, relations, triples, and usage status from dietary-supplement notes at F1 scores that match or beat single-task models.

desk verdict A plausible multi-task/RAG framework for dietary supplement IE, but the RAG benefit may be partly memorization and the reporting needs cleanup. read the letter →

arxiv 2411.15700 v1 pith:OJ5FMALH submitted 2024-11-24 cs.CL cs.AIcs.CE

Zaifu Zhan , Shuang Zhou , Mingchen Li , Rui Zhang This is my paper · ORCID
classification cs.CLcs.AIcs.CE
keywords dietarysupplementsinformationextractionmulti-tasklearningretrieval-augmentedgenerationlargelanguagemodelsclinicalrecordsinstructionfine-tuningnamedentityrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dietary supplement information sits in unstructured clinical notes, and extracting it cleanly usually requires separate models for different jobs. This paper proposes RAMIE, a framework that trains one instruction-tuned large language model to do four jobs at once: name supplement and adverse-event entities, classify relations between them, output subject-predicate-object triples, and label whether supplement use started, continued, was discontinued, or is uncertain. The central claim is that retrieval augmentation—showing the model a semantically similar sentence-response pair retrieved from the training set at both training and inference time—raises accuracy, while multi-task learning keeps the whole system to a single cheaper model. In the reported experiments, the best RAMIE setups reach F1 scores of 87.39 on entity recognition, 93.74 on relation extraction, 79.45 on triple extraction, and 93.45 on usage classification, generally matching or beating the same models trained one task at a time.

What carries the argument

The load-bearing mechanism is the retrieval-augmented prompt built from the training set. For each incoming sentence, a retriever compares sentence embeddings by cosine similarity and selects the closest sentence-response pair from the same task's training split; that example is inserted into a task-specific instruction prompt, and the model is fine-tuned with low-rank adaptation on these prompts. Retrieval is restricted during training so the model cannot copy the input sentence's own answer, but at test time the retriever searches the full training set. Three dense retrievers—MedCPT, Contriever, and BMRetriever—supply the similarity search, and instruction fine-tuning supplies the task-switching behavior, so one model can answer all four output formats.

What would settle it

Recompute the reported RAG gains after deduplicating or removing training sentences that are near-duplicates of test sentences; if the advantage of retrieved examples over random examples disappears, the claimed RAG boost is an artifact of dataset redundancy rather than retrieval-based generalization.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that retrieval-augmented generation is what makes multi-task instruction fine-tuning affordable for dietary-supplement information extraction. With instruction fine-tuning alone, training one model on all four tasks costs an average F1 drop of about 2.72% relative to single-task training, with the largest loss on triple extraction; when the prompt is augmented with a retrieved training example, many of those losses disappear. A random example does not help and can hurt, but each of the three dense retrievers consistently improves scores over the no-retrieval baseline. Across 96 model-retriever-task setups, 46 beat the corresponding single-task fine-tuned model, and the best per-task numbers are 87.39 F1 for NER, 93.74 for RE, 79.45 for TE, and 93.45 for UC. The paper concludes that a single RAMIE-tuned model is comparable to single-task models while being much cheaper to store and train.

Load-bearing premise

The RAG benefit assumes that the retrieved nearest training example at test time is helping the model learn the task, rather than giving away the answer from a near-duplicate sentence; the paper does not report how similar train and test sentences are.

Editorial extensions

If this is right

  • A single RAMIE-tuned LLM can replace four separately trained information-extraction models, cutting storage and training cost in a clinical NLP pipeline.
  • Retrieval-augmented prompts with dense retrievers beat both zero-shot and random-example prompting, so the same recipe is worth testing in other biomedical extraction tasks.
  • Multi-task learning alone costs a small F1 drop, but pairing it with retrieval augmentation recovers most of the loss, making multi-task deployment practical.
  • Relation extraction is the most multi-task-friendly task, while triple extraction is the persistent bottleneck.
  • Because RAMIE is not tied to dietary supplement vocabulary, the framework is claimed to transfer to other domains with annotated training sets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stricter test of the RAG claim would split the data by patient or document rather than by sentence; if the retrieval gain vanishes, the retriever is exploiting repeated phrasing instead of improving generalization.
  • A retriever fine-tuned on dietary-supplement text could push performance further, since all three retrievers used here are general biomedical tools and the paper notes none is supplement-specific.
  • The consistently larger multi-task drop on triple extraction suggests grouping tasks by output format or giving triple extraction extra capacity could reduce negative transfer beyond what RAG already repairs.
  • The finding that random examples hurt while retrieved examples help implies that example relevance, not just few-shot formatting, drives the improvement, so retrieval quality thresholds deserve explicit study.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes RAMIE, a retrieval-augmented multi-task instruction fine-tuning framework for extracting dietary supplement information from clinical text, covering named entity recognition, relation extraction, triple extraction, and usage classification. The authors evaluate eight LLMs under three settings: single-task instruction fine-tuning, the full RAMIE framework (multi-task learning plus retrieval-augmented generation), and multi-task instruction fine-tuning without retrieval. The main claimed findings are that RAG consistently improves accuracy over multi-task learning without RAG, that multi-task learning incurs only a small performance drop while saving storage and training cost, and that RAMIE yields per-task F1 scores up to 87.39 (NER), 93.74 (RE), 79.45 (TE), and 93.45 (UC). The paper includes an ablation study with three retrievers plus a random-example baseline.

Significance. If the central claim is correct, the paper is a useful engineering contribution: it shows that a single instruction-tuned LLM can handle four clinical information extraction tasks with a modest performance trade-off, and that retrieving similar training examples at inference time can help. The internal comparisons are transparent, the RAG-versus-no-RAG ablation is consistently positive across all eight models, and the inclusion of a random-example baseline is a good control. The main caveats are that the evaluation is built on single-run point estimates, the headline results are selected across different retriever configurations, and the RAG benefit may be inflated by near-duplicate training/test sentences from the same clinical data repository. These issues are addressable with additional analysis, so the paper's core idea remains plausible but is not yet fully supported.

major comments (3)
  1. [Section 2.3 B, Tables 3 and 4] The central claim that RAG significantly boosted overall accuracy (abstract, Section 3.3, Fig. 2) depends on the retrieved training examples being helpful analogies rather than near-duplicate answers. The paper says that during testing the retriever may select from the entire training set and that there is no overlap between input and retrieved examples, but 'no overlap' only excludes exact sentence identity. The datasets are drawn from a single CDR with templated clinical language, and the paper reports no deduplication or sentence-level similarity analysis between training and test sentences. If a test sentence retrieves a training sentence with nearly identical wording and the same gold labels, the model can copy the response, inflating the RAG scores and invalidating the comparison with the no-RAG ablation. I request: (a) the distribution of cosine similarities between each test sentence and its retrieved training sentence; (b) an ablation that removes or thresholds near-duplicate retrieved examples; and (c) a report of the RAG gain restricted to test instances whose retrieved neighbor is not near-duplicate. Without this analysis, the headline RAG improvement is not distinguished from memorization.
  2. [Abstract and Table 3] The headline numbers in the abstract are not produced by a single RAMIE configuration. Llama2-13B's NER F1 of 87.39 comes from the MedCPT row, while the same model's RE F1 of 93.74 comes from the BMRetriever row; Llama2-7B's TE F1 of 79.45 is from the BMRetriever row, and MedAlpaca-7B's UC F1 of 93.45 is from the Contriever row. Thus the abstract's phrasing 'with the aid of the RAMIE framework, Llama2-13B achieved...' is misleading because no single model-plus-retriever combination attains all of these scores. The comparison against single-task fine-tuning and against MTL without RAG should be made for a fixed configuration (for example, the retriever selected on the development set), or the paper should explicitly state that the per-task results are oracle selections across retrievers and models. As reported, the performance advantage of RAMIE is overstated relative to any single deployable instantiation of the framework.
  3. [Section 2.4 and Section 3.3] All results in Tables 2-4 appear to be single-run point estimates, with no confidence intervals, standard deviations, or significance tests. The abstract and Section 3.3 use the word 'significantly' ('RAG significantly boosted overall accuracy'), but the observed differences are often small (for example, 0.94% for UC) and could be within sampling noise. I request that the authors either run multiple seeds and report mean and variance, or soften the language to 'consistently higher in these experiments' with an explicit caveat that the differences were not assessed for statistical significance. This is important because the central claim is a quantitative comparison between conditions.
minor comments (4)
  1. [Abstract and Table 4] The improvement percentages in the abstract appear to be computed as (new - old) / new, whereas the performance-drop percentages in Table 4 are computed as (single - multi) / single. For example, the reported 14.26% TE improvement for Llama2-7B corresponds to (79.45 - 68.12) / 79.45, not the conventional relative improvement (79.45 - 68.12) / 68.12 = 16.6%. Please use a single, clearly stated convention, preferably relative to the baseline.
  2. [Table 3] The header of Table 3 lists 'PMC-Llama-7B', but the methods text and Table 2 refer to PMC-Llama-13B. Please correct this inconsistency.
  3. [General] The manuscript does not mention release of code or data. Since the paper claims to be the first benchmark work for LLM-based DS information extraction, releasing the split datasets and evaluation scripts would substantially aid reproducibility and comparison by future work.
  4. [Tables 2 and 3] There are several typographical errors, including 'Recongition' instead of 'Recognition' in the table captions, 'The motivation for to proposed RAMIE framework' in Section 2.3, and the unfinished question 'The relationship between melatonin and tinnitus is?' in the Table 1 RE example.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RAMIE is an empirical benchmark with no derivation that reduces to its own inputs.

full rationale

This paper is an empirical evaluation, not a mathematical derivation, so the circularity patterns of fitted parameters being renamed as predictions or equations reducing to their own inputs do not apply. The central claim that RAG boosts accuracy (Section 3.3, Figure 2) is supported by direct experimental comparison of retrieval-augmented versus non-retrieval settings on held-out test splits, and the F1 numbers are obtained from actual model outputs rather than from a fitted formula. The self-citations to LEAP (ref 19) and the biomedical RAG benchmark (ref 20) are used to justify prompt design and retrieval methodology, but the reported improvements are measured independently against BERT and single-task instruction fine-tuning, so these citations are not load-bearing circular premises. One legitimate concern, noted in the reader's take, is that test-time retrieval might surface near-duplicate training sentences because the datasets come from a single clinical repository and no deduplication is reported; however, this is a data-leakage or generalization validity concern, not a circularity of the paper's own derivation, and flagging it as circularity would require speculation beyond what the paper's text establishes. Therefore, no specific circular step can be quoted and exhibited, and the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the accuracy of the CDR annotations, on a random split that avoids train and test near-duplicates when retrieving examples at test time, and on the exact-match evaluation metric being meaningful. No new physical or formal entities are introduced.

free parameters (4)
  • LoRA configuration = rank=64, alpha=32, dropout=0.1
    Chosen by hand without reported tuning; these values affect all F1 results.
  • Optimizer and training schedule = AdamW, learning rate 1e-5, 5000 steps, batch size 4
    Uniform hyperparameters set for all models; no search is reported.
  • Checkpoint selection rule = best development F1 among evaluations every 1000 steps
    Selecting the best development checkpoint is a model selection procedure that can interact with retriever choice.
  • Number of retrieved examples per prompt = not reported (description indicates one pair)
    The method text refers to 'the most relevant sentence-response pair', but the prompt template's example count is not explicitly specified.
assumptions (3)
  • domain assumption The CDR dietary supplement annotations (entities, relations, triples, usage) are accurate and complete.
    The paper treats Fan et al. (2021) and Fan and Zhang (2018) annotations as ground truth and does not report annotator agreement.
  • domain assumption The random 8:1:1 split means test sentences are not near-duplicates of training sentences.
    Test-time retrieval from the training set is claimed to add informative examples, not leak answer strings; no deduplication or similarity analysis is provided (Section 2.3 B).
  • domain assumption Exact whole-output match is a faithful metric for NER and TE.
    A prediction is correct only if the entire output matches ground truth (Section 2.4), a strict sentence-level criterion that may favor generative LLMs over token classifiers like BERT.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplements." pith.science (2026). https://pith.science/paper/OJ5FMALH

@misc{pith2026241115700,
  author       = {Pith},
  title        = {Pith review of: RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplements},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OJ5FMALH}},
  note         = {Machine review of arXiv:2411.15700}
}
read the original abstract

\textbf{Objective:} We aimed to develop an advanced multi-task large language model (LLM) framework to extract multiple types of information about dietary supplements (DS) from clinical records. \textbf{Methods:} We used four core DS information extraction tasks - namely, named entity recognition (NER: 2,949 clinical sentences), relation extraction (RE: 4,892 sentences), triple extraction (TE: 2,949 sentences), and usage classification (UC: 2,460 sentences) as our multitasks. We introduced a novel Retrieval-Augmented Multi-task Information Extraction (RAMIE) Framework, including: 1) employed instruction fine-tuning techniques with task-specific prompts, 2) trained LLMs for multiple tasks with improved storage efficiency and lower training costs, and 3) incorporated retrieval augmentation generation (RAG) techniques by retrieving similar examples from the training set. We compared RAMIE's performance to LLMs with instruction fine-tuning alone and conducted an ablation study to assess the contributions of multi-task learning and RAG to improved multitasking performance. \textbf{Results:} With the aid of the RAMIE framework, Llama2-13B achieved an F1 score of 87.39 (3.51\% improvement) on the NER task and demonstrated outstanding performance on the RE task with an F1 score of 93.74 (1.15\% improvement). For the TE task, Llama2-7B scored 79.45 (14.26\% improvement), and MedAlpaca-7B achieved the highest F1 score of 93.45 (0.94\% improvement) on the UC task. The ablation study revealed that while MTL increased efficiency with a slight trade-off in performance, RAG significantly boosted overall accuracy. \textbf{Conclusion:} This study presents a novel RAMIE framework that demonstrates substantial improvements in multi-task information extraction for DS-related data from clinical records. Our framework can potentially be applied to other domains.

Figures

Figures reproduced from arXiv: 2411.15700 by the authors.

Figure 1
Figure 1. The retrieval-augmented multi-task information extraction framework(RAMIE). 3. TE: TE is an end-to-end RE task by structuring information into subject-predicate-object triples directly from the inputs. This task identified the named entities and then determined the relationships between identified entities into triples, which can be used for further analysis, such as creating knowledge graphs. In a sentence like "Cr… view at source ↗
Figure 2
Figure 2. Performance comparison (average F1 score) of RAMIE framework and the MTL without RAG. We averaged the F1 scores of four tasks for each model. performance drop across all models was 2.72%. Specifically, for the NER, TE, and UC tasks, the average drops across the eight LLMs were 2.77%, 5.64%, and 3.31%, respectively. Interestingly, the RE task showed a slight benefit from the multi-task instruction fine-tuning. 4 Disc… view at source ↗
Figure 3
Figure 3. Average F1 scores comparison between single-task learning and multi-task learning. for using examples. Interestingly, using a random example in the prompt actually decreases model performance, while all retrievers—MedCPT, Contriever, and BMRetriever—consistently improved the performance of LLMs. Contriever shows the best overall performance, with six out of eight models achieving their highest scores when combined w… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 45 canonical work pages

  1. [1]

    https://www.crnusa.org/2023survey/infographics (2023)

    Council for responsible nutrition 2023 survey. https://www.crnusa.org/2023survey/infographics (2023)

    work page 2023

  2. [2]

    Fu, P. P.et al. Quality assurance and safety of herbal dietary supplements. J. Environ. Sci. Heal. Part C 27, 91–119 (2009)

    work page 2009

  3. [3]

    & Kaufman, A

    Dodge, T., Litt, D. & Kaufman, A. Influence of the dietary supplement health and education act on consumer beliefs about the safety and effectiveness of dietary supplements. J. health communication 16, 230–244 (2011)

    work page 2011

  4. [4]

    & Naughton, D

    Petroczi, A., Taylor, G. & Naughton, D. Mission impossible? regulatory and enforcement issues to ensure safety of dietary supplements. Food Chem. Toxicol.49, 393–402 (2011)

    work page 2011

  5. [5]

    https://www.fda.gov/news-events/rumor-control/facts-about-dietary-supplements (2023)

    Facts about dietary supplements. https://www.fda.gov/news-events/rumor-control/facts-about-dietary-supplements (2023)

    work page 2023

  6. [6]

    & Kumar, M

    Tucker, J., Fischer, T., Upjohn, L., Mazzera, D. & Kumar, M. Unapproved pharmaceutical ingredients included in dietary supplements associated with us food and drug administration warnings. JAMA Netw. Open 1, e183337–e183337, DOI: 10.1001/jamanetworkopen.2018.3337 (2018). https://jamanetwork.com/journals/jamanetworkopen/articlepdf/2706496/ tucker_2018_oi_1...

    work page arXiv 2018

  7. [7]

    D’Cunha, N. M. et al. Effect of long-term nutraceutical and dietary supplement use on cognition in the elderly: a 10-year systematic review of randomised controlled trials. Br. J. Nutr. 119, 280–298 (2018)

    work page 2018

  8. [8]

    Van Norman, G. A. Limitations of animal studies for predicting toxicity in clinical trials: is it time to rethink our current approach? JACC: Basic to Transl. Sci. 4, 845–854 (2019)

    work page 2019

Show all 63 references
  1. [9]

    Palmer, M. E. et al. Adverse events associated with dietary supplements: an observational study. The Lancet 361, 101–106 (2003)

    2003

  2. [10]

    Timbo, B. B. et al. Dietary supplement adverse event report data from the fda center for food safety and applied nutrition adverse event reporting system (caers), 2004-2013. Annals Pharmacother. 52, 431–438 (2018)

    2018

  3. [11]

    Detecting signals of interactions between warfarin and dietary supplements in electronic health records

    Fan, Y .et al. Detecting signals of interactions between warfarin and dietary supplements in electronic health records. In MEDINFO 2017: Precision Healthcare through Informatics, 370–374 (IOS Press, 2017)

    2017

  4. [12]

    Zhang, R. et al. Evaluating term coverage of herbal and dietary supplements in electronic health records. In AMIA Annual Symposium Proceedings, vol. 2015, 1361 (American Medical Informatics Association, 2015)

    2015

  5. [13]

    & Zhang, R

    Fan, Y . & Zhang, R. Using natural language processing methods to classify use status of dietary supplements in clinical notes. BMC medical informatics decision making 18, 15–22 (2018)

    2018

  6. [14]

    J., Bihorac, A

    Shickel, B., Tighe, P. J., Bihorac, A. & Rashidi, P. Deep ehr: A survey of recent advances in deep learning techniques for electronic health record (ehr) analysis. IEEE J. Biomed. Heal. Informatics 22, 1589–1604, DOI: 10.1109/JBHI.2017. 2767063 (2018)

    2018 doi

  7. [15]

    Cowie, M. R. et al. Electronic health records to facilitate clinical research. Clin. Res. Cardiol. 106, 1–9 (2017)

    2017

  8. [16]

    Jha, A. K. et al. Use of electronic health records in us hospitals. New Engl. J. Medicine 360, 1628–1638 (2009)

    2009

  9. [17]

    & Zhang, R

    Fan, Y ., Zhou, S., Li, Y . & Zhang, R. Deep learning approaches for extracting adverse events and indications of dietary supplements from clinical text. J. Am. Med. Informatics Assoc. 28, 569–577 (2021)

    2021

  10. [18]

    Lee, L.-H. & Lu, Y . Multiple embeddings enhanced multi-graph neural networks for chinese healthcare named entity recognition. IEEE J. Biomed. Heal. Informatics 25, 2801–2810 (2021)

    2021

  11. [19]

    & Zhang, R

    Zhou, H., Li, M., Xiao, Y ., Yang, H. & Zhang, R. Leap: Llm instruction-example adaptive prompting framework for biomedical relation extraction. J. Am. Med. Informatics Assoc. ocae147 (2024)

    2024

  12. [20]

    Li, M. et al. Benchmarking retrieval-augmented large language models in biomedical nlp: Application, robustness, and self-awareness. arXiv preprint arXiv:2405.08151 (2024)

    2024

  13. [21]

    Holper, S. et al. Ambiguous medical abbreviation study: challenges and opportunities. Intern. medicine journal 50, 1073–1078 (2020)

    2020

  14. [22]

    Grossman Liu, L. et al. A deep database of medical abbreviations and acronyms for natural language processing. Sci. Data 8, 149 (2021)

    2021

  15. [23]

    Using word embeddings to expand terminology of dietary supplements on clinical notes

    Fan, Y .et al. Using word embeddings to expand terminology of dietary supplements on clinical notes. JAMIA open 2, 246–253 (2019)

    2019

  16. [24]

    Rizvi, R. F. et al. idisk: the integrated dietary supplements knowledge base. J. Am. Med. Informatics Assoc. 27, 539–548 (2020)

    2020

  17. [25]

    & Uzuner, O

    Dang, H., Lee, K., Henry, S. & Uzuner, O. Ensemble bert for classifying medication-mentioning tweets. In Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task, 37–41 (2020)

    2020

  18. [26]

    Singh, E. et al. A conversational agent system for dietary supplements use. BMC medical informatics decision making 22, 153 (2022)

    2022

  19. [27]

    Schutte, D. et al. Discovering novel drug-supplement interactions using a dietary supplements knowledge graph generated from the biomedical literature. arXiv preprint arXiv:2106.12741 (2021)

    2021 arXiv

  20. [28]

    Zhou, S. et al. Identification of dietary supplement use from electronic health records using transformer-based language models. In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI), 513–514 (IEEE, 2021)

    2021

  21. [29]

    Improving language understanding by generative pre-training

    Radford, A. Improving language understanding by generative pre-training. OpenAI (2018)

    2018

  22. [30]

    Radford, A. et al. Language models are unsupervised multitask learners. OpenAI blog 1, 9 (2019)

    2019

  23. [31]

    Brown, T. B. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

    2020 arXiv

  24. [32]

    Touvron, H. et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023). 9/15

    2023 arXiv

  25. [33]

    Bent, A. A. Large language models: Ai’s legal revolution. Pace Law Rev. 44, 91 (2023)

    2023

  26. [34]

    Zhou, S. et al. Large language models for disease diagnosis: A scoping review. arXiv preprint arXiv:2409.00097 (2024)

    2024 arXiv

  27. [35]

    Goel, A. et al. Llms accelerate annotation for medical information extraction. In Machine Learning for Health (ML4H), 82–100 (PMLR, 2023)

    2023

  28. [36]

    Llms in biomedicine: A study on clinical named entity recognition.arXiv preprint arXiv:2404.07376 (2024)

    Monajatipoor, M.et al. Llms in biomedicine: A study on clinical named entity recognition.arXiv preprint arXiv:2404.07376 (2024)

    2024 arXiv

  29. [37]

    & Choi, D

    Chen, R., Qin, C., Jiang, W. & Choi, D. Is a large language model a good annotator for event extraction? In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 17772–17780 (2024)

    2024

  30. [38]

    Sun, Q. et al. Consistency guided knowledge retrieval and denoising in llms for zero-shot document-level relation triplet extraction. In Proceedings of the ACM on Web Conference 2024, 4407–4416 (2024)

    2024

  31. [39]

    Pushing the limit of llm capacity for text classification

    Zhang, Y .et al. Pushing the limit of llm capacity for text classification. arXiv preprint arXiv:2402.07470 (2024)

    2024 arXiv

  32. [40]

    Zhou, H. et al. Complementary and integrative health information in the literature: its lexicon and named entity recognition. J. Am. Med. Informatics Assoc. 31, 426–434 (2024)

    2024

  33. [41]

    Zhao, T., Yan, Z., Cao, Y . & Li, Z. A unified multi-task learning framework for joint extraction of entities and relations. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 14524–14531 (2021)

    2021

  34. [42]

    Zheng, S. et al. Joint extraction of entities and relations based on a novel tagging scheme. arXiv preprint arXiv:1706.05075 (2017)

    2017 arXiv

  35. [43]

    & Song, Y

    Qin, H., Tian, Y . & Song, Y . Enhancing relation extraction via adversarial multi-task learning. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, 6190–6199 (2022)

    2022

  36. [44]

    & Yang, Q

    Chen, S., Zhang, Y . & Yang, Q. Multi-task learning in natural language processing: An overview.ACM Comput. Surv. 56, 1–32 (2024)

    2024

  37. [45]

    An overview of multi-task learning in deep neural networks

    Ruder, S. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017)

    2017 arXiv

  38. [46]

    & Sha, F

    Kang, Z., Grauman, K. & Sha, F. Learning with whom to share in multi-task feature learning. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), 521–528 (2011)

    2011

  39. [47]

    & Liu, X

    Yin, W., Xu, M., Li, Y . & Liu, X. Llm as a system service on mobile devices.arXiv preprint arXiv:2403.11805 (2024)

    2024 arXiv

  40. [48]

    & McInnes, B

    Mulyar, A., Uzuner, O. & McInnes, B. Mt-clinical bert: scaling clinical information extraction with multitask learning. J. Am. Med. Informatics Assoc. 28, 2108–2115 (2021)

    2021

  41. [49]

    Lewis, P. et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Adv. Neural Inf. Process. Syst. 33, 9459–9474 (2020)

    2020

  42. [50]

    Wu, S. et al. Stark: Benchmarking llm retrieval on textual and relational knowledge bases.arXiv preprint arXiv:2404.13207 (2024)

    2024 arXiv

  43. [51]

    & Zhang, R

    Hou, Y . & Zhang, R. Enhancing dietary supplement question answer via retrieval-augmented generation (rag) with llm. medRxiv 2024–09 (2024)

    2024

  44. [52]

    Jiang, A. Q. et al. Mistral 7b. arXiv preprint arXiv:2310.06825 (2023)

    2023 arXiv

  45. [53]

    https://huggingface.co/meta-llama/Meta-Llama-3-8B (2024)

    Meta-llama 3 8b on hugging face. https://huggingface.co/meta-llama/Meta-Llama-3-8B (2024)

    2024

  46. [54]

    Biomistral: A collection of open-source pretrained large language models for medical domains

    Labrak, Y .et al. Biomistral: A collection of open-source pretrained large language models for medical domains. arXiv preprint arXiv:2402.10373 (2024)

    2024 arXiv

  47. [55]

    Wu, C. et al. Pmc-llama: toward building open-source language models for medicine. J. Am. Med. Informatics Assoc. ocae045 (2024)

    2024

  48. [56]

    Han, T. et al. Medalpaca–an open-source collection of medical conversational ai models and training data. arXiv preprint arXiv:2304.08247 (2023)

    2023 arXiv

  49. [57]

    Jin, Q. et al. Medcpt: Contrastive pre-trained transformers with large-scale pubmed search logs for zero-shot biomedical information retrieval. Bioinformatics 39, btad651 (2023)

    2023

  50. [58]

    Izacard, G. et al. Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118 (2021)

    2021 arXiv

  51. [59]

    Xu, R. et al. Bmretriever: Tuning large language models as better biomedical text retrievers. arXiv preprint arXiv:2404.18443 (2024). 10/15

    2024 arXiv

  52. [60]

    Longpre, S. et al. The flan collection: Designing data and methods for effective instruction tuning. In International Conference on Machine Learning, 22631–22648 (PMLR, 2023)

    2023

  53. [61]

    Zhang, S. et al. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792 (2023)

    2023

  54. [62]

    Hu, E. J. et al. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  55. [63]

    & Farahmand, A

    Hao, Z., AghaKouchak, A., Nakhjiri, N. & Farahmand, A. Global integrated drought monitoring and prediction system (GIDMaPS) data sets. figshare http://dx.doi.org/10.6084/m9.figshare.853801 (2014). 11/15 Task Statistics / Annotations / Examples NER Train / Dev / Test size: 2365...

    2014

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.