Pith. sign in

REVIEW 5 major objections 3 minor 35 references

Clinical trial cohort selection using Large Language Models on n2c2 Challenges

T0 review · 5 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Open-source LLMs fall short on nuanced patient-selection criteria

desk verdict A useful small-model benchmark on n2c2 cohort selection, overbroad in its 'generative LLMs underperform' conclusion; the empirical pattern is real but confined to the tested models and prompt family. read the letter →

arxiv 2501.11114 v1 pith:WFXVTUYQ submitted 2025-01-19 cs.CL cs.AI

Chi-en Amy Tai , Xavier Tannier This is my paper · ORCID
classification cs.CLcs.AI
keywords textextractionclassificationclinicaltrialcohortselectionlargelanguagemodelsn2c2challengesfew-shotpromptingchain-of-thoughtreasoningopen-sourceLLMs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish how far open-source large language models are from being usable for clinical trial cohort selection. The authors benchmark seven models on three standard clinical-text tasks—smoking status, obesity co-morbidities, and thirteen trial eligibility criteria—using the same basic, few-shot, and iterated few-shot prompts. They find that the models match or beat earlier non-LLM systems on criteria that are explicitly stated, such as smoking class or aspirin use, but score well below the original challenge winners on criteria that require fine-grained medical reasoning, such as prior abdominal surgery, drug abuse, or the patient's decision-making capacity. The paper reads this as evidence against the common assumption that generative LLMs generally outperform existing methods or will solve patient selection outright. The central claim, in the authors' own words, is that generative LLMs do not generally outperform existing methods and are not likely to solve the problem of patient selection once and for all.

What carries the argument

The argument is carried by a two-stage benchmark comparison. Stage one scores seven open-source LLMs on the 2018 cohort-selection task under three prompt recipes—a basic prompt, few-shot learning, and an iterated few-shot version whose examples are selected from earlier mistakes—each run with five-fold cross-validation and measured by per-criterion F1. Stage two takes the best-performing model from stage one and applies the same iterated few-shot procedure to the 2006 smoking and 2008 obesity tasks. The decisive comparison is between these generic-prompt LLM scores and the published best results from the original shared-task campaigns, especially the per-criterion breakdown on the 2018 task, which exposes exactly where the models fail.

What would settle it

Run the same three n2c2 tasks through a current larger open-weight model, or a commercial API on de-identified records that comply with the data agreement, using the same five-shot chain-of-thought prompts. If that model meets or beats the original challenge winners on the nuanced 2018 criteria—abdominal surgery, recent myocardial infarction, drug abuse, and decision-making capacity—while keeping the simple-criterion scores high, the paper's claim that generative LLMs do not generally outperform existing methods would be refuted for that model class.

Watch

Extended reading notes

Core claim

Using the three n2c2 challenge datasets as a controlled benchmark, the paper's central empirical claim is that open-source LLMs with generic prompting perform poorly as selection criteria become nuanced. On the 2018 cohort-selection test set, the best of the seven tested models post F1 scores ranging from below 0.1 to about 0.6 on nuanced criteria such as abdominal surgery, myocardial infarction within six months, drug abuse, and decision-making capacity, while the original challenge winners scored above 0.87 on those same criteria; on plainly worded criteria such as aspirin use the LLMs match or exceed the campaign baseline. The same split appears on the 2006 and 2008 tasks: smoking status and several co-morbidity labels are handled reasonably, while under-specified or judgment-heavy categories lose ground. The authors therefore conclude that their results run counter to the view that generative LLMs generally outperform existing methods or are likely to solve patient selection once and for all.

Load-bearing premise

The conclusions assume that seven open-source models prompted with one generic recipe stand in for 'LLMs' broadly; a newer, larger, or differently prompted model could still outperform the traditional systems on the nuanced criteria.

Editorial extensions

If this is right

  • Simple eligibility criteria such as smoking status or aspirin use can be screened automatically with these models, reducing manual chart review for those parts of a protocol.
  • Criteria requiring inference or judgment—abdominal surgery history, drug abuse, decision-making capacity—should be routed to human review or rule-based verification rather than accepted from an LLM.
  • Choosing and iteratively curating few-shot examples is a measurable performance lever; the iterated few-shot prompt consistently beat the basic and plain few-shot prompts.
  • Medical-domain pretraining did not automatically help: the general-text models outperformed the medical-specific model on the 2018 tasks.
  • Claims that LLMs beat existing patient-selection systems need the same per-criterion benchmark comparison used here, because overall scores can hide failures on hard criteria.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's scope is deliberately limited to models that can run on an academic computing cluster, so its negative conclusion is a statement about that model generation and prompt recipe, not a law; the same protocol applied to newer or larger models could shift the balance on nuanced criteria.
  • A practical deployment that follows from the results is a two-tier screen: let an LLM flag patients who clearly pass or fail simple criteria, and send only ambiguous and judgment-heavy cases to human reviewers, cutting manual workload without trusting LLMs on the hard criteria.
  • The per-criterion F1 gaps on the 2018 task are a ready-made diagnostic for future model releases: any model that closes those gaps while keeping simple-criterion performance would be evidence of genuine reasoning gains rather than better pattern matching.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 3 minor

Summary. The paper evaluates seven open-weight LLMs on three n2c2 clinical cohort-selection benchmarks (2006 smoking status, 2008 obesity comorbidities, 2018 trial eligibility criteria) using basic, few-shot, and iterated few-shot prompting, with 5-fold cross-validation on the training data and comparison to official challenge winners. The main finding is that the best tested models (vicuna-13b and mistral-7b-instruct) achieve competitive or strong F1 on simple criteria (e.g., smoking class, aspirin use) but perform poorly on nuanced criteria such as MI-6MOS (0.414), ALCOHOL-ABUSE (0.375), and DRUG-ABUSE (0.222), and the paper concludes in Section 6.1 that generative LLMs do not generally outperform existing methods for patient selection.

Significance. If the claim is scoped to the tested models and prompting regime, the paper provides a useful, reproducible benchmark: it uses public datasets, reports per-criterion F1, compares to official campaign winners, and uses open-weight models that can be run locally. The per-task breakdown in Tables 2-6 is informative, and the negative result on fine-grained criteria is a valuable cautionary data point. However, the broad conclusion about 'generative LLMs' is not supported by the evidence, as only seven small open-weight models and one generic prompting family were tested, which the authors acknowledge in Section 6.2. With a scoped conclusion and a clarified evaluation protocol, the paper would be a solid empirical contribution.

major comments (5)
  1. [Section 6.1] The sentence 'our results run counter to a common view that generative LLMs generally outperform existing methods, or are even likely to solve the problem of patient selection once and for all' overreaches beyond the evidence. The evaluation covers seven open-weight models, the largest being vicuna-13b, with a single generic prompt family and no commercial or larger open models, as Section 6.2 concedes. The empirical pattern is real for these specific models, but the general claim requires invariance across model scale, instruction tuning, and prompting. Please restrict the conclusion to 'the small open-source LLMs and generic prompts evaluated here' or substantially broaden the evaluation.
  2. [Section 4.1 and Tables 2/3] The method text says 'the average and standard deviation of their F1 score across the five folds', while the table captions say 'median and standard deviation over 5 runs' and report a single value with a parenthesized value. With the skewed per-fold results visible for models like open-orca-mistral-7b (0.000 on several criteria), the median and mean can diverge substantially. Please state explicitly which statistic is reported, report both where the distribution is skewed, and describe how the standard deviation is computed for a median.
  3. [Tables 2-6 and Section 6.1] Within-model and between-model comparisons are made using means/medians and standard deviations without any significance test, bootstrap confidence intervals, or correction for multiple comparisons. The claim in Section 6.1 that vicuna-13b has 'more stability (lower standard deviation)' is not formally tested. Given that five runs per fold are reported, please add bootstrap confidence intervals or a paired significance test across folds, and state whether sampling temperature and random seeds were fixed or varied.
  4. [Section 4.2] The section is titled 'Fine-tuning' but describes iterative selection of few-shot examples, not gradient-based fine-tuning. No training hyperparameters, epochs, losses, or updated weights are mentioned. As written, the stage-2 results in Tables 4-6 could be mistaken for fine-tuned outputs. Please either retitle the section (e.g., 'Few-shot example selection for n2c2-2006/2008') or provide the actual fine-tuning setup.
  5. [Section 4.1 and 5-fold protocol] The relationship between the five-fold cross-validation, the train/validation split, and the reported results on the 'val set' (Table 2) and 'test set' (Table 3) is ambiguous. It is unclear whether the five folds are formed from the original training set, how many folds are used to select few-shot examples, and whether the reported scores are averaged over folds for a single validation/test split. Please specify the exact data flow: how the few-shot examples are selected from each fold, how the validation set is used for model selection, and how the official test set is evaluated.
minor comments (3)
  1. [Section 3.1] The word 'interannotor' should be 'inter-annotator'; similarly, 'critera' appears in Section 2 (first paragraph) and should be 'criteria'.
  2. [Table 3] The row 'best from campaign' for ASP-MI (0.770) is lower than several LLM scores (e.g., vicuna-7b 0.878, mistral-7b-instruct 0.875); this is worth a sentence in the discussion, as it qualifies the overall underperformance narrative.
  3. [Section 5] The text says results are reported for 'the n2c2-2006 and n2c2-2008 challenge test dataset for the vicuna-13b model', but Table 4 also reports mistral-7b-instruct; align the prose with the tables.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study is an empirical benchmark evaluation whose claims are grounded in external n2c2 test-set comparisons, not in assumptions equivalent to its conclusions.

full rationale

The paper's central claim — that open-source LLMs with generic prompts perform poorly on nuanced eligibility criteria — is derived from measured F1 scores on held-out n2c2 test sets (Tables 3, 5, and 6) compared with official campaign results. No parameter is fitted to the target outcome and then renamed as a prediction; the 'best LLM' selection uses a validation split of n2c2-2018, and the reported test-set numbers are out-of-sample. The authors' own prior work appears only as literature context: [21] Zaghir et al. and [31] Naguib et al. are cited for prompt-engineering patterns and for the separate claim that masked language models outperform LLM prompting in few-shot clinical entity recognition. Neither citation is load-bearing for the cohort-selection conclusion, because the paper's own tables carry the argument. The limitations section explicitly concedes the scope (no commercial LLMs, no prompt optimization, no chain-of-thought), which makes the conclusion narrow but does not make it circular. There is no derivation chain in which an output equals an input by construction, and no uniqueness theorem or ansatz is imported from the authors' prior work. The evaluation is self-contained against external benchmarks, so the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no fitted parameters or new entities. It relies on the external benchmark annotations and the reported scores of past challenge winners as ground truth, and on the assumption that the validation-based model and example selection does not leak information into the test set.

assumptions (3)
  • domain assumption The n2c2 gold annotations are correct and consistent.
    The paper uses these labels as ground truth for F1 computation; if the labels contain errors, reported scores are biased.
  • domain assumption Reported 'best from campaign' scores are reliable and comparable.
    The comparison in Tables 3-6 assumes the challenge winners' scores were computed on the same test set and with the same metrics, though the original campaign papers may have allowed ensembling or additional validation data.
  • domain assumption Validation-based few-shot example selection does not leak test information.
    The iterated few-shot selection uses validation F1; the test set should be untouched for a fair evaluation, and the paper provides no explicit evidence that test data was not inspected during example selection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Clinical trial cohort selection using Large Language Models on n2c2 Challenges." pith.science (2026). https://pith.science/paper/WFXVTUYQ

@misc{pith2026250111114,
  author       = {Pith},
  title        = {Pith review of: Clinical trial cohort selection using Large Language Models on n2c2 Challenges},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WFXVTUYQ}},
  note         = {Machine review of arXiv:2501.11114}
}
read the original abstract

Clinical trials are a critical process in the medical field for introducing new treatments and innovations. However, cohort selection for clinical trials is a time-consuming process that often requires manual review of patient text records for specific keywords. Though there have been studies on standardizing the information across the various platforms, Natural Language Processing (NLP) tools remain crucial for spotting eligibility criteria in textual reports. Recently, pre-trained large language models (LLMs) have gained popularity for various NLP tasks due to their ability to acquire a nuanced understanding of text. In this paper, we study the performance of large language models on clinical trial cohort selection and leverage the n2c2 challenges to benchmark their performance. Our results are promising with regard to the incorporation of LLMs for simple cohort selection tasks, but also highlight the difficulties encountered by these models as soon as fine-grained knowledge and reasoning are required.

Figures

Figures reproduced from arXiv: 2501.11114 by the authors.

Figure 1
Figure 1. The n2c2-2008 dataset distribution for the fifteen textual criteria. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. The n2c2-2008 dataset distribution for the fifteen intuitive criteria. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The n2c2-2018 dataset distribution for the thirteen criteria. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 31 canonical work pages

  1. [1]

    Optimizing clinical research participant selection with infor- matics,

    C. Weng, “Optimizing clinical research participant selection with infor- matics,” Trends in pharmacological sciences, vol. 36, no. 11, pp. 706–709, 2015

    work page 2015

  2. [2]

    Piloting the ehr4cr feasibility platform across europe,

    J. Doods, R. Bache, M. McGilchrist, C. Daniel, M. Dugas, and F. Fritz, “Piloting the ehr4cr feasibility platform across europe,” Methods of in- formation in medicine , vol. 53, no. 04, pp. 264–268, 2014

    work page 2014

  3. [3]

    Efficiency and effectiveness eval- uation of an automated multi-country patient count cohort system,

    I. Soto-Rey, B. Trinczek, Y. Girardeau, E. Zapletal, N. Ammour, J. Doods, M. Dugas, and F. Fritz, “Efficiency and effectiveness eval- uation of an automated multi-country patient count cohort system,” BMC medical research methodology, vol. 15, pp. 1–10, 2015. 16

    work page 2015

  4. [4]

    Leveraging the ehr4cr platform to support patient inclusion in academic studies: challenges and lessons learned,

    Y. Girardeau, J. Doods, E. Zapletal, G. Chatellier, C. Daniel, A. Bur- gun, M. Dugas, and B. Rance, “Leveraging the ehr4cr platform to support patient inclusion in academic studies: challenges and lessons learned,” BMC medical research methodology, vol. 17, pp. 1–10, 2017

    work page 2017

  5. [5]

    Formal representation of eligibility criteria: a literature review,

    C. Weng, S. W. Tu, I. Sim, and R. Richesson, “Formal representation of eligibility criteria: a literature review,” Journal of biomedical infor- matics, vol. 43, no. 3, pp. 451–467, 2010

    work page 2010

  6. [6]

    Dynamic categorization of clinical research eligibility criteria by hierarchical clustering,

    Z. Luo, M. Yetisgen-Yildiz, and C. Weng, “Dynamic categorization of clinical research eligibility criteria by hierarchical clustering,” Journal of biomedical informatics , vol. 44, no. 6, pp. 927–935, 2011

    work page 2011

  7. [7]

    Developing a data element repository to support ehr-driven phenotype algorithm authoring and execution,

    G. Jiang, R. C. Kiefer, L. V. Rasmussen, H. R. Solbrig, H. Mo, J. A. Pacheco, J. Xu, E. Montague, W. K. Thompson, J. C. Denny, et al. , “Developing a data element repository to support ehr-driven phenotype algorithm authoring and execution,” Journal of biomedical informatics , vol. 62, pp. 232–242, 2016

    work page 2016

  8. [8]

    Cross border semantic interoperability for clinical research: the ehr4cr semantic re- sources and services,

    C. Daniel, D. Ouagne, E. Sadou, K. Forsberg, M. Mc Gilchrist, E. Zaple- tal, N. Paris, S. Hussain, M.-C. Jaulent, and D. Kalra, “Cross border semantic interoperability for clinical research: the ehr4cr semantic re- sources and services,” AMIA Summits on Translational Science Proceed- ings, vol. 2016, p. 51, 2016

    work page 2016

Show all 35 references
  1. [9]

    Phekb: a catalog and workflow for creating electronic phenotype algo- rithms for transportability,

    J. C. Kirby, P. Speltz, L. V. Rasmussen, M. Basford, O. Gottesman, P. L. Peissig, J. A. Pacheco, G. Tromp, J. Pathak, D. S. Carrell, et al. , “Phekb: a catalog and workflow for creating electronic phenotype algo- rithms for transportability,” Journal of the American Medical In...

    2016

  2. [10]

    n2c2 nlp research data sets

    H. M. School, “n2c2 nlp research data sets.” https://portal.dbmi. hms.harvard.edu/projects/n2c2-nlp/. Accessed: 2024-07-20

    2024

  3. [11]

    Identifying patient smoking status from medical discharge records,

    ¨O. Uzuner, I. Goldstein, Y. Luo, and I. Kohane, “Identifying patient smoking status from medical discharge records,” Journal of the Ameri- can Medical Informatics Association , vol. 15, no. 1, pp. 14–24, 2008

    2008

  4. [12]

    Recognizing obesity and comorbidities in sparse data,

    ¨O. Uzuner, “Recognizing obesity and comorbidities in sparse data,” Journal of the American Medical Informatics Association, vol. 16, no. 4, pp. 561–570, 2009. 17

    2009

  5. [13]

    Cohort selection for clinical trials: n2c2 2018 shared task track 1,

    A. Stubbs, M. Filannino, E. Soysal, S. Henry, and ¨O. Uzuner, “Cohort selection for clinical trials: n2c2 2018 shared task track 1,” Journal of the American Medical Informatics Association, vol. 26, no. 11, pp. 1163– 1171, 2019

    2018

  6. [14]

    Gpts are gpts: An early look at the labor market impact potential of large language models,

    T. Eloundou, S. Manning, P. Mishkin, and D. Rock, “Gpts are gpts: An early look at the labor market impact potential of large language models,” arXiv preprint arXiv:2303.10130 , 2023

    2023 arXiv

  7. [15]

    A survey of gpt-3 family large language models including chatgpt and gpt-4,

    K. S. Kalyan, “A survey of gpt-3 family large language models including chatgpt and gpt-4,” Natural Language Processing Journal , p. 100048, 2023

    2023

  8. [16]

    Embracing large language models for medical applications: opportunities and challenges,

    M. Karabacak and K. Margetis, “Embracing large language models for medical applications: opportunities and challenges,” Cureus, vol. 15, no. 5, 2023

    2023

  9. [17]

    Transforming clinical trials: the emerging roles of large language models,

    J.-L. Ghim and S. Ahn, “Transforming clinical trials: the emerging roles of large language models,” Translational and Clinical Pharmacology , vol. 31, pp. 131 – 138, 2023

    2023

  10. [18]

    Scaling clinical trial matching using large language models: A case study in oncology,

    C. Wong, S. Zhang, Y. Gu, C. Moung, J. Abel, N. Usuyama, R. K. Weerasinghe, B. D. Piening, T. Naumann, C. B. Bifulco, and H. Poon, “Scaling clinical trial matching using large language models: A case study in oncology,” in Machine Learning in Health Care , 2023

    2023

  11. [19]

    Large language models for healthcare data augmentation: An example on patient-trial matching.,

    J. Yuan, R. Tang, X. Jiang, and X. Hu, “Large language models for healthcare data augmentation: An example on patient-trial matching.,” AMIA Annual Symposium proceedings., vol. 2023, pp. 1324–1333, 2023

    2023

  12. [20]

    Distilling large language models for matching patients to clinical trials,

    M. Nievas, A. Basu, Y. Wang, and H. Singh, “Distilling large language models for matching patients to clinical trials,” Journal of the American Medical Informatics Association : JAMIA , 2023

    2023

  13. [21]

    Prompt engineering paradigms for medical applications: scop- ing review and recommendations for better practices,

    J. Zaghir, M. Naguib, M. Bjelogrlic, A. N´ ev´ eol, X. Tannier, and C. Lo- vis, “Prompt engineering paradigms for medical applications: scop- ing review and recommendations for better practices,” arXiv preprint arXiv:2405.01249, 2024

    2024 arXiv

  14. [22]

    Building community knowledge in online competitions: motivation, practices and challenges,

    R. Cheng and M. Zachry, “Building community knowledge in online competitions: motivation, practices and challenges,” Proceedings of the 18 ACM on Human-Computer Interaction , vol. 4, no. CSCW2, pp. 1–22, 2020

    2020

  15. [23]

    Clinical concept extrac- tion using transformers,

    X. Yang, J. Bian, W. R. Hogan, and Y. Wu, “Clinical concept extrac- tion using transformers,” Journal of the American Medical Informatics Association, vol. 27, no. 12, pp. 1935–1942, 2020

    1935

  16. [24]

    Zero-shot clinical trial patient matching with llms,

    M. Wornow, A. Lozano, D. Dash, J. Jindal, K. W. Mahaffey, and N. H. Shah, “Zero-shot clinical trial patient matching with llms,” 2024

    2024

  17. [25]

    Utilizing large language models for enhanced clinical trial matching: A study on automation in patient screening,

    J. Beattie, S. Neufeld, D. Yang, C. Chukwuma, A. Gul, N. Desai, S. Jiang, and M. Dohopolskiet, “Utilizing large language models for enhanced clinical trial matching: A study on automation in patient screening,” Cureus, vol. 16, no. 5, 2024

    2024

  18. [26]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  19. [27]

    Medalpaca–an open- source collection of medical conversational ai models and training data,

    T. Han, L. C. Adams, J.-M. Papaioannou, P. Grundmann, T. Ober- hauser, A. L¨ oser, D. Truhn, and K. K. Bressem, “Medalpaca–an open- source collection of medical conversational ai models and training data,” arXiv preprint arXiv:2304.08247 , 2023

    2023 arXiv

  20. [28]

    Why johnny can’t prompt: how non-ai experts try (and fail) to design llm prompts,

    J. Zamfirescu-Pereira, R. Y. Wong, B. Hartmann, and Q. Yang, “Why johnny can’t prompt: how non-ai experts try (and fail) to design llm prompts,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , pp. 1–21, 2023

    2023

  21. [29]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24824–24837, 2022

    2022

  22. [30]

    Llms are not zero-shot reasoners for biomedical information extraction,

    A. Nagar, V. Schlegel, T.-T. Nguyen, H. Li, Y. Wu, K. Binici, and S. Winkler, “Llms are not zero-shot reasoners for biomedical information extraction,” 2024. 19

    2024

  23. [31]

    Few shot clinical entity recog- nition in three languages: Masked language models outperform llm prompting,

    M. Naguib, X. Tannier, and A. N´ ev´ eol, “Few shot clinical entity recog- nition in three languages: Masked language models outperform llm prompting,” 2024

    2024

  24. [32]

    Optimizing instructions and demonstrations for multi-stage language model programs,

    K. Opsahl-Ong, M. J. Ryan, J. Purtell, D. Broman, C. Potts, M. Za- haria, and O. Khattab, “Optimizing instructions and demonstrations for multi-stage language model programs,” 2024

    2024

  25. [33]

    Fine-tuning and prompt optimiza- tion: Two great steps that work better together,

    D. Soylu, C. Potts, and O. Khattab, “Fine-tuning and prompt optimiza- tion: Two great steps that work better together,” 2024

    2024

  26. [34]

    Autocriteria: a generalizable clinical trial eligibility criteria extraction system powered by large lan- guage models,

    S. Datta, K. Lee, H. Paek, F. J. Manion, N. Ofoegbu, J. Du, Y. Li, L.-C. Huang, J. Wang, B. Lin, et al. , “Autocriteria: a generalizable clinical trial eligibility criteria extraction system powered by large lan- guage models,” Journal of the American Medical Informatics Assoc...

    2024

  27. [35]

    Clinicalmamba: A generative clinical language model on longitudinal clinical notes,

    Z. Yang, A. Mitra, S. Kwon, and H. Yu, “Clinicalmamba: A generative clinical language model on longitudinal clinical notes,” arXiv preprint arXiv:2403.05795, 2024. Supplementary Material A: Prompt examples {} is used to denote the location of the selected patient text for infe...

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.