Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

The Aloe Family Recipe for Open and Specialized Healthcare LLMs

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The Aloe Beta recipe produces open healthcare LLMs competitive with private models on medical MCQA benchmarks.

desk verdict A solid open-model engineering report with real artifacts, but the headline MCQA parity claim leans on an unvalidated decontamination step that needs to be checked before the numbers are taken at face value. read the letter →

arxiv 2505.04388 v2 pith:EF72M3OM submitted 2025-05-07 cs.CL cs.AI

Dario Garcia-Gasulla , Jordi Bayarri-Planas , Ashwin Kumar Gururajan , Enrique Lopez-Cuena , Adrian Tormos , Daniel Hinjos , Pablo Bernabeu-Perez , Anna Arias-Duart
show 5 more authors
Pablo Agustin Martin-Torres Marta Gonzalez-Mallo Sergio Alvarez-Napagao Eduard Ayguadé-Parra Ulises Cortés
This is my paper · ORCID
classification cs.CLcs.AI
keywords healthcareLLMsopen-sourcemodelschain-of-thoughtdirectpreferenceoptimizationmodelmergingretrieval-augmentedgenerationjailbreaksafetymedicalbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a fully open, permissively licensed recipe for healthcare LLMs can reach competitive performance with closed systems. The authors build four Aloe Beta models on Llama 3.1 and Qwen 2.5 base models, expanding public medical multiple-choice data with synthetic chain-of-thought reasoning, merging with the official instruct versions, and aligning with preference optimization plus a jailbreak-focused red-teaming stage. They report that the 72B model is the highest-performing open model on the medical MCQA benchmarks they evaluate, that its accuracy matches or exceeds reported numbers for private systems, and that the large models resist jailbreaking far better than their instruct baselines. If correct, the practical consequence is that institutions can deploy, inspect, and adapt healthcare LLMs of near-top accuracy without relying on closed APIs. The paper frames its four-way evaluation (close-ended, open-ended, expert preference, and safety) as part of the contribution, and it notes that the closed-model numbers were not reproduced by the authors, so the parity claim rests on the MCQA setting.

What carries the argument

The mechanism is the full recipe, not any single model. The supervised fine-tuning corpus contains 2M samples, including about 420K synthetic chain-of-thought solutions generated by Llama-3.1-70B-Instruct from the training splits of MedQA, MedMCQA, PubMedQA, HeadQA, MMLU auxiliary, and PolyMed, converting simple letter-answer data into step-by-step reasoning. The training then uses DARE-TIES merging, which drops low-magnitude parameters and rescales the rest, to combine the Aloe SFT weights with the official instruct weights, and a two-stage DPO where the final stage is trained only on a curated jailbreak dataset built from red-teaming templates to harden safety without eroding accuracy. At inference, a Medprompt-style retrieval-augmented pipeline retrieves five chain-of-thought few-shot examples from a 192K-example database generated from the same benchmark training sets, then pools twenty self-consistency samples with choice shuffling.

What would settle it

Compare the released Aloe Beta training corpus against the official test splits of MedQA, MedMCQA, PubMedQA, MMLU, and CareQA using exact-match, paraphrase, and embedding-similarity methods, and check whether any retained training example matches a test question. A nonzero surviving match rate would inflate the reported parity numbers; zero matches under all methods would support the claim.

Watch

Extended reading notes

Core claim

The central claim is that an open training and evaluation recipe can make an open 72B model the best open performer on five medical multiple-choice benchmarks and competitive with the best private alternatives reported in the literature. In the paper's own comparison, Qwen2.5-Aloe-Beta-72B averages 82.54 accuracy across MultiMedQA, MedMCQA, MedQA, MMLU, and CareQA, and on the shared benchmark subsets it matches or outperforms the numbers reported for GPT-4 and MedPalm-2; adding a Medprompt-style retrieval pipeline raises it to 85.68 and closes most of the remaining gap. A second reported result is safety: the two-stage DPO alignment reduces the average attack success rate on the S-Eval jailbreak benchmark to below 9% for both large Aloe models, down from 14-15% in the base instruct models. The human evaluation, based on 49 physicians judging 695 pairs of answers to Reddit health questions, showed no statistically significant preference between the Aloe models and their instruct bases, which the authors read as evidence that current generalist LLMs can give reliable advice on simple primary-care questions.

Load-bearing premise

The headline comparison rests on the assumption that the decontamination step, an LLM judge that flags and removes instructions, fully eliminated every overlapping or near-duplicate question between the benchmark training sets used for synthetic chain-of-thought generation and the test sets used in the final MCQA evaluation.

Editorial extensions

If this is right

  • Open healthcare LLMs can reach benchmark parity with closed systems while releasing weights, data, and training details, shifting deployment options for hospitals and researchers.
  • The two-stage DPO with red-teaming data can cut jailbreak success rates dramatically for large models (from an average above 14% to below 9%) without lowering measured MCQA accuracy in the reported results.
  • Adding retrieval-augmented prompting boosts small Aloe models by roughly 6 to 9 accuracy points and large models by roughly 3 to 4 points, so one model family can serve both lightweight and high-accuracy deployments.
  • The reported field-by-field accuracy table, with the 72B model achieving top performance in 13 of 17 medical specialties, can serve as a model-selection guide for specialty-specific applications.
  • The recipe is stated as transferable to future strong open base models, so the same data, merging, and alignment stages could be reapplied when newer open models appear.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The open-ended evaluation results are noisy and disagree across metrics, so a natural extension beyond the paper is to anchor open-ended assessment in clinician-rated outcome measures rather than n-gram or perplexity scores.
  • Because the synthetic chain-of-thought data are generated from the same benchmarks used for evaluation, the cleanest test of the parity claim is to rerun training with the official test sets excluded under stricter near-duplicate removal than the reported LLM-judge decontamination.
  • The same merge-plus-DPO sequence could transfer to other safety-critical domains such as finance or legal advice, but that transfer is not demonstrated in this paper.
  • The retrieval database is built from benchmark training sets, so if retrieval ever returns a near-duplicate of a test question, the in-context-learning scores could be optimistic; the paper does not quantify this risk.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. Aloe Beta is a family of four healthcare-specialized LLMs (Llama-3.1-8B/70B and Qwen2.5-7B/72B) built with a publicly documented recipe: curation of roughly 2M SFT samples (about 60% medical instructions from public datasets, 420K synthetic chain-of-thought samples generated from the training splits of MedQA, MedMCQA, PubMedQA, HeadQA, MMLU-medical, and PolyMed, plus 400K general-domain samples), four-epoch SFT, DARE-TIES merging with the official instruct checkpoints, two-stage DPO alignment with a final red-teaming-only stage, and a Medprompt-style retrieval-augmented inference pipeline. Evaluation covers MCQA (Table 9), open-ended tasks, human expert pairwise preferences, and attack-success-rate safety testing (S-Eval). The paper claims that Qwen2.5-Aloe-Beta-72B is the highest-performing open model examined (82.54 MCQA average), that the largest Aloe models are competitive with GPT-4 and MedPalm-2 on medical MCQA, that DPO alignment substantially improves jailbreak resistance in large models, and that all models, datasets, code, and a healthcare-specific risk assessment are released openly.

Significance. If the central claims hold, this is a strong systems contribution to open healthcare LLMs. The complete release of weights, training data, and RAG components, combined with unusually detailed reporting of data provenance, hyperparameters, compute, and carbon cost, makes the recipe directly reusable. The Qwen-based Aloe models show consistent MCQA gains over their instruct baselines (Table 9: +3.4 for 7B, +2.2 for 72B); the large-model safety results are large and internally consistent (average ASR below 9% in Tables 13-14); and the body text is candid about non-reproduced closed-model numbers and statistically non-significant human preferences. The significance is contingent, however: the MCQA parity claim rests on a decontamination step that is neither specified nor validated (Major Comment 1), the abstract overstates the human-evaluation and closed-model-comparison findings (Major Comments 2-3), and the data-count tables contain internal inconsistencies. With the contamination audit performed and the claims tempered accordingly, this would be a valuable and citable recipe paper.

major comments (3)
  1. [§4.1.1, §4.1.2, Table 9] The central claim that Aloe-Beta-72B is 'competitive with the best private alternatives' (Abstract, §7.1, §8) rests on the assumption that the Table 9 test sets are absent from training. However, §4.1.2 constructs 419,938 synthetic CoT samples by prompting Llama-3.1-70B-Instruct with the training splits of exactly the MCQA task families used in Table 9: MedQA (10,178), MedMCQA (182,736), PubMedQA (210,257), HeadQA (6,600), and MMLU-medical (4,321). The only barrier is the decontamination step in §4.1.1, described in a single sentence: Nous-Hermes-2-Yi-34B is used as a judge and 'all instructions that it flags' are removed. No judge prompt, no threshold, no count of removed samples, and no validation of judge recall are reported; moreover, the judge is a 34B open model while the generator is a 70B instruct model, so its recall on paraphrased items is not assured. Because the conversion to CoT paraphrases each question and appends a long reasoning chain, near-duplicates of test items can plausibly evade the judge, and the same concern applies to the RAG database of §6, which is built from the MedQA and MedMCQA training sets. Since the datasets are released, a direct overlap check is feasible. I request: (a) the full decontamination configuration and per-source removal counts; (b) a recall test in which known test items are spiked into the pipeline; and (c) confirmation of the headline gains and parity on a benchmark not derived from the selected training sources (e.g., the CareQA 2024 items already used in Table 9, or another recent exam set). Until then, the parity claim is not established.
  2. [§7.3, Abstract] The abstract claims that the Aloe models 'are often preferred by healthcare professionals,' but §7.3 reports binomial tests in which no pairwise battle reached statistical significance (p<0.05), and the text concludes from 695 expert responses that the choice of one model over the other 'is relegated to the personal preference of the doctors, with limited statistical relevance.' The abstract should be revised to state that experts found the Aloe models comparable to, rather than preferred over, the instruct baselines.
  3. [Table 9, §7.1] The prose in §7.1 ('Aloe-Beta-72B matches or outperforms GPT-4 and MedPalm-2'; 'the performance gap between open and closed models seems minimal') compares 0-shot Aloe accuracy with closed-model numbers that the table footnote correctly says are 'not reproduced' and that were obtained under the protocols of the Medprompt and Med-PaLM papers. The footnote discloses the non-reproduction, but the surrounding text does not carry the caveat into the abstract, where 'competitive with the best private alternatives' is asserted without protocol qualification. Please either evaluate the closed models under the Aloe protocol, or state explicitly in the abstract and §8 that the comparison is against published numbers obtained with different prompting and ensembling setups.
minor comments (7)
  1. [Table 5] The column for Aloe-Beta-72B lists the base model as 'Llama-3.1-72B', which contradicts the text and all other tables identifying it as Qwen2.5-72B; no Llama-3.1-72B model exists.
  2. [Table 8] The 70B entry for 'Total training time' is 0.98, but the sum of the first-stage (12.76, Table 6) and second-stage (0.98, Table 7) training times is 13.74, which is the value actually used in Appendix D Eq. (D12).
  3. [Appendix D, Eq. (D6)] The computation 22.68 kW × 15.30 h × 0.158 kg/kWh equals 54.8 kg CO2, not the stated 61.42 kg; the stated value corresponds to 17.14 h, i.e., the 8B training time. The 7B carbon figures should be recomputed consistently with Table 5.
  4. [§4.1, Tables 3 and A1] Dataset counts and percentages do not reconcile: Table 3 totals 1,622,404 samples while Table A1 totals 1,603,732; the synthetic CoT MCQA share is given as 31.54% in the text of §4.1 but 31.17% in Table 3; and the '1.2M medical + 225K synthetic + 420K synthetic' breakdown in §4.1 does not obviously match either table. Please reconcile the data card.
  5. [Table 9] The 'Avg.' row averages across columns that include both MultiMedQA and its constituent benchmarks (MedQA, MedMCQA, MMLU, and PubMedQA), so those benchmarks are counted twice in the reported average; please report the composition of the MultiMedQA column and either recompute the average over non-overlapping benchmarks or disclose the double counting.
  6. [§6] The Medprompt retrieval database (192,084 examples) is constructed from the training sets of MedQA and MedMCQA; please state explicitly whether this database was decontaminated against the corresponding test sets and how it relates to the synthetic SFT CoT data described in §4.1.2.
  7. [Declarations] The statement that Aloe Beta is 'significantly better in every aspect' than Aloe Alpha is not supported by any comparison in this manuscript; a brief quantitative comparison against Aloe Alpha would substantiate the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the MCQA training/evaluation overlap is a contamination concern, not a constructional reduction.

full rationale

Walking the claimed derivation chain: the paper's contribution is an empirical training recipe (SFT on curated and synthetic data, DARE-TIES merging, DPO, and optional RAG) evaluated on standard benchmarks; it does not claim to derive benchmark scores from training inputs. The synthetic CoT data in §4.1.2 is generated from the training splits of MedQA, MedMCQA, PubMedQA, HeadQA, and the MMLU auxiliary set, and Table 9 reports accuracy on the same benchmark families. This overlap is real, but it is not circular by construction: the reported numbers are measured on held-out test splits, and §4.1.1 describes an LLM-judge decontamination pass specifically intended to remove validation/test leakage. Whether Nous-Hermes-2-Yi-34B has perfect recall is an empirical contamination risk, and the paper gives no judge prompt, threshold, or removal count, but that is a correctness/validity concern, not an equivalence between prediction and input. The comparison with closed models uses external numbers from [1] and [2], and the paper openly states that with RAG the closed models 'slightly outperform' Aloe; there is no fitted parameter renamed as a prediction. Self-citations ([10], [17], [46], [63]) are used for context, evaluation-suite provenance, safety-judge validation, and ICL configuration; none is invoked to forbid alternatives or to force the central MCQA result. The risk assessment is declared to be reproduced from Aloe Alpha, which is transparency rather than circularity. Therefore no circular step meets the bar of Eq. X = Eq. Y by construction, and the paper is best assessed as self-contained against external benchmarks, with the decontamination efficacy flagged as a separate empirical risk.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on assumptions about data reliability, contamination control, and comparability of external benchmark scores. These are not proven by the paper, but many are standard in the field. The listed hyperparameters are recipe choices rather than fitted physical constants, and no new theoretical entities are introduced.

free parameters (5)
  • SFT learning rates = 1e-5 (Qwen), 2e-5 (Llama)
    Selected after additional runs for model selection (§5.1); the central recipe depends on these values.
  • DPO beta and learning rates = beta=0.1; LR 2e-7 then 1e-7
    Hyperparameters chosen in §5.3; beta controls preference strength and the two-stage schedule shapes safety behavior.
  • LSH deduplication thresholds = 0.72 (QA), 0.77 (multi-turn)
    Tuned thresholds in §4.1.1; they alter dataset composition and hence model behavior.
  • DEITA bottom-10% filter = 10%
    The bottom 10% of samples by evol score are discarded; this threshold is chosen by the authors with a justificatory discussion in Appendix B.
  • Medprompt retrieval configuration = 5 few-shots, 20 self-consistency iterations, SFR-Embedding-Mistral
    Configuration follows prior work [63]; it directly affects the ICL results in Table 9.
assumptions (5)
  • domain assumption Synthetic CoT data generated by Llama-3.1-70B-Instruct with the correct answer supplied is factually reliable enough to improve medical reasoning.
    Used throughout §4.1.2 to create 420K training samples; no independent verification of correctness for generated CoT is provided beyond reliance on the generator and the supplied answer.
  • domain assumption The LLM-based decontamination judge (Nous-Hermes-2-Yi-34B) removes all train/test leakage between synthetic CoT sources and evaluation benchmarks.
    Invoked in §4.1.1; no contamination audit or leakage measurement is reported.
  • domain assumption Llama Guard 3 8B's attack success rate labels are a valid safety measure.
    §7.4 safety numbers depend on this proxy; human verification of the safety classifications is not provided.
  • domain assumption Closed model scores in Table 9 from Medprompt and MedPaLM-2 papers are comparable to the authors' own evaluation runs.
    Table 9 notes state these closed numbers are not reproduced; the open-vs-closed parity claim relies on cross-paper comparability.
  • domain assumption Reddit HealthAdvice questions are representative of real primary-care LLM use and do not appear in training.
    §7.3 human evaluation design relies on this assumption; no contamination check on Reddit questions is reported beyond their recency.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Aloe Family Recipe for Open and Specialized Healthcare LLMs." pith.science (2026). https://pith.science/paper/EF72M3OM

@misc{pith2026250504388,
  author       = {Pith},
  title        = {Pith review of: The Aloe Family Recipe for Open and Specialized Healthcare LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EF72M3OM}},
  note         = {Machine review of arXiv:2505.04388}
}
read the original abstract

Purpose: With advancements in Large Language Models (LLMs) for healthcare, the need arises for competitive open-source models to protect the public interest. This work contributes to the field of open medical LLMs by optimizing key stages of data preprocessing and training, while showing how to improve model safety (through DPO) and efficacy (through RAG). The evaluation methodology used, which includes four different types of tests, defines a new standard for the field. The resultant models, shown to be competitive with the best private alternatives, are released with a permisive license. Methods: Building on top of strong base models like Llama 3.1 and Qwen 2.5, Aloe Beta uses a custom dataset to enhance public data with synthetic Chain of Thought examples. The models undergo alignment with Direct Preference Optimization, emphasizing ethical and policy-aligned performance in the presence of jailbreaking attacks. Evaluation includes close-ended, open-ended, safety and human assessments, to maximize the reliability of results. Results: Recommendations are made across the entire pipeline, backed by the solid performance of the Aloe Family. These models deliver competitive performance across healthcare benchmarks and medical fields, and are often preferred by healthcare professionals. On bias and toxicity, the Aloe Beta models significantly improve safety, showing resilience to unseen jailbreaking attacks. For a responsible release, a detailed risk assessment specific to healthcare is attached to the Aloe Family models. Conclusion: The Aloe Beta models, and the recipe that leads to them, are a significant contribution to the open-source medical LLM field, offering top-of-the-line performance while maintaining high ethical requirements. This work sets a new standard for developing and reporting aligned LLMs in healthcare.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Benchmarking the Safety of Large Language Models for Robotic Health Attendant Control

    cs.AI 2026-04 unverdicted novelty 6.0 of 10

    Most of 72 tested LLMs complied with harmful medical-robot orders over half the time, with open-weight models far worse than proprietary ones.

Reference graph

Works this paper leans on

102 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Nori, H., Lee, Y.T., Zhang, S., Carignan, D., Edgar, R., Fusi, N., King, N., Larson, J., Li, Y., Liu, W., Luo, R., McKinney, S.M., Ness, R.O., Poon, H., Qin, T., Usuyama, N., White, C., Horvitz, E.: Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine (2023)

    2023

  2. [2]

    https://arxiv.org/abs/2305.09617

    Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., Schaekermann, M., Wang, A., Amin, M., Lachgar, S., Mansfield, P., Prakash, S., Green, B., Dominowska, E., Arcas, B.A., Tomasev, N., Liu, Y., Wong, R., Semturs, C., Mahdavi, S.S., Barral, J., Web- ster, D., Corrado, G.S., Matias, Y., Azizi, ...

    arXiv 2023

  3. [3]

    https://arxiv.org/abs/2404.18416

    Saab, K., Tu, T., Weng, W.-H., Tanno, R., Stutz, D., Wulczyn, E., Zhang, F., Strother, T., Park, C., Vedadi, E., Chaves, J.Z., Hu, S.-Y., Schaekermann, M., Kamath, A., Cheng, Y., Barrett, D.G.T., Cheung, C., Mustafa, B., Palepu, A., McDuff, D., Hou, L., Golany, T., Liu, L., Alayrac, J.-b., Houlsby, N., Tomasev, N., Freyberg, J., Lau, C., Kemp, J., Lai, J....

    arXiv 2024

  4. [4]

    arXiv preprint arXiv:2304.08247 (2023)

    Han, T., Adams, L.C., Papaioannou, J.-M., Grundmann, P., Oberhauser, T., L¨ oser, A., Truhn, D., Bressem, K.K.: Medalpaca–an open-source collec- tion of medical conversational ai models and training data. arXiv preprint arXiv:2304.08247 (2023)

    arXiv 2023

  5. [5]

    preprint arXiv:2304.14454 (2023)

    Wu, C., Zhang, X., Zhang, Y., et al.: Pmc-llama: Further finetuning llama on medical papers. preprint arXiv:2304.14454 (2023)

    arXiv 2023

  6. [6]

    preprint arXiv:2311.16079 (2023)

    Chen, Z., Cano, A.H., Romanou, A., et al.: Meditron-70b: Scaling medical pretraining for large language models. preprint arXiv:2311.16079 (2023)

    arXiv 2023

  7. [7]

    preprint arXiv:2402.13963 (2024)

    Qiu, P., Wu, C., et al.: Towards building multilingual language model for medicine. preprint arXiv:2402.13963 (2024)

    arXiv 2024

  8. [8]

    preprint arXiv:2402.10373 (2024)

    Labrak, Y., Bazoge, A., Morin, E., et al.: Biomistral: A collection of open-source pretrained large language models for medical domains. preprint arXiv:2402.10373 (2024)

    arXiv 2024

Show all 102 references
  1. [9]

    Hugging Face (2024)

    Ankit Pal, M.S.: OpenBioLLMs: Advancing Open-Source Large Language 74 Models for Healthcare and Life Sciences. Hugging Face (2024)

    2024

  2. [10]

    https: //arxiv.org/abs/2405.01886

    Gururajan, A.K., Lopez-Cuena, E., Bayarri-Planas, J., Tormos, A., Hinjos, D., Bernabeu-Perez, P., Arias-Duart, A., Martin-Torres, P.A., Urcelay-Ganzabal, L., Gonzalez-Mallo, M., Alvarez-Napagao, S., Ayguad´ e-Parra, E., Garcia-Gasulla, U.C.D.: Aloe: A Family of Fine-tuned Open...

    2024 arXiv

  3. [11]

    https://arxiv.org/abs/2406

    Zhang, K., Zeng, S., Hua, E., Ding, N., Chen, Z.-R., Ma, Z., Li, H., Cui, G., Qi, B., Zhu, X., Lv, X., Jinfang, H., Liu, Z., Zhou, B.: UltraMedical: Build- ing Specialized Generalists in Biomedicine (2024). https://arxiv.org/abs/2406. 03949

    2024

  4. [12]

    https://arxiv.org/abs/2408.06142

    Christophe, C., Kanithi, P.K., Raha, T., Khan, S., Pimentel, M.A.: Med42-v2: A Suite of Clinical LLMs (2024). https://arxiv.org/abs/2408.06142

    2024 arXiv

  5. [13]

    https: //arxiv.org/abs/2412.18925

    Chen, J., Cai, Z., Ji, K., Wang, X., Liu, W., Wang, R., Hou, J., Wang, B.: HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs (2024). https: //arxiv.org/abs/2412.18925

    2024 arXiv

  6. [14]

    preprint arXiv:2307.15343 (2023)

    Umapathi, L.K., Pal, A., Sankarasubbu, M.: Med-halt: Medical domain halluci- nation test for large language models. preprint arXiv:2307.15343 (2023)

    2023 arXiv

  7. [15]

    medRxiv, 2024–04 (2024)

    Grabb, D., Lamparth, M., Vasan, N.: Risks from language models for automated mental healthcare: Ethics and structure for implementation. medRxiv, 2024–04 (2024)

    2024

  8. [16]

    preprint arXiv:2403.12025 (2024)

    Pfohl, S.R., Cole-Lewis, H., Sayres, R., et al.: A toolbox for surfacing health equity harms and biases in large language models. preprint arXiv:2403.12025 (2024)

    2024 arXiv

  9. [17]

    In: Chiruzzo, L., Ritter, A., Wang, L

    Arias-Duart, A., Martin-Torres, P.A., Hinjos, D., Bernabeu-Perez, P., Ganza- bal, L.U., Mallo, M.G., Gururajan, A.K., Lopez-Cuena, E., Alvarez-Napagao, S., Garcia-Gasulla, D.: Automatic evaluation of healthcare LLMs beyond question- answering. In: Chiruzzo, L., Ritter, A., Wan...

    2025

  10. [18]

    https://huggingface.co/spaces/ HuggingFaceFW/blogpost-fine-tasks

    Kydl´ ıˇ cek, H., Penedo, G., Fourier, C., Habib, N., Wolf, T.: FineTasks: Finding signal in a haystack of 200+ multilingual tasks. https://huggingface.co/spaces/ HuggingFaceFW/blogpost-fine-tasks

  11. [19]

    In: International Conference on Machine Learning, pp

    Wortsman, M., Ilharco, G., Gadre, S.Y., et al.: Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In: International Conference on Machine Learning, pp. 23965–23998 (2022). 75 PMLR

    2022

  12. [20]

    In: International Conference on Computational Linguistics (2020)

    Sun, J., Wang, S., Zhang, J., Zong, C.: Distill and replay for continual language learning. In: International Conference on Computational Linguistics (2020)

    2020

  13. [21]

    Rao, B., Zhu, E.: Searching web data using minhash lsh, 2257–2258 (2016)

    2016

  14. [22]

    GitHub (2024)

    Penedo, G., Cappelli, A., Wolf, T., Sasko, M.: DataTrove: large scale data processing. GitHub (2024). https://github.com/huggingface/datatrove

    2024

  15. [23]

    Yang, S., Chiang, W.-L., Zheng, L., et al.: Rethinking Benchmark and Contam- ination for Language Models with Rephrased Samples (2023)

    2023

  16. [24]

    preprint arXiv:2312.15685 (2023)

    Liu, W., Zeng, W., He, K., et al.: What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. preprint arXiv:2312.15685 (2023)

    2023 arXiv

  17. [25]

    Mukherjee, S., Mitra, A., Jawahar, G., Agarwal, S., Palangi, H., Awadallah, A.: Orca: Progressive Learning from Complex Explanation Traces of GPT-4 (2023)

    2023

  18. [26]

    Proceedings of the National Academy of Sciences120(30), 2305016120 (2023)

    Gilardi, F., Alizadeh, M., Kubli, M.: Chatgpt outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences120(30), 2305016120 (2023)

    2023

  19. [27]

    preprint arXiv:2305.14233 (2023)

    Ding, N., Chen, Y., Xu, B., Qin, Y., Zheng, Z., Hu, S., Liu, Z., Sun, M., Zhou, B.: Enhancing chat language models by scaling high-quality instructional conversations. preprint arXiv:2305.14233 (2023)

    2023 arXiv

  20. [28]

    preprint arXiv:2402.10176 (2024)

    Toshniwal, S., Moshkov, I., Narenthiran, S., Gitman, D., Jia, F., Gitman, I.: Openmathinstruct-1: A 1.8 million math instruction tuning dataset. preprint arXiv:2402.10176 (2024)

    2024 arXiv

  21. [29]

    preprint arXiv:2312.02120 (2023)

    Wei, Y., Wang, Z., Liu, J., Ding, Y., Zhang, L.: Magicoder: Source code is all you need. preprint arXiv:2312.02120 (2023)

    2023 arXiv

  22. [30]

    preprint arXiv:2404.07503 (2024)

    Liu, R., Wei, J., Liu, F., Si, C., Zhang, Y., Rao, J., Zheng, S., Peng, D., Yang, D., Zhou, D., et al.: Best practices and lessons learned on synthetic data for language models. preprint arXiv:2404.07503 (2024)

    2024 arXiv

  23. [31]

    Tang, R., Han, X., Jiang, X., Hu, X.: Does synthetic data generation of llms help clinical text mining? preprint arXiv:2303.04360 (2023)

    2023 arXiv

  24. [32]

    In: Proceedings of the Con- ference on Empirical Methods in Natural Language Processing

    Li, R., Wang, X., Yu, H.: Two directions for clinical data generation with large language models: Data-to-label and label-to-data. In: Proceedings of the Con- ference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Proce...

    2023

  25. [33]

    NPJ Digital Medicine 6(1), 210 (2023)

    Peng, C., Yang, X., Chen, A., Smith, K.E., PourNejatian, N., Costa, A.B., Mar- tin, C., Flores, M.G., Zhang, Y., Magoc, T., et al.: A study of generative large language model for medical research and healthcare. NPJ Digital Medicine 6(1), 210 (2023)

    2023

  26. [34]

    : PubMedQA: A Dataset for Biomedical Research Question Answering

    Jin, Q., Dhingra, B., Liu, Z., et al. : PubMedQA: A Dataset for Biomedical Research Question Answering. In: Proceedings of the 2019 Conference on Empir- ical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP...

    2019

  27. [35]

    https://arxiv.org/abs/2009.13081

    Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., Szolovits, P.: What Dis- ease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams (2020). https://arxiv.org/abs/2009.13081

    2020 arXiv

  28. [36]

    Qiu, P., Wu, C., Zhang, X., et al.: Towards Building Multilingual Language Model for Medicine (2024)

    2024

  29. [37]

    In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp

    Vilares, D., G´ omez-Rodr´ ıguez, C.: HEAD-QA: A healthcare dataset for com- plex reasoning. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 960–966. Association for Computational Lin- guistics, Florence, Italy (2019). https://d...

    2019 doi

  30. [38]

    Zenodo, ??? (2023)

    Ju, C.-Y., Lee, D.-h.: PolyMed: A Medical Dataset Addressing Disease Imbal- ance for Robust Automatic Diagnosis Systems. Zenodo, ??? (2023). https: //doi.org/10.5281/zenodo.7866103 . https://doi.org/10.5281/zenodo.7866103

    2023 doi

  31. [39]

    Zhang, K., Zeng, S., Hua, E., Ding, N., Chen, Z.-R., Ma, Z., Li, H., Cui, G., Qi, B., Zhu, X., Lv, X., Jinfang, H., Liu, Z., Zhou, B.: UltraMedical: Building Specialized Generalists in Biomedicine (2024)

    2024

  32. [40]

    https://huggingface.co/ datasets/BAAI/Infinity-Preference (2024)

    Artificial Intelligence, B.A.: Infinity-Preference. https://huggingface.co/ datasets/BAAI/Infinity-Preference (2024)

    2024

  33. [41]

    arXiv preprint arXiv:2410.18451 (2024)

    Liu, C.Y., Zeng, L., Liu, J., Yan, R., He, J., Wang, C., Yan, S., Liu, Y., Zhou, Y.: Skywork-reward: Bag of tricks for reward modeling in llms. arXiv preprint arXiv:2410.18451 (2024)

    2024 arXiv

  34. [42]

    https://arxiv.org/abs/2311.08592

    Radharapu, B., Robinson, K., Aroyo, L., Lahoti, P.: AART: AI-Assisted Red- Teaming with Diverse Data Generation for New LLM-powered Applications (2023). https://arxiv.org/abs/2311.08592

    2023 arXiv

  35. [43]

    In: Graham, Y., Purver, M

    Wang, Y., Li, H., Han, X., Nakov, P., Baldwin, T.: Do-not-answer: Evaluating safeguards in LLMs. In: Graham, Y., Purver, M. (eds.) Findings of the Associ- ation for Computational Linguistics: EACL 2024, pp. 896–911. Association for Computational Linguistics, St. Julian’s, Malt...

    2024

  36. [44]

    Do Anything Now

    Shen, X., Chen, Z., Backes, M., Shen, Y., Zhang, Y.: “Do Anything Now”: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Lan- guage Models. In: ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM, ??? (2024)

    2024

  37. [45]

    Chen, S., Han, Z., He, B., Ding, Z., Yu, W., Torr, P., Tresp, V., Gu, J.: Red team- ing gpt-4v: Are gpt-4v safe against uni/multi-modal jailbreak attacks? preprint arXiv:2404.03411 (2024)

    2024 arXiv

  38. [46]

    arXiv preprint arXiv:2502.13603 (2025)

    Garcia-Gasulla, D., Arias-Duart, A., Tormos, A., Hinjos, D., Molina-Sedano, O., Gururajan, A.K., Cardello, M.E.: Efficient safety retrofitting against jailbreaking for llms. arXiv preprint arXiv:2502.13603 (2025)

    2025 arXiv

  39. [47]

    Nature Machine Intelligence, 1–10 (2025)

    Akiba, T., Shing, M., Tang, Y., Sun, Q., Ha, D.: Evolutionary optimization of model merging recipes. Nature Machine Intelligence, 1–10 (2025)

    2025

  40. [48]

    arXiv preprint arXiv:2408.07666 (2024)

    Yang, E., Shen, L., Guo, G., Wang, X., Cao, X., Zhang, J., Tao, D.: Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities. arXiv preprint arXiv:2408.07666 (2024)

    2024 arXiv

  41. [49]

    https://arxiv.org/abs/2203.05482

    Wortsman, M., Ilharco, G., Gadre, S.Y., Roelofs, R., Gontijo-Lopes, R., Mor- cos, A.S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., Schmidt, L.: Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022). h...

    2022 arXiv

  42. [50]

    https://arxiv.org/abs/2306

    Yadav, P., Tam, D., Choshen, L., Raffel, C., Bansal, M.: TIES-Merging: Resolv- ing Interference When Merging Models (2023). https://arxiv.org/abs/2306. 01708

    2023

  43. [51]

    preprint arXiv:2311.03099 (2023)

    Yu, L., Yu, B., Yu, H., Huang, F., Li, Y.: Language models are super mario: Absorbing abilities from homologous models as a free lunch. preprint arXiv:2311.03099 (2023)

    2023 arXiv

  44. [52]

    https: //arxiv.org/abs/2212.04089

    Ilharco, G., Ribeiro, M.T., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., Farhadi, A.: Editing Models with Task Arithmetic (2023). https: //arxiv.org/abs/2212.04089

    2023 arXiv

  45. [53]

    https://arxiv.org/abs/2403.19522

    Jang, D.-H., Yun, S., Han, D.: Model Stock: All we need is just a few fine-tuned models (2024). https://arxiv.org/abs/2403.19522

    2024 arXiv

  46. [54]

    https://arxiv.org/abs/2312.06795

    Davari, M., Belilovsky, E.: Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Masks (2024). https://arxiv.org/abs/2312.06795

    2024 arXiv

  47. [55]

    preprint arXiv:2306.01708 (2023) 78

    Yadav, P., Tam, D., Choshen, L., Raffel, C., Bansal, M.: Resolving interference when merging models. preprint arXiv:2306.01708 (2023) 78

    2023 arXiv

  48. [56]

    preprint arXiv:2403.13257 (2024)

    Goddard, C., Siriwardhana, S., Ehghaghi, M., et al.: Arcee’s mergekit: A toolkit for merging large language models. preprint arXiv:2403.13257 (2024)

    2024 arXiv

  49. [57]

    Christiano, P., Leike, J., Brown, T.B., et al.: Deep reinforcement learning from human preferences (2023)

    2023

  50. [58]

    Rafailov, R., Sharma, A., Mitchell, E., et al.: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023)

    2023

  51. [59]

    Tunstall, L., Beeching, E., Lambert, N., et al.: Zephyr: Direct Distillation of LM Alignment (2023)

    2023

  52. [60]

    arXiv preprint arXiv:2405.11143 (2024)

    Hu, J., Wu, X., Zhu, Z., Xianyu, Wang, W., Zhang, D., Cao, Y.: Openrlhf: An easy-to-use, scalable and high-performance rlhf framework. arXiv preprint arXiv:2405.11143 (2024)

    2024 arXiv

  53. [61]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2024)

    Meng, Y., Xia, M., Chen, D.: Simpo: Simple preference optimization with a reference-free reward. In: Advances in Neural Information Processing Systems (NeurIPS) (2024)

    2024

  54. [62]

    https://arxiv.org/abs/2201.11903

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., Zhou, D.: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2023). https://arxiv.org/abs/2201.11903

    2023 arXiv

  55. [63]

    https://arxiv

    Bayarri-Planas, J., Gururajan, A.K., Garcia-Gasulla, D.: Pareto-Optimized Open-Source LLMs for Healthcare via Context Retrieval (2025). https://arxiv. org/abs/2409.15127

    2025 arXiv

  56. [64]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp

    Alzahrani, N., Alyahya, H., Alnumay, Y., Alrashed, S., Alsubaie, S., Almushayqih, Y., Mirza, F., Alotaibi, N., Al-Twairesh, N., Alowisheq, A., et al.: When benchmarks are targets: Revealing the sensitivity of large language model leaderboards. In: Proceedings of the 62nd Annua...

    2024

  57. [65]

    : Evaluation and mitigation of the limitations of large language models in clinical decision-making

    Hager, P., Jungmann, F., Holland, R., Bhagat, K., Hubrecht, I., Knauer, M., Vielhauer, J., Makowski, M., Braren, R., Kaissis, G., et al. : Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature medicine 30(9), 2613–2622 (2024)

    2024

  58. [66]

    In: Text Summarization Branches Out, pp

    Lin, C.-Y.: Rouge: A package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81 (2004)

    2004

  59. [67]

    In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp

    Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)

    2002

  60. [68]

    The Journal of the Acoustical Society of America 62(S1), 63–63 (1977)

    Jelinek, F., Mercer, R.L., Bahl, L.R., Baker, J.K.: Perplexity—a measure of the 79 difficulty of speech recognition tasks. The Journal of the Acoustical Society of America 62(S1), 63–63 (1977)

    1977

  61. [69]

    Nature Machine Intelligence, 1–11 (2025)

    Steyvers, M., Tejeda, H., Kumar, A., Belem, C., Karny, S., Hu, X., Mayer, L.W., Smyth, P.: What large language models know and what people think they know. Nature Machine Intelligence, 1–11 (2025)

    2025

  62. [70]

    arXiv preprint arXiv:2404.04067 (2024)

    Dada, A., Bauer, M., Contreras, A.B., Kora¸ s, O.A., Seibold, C.M., Smith, K.E., Kleesiek, J.: Clue: A clinical language understanding evaluation for llms. arXiv preprint arXiv:2404.04067 (2024)

    2024

  63. [71]

    arXiv preprint arXiv:2409.07314 (2024)

    Kanithi, P.K., Christophe, C., Pimentel, M.A., Raha, T., Saadi, N., Javed, H., Maslenkova, S., Hayat, N., Rajan, R., Khan, S.: Medic: Towards a compre- hensive framework for evaluating llms in clinical applications. arXiv preprint arXiv:2409.07314 (2024)

    2024 arXiv

  64. [72]

    In: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, pp

    Ben Abacha, A., Yim, W.-w., Fan, Y., Lin, T.: An empirical study of clini- cal note generation from doctor-patient encounters. In: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, pp. 2291–2302. Association for Comp...

    2023

  65. [73]

    Nature Scientific Data 10 (2023)

    Yim, W., Fu, Y., Ben Abacha, A., Snider, N., Lin, T., Yetisgen, M.: Aci-bench: a novel ambient clinical intelligence dataset for benchmarking automatic visit note generation. Nature Scientific Data 10 (2023)

    2023

  66. [74]

    In: Proceedings of the 2022 6th International Conference on Natural Language Processing and Informa- tion Retrieval

    Schopf, T., Braun, D., Matthes, F.: Evaluating unsupervised text classifica- tion: Zero-shot and similarity-based approaches. In: Proceedings of the 2022 6th International Conference on Natural Language Processing and Informa- tion Retrieval. NLPIR ’22, pp. 6–15. Association f...

    2023

  67. [75]

    arXiv preprint arXiv:2405.12701 (2024)

    Jeong, M., Hwang, H., Yoon, C., Lee, T., Kang, J.: Olaph: Improving factuality in biomedical long-form question answering. arXiv preprint arXiv:2405.12701 (2024)

    2024 arXiv

  68. [76]

    In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp

    Zeng, G., Yang, W., Ju, Z., Yang, Y., Wang, S., Zhang, R., Zhou, M., Zeng, J., Dong, X., Zhang, R., et al.: Meddialog: Large-scale medical dialogue datasets. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 9241–9250 (2020)

    2020

  69. [77]

    In: ACL-BioNLP 2019 (2019)

    Ben Abacha, A., Shivade, C., Demner-Fushman, D.: Overview of the mediqa 2019 shared task on textual inference, question entailment and question answer- ing. In: ACL-BioNLP 2019 (2019)

    2019

  70. [78]

    Briefings in Bioinformatics 23(5), 282 (2022)

    Luo, L., Lai, P.-T., Wei, C.-H., Arighi, C.N., Lu, Z.: Biored: a rich biomedical 80 relation extraction dataset. Briefings in Bioinformatics 23(5), 282 (2022)

    2022

  71. [79]

    Scientific data 3(1), 1–9 (2016)

    Johnson, A.E., Pollard, T.J., Shen, L., Lehman, L.-w.H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., Mark, R.G.: Mimic-iii, a freely accessible critical care database. Scientific data 3(1), 1–9 (2016)

    2016

  72. [80]

    arXiv preprint arXiv:2405.14191 (2024)

    Yuan, X., Li, J., Wang, D., Chen, Y., Mao, X., Huang, L., Xue, H., Wang, W., Ren, K., Wang, J.: S-eval: Automatic and adaptive test generation for benchmarking safety evaluation of large language models. arXiv preprint arXiv:2405.14191 (2024)

    2024 arXiv

  73. [81]

    Kapoor, S., Bommasani, R., Klyman, K., et al.: On the societal impact of open foundation models (2024)

    2024

  74. [82]

    https://arxiv.org/abs/2408.12547

    Wu, C., Qiu, P., Liu, J., Gu, H., Li, N., Zhang, Y., Wang, Y., Xie, W.: Towards Evaluating and Building Versatile Large Language Models for Medicine (2024). https://arxiv.org/abs/2408.12547

    2024 arXiv

  75. [83]

    GitHub (2024)

    Zhang, K., Ding, N., Qi, B., Zeng, S., Li, H., Zhu, X., Chen, Z.-R., Zhou, B.: UltraMedical: Building Specialized Generalists in Biomedicine. GitHub (2024)

    2024

  76. [84]

    https://huggingface.co/ datasets/omi-health/medical-dialogue-to-soap-summary (2024)

    OMI-Health: medical-dialogue-to-soap-summary. https://huggingface.co/ datasets/omi-health/medical-dialogue-to-soap-summary (2024)

    2024

  77. [85]

    https://arxiv

    Chen, J., Gui, C., Gao, A., Ji, K., Wang, X., Wan, X., Wang, B.: CoD, Towards an Interpretable Medical Agent using Chain of Diagnosis (2024). https://arxiv. org/abs/2407.13301

    2024 arXiv

  78. [86]

    In: TREC 2017 (2017)

    Ben Abacha, A., Agichtein, E., Pinter, Y., Demner-Fushman, D.: Overview of the medical question answering task at trec 2017 liveqa. In: TREC 2017 (2017)

    2017

  79. [87]

    Zhang, X., Tian, C., Yang, X., Chen, L., Li, Z., Petzold, L.R.: AlpaCare:Instruction-tuned Large Language Models for Medical Application (2023)

    2023

  80. [88]

    In: Cohn, T., He, Y., Liu, Y

    Zhu, M., Ahuja, A., Juan, D.-C., Wei, W., Reddy, C.K.: Question answering with long multiple-span answers. In: Cohn, T., He, Y., Liu, Y. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3840–3849. Asso- ciation for Computational Linguistics, On...

    2020

  81. [89]

    BMC Bioinform

    Ben Abacha, A., Demner-Fushman, D.: A question-entailment approach to question answering. BMC Bioinform. 20(1), 511–151123 (2019)

    2019

  82. [90]

    Cureus 15(6) (2023) 81

    Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S., Zhang, Y.: Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus 15(6) (2023) 81

    2023

  83. [91]

    https://huggingface.co/datasets/gamino/wiki medical terms (2022)

    Gamino: wiki medical terms. https://huggingface.co/datasets/gamino/wiki medical terms (2022)

    2022

  84. [92]

    Scientific Data10, 170 (2023)

    Krithara, A., Nentidis, A., Bougiatiotis, K., Paliouras, G.: Bioasq-qa: A manually curated corpus for biomedical question answering. Scientific Data10, 170 (2023)

    2023

  85. [93]

    https://huggingface.co/datasets/BI55/MedText (2023)

    BI55: MedText. https://huggingface.co/datasets/BI55/MedText (2023)

    2023

  86. [94]

    https://huggingface.co/datasets/ ZahrizhalAli/mental health conversational dataset (2023)

    Ali, Z.: mental health conversational dataset. https://huggingface.co/datasets/ ZahrizhalAli/mental health conversational dataset (2023)

    2023

  87. [95]

    https://huggingface.co/datasets/mlabonne/ FineTome-100k (2024)

    Labonne, M.: FineTome-100k. https://huggingface.co/datasets/mlabonne/ FineTome-100k (2024)

    2024

  88. [96]

    https://arxiv.org/abs/2406.08464

    Xu, Z., Jiang, F., Niu, L., Deng, Y., Poovendran, R., Choi, Y., Lin, B.Y.: Mag- pie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing (2024). https://arxiv.org/abs/2406.08464

    2024 arXiv

  89. [97]

    https: //huggingface.co/NousResearch/hermes-function-calling-v1

    ”interstellarninja”, T.: Hermes-Function-Calling-Dataset-V1. https: //huggingface.co/NousResearch/hermes-function-calling-v1

  90. [98]

    Zeng, A., Liu, M., Lu, R., Wang, B., Liu, X., Dong, Y., Tang, J.: AgentTuning: Enabling Generalized Agent Abilities for LLMs (2023)

    2023

  91. [99]

    arXiv preprint arXiv:2406.18518 (2024)

    Liu, Z., Hoang, T., Zhang, J., Zhu, M., Lan, T., Kokane, S., Tan, J., Yao, W., Liu, Z., Feng, Y., et al.: Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. arXiv preprint arXiv:2406.18518 (2024)

    2024 arXiv

  92. [100]

    arXiv preprint arXiv:2408.07055 (2024)

    Bai, Y., Zhang, J., Lv, X., Zheng, L., Zhu, S., Hou, L., Dong, Y., Tang, J., Li, J.: Longwriter: Unleashing 10,000+ word generation from long context llms. arXiv preprint arXiv:2408.07055 (2024)

    2024 arXiv

  93. [101]

    https://arxiv.org/abs/2401.18058

    Bai, Y., Lv, X., Zhang, J., He, Y., Qi, J., Hou, L., Tang, J., Dong, Y., Li, J.: LongAlign: A Recipe for Long Context Alignment of Large Language Models (2024). https://arxiv.org/abs/2401.18058

    2024 arXiv

  94. [102]

    arXiv preprint arXiv:2409.02897 (2024) 82

    Zhang, J., Bai, Y., Lv, X., Gu, W., Liu, D., Zou, M., Cao, S., Hou, L., Dong, Y., Feng, L., Li, J.: Longcite: Enabling llms to generate fine-grained citations in long-context qa. arXiv preprint arXiv:2409.02897 (2024) 82

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.