Pith. sign in

REVIEW 3 major objections 6 minor 46 references

A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 3.8B clinical language model built from merged experts and synthetic alignment matches much larger medical models and beats GPT-4-0125 on ICD-10 coding.

desk verdict Solid modular recipe for clinical SLMs with useful new resources, but headline gains on two rewritten benchmark datasets need to be re-examined before the magnitudes are taken at face value. read the letter →

arxiv 2505.10717 v2 pith:O4SBU4ON submitted 2025-05-15 cs.CL cs.AI

Jean-Philippe Corbeil , Amin Dada , Jean-Michel Attendu , Asma Ben Abacha , Alessandro Sordoni , Lucas Caccia , François Beaulieu , Thomas Lin
show 2 more authors
Jens Kleesiek Paul Vozila
This is my paper · ORCID
classification cs.CLcs.AI
keywords clinicalNLPsmalllanguagemodelspre-instructiontuningmodelmergingsyntheticinstructiondatadirectpreferenceoptimizationICD-10codingCLUE+benchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a 3.8B-parameter model, MediPhi-Instruct, can reach near-parity with 8B models and beat GPT-4-0125 on ICD-10 coding while using no real clinical records for training. It builds this from three steps: pre-instruction tuning of five experts on permissively licensed medical corpora, merging those experts back into one model, and aligning the merged model on MediFlow, a synthetic dataset of 2.5 million instructions. The authors report relative gains over the base model of 64.3% on medical entities, 49.5% on radiology reports, and 44% on ICD-10 coding, with a further 18.9% average gain from alignment. The paper also contributes CLUE+, a doubled clinical benchmark, and the MediFlow dataset. If the results hold, small models become a practical route to deployable clinical NLP without touching sensitive patient data.

What carries the argument

The central machinery is pre-instruction tuning (PIT) — a two-phase continual pre-training that first fine-tunes on GPT-4o-generated task outputs for each document (summaries, QA, entities, relations) and then on the concatenation of those task data with the original corpus — followed by model merging and synthetic-data alignment. Each of five experts is trained on one corpus group (PubMed, Clinical, MedCode, Guidelines, MedWiki), then merged back with the base model via SLERP to restore general skills lost to catastrophic forgetting, and finally all experts are unified with BreadCrumbs merging guided by an evolutionary search over synthetic validation sets. The aligned model is produced by SFT on the 2.5M-instruction MediFlow set (quality-filtered to 800K) and then DPO on 130K triplets with GPT-4o-generated marginally wrong rejected responses. This stack converts domain corpora into a single small model without using real clinical records.

What would settle it

Re-run the CLUE+ evaluation on the original, unmodified MedicationQA answers and MEDIQA-RRS impressions; if the 18.9% average gain or the GPT-4 comparison shrinks materially, the reported improvements depend on the reformulation step rather than on the model itself.

Watch

Extended reading notes

Core claim

The paper claims to construct a 3.8B clinical language model, MediPhi-Instruct, whose modular pipeline — pre-instruction tuning of five domain experts, model merging, and SFT plus DPO alignment on synthetic instructions — yields relative gains of 18.9% over its base model on CLUE+, a new 12-dataset clinical benchmark it introduces. The merged experts already improve over the base without task-specific fine-tuning by 64.3% on medical entities, 49.5% on radiology reports, and 44% on ICD-10 coding, with the coding expert surpassing GPT-4-0125 by 14% relative. After alignment, the model clips 3.8B parameters within 1% of Llama-3-8B on CLUE+ while beating it on four datasets, and exceeds Llama3-Med42-8B on four clinical tasks. The authors treat these results as evidence that domain adaptation for small models can be achieved modularly, with permissively licensed data and synthetic tasks replacing sensitive clinical records.

Load-bearing premise

The reported gains rest on a benchmark whose answers for MedicationQA and MEDIQA-RRS were rewritten by Llama-3.3-70B, with content preservation confirmed only by exact string matching for the radiology QA reformulation.

Editorial extensions

If this is right

  • A single 3.8B model, MediPhi-Instruct, approaches the CLUE+ average of an 8B Llama-3 model while using roughly half the parameters.
  • The pipeline needs no real clinical records: permissively licensed corpora plus a synthetic instruction dataset suffice for the reported gains.
  • Merging experts back with the base model after pre-instruction tuning recovers general abilities that continual pre-training degrades.
  • Specialized expert models, such as the ICD-10 coding expert, already surpass GPT-4-0125 without any task-specific fine-tuning.
  • Alignment on a quality-filtered subset of synthetic instructions (800K of 2.5M) beats using the full set, supporting data quality over quantity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Re-running CLUE+ on the original, untouched MedicationQA answers and MEDIQA-RRS impressions would test whether the 18.9% gain is a property of the model or of the answer reformulation step.
  • The modular recipe — pre-instruction tuning on permissively licensed corpora, merging, and DPO on synthetic tasks — looks transplantable to other data-scarce, license-constrained domains such as legal document analysis, where an instruction-following teacher can synthesize tasks from public corpora.
  • Because MediFlow spans 98 document types but only 14 task types, its unused formats such as nursing notes, laboratory reports, and consent forms are natural next evaluation targets the paper has not measured.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a modular framework for adapting a 3.8B-parameter SLM (Phi-3.5 mini) to clinical NLP tasks without using real clinical records for training. The pipeline has three stages: (1) pre-instruction tuning (PIT) of five domain experts on PubMed, clinical notes, medical coding pages, guidelines, and MedWiki; (2) model merging, first SLERP-merging each expert with the base model and then combining all experts into a unified model (MediPhi) via BreadCrumbs merging selected by an evolutionary search; and (3) alignment on MediFlow, a synthetic instruction dataset of 2.5M examples, using SFT and DPO to obtain MediPhi-Instruct. The authors introduce CLUE+, a 12-dataset benchmark extending CLUE, and report that the experts improve over the base model on several tasks (64.3% relative on SDoH, 49.5% on RRS QA, 44% on ICD-10 coding, with the latter surpassing GPT-4-0125 by 14%), and that alignment adds 18.9% average relative gain. The paper also releases the models, the MediFlow corpus, and synthetic validation sets.

Significance. If the results hold, this is a strong practical contribution: a 3.8B model that approaches or exceeds 8B and much larger models on a broad set of clinical tasks, built without real clinical records and with a modular, reusable pipeline. The paper's strengths include careful ablations (DAPT vs. PIT vs. Explainer, SLERP vs. no merging), transparent flagging of the fine-tuned Med42 comparison, and the release of models, the 2.5M-instruction MediFlow corpus, and synthetic validation sets that enable reproducibility of the merge selection. The CLUE+ extension itself is a useful benchmarking resource. The quantitative claims are credible relative to the base model on the datasets that were not modified, but the benchmark-label rewrites described in Appendix A.8 create a correctness risk for part of the headline results; the lack of uncertainty estimates is an additional concern.

major comments (3)
  1. [Appendix A.8.1 and A.8.2] The CLUE+ evaluation modifies two test sets with Llama-3.3-70B: MedicationQA is reduced from 674 to 485 pairs with rewritten answers, and MEDIQA-RRS impressions are converted from summarization targets into generated QA pairs. The reported headline gains on RRS QA (49.5% relative) and on MedicationQA (11.2 to 19.3), as well as part of the 18.9% average alignment gain, are computed on these reformulated labels. The verification for RRS QA only uses exact-string matching on answer substrings, which does not establish that the generated questions faithfully cover the impression or that the reformulation is unbiased across models. Because MediPhi-Instruct is aligned on GPT-4o-generated QA-style synthetic instructions, the measured gains may partly reflect better mimicry of the rewrite style rather than improved clinical capability. The paper should report results on the original task formats—e.g., MEDIQA-RRS summarization with the original impressions scored by ROUGE, and MedicationQA on all 674 original pairs with a reference-based metric—or provide human validation of the reformulations and show that the rankings are stable.
  2. [Tables 2, 3, 4, and 5] All reported accuracies are single-run point estimates with no error bars, confidence intervals, or significance tests. Several decisive-looking differences are small in absolute terms (e.g., MediPhi-SFT 43.0 vs. MediPhi-Instruct 43.4; Task-Arithmetic 39.4 vs. Ties 39.3; MediPhi 39.3 vs. MedWiki 39.7), and without variance estimates it is not possible to judge whether the reported relative improvements (64.3%, 49.5%, 44%, 18.9%) are stable. I ask the authors to report multiple seeds (at least 3) for the main model comparisons, or at minimum bootstrap confidence intervals over the benchmark datasets, and to indicate which differences are statistically reliable.
  3. [Section 3.1.4 and Appendix A.3] The selection of the merge operator, the evolutionary search budget, and the SLERP proportions is performed on GPT-4o-generated synthetic validation sets that are explicitly designed to be consistent with the CLUE+ tasks, and the MediFlow alignment data covers the same task types and output formats as CLUE+. While this is not test-set fitting, it means the reported CLUE+ numbers are not a fully independent held-out evaluation: the pipeline is tuned on benchmark-aligned synthetic data. The claim of 'no task-specific fine-tuning' should be qualified to state that merge weights, operator choice, and data filters were selected using synthetic sets aligned with the evaluation distribution. An evaluation on an external clinical benchmark would substantially strengthen the generality claim.
minor comments (6)
  1. [Introduction, footnote 2] The contribution bullet claims the models and datasets are released under a 'commercially permissive license,' but the footnote for this claim reads 'TBD.' This is internally inconsistent and should be resolved before publication.
  2. [Table 4] The rows for 'Phi3.5 mini +SFT 800K' and '+DPO' both report 42.2. If the DPO row means SFT followed by DPO, the notation is unclear; please label the training stages explicitly and verify that the duplicated value is not a typo.
  3. [Table 5 and surrounding text] The Med42 comparison is fine-tuned on the ACI-Bench training set, which the authors mark with an asterisk. This is good transparency, but the main text states that 'MediPhi surpasses Med42 in relative percentages on four tasks' and 'outperforms LLaMA3 on four key datasets' without repeating the caveat; the limitation of the Med42 comparison should be stated in the results discussion itself, not only in the table footnote.
  4. [Figures 4 and 5] The legend labels 'GPT-4o-0806' and 'GPT-4-0125' are inconsistent with the abstract and Section 4.2, which mention GPT-4-0125; please verify the model versions and use one consistent naming convention.
  5. [Section 4.2] When stating that easy and medium difficulty ICD10CM questions were excluded, the paper should report the number of remaining test examples and confirm that the same subset is used for the GPT-4 comparison.
  6. [Appendix A.1] In Table 1, the PubMed group lists both PMC full articles (3.8M docs) and PMC abstracts (36M docs). Clarify whether the full articles include the abstracts, to avoid apparent double counting of token totals.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: the reported gains are measured on an independently constructed (though partly LLM-rewritten) benchmark, and the synthetic validation sets are a transparent proxy rather than a re-used test set.

full rationale

The paper's derivation chain is not circular under the definitions used here. Training proceeds from public corpora (PubMed, MedWiki, guidelines, MedCode, clinical sources) through PIT, SLERP/multi-expert merging, and SFT/DPO on the synthetic MediFlow instructions; evaluation is then performed on CLUE+, a benchmark assembled from published datasets plus six additional public datasets. No equation or construction equates the CLUE+ evaluation metric with the training objective, the merge-validation score, or the MediFlow generation loss. The closest candidate is Section 3.1.4, where GPT-4o-generated multiple-choice validation sets 'aligned with our benchmark tasks' guide the evolutionary merge search, while final claims are reported on CLUE+. This is benchmark-aligned model selection on a proxy distribution, not a reduction of the benchmark to the validation set by construction: the validation sets are newly generated items, not CLUE+ test inputs or labels, and the headline gains on RRS QA, SDoH, and ACI-Bench involve non-multiple-choice outputs not directly optimized by that proxy. Similarly, MediFlow intentionally covers the same task families as CLUE+ (QA, summarization, NER, textual entailment, error detection), but that is standard instruction-tuning coverage rather than a fitted re-use of test data. The reformulation of MedicationQA and MEDIQA-RRS answers by Llama-3.3-70B (Appendices A.8.1, A.8.2) is a real validity threat to external comparisons with GPT-4 and to the absolute benchmark numbers, but it is a correctness/benchmark-integrity issue, not an internal circularity pattern: the paper never defines its training or selection objective in terms of those rewritten labels. Self-citations to CLUE, MEDIQA-RRS, MEDEC, and ACI-Bench are dataset provenance citations to published resources; no uniqueness theorem or unverified prior result is imported as load-bearing support. The central empirical claims therefore stand on their own measured pipeline, and no specific reduction from output to input can be exhibited.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central result rests on assumptions about synthetic data quality, benchmark-aligned model selection, and the validity of CLUE+ after LLM-based test reformulation. The main free parameters are merge proportions, data filtering thresholds, and generation settings, all chosen by validation performance rather than derived from theory.

free parameters (5)
  • SLERP merge proportion per expert = 10%, 25%, or 50% (per-expert values not listed)
    Selected on GPT-4o-generated validation sets (Section 3.1.3). Controls how much expert knowledge survives versus base skills.
  • Multi-expert merge operator and evolutionary search budget = Task-Arithmetic, TIES, BreadCrumbs; 500 evaluations
    BreadCrumbs was chosen as MediPhi because it balanced average accuracy and #DG on validation sets; other operators differ by only about 0.1 points.
  • MediFlow quality filter thresholds = 800k for SFT, 130,852 for DPO
    Filtering to top 800k improves CLUE+ from 41.9 to 43.0; the threshold was chosen after observing this difference.
  • Generation temperatures and judge sample count = T=1.0 (70%) / 1.25 (30%); M=5
    Hand-chosen to balance quality and diversity; not ablated.
  • PIT task type used for all experts = Summarization
    Chosen after the ICD10CM case study where Summary PIT beat QA, NER, and relation extraction (Figures 4-5).
assumptions (5)
  • domain assumption Synthetic instructions and judge scores from GPT-4o and GPT-4o-mini are a reliable proxy for human clinical quality.
    MediFlow is filtered and DPO pairs are created without human annotation; the alignment gains depend on these labels being correct enough to transfer to real clinical text.
  • domain assumption GPT-4o-generated validation sets are a faithful proxy for CLUE+ test performance.
    Used to select merge weights in Section 3.1.4; no correlation or calibration study is provided.
  • domain assumption Model merging preserves expert capabilities and restores base skills.
    The method relies on SLERP, TIES, and BreadCrumbs properties from prior work; the paper only evaluates clinical tasks, not general capabilities (acknowledged in Limitations).
  • domain assumption The CLUE+ protocol, including LLM-reformulated test answers, measures the intended clinical abilities.
    MedicationQA and RRS QA test sets were modified with an LLM; if reformulation changes labels, benchmark numbers shift.
  • domain assumption PIT benefits extend from QA to summarization, NER, and relation extraction.
    The paper generalizes Jiang et al. (2024) based on an ICD10CM case study; this underlies all five experts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment." pith.science (2026). https://pith.science/paper/O4SBU4ON

@misc{pith2026250510717,
  author       = {Pith},
  title        = {Pith review of: A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O4SBU4ON}},
  note         = {Machine review of arXiv:2505.10717}
}
read the original abstract

High computation costs and latency of large language models such as GPT-4 have limited their deployment in clinical settings. Small language models (SLMs) offer a cost-effective alternative, but their limited capacity requires biomedical domain adaptation, which remains challenging. An additional bottleneck is the unavailability and high sensitivity of clinical data. To address these challenges, we propose a novel framework for adapting SLMs into high-performing clinical models. We introduce the MediPhi collection of 3.8B-parameter SLMs developed with our novel framework: pre-instruction tuning of experts on relevant medical and clinical corpora (PMC, Medical Guideline, MedWiki, etc.), model merging, and clinical-tasks alignment. To cover most clinical tasks, we extended the CLUE benchmark to CLUE+, doubling its size. Our expert models deliver relative improvements on this benchmark over the base model without any task-specific fine-tuning: 64.3% on medical entities, 49.5% on radiology reports, and 44% on ICD-10 coding (outperforming GPT-4-0125 by 14%). We unify the expert models into MediPhi via model merging, preserving gains across benchmarks. Furthermore, we built the MediFlow collection, a synthetic dataset of 2.5 million high-quality instructions on 14 medical NLP tasks, 98 fine-grained document types, and JSON format support. Alignment of MediPhi using supervised fine-tuning and direct preference optimization achieves further gains of 18.9% on average.

Figures

Figures reproduced from arXiv: 2505.10717 by the authors.

Figure 1
Figure 1. Our approach in two steps: 1) continual pre [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of model merging techniques. a. Domain-specific merging via SLERP: An expert model is obtained by fine-tuning the base model on domain￾specific data (step i). The expert and base models are then merged using spherical linear interpolation (SLERP) to produce the final merged model (step ii). b. Multi-expert merging: Multiple expert models are independently derived from the base model via domain￾specific pret… view at source ↗
Figure 3
Figure 3. Schema of MediFlow generation. i) Given a randomly sampled set of parameter values, we prompt GPT-4o for N instructions to obtain triplets (instruction, inputs, outputs) in which we have P pairs of inputs (I) and outputs (O) each. ii) We prompt GPT-4o mini with LLM-as-a-Judge and self-consistency on M samples. iii) We define a heuristic to filter a di￾verse, high-quality subset for alignment purposes. Generation of … view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Performance of different continual pre￾training methods: domain adaptation pre-training (DAPT) on ICD10CM webpages, fine-tuning on syn￾thetic textbook-like material generated with GPT-4o (Explainer), and pre-instruction tuning (PIT) with differ￾ent synthetically genera…
Figure 5
Figure 5. Figure 5: Impact of SLERP merging on the perfor￾mance of ICD10CM. Merging back with the base model (SLERP 50%) systematically results in gains and these are more pronounced for PIT. ing GPT-4 by 8% (14% relative). Based on these results, the rest of the paper will apply PIT with…
Figure 6
Figure 6. Figure 6: Summary of improvements from Phi3.5 to MediP hi-Instruct (SFT+DPO). DataM ix is adapted on all corpora. SLERP P IT AV G is the average of five experts trained with PIT on each dataset. MediP hi is the unified expert from the five experts. MediP hi-SF T is an instruct m…
Figure 7
Figure 7. Figure 7: Distribution of instruction tokens in MediFlow [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Distribution of input tokens in MediFlow with [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Distribution of output tokens in MediFlow [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Judge Score Distributions across MediFlow. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: t-SNE 2D scatterplot of MediFlow (2.5M) using OpenAI [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: t-SNE 2D scatterplot of MediFlow (2.5M) using OpenAI [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: t-SNE 2D scatterplots of MediFlow (2.5M) [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: An example of a verbose answer in MedicationQA and the reformulated answer we replaced it with. [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: An example of a formulation of questions based on the impression section. [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 36 canonical work pages

  1. [1]

    Discharge Summary (6)

    work page

  2. [2]

    SOAP Clinical Note (5)

    work page

  3. [3]

    Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al

    Clinicalbench: Can llms beat traditional ml models in clinical prediction? In GenAI for Health: Potential, Trust and Policy Compliance. Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al. 2023. Meditron-70b: Scaling medical pretrai...

    arXiv 2023

  4. [4]

    In Forty-first International Conference on Machine Learning

    Fewer truncations improve language modeling. In Forty-first International Conference on Machine Learning. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. Enhancing chat language models by scaling high-quality instructional conversations. In Proceedings of the 2023 Conference on Empiri- cal Methods ...

    arXiv 2023

  5. [5]

    entity": The specific category of the extracted information, chosen from the predefined taxonomy below. -

    ASSESSMENT AND PLAN (A/P) Diagnosis & Differential Diagnosis: Explain why the most likely condition was chosen. Plan: - Medications: Name, dose, frequency, and rationale. - Additional Tests: Imaging, lab work, specialist referrals. - Follow-up Plan: Next steps, expected outcomes. - Patient Education: Instructions, lifestyle modifications. - Ensure that ev...

    work page

  6. [6]

    Scientific Article (8)

    work page

  7. [7]

    Advances in Neural Information Processing Systems, 36:50358–50376

    Scaling data-constrained language models. Advances in Neural Information Processing Systems, 36:50358–50376. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawa- har, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. Orca: Progressive learning from complex explanation traces of gpt-4. Preprint, arXiv:2306.02707. Clara Na, Ian Magnusson, Ananya Harsh J...

    arXiv 2023

  8. [8]

    Medicine, 84(88.3):77–3

    Can generalist foundation models outcom- pete special-purpose tuning? case study in medicine. Medicine, 84(88.3):77–3. Harsha Nori, Naoto Usuyama, Nicholas King, Scott Mayer McKinney, Xavier Fernandes, Sheng Zhang, and Eric Horvitz. 2024. From medprompt to o1: Exploration of run-time strategies for medi- cal challenge problems and beyond. arXiv preprint a...

    arXiv 2024

Show all 46 references
  1. [9]

    Monitoring Data of Vital Signs (1)

  2. [10]

    Junda Wang, Zonghai Yao, Zhichao Yang, Huixue Zhou, Rumeng Li, Xun Wang, Yucheng Xu, and Hong Yu

    Will we run out of data? limits of llm scaling based on human-generated data. Junda Wang, Zonghai Yao, Zhichao Yang, Huixue Zhou, Rumeng Li, Xun Wang, Yucheng Xu, and Hong Yu

  3. [11]

    arXiv preprint arXiv:2310.15959

    Notechat: a dataset of synthetic doctor-patient conversations conditioned on clinical notes. arXiv preprint arXiv:2310.15959. Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Mor- cos, Hongseok Namkoong, Ali Farhadi, Yair Carmon...

    arXiv

  4. [12]

    Journal of the Ameri- can Medical Informatics Association: JAMIA, pages ocae045–ocae045

    Pmc-llama: toward building open-source lan- guage models for medicine. Journal of the Ameri- can Medical Informatics Association: JAMIA, pages ocae045–ocae045. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yun- tian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. 2024. Mag...

    2024 arXiv

  5. [13]

    Radiology Report (3)

  6. [14]

    Doctor-Patient Conversation (5)

  7. [15]

    Nurse-Patient Dialog (5)

  8. [17]

    Emergency Department Note (7)

  9. [18]

    Laboratory Report (1)

  10. [19]

    Prescription Note (1)

  11. [20]

    Preoperative Assessment (1)

  12. [21]

    Postoperative Note (1)

  13. [22]

    Consultation Note (1)

  14. [23]

    Pathology Report (1)

  15. [24]

    Screening Report (1)

  16. [27]

    Immunization Record (1)

  17. [28]

    Psychiatric Evaluation (5)

  18. [29]

    Social Work Note (1)

  19. [30]

    End-of-Life Care Documentation (1)

  20. [33]

    Home Health Care Report (1)

  21. [34]

    Genetic Testing Report (1)

  22. [35]

    Chest pain for two days

    CHIEF COMPLAINT A precise and natural-language statement describing the primary reason for the visit. Usually written in the patient’s own words (e.g., "Chest pain for two days.")

  23. [36]

    rationale

    Patient Education Material (1) In Figures 7, 8 and 9, we present the histograms of tokens for generated instructions, input and out- put, respectively. Figure 7: Distribution of instruction tokens in MediFlow with y-axis in log scale. The average is301±295 tokens. Figure 8: Di...

    2019

  24. [37]

    - Duration: Progression over time

    HISTORY OF PRESENT ILLNESS Provide a detailed, full nar- rative including: - Onset: Exact time course (sudden/gradual, exact duration). - Duration: Progression over time. - Severity: Patient’s description or numeric scale (1-10). - Location: Anatomical speci- ficity. - Modifyi...

  25. [38]

    Describe specific observations for each system: - Vital Signs: BP, HR, Temp, O2 Sat, RR

    PHYSICAL EXAM Expand on every finding instead of using short labels. Describe specific observations for each system: - Vital Signs: BP, HR, Temp, O2 Sat, RR. - General Appearance: Pa- tient’s demeanor, level of distress. - Neurological: Reflexes, motor is it nessasary to ween ...

  26. [39]

    List both abnormal and pertinent normal find- ings

    RESULTS Include all relevant diagnostic data, explaining why each result matters. List both abnormal and pertinent normal find- ings

  27. [40]

    entity":

    Employment - "entity": "Employment" (general employment-related mention) - "label": "StatusEmploy" → "employed", "unemployed", "retired", "on disability", "student", "homemaker" - "label": "Duration" → "for the last five years", "since 2010" - "label": "History" → "15 years ag...

    2010

  28. [41]

    entity":

    Living Status -"entity": "LivingStatus" (mentions of where and how someone lives) - "label": "StatusTime" → "current", "past", "future" - "label": "TypeLiving" → "alone", "with family", "with others", "homeless" - "label": "Duration" → "for the past ten years", "since 2015" - ...

    2015

  29. [42]

    entity":

    Substance Use - "entity": "Alcohol", "Drug", "Tobacco" (mentions of substance use) - "label": "StatusTime" → "none", "current", "past" - "label": "Duration" → "for the past eight years" - "label": "History" → "seven years ago", "in 2005" - "label": "Method" → "smoke", "snort",...

    2005

  30. [43]

    Extract only explicitly mentioned entities—do not infer informa- tion

  31. [44]

    Use exact text from the document—the "value" must match the original wording

  32. [45]

    entity" and

    Categorize precisely—select the most appropriate "entity" and "label"

  33. [46]

    Your final output should be a list of JSON objects containing only the entities present in the document

    Ensure valid JSON format—return structured, machine-readable output. Your final output should be a list of JSON objects containing only the entities present in the document. A.8.5 MEDEC MEDEC System Prompt Task: Medical Error Detection in Clinical Text Role: You are an expert ...

    2004

  34. [2019]

    https://mtsamples.com/

    Mtsamples. https://mtsamples.com/. Ac- cessed: 2024-11-24. Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024a. Phi-3 technical report: A highly capable language model l...

    2024 arXiv

  35. [2020]

    In International Conference on Learning Representations

    Measuring massive multitask language under- standing. In International Conference on Learning Representations. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Worts- man, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In The Eleventh ...

    2024 arXiv

  36. [2022]

    zero-shot

    Training language models to follow instruc- tions with human feedback. Advances in neural in- formation processing systems, 35:27730–27744. Ankit Pal, Logesh Kumar Umapathi, and Malaikan- nan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for med...

    2022

  37. [2023]

    Journal of the Amer- ican Medical Informatics Association, 30(8):1367– 1378

    The 2022 n2c2/uw shared task on extracting social determinants of health. Journal of the Amer- ican Medical Informatics Association, 30(8):1367– 1378. Leland McInnes, John Healy, Steve Astels, et al. 2017. hdbscan: Hierarchical density based clustering. J. Open Source Softw., ...

    2022 arXiv

  38. [2024]

    arXiv preprint arXiv:2412.19260

    Medec: A benchmark for medical error detec- tion and correction in clinical notes. arXiv preprint arXiv:2412.19260. Yekun Chai. 2019. eval4ner: An all-round evalua- tion for named entity recognition. https://github. com/cyk1337/eval4ner. Canyu Chen, Jian Yu, Shan Chen, Che Liu...

    2019 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.