pith. sign in

arxiv: 2606.26879 · v2 · pith:GATBTF2Tnew · submitted 2026-06-25 · 💻 cs.AI

A Pipeline for Generating Longitudinal Synthetic Clinical Notes Using Large Language Models

William Poulett , Alice Waterhouse , Ben Wallace , Scarlett Kynoch , Amaia Imaz Blanco , Michael Spence , Jonathan Pearson This is my paper

Pith reviewed 2026-06-29 04:59 UTC · model grok-4.3

classification 💻 cs.AI
keywords synthetic clinical noteslarge language modelslongitudinal patient recordshealthcare AI developmentprivacy-preserving data generationpatient journey simulationclinical note generation pipelinesynthetic dataset release
0
0 comments X

The pith

A modular pipeline uses large language models to generate internally consistent longitudinal synthetic clinical notes across full patient journeys.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a pipeline that starts with structured patient profiles, simulates semi-structured journeys through hospital care, and then uses large language models to produce the actual clinical notes. It adds validation and augmentation steps to keep the notes consistent over time while allowing variation in how they are written and what details they include. The goal is to give researchers a dataset they can use to build and test AI tools for tasks like note summarization or coding without needing access to real patient records. A release of 70 synthetic patients, each with 20-50 notes, is provided at different levels of quality control so users can choose the trade-off between scale and realism.

Core claim

The pipeline produces internally consistent longitudinal synthetic clinical notes that capture variation in writing style, note structure, and clinical detail, enabling development of clinical AI systems without reliance on real patient data. It does this through a modular design that combines structured patient generation, semi-structured patient journey simulation, and unstructured note generation with large language models, plus additional LLM-based validation and augmentation mechanisms to improve faithfulness, realism, and diversity.

What carries the argument

The modular pipeline that links structured patient generation, semi-structured journey simulation, and LLM-driven note generation, together with separate validation and augmentation steps.

If this is right

  • The released dataset of 70 patients with 20-50 notes each can be used directly to develop and evaluate summarisation tools, coding models, and decision support systems.
  • Users can select different validation levels of the data to balance realism against the volume needed for their specific use case.
  • The approach removes the need for real patient data, thereby avoiding associated privacy risks in clinical AI development.
  • Internal consistency across longitudinal records is maintained while still allowing variation in style and clinical detail.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same modular structure of patient generation followed by journey simulation and note creation could be tested in other domains that require longitudinal records, such as legal case files.
  • If the pipeline's consistency mechanisms scale reliably, it might support repeated generation of larger cohorts for rare-disease or low-prevalence scenarios.
  • Performance gaps between synthetic-trained and real-data-trained models could be measured on standard clinical NLP benchmarks to quantify the pipeline's practical limits.

Load-bearing premise

That the LLM generation steps plus validation and augmentation produce notes with enough faithfulness, realism, and diversity to be practically useful for training and evaluating clinical AI tools.

What would settle it

A blinded test in which clinicians rate the synthetic notes for realism and consistency against real notes, or an experiment showing whether models trained on the synthetic dataset achieve performance comparable to models trained on real clinical notes for tasks like summarization or coding.

Figures

Figures reproduced from arXiv: 2606.26879 by Alice Waterhouse, Amaia Imaz Blanco, Ben Wallace, Jonathan Pearson, Michael Spence, Scarlett Kynoch, William Poulett.

Figure 1
Figure 1. Figure 1: A simple overview of our synthetic data pipeline. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Generating a population of patients. Synthea data is used alongside an LLM to create a [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Generating reasons for admission for patients. Admissions can either be elective admis [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Generating patient journeys. Patient journeys are generated using a series of LLM calls. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Generating clinical notes. Our clinical note generation stage uses very versatile prompts [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: We augment our clinical notes by optionally adding typos, abbreviations, and staff [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Generation of an evaluation report using traditional metrics and LLM Judges. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
read the original abstract

Synthetic data is increasingly used to enable the development and evaluation of AI systems in domains where access to real-world data is restricted. In healthcare, clinical documentation presents particular challenges due to its sensitivity. This work introduces a synthetic clinical notes pipeline and dataset designed to support the development of clinical AI tools while avoiding the privacy risks associated with real patient data. The dataset is generated using a modular pipeline that combines structured patient generation, semi-structured patient journey simulation, and unstructured clinical note generation using large language models. The pipeline is designed to prioritise internal consistency across longitudinal patient records, while also capturing variation in writing style, note structure, and clinical detail. Additional mechanisms, including LLM-based validation and augmentation steps, are used to improve faithfulness, realism, and diversity of the generated notes. We release a dataset of 70 synthetic patients, each associated with 20-50 clinical notes spanning a full hospital journey. The dataset is provided at multiple levels of validation, enabling users to balance realism and scalability depending on their use case. This dataset supports the development, testing, and evaluation of clinical AI systems, including summarisation tools, coding models, and decision support systems, without reliance on real patient data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript describes a modular pipeline for generating longitudinal synthetic clinical notes using large language models. It combines structured patient generation, semi-structured patient journey simulation, and unstructured clinical note generation, augmented by LLM-based validation and augmentation steps intended to enforce internal consistency, realism, and diversity. The work releases a dataset of 70 synthetic patients, each with 20-50 notes spanning a full hospital journey, provided at multiple validation levels to support clinical AI development without real patient data.

Significance. If the pipeline's validation mechanisms demonstrably produce notes with sufficient faithfulness and longitudinal consistency, the released dataset could enable privacy-preserving development of clinical AI tools such as summarizers and coding models. The absence of any quantitative metrics, comparisons to real EHR distributions, or expert evaluations in the provided description, however, prevents assessment of whether these properties hold at scale.

major comments (1)
  1. [Abstract] Abstract: The claims that the pipeline produces 'internally consistent' longitudinal records that 'capture variation in writing style, note structure, and clinical detail' and that the LLM-based validation steps 'improve faithfulness, realism, and diversity' are presented without any supporting quantitative metrics, human clinician ratings, or distributional comparisons to real clinical corpora. This directly undercuts the central assertion that the notes are suitable for training and evaluating clinical AI systems.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review. We address the major comment below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The claims that the pipeline produces 'internally consistent' longitudinal records that 'capture variation in writing style, note structure, and clinical detail' and that the LLM-based validation steps 'improve faithfulness, realism, and diversity' are presented without any supporting quantitative metrics, human clinician ratings, or distributional comparisons to real clinical corpora. This directly undercuts the central assertion that the notes are suitable for training and evaluating clinical AI systems.

    Authors: We agree that the abstract asserts these properties without quantitative support, metrics, ratings, or distributional comparisons. The manuscript describes the pipeline's modular design and validation mechanisms as intended to achieve internal consistency, stylistic variation, and improved faithfulness/realism/diversity, but does not include empirical evaluations of these outcomes. In revision we will edit the abstract to frame these as design goals of the pipeline rather than demonstrated results, and we will add a limitations section explicitly noting the lack of such quantitative validation or real-EHR comparisons. This will align the claims with the paper's actual scope as a pipeline and dataset release. revision: yes

Circularity Check

0 steps flagged

No circularity: descriptive engineering pipeline with no derivations or self-referential reductions

full rationale

The paper describes a modular pipeline for synthetic note generation using LLMs, structured patient simulation, and validation steps. No equations, fitted parameters, predictions, or uniqueness theorems appear. Claims rest on pipeline design choices and released data rather than any reduction of outputs to inputs by construction. No self-citation load-bearing steps or ansatz smuggling are present. The work is self-contained as an engineering contribution without mathematical derivation chains.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work rests on the domain assumption that current LLMs can produce usable clinical text when suitably prompted and validated; no free parameters, invented entities, or additional axioms are introduced beyond standard LLM capabilities.

axioms (1)
  • domain assumption LLMs can generate realistic and consistent clinical notes when guided by structured patient data and validation steps
    This underpins the unstructured note generation and augmentation components described in the abstract.

pith-pipeline@v0.9.1-grok · 5753 in / 1212 out tokens · 27061 ms · 2026-06-29T04:59:24.419149+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 2 canonical work pages

  1. [1]

    Robust de-anonymization of large sparse datasets

    Arvind Narayanan and Vitaly Shmatikov. Robust de-anonymization of large sparse datasets. In2008 IEEE Symposium on Security and Privacy (SP 2008), pages 111–125. IEEE, 2008

    2008

  2. [2]

    Synthetic data generation: State of the art in health care domain.Computer Science Review, 48:100546, 2023

    Hajra Murtaza, Musharif Ahmed, Naurin Farooq Khan, Ghulam Murtaza, Saad Zafar, and Ambreen Bano. Synthetic data generation: State of the art in health care domain.Computer Science Review, 48:100546, 2023

    2023

  3. [3]

    Pezoulas, Dimitrios I

    Vasileios C. Pezoulas, Dimitrios I. Zaridis, Eugenia Mylona, Christos Androutsos, Kosmas Apostolidis, Nikolaos S. Tachos, and Dimitrios I. Fotiadis. Synthetic data generation methods in healthcare: A review on open-source tools and methods.Computational and Structural Biotechnology Journal, 23:2892–2910, 2024

    2024

  4. [4]

    Olatunji, Jens Rauch, Matthias Katzensteiner, and Megha Khosla

    Iyiola E. Olatunji, Jens Rauch, Matthias Katzensteiner, and Megha Khosla. A review of anonymization for healthcare data.CoRR, abs/2104.06523, 2021

    work page arXiv 2021

  5. [5]

    Artificial data pilot, 2025

    NHS England Digital. Artificial data pilot, 2025. Accessed: 2026-05-13

    2025

  6. [6]

    Synthea: Synthetic patient population simulator, 2026

    The MITRE Corporation. Synthea: Synthetic patient population simulator, 2026. Accessed: 2026-05-13

    2026

  7. [7]

    Agent hospital: A simulacrum of hospital with evolvable medical agents, 2025

    Junkai Li, Yunghwei Lai, Weitao Li, Jingyi Ren, Meng Zhang, Xinhui Kang, Siyu Wang, Peng Li, Ya-Qin Zhang, Weizhi Ma, and Yang Liu. Agent hospital: A simulacrum of hospital with evolvable medical agents, 2025

    2025

  8. [8]

    Sdv (synthetic data vault) developer documentation, 2026

    DataCebo. Sdv (synthetic data vault) developer documentation, 2026. Accessed: 2026-05-13

    2026

  9. [9]

    Swpc synthea: Uk adaptation of the synthea synthetic patient generator, 2026

    NHS England. Swpc synthea: Uk adaptation of the synthea synthetic patient generator, 2026. Accessed: 2026-05-13. 16

    2026

  10. [10]

    Springer Nature Switzerland, 2024

    Gleb Kumichev, Pavel Blinov, Yulia Kuzkina, Vasily Goncharov, Galina Zubkova, Nikolai Zenovkin, Aleksei Goncharov, and Andrey Savchenko.MedSyn: LLM-Based Synthetic Medical Text Generation Framework, page 215–230. Springer Nature Switzerland, 2024

    2024

  11. [11]

    A comprehensive taxonomy of hallucinations in large language models, 2025

    Manuel Cossio. A comprehensive taxonomy of hallucinations in large language models, 2025

    2025

  12. [12]

    Medical hallucination in foundation models and their impact on healthcare.medRxiv, 2025

    Yubin Kim, Hyewon Jeong, Shan Chen, Shuyue Stella Li, Mingyu Lu, Kumail Alhamoud, Jimin Mun, Cristina Grau, Minseok Jung, Rodrigo Gameiro, Lizhou Fan, Eugene Park, Tristan Lin, Joonsik Yoon, Wonjin Yoon, Maarten Sap, Yulia Tsvetkov, Paul Liang, Xuhai Xu, Xin Liu, Daniel McDuff, Hyeonhoon Lee, Hae Won Park, Samir Tulebaev, and Cynthia Breazeal. Medical hal...

    2025

  13. [13]

    Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, Juanma Zambrano Chaves, Szu-Yeu Hu, Mike Schaekermann, Aishwarya Kamath, Yong Cheng, David G. T. Barrett, Cathy Cheung, Basil Mustafa, Anil Palepu, Daniel McDuff, Le Hou, Tomer Golany, Luyang Liu, Jean baptiste Alayrac, Nei...

    2024

  14. [14]

    Smith, Nima PourNejatian, Anthony B

    Cheng Peng, Xi Yang, Aokun Chen, Kaleb E. Smith, Nima PourNejatian, Anthony B. Costa, Cheryl Martin, Mona G. Flores, Ying Zhang, Tanja Magoc, Gloria Lipori, Duane A. Mitchell, Naykky S. Ospina, Mustafa M. Ahmed, William R. Hogan, Elizabeth A. Shenkman, Yi Guo, Jiang Bian, and Yonghui Wu. A study of generative large language model for medical research and ...

    2023

  15. [15]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023

    2023

  16. [16]

    Diego Gosmar and Deborah A. Dahl. Hallucination mitigation using agentic ai natural language-based frameworks.ArXiv, abs/2501.13946, 2025

    work page arXiv 2025

  17. [17]

    A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics.Information Fusion, 118:102963, 2025

    Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and Erik Cambria. A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics.Information Fusion, 118:102963, 2025

    2025

  18. [18]

    Benchmarking retrieval-augmented generation for medicine, 2024

    Guangzhi Xiong, Qiao Jin, Zhiyong Lu, and Aidong Zhang. Benchmarking retrieval-augmented generation for medicine, 2024. 17

    2024

  19. [19]

    Reasoning-enhanced healthcare predictions with knowledge graph community retrieval, 2025

    Pengcheng Jiang, Cao Xiao, Minhao Jiang, Parminder Bhatia, Taha Kass-Hout, Jimeng Sun, and Jiawei Han. Reasoning-enhanced healthcare predictions with knowledge graph community retrieval, 2025

    2025

  20. [20]

    A survey on llm-as-a-judge, 2025

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. A survey on llm-as-a-judge, 2025

    2025

  21. [21]

    Justice or prejudice? quantifying biases in llm-as-a-judge, 2024

    Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, Nitesh V Chawla, and Xiangliang Zhang. Justice or prejudice? quantifying biases in llm-as-a-judge, 2024

    2024

  22. [22]

    typo: A python package to simulate typographical errors, 2023

    Ranvijay Kumar. typo: A python package to simulate typographical errors, 2023. Version 0.1.7. Accessed: 2026-05-13

    2023

  23. [23]

    Evaluating gender bias in large language models in long-term care.BMC Medical Informatics and Decision Making, 25(1):274, 2025

    Sam Rickman. Evaluating gender bias in large language models in long-term care.BMC Medical Informatics and Decision Making, 25(1):274, 2025. Published: 2025-08-11. 18

    2025