{"id":"5dc11b58-2515-4b89-8c12-514c3dfca951","arxiv_id":"2412.05153","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A frozen LLM prompted with a text description and one average patient example generates synthetic Parkinson's and Alzheimer's cohorts with preserved correlations, but with lower fidelity than models trained on original data.","lead":"This paper tests whether a large language model can invent fake but realistic patient records from a text description alone, without ever seeing the real records. The method works well enough for education and early prototyping, but it cannot yet match generators that train on real data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation leaks target-cohort summary statistics into the prompt via the example patient; the claimed zero-shot generalizability to new populations is untested.","rationale":"The reader's weakest_assumption correctly identifies that the single example patient, constructed from published averages of the same PPMI and ADNI cohorts later used as ground truth, is a load-bearing component; the ablation confirms it materially improves fidelity. My stress-test sharpens this into a concrete evaluation-leakage concern: the prompt as evaluated contains summary statistics of the target distribution, so the experiments cannot support the claim that a text description alone suffices for an arbitrary new population. This does not invalidate the method's practical utility for prototyping or education, but it does mean the central 'no original data' claim is conditional on the availability of cohort-specific aggregate information. I agree with the reader's verdict of CONDITIONAL: the paper's contribution is real and clearly presented, but the generality of the zero-shot claim requires an external-population test. I proposed one such test that would settle the concern. No other objection — such as the lower fidelity compared to Gaussian Copulas or the odd LogisticDetection behavior — is as load-bearing, because the method's stated value proposition is specifically its ability to work without original patient-level data.","tokens_in":18540,"tokens_out":2811,"duration_ms":30289,"concrete_test":"Select a target clinical cohort not among PPMI/ADNI, e.g., a sepsis cohort from MIMIC-III or eICU. Construct two prompts: (A) database description only, with no example patient; (B) description plus a single example patient built from published summary statistics of a *different* disease/population (or, if available, from the target cohort's own published aggregates — record which). Generate 1000 synthetic rows with GPT-4 for each prompt and compute Column Shape, Column Pair Trends, and LogisticDetection against the real cohort. If prompt (A) or (B) achieves Column Shape comparable to the paper's 0.778 and the target-derived example is not required, the zero-shot claim holds; if removing the target-derived example drops Column Shape below roughly 0.6, the method's fidelity depends on cohort-specific aggregate information and the 'only a description' claim fails for new populations.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that the method 'does not require access to the original data, but only a description of the desired database' is only partially supported by the evaluation. In Section III.C.2, the single context example is explicitly 'created using the average values of the selected variables extracted from publications on #PPMI2020 [35] and on the ADNI dataset [36], [37]' — the same PPMI and ADNI cohorts that serve as ground truth in Tables I and II. The ablation in Table V shows that removing this example lowers Column Shape from 0.70 to 0.64 and ContingencySimilarity from 0.90 to 0.79, demonstrating that the example injects substantial cohort-specific information about marginal and joint distributions. Thus, the evaluated configuration is not purely 'text-to-tabular' in the sense claimed: it requires that aggregate statistics of the exact target population be available in the literature or from experts. For a genuinely new or rare disease population, where such published averages do not exist, the method's fidelity is untested, and the paper's own Discussion admits this limitation. This is not an internal inconsistency — the authors transparently report the prompt — but it means the headline generalization claim goes beyond what the experiments establish.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a text-to-tabular method for generating synthetic patient data with frozen large language models (GPT-3.5 and GPT-4). The method builds a prompt containing prior knowledge about the disease and database, detailed data specifications and formatting instructions, and optionally a single fictitious example patient, and then samples synthetic tables from the LLM. The authors evaluate the approach on PPMI Parkinson's disease data and ADNI Alzheimer's disease data, comparing it against CTGAN, TVAE, and a Gaussian copula model across fidelity, privacy, and utility metrics, with five data splits and five synthetic datasets each, plus an ablation study on prompt components. The central claim is that the method does not require access to the original patient-level data but only a description of the desired database, enabling zero-shot generation in low-resource settings.","tokens_in":18727,"tokens_out":6651,"duration_ms":63380,"significance":"If the central claim holds, the method offers a low-cost, privacy-preserving route to synthetic clinical cohorts for well-characterized diseases, which is genuinely useful for early pipeline development and educational purposes. The paper has several concrete strengths: the evaluation framework is thorough, with multiple splits, multiple synthetic datasets, and a broad metric suite covering fidelity, privacy, and utility; the prompts and code are provided openly; the ablation study isolates key prompt components; and the comparison against three established baselines is informative. The main weakness is that the evaluation setting is narrower than the abstract's headline claim: the single in-context example is derived from published averages of the same PPMI and ADNI studies that serve as ground truth, so the method as tested already receives cohort-specific aggregate information. The paper is transparent about its limitations in the Discussion, but the abstract and introduction currently overstate the scope of what is demonstrated.","major_comments":[{"comment":"The central claim that the method requires only a description of the desired database is not fully supported by the evaluated configuration. Section III.C.2 states that the single context example was created using average values of the selected variables extracted from publications on PPMI and ADNI, which are the same studies whose restricted-access data are used as ground truth in Tables I and II. For PPMI, the example comes from the PPMI2020 datacut while the evaluation uses PPMI2024, but both are part of the same study; for ADNI, the example comes from ADNI-based publications. The ablation in Table V quantifies the impact: for GPT-3.5, adding the example row raises Column Shape from 0.64 to 0.70 and ContingencySimilarity from 0.79 to 0.90. This shows that the example injects substantial cohort-specific information about marginal and joint distributions, so the method as evaluated is not purely text-to-tabular in the zero-shot sense claimed. I would ask the authors to either (a) test a configuration in which the example is not derived from the target cohort, such as an example constructed from a different disease or from expert knowledge independent of PPMI/ADNI, or (b) revise the abstract and introduction to state that the method uses a textual description plus target-cohort aggregate statistics when such statistics are available.","section":"Section III.C.2 and Tables I, II, V"},{"comment":"The Discussion explicitly acknowledges that the approach was only tested on well-documented diseases and reference databases, that rare diseases are untested, and that applying the method without access to the original patient data may require a validation framework using aggregated data from the literature or medical experts. This is an appropriate limitation, but it directly qualifies the contribution stated in the abstract, which says the method requires 'only a description of the desired database' with no such caveat. The paper should bring this scope condition into the abstract and Section I, either as a stated assumption or as an explicit limitation of the current evaluation. This is not an internal inconsistency, but the current framing makes the headline claim stronger than the evidence supports.","section":"Section V (Discussion)"},{"comment":"The ablation narrative is not fully supported by the numbers in Table V. The 'Best parameters' row, which combines GPT-4 with the example row, reports ContingencySimilarity 0.75, lower than the reference GPT-3.5 no-example value of 0.79 and substantially lower than the GPT-3.5 example-row value of 0.90. The text says that combining the elements that improved fidelity produced the best distribution shapes despite a slight deterioration in correlations, but for ContingencySimilarity the deterioration relative to the reference is not slight; it is the lowest value among all rows except the sampling-1-by-1 condition. The paper should clarify whether this is considered acceptable, and should report the full metric-by-metric comparison against the reference configuration in the text, not only in the table.","section":"Section IV.C and Table V"}],"minor_comments":[{"comment":"The metric names contain typos: 'Wassersein' should be 'Wasserstein' and 'Jenssen-Shannon' should be 'Jensen-Shannon'.","section":"Tables I and II"},{"comment":"The text writes 'Gaussain copula'; this should be 'Gaussian copula'. In addition, the baseline abbreviation 'V AE' should be written 'TVAE' consistently.","section":"Section III.D.1"},{"comment":"The WD row uses a comma as a thousands separator ('4,618±932') while all other numbers in the paper use periods; the formatting should be unified.","section":"Table II"},{"comment":"The header 'Experience' should read 'Experiment'. The 'Sampling 1 by 1' row should state explicitly whether n=1 is used for every generated table or only for the final iteration after generating m tables of size n.","section":"Table V"},{"comment":"The first column label 'SDG Model on #PPMI2024 Dtrain (Ref.)' is misleading because that column is not a synthetic-data model but the reference metric computed on the real training data; the caption should state this explicitly.","section":"Tables I and II"},{"comment":"The sentence about 'left and right putamen variables for PPMI and the ventricles and intracranial volumes for ADNI' refers to imaging variables, but this is only implicit; the text should state that these are the imaging variables shown in the supplementary distribution plots.","section":"Section IV.A"}],"recommendation":"major_revision","confidential_remarks":"The paper is well within the scope of the journal and the experimental work is solid and transparent. The main issue is that the abstract and introduction overstate the zero-shot claim: the evaluated configuration already receives target-cohort aggregate statistics through the example patient, and the authors' own Discussion admits that rare diseases and new populations are untested. This is fixable either by adding an evaluation without target-cohort-derived examples or by carefully scoping the claims. I also noted a discrepancy between the ablation text and Table V for ContingencySimilarity in the best configuration, which should be corrected. With those changes, the paper would be a reasonable contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a solid applied study of frozen-LLM synthetic tabular data generation, but the headline 'no original data' claim is softer than the abstract suggests: the single in-context example is built from published averages of the very PPMI and ADNI cohorts used as ground truth, and the ablation shows it materially improves fidelity. The stress-test note lands. The authors are transparent about it, which helps, but the zero-shot generalization to genuinely new populations is untested.\n\nWhat's new is real: a text-to-tabular setup with a frozen LLM, no fine-tuning and no patient-level data, evaluated against CTGAN, TVAE and a Gaussian copula. The evaluation framework is thorough — five splits, five synthetic sets, fidelity, privacy, utility — and the code and prompts are public. The utility numbers (TSTR F1 ~0.98 for PPMI) show the synthetic data can support a realistic prediction task, which is a genuinely useful outcome for prototyping and education.\n\nSoft spots, in proportion: first, the circularity above is the main one, but it's a limitation, not a fatal flaw. You still need some aggregate knowledge of the target population; for a rare disease with no published averages, the method's behavior is unknown. The paper itself says this in the Discussion. Second, they cite GReaT as the pioneering LLM SDG method but don't include it as a baseline; that comparison would have been informative even if expected to favor GReaT. Minor: imaging variables are visibly off, but that's consistent with their explanation about sparse literature values.\n\nWho this is for: applied health-informatics and ML readers who need cheap, quick synthetic cohorts for pipeline development or teaching. It is not a theory-changing result. The paper deserves a serious referee; I'd send it out with a request to reframe the 'no original data' claim and to add a no-example or external-example condition. The central argument is defensible under those conditions.","headline":"Frozen-LLM text-to-tabular SDG is a genuinely new zero-shot framing with thorough evaluation, but the in-context example leaks target-cohort aggregates, so the 'no original data' claim is only partially tested.","tokens_in":19324,"tokens_out":5348,"would_cite":false,"duration_ms":47839,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Text prompts alone can generate realistic synthetic patient cohorts.","keywords":["synthetic data generation","large language models","tabular data","patient data","zero-shot generation","in-context learning","clinical correlations","data privacy"],"falsifier":"Run the method on a rare or private disease cohort with no published average values in the prompt, then compare the correlation matrix and marginal distributions of the synthetic cohort against the real cohort; the claim fails if strongly coupled clinical variables show near-zero correlation in the synthetic data while the real data show strong correlation.","tokens_in":18310,"feed_emoji":"🩺","tokens_out":6207,"duration_ms":60964,"temperature":0.7,"pith_summary":"The paper proposes generating synthetic tabular patient data with a large language model that never sees the original dataset. Instead, the model receives a text description of the desired database, detailed column definitions, and one fictitious patient example built from published average values, and it produces whole tables of patients. The authors test this on Parkinson's and Alzheimer's cohorts and report that the generated patients preserve the shape of the real distributions and, notably, the correlations between clinical variables, though less accurately than models trained on the real data. The payoff would be a low-skill, zero-shot way to fabricate realistic patient cohorts for pilot studies, pipeline development, and teaching, without the privacy hurdles of sharing real records.","feed_headline":"No patient records needed: LLMs generate realistic synthetic cohorts","feed_subtitle":"A text description and one example patient yield cohorts that preserve clinical correlations for Parkinson's and Alzheimer's.","key_machinery":"The carrying mechanism is the structured prompt, assembled from three parts: prior knowledge describing the medical database and disease, instructions giving column names, types, and category encodings plus the requested row count and JSON format, and one fictitious example patient constructed from published average values of the same cohort. The prompt is sent to GPT-3.5 or GPT-4 at temperature 1; because LLMs truncate long repetitive outputs, the desired cohort is generated as many small tables of ten rows, with column order randomly permuted at each generation. The single example patient is the load-bearing context that anchors the model to the target population's typical values.","core_discovery":"On the paper's own terms, the discovery is that a frozen, off-the-shelf LLM prompted only with a database description and a single prototypical patient row generates synthetic cohorts whose univariate distributions and pairwise clinical correlations are broadly faithful to real restricted-access cohorts. Compared with CTGAN, TVAE, and a Gaussian copula trained on the original training splits, the LLM's fidelity scores are lower overall, but in several correlation metrics it is competitive with or better than CTGAN, and a classifier trained on the synthetic data reaches test F1 scores comparable to one trained on real data. Because the original patient-level data is never input to the model, the method carries a structural privacy guarantee that data-dependent generators do not.","pith_inferences":["We infer that the method's fidelity will degrade sharply for rare diseases or populations with no published summary statistics, because the single anchor example then cannot represent the target distribution.","We infer that the same prompt structure could be extended to longitudinal or multimodal outputs, since the LLM is already conditioned on text and could emit sequences of records or nested JSON.","We infer that a testable next step is to use a private benchmark dataset whose aggregate statistics are withheld, prompting only with disease knowledge, to measure how much of the fidelity is carried by the anchor example rather than by the model's pretrained knowledge.","We infer that the Wasserstein-distance gaps on imaging variables will persist unless the prompt provides external calibration values for those measures, since the model tends to produce prototypical values with low variance."],"forward_implications":["Synthetic cohorts can be produced before any real data is in hand, so ML pipelines can be built and tested in parallel with data-access approvals.","Because no patient-level record is ever passed to the model, the generated data is structurally private rather than relying on differential-privacy machinery.","The method transfers to a second disease with a different variable set, suggesting the prompt template generalizes across well-documented conditions.","In the train-synthetic-test-real evaluation, classifiers trained on the synthetic Parkinson's data scored as well as classifiers trained on real data, indicating the synthetic cohort can stand in for real data in at least some prediction tasks.","The ablation shows that upgrading the model and adding one example patient are the prompt changes that matter most for distribution-shape fidelity."],"supporting_citations":[{"why":"Establishes that pre-trained LLMs can serve as tabular data generators; the paper adopts its frozen-LLM setup and column-permutation strategy.","marker":"[11]"},{"why":"Introduces CTGAN and TVAE, the deep generative baselines the paper compares against, and supplies their training configuration.","marker":"[18]"},{"why":"Introduces the Gaussian copula baseline used as the strongest fidelity comparator.","marker":"[16]"},{"why":"Provides the published average Parkinson's cohort values from which the single in-context example patient is constructed.","marker":"[35]"},{"why":"Provides published Alzheimer's imaging averages used to build the context example.","marker":"[36]"},{"why":"Supplies published Alzheimer's clinical averages for the context example.","marker":"[37]"},{"why":"Supplies the fidelity and privacy metrics (Column Shape, Column Pair Trend, DCR, NNDR) used in the evaluation.","marker":"[38]"},{"why":"Defines the nearest-neighbour distance ratio privacy metric used to assess how close synthetic patients sit to real records.","marker":"[41]"},{"why":"Motivates the choice of restricted-access cohorts by showing LLMs can memorize tabular training data.","marker":"[34]"}],"fun_headline_variants":["LLMs create patient data from a text description alone","No patient records needed for LLM synthetic cohorts","Text-to-tabular LLM yields realistic synthetic patient data","LLM generates synthetic patients preserving clinical correlations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that one fictitious patient assembled from published average values of the target cohort, plus the model's pretrained biomedical knowledge, is enough to recreate the joint distribution of the real database; for a population with no published averages or a different distribution, the anchor no longer holds.","fun_headline_variants_meta":{"raw":{"variants":["LLMs create patient data from a text description alone","No patient records needed for LLM synthetic cohorts","Text-to-tabular LLM yields realistic synthetic patient data","LLM generates synthetic patients preserving clinical correlations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000291,"raw_usage":{"total_tokens":1689,"prompt_tokens":924,"completion_tokens":765,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":703}},"tokens_in":540,"tokens_out":765,"duration_ms":6978,"temperature":1.0,"reasoning_tokens":703,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:49:06.127779+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the method on a rare or private disease cohort with no published average values in the prompt, then compare the correlation matrix and marginal distributions of the synthetic cohort against the real cohort; the claim fails if strongly coupled clinical variables show near-zero correlation in the synthetic data while the real data show strong correlation.","supporting_citations":[{"cited_title":"The Parkinson’s progression markers initiative (PPMI) – establishing a PD biomarker cohort,","cited_arxiv_id":null,"evidence_quote":"Provides the published average Parkinson's cohort values from which the single in-context example patient is constructed."},{"cited_title":"A Comparison of Accelerated and Non-accelerated MRI Scans for Brain V olume and Boundary Shift Integral Measures of V olume Change: Evidence from the ADNI Dataset,","cited_arxiv_id":null,"evidence_quote":"Provides published Alzheimer's imaging averages used to build the context example."},{"cited_title":"Identification of mild cognitive impairment subtypes predicting conversion to Alzheimer’s disease using multimodal data,","cited_arxiv_id":null,"evidence_quote":"Supplies published Alzheimer's clinical averages for the context example."},{"cited_title":"[Online]","cited_arxiv_id":null,"evidence_quote":"Supplies the fidelity and privacy metrics (Column Shape, Column Pair Trend, DCR, NNDR) used in the evaluation."}],"review_version":1}