{"id":"0f38a646-4e5c-4b7b-9f4f-3a10295d7c71","arxiv_id":"2509.10882","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Term2Note synthesizes differentially private clinical notes by generating section-wise text conditioned on privately perturbed medical terms, achieving fidelity and downstream utility close to real notes.","lead":"Term2Note is a method for creating synthetic hospital discharge notes that carry formal privacy guarantees. It separates medical terms from writing style, adds privacy noise to the terms, and uses a language model to write new notes that preserve structure and vocabulary while hiding individual patients.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DPRP* is asserted, not proven: Algorithm 2 has no sensitivity bound or noise calibration, its output line is dimensionally ill-formed, and the Appendix proof assumes rather than establishes the (epsilon_t, delta_t) guarantee that the central claim depends on.","rationale":"I agree with the reader that the weakest point is the unproved DP-ness of DPRP*. The paper's evaluations (fidelity, ICD utility, physician preference) are extensive and the method appears practically promising, but empirical quality cannot establish a formal privacy guarantee. The Appendix's proof is a lemma about parallel composition, not a proof that DPRP* is DP; M2's DP status is simply asserted. I add two sharper observations that make the concern concrete rather than vague: the pseudocode is dimensionally invalid, and the Gaussian noise added to E and E^T E has no stated sensitivity bound, so the (epsilon_t, delta_t) numbers in Table 1 are not connected to actual noise scales. There are secondary formal gaps too (section titles and segmentation are derived from the private note and used as conditioning without perturbation; per-note DPRP* would compose across records), but the DPRP* issue alone is sufficient to make the central claim unsupported. Because the mechanism could in principle be repaired by clipping embeddings, supplying a correct projection expression, and proving sensitivity, the appropriate verdict remains CONDITIONAL rather than REJECT: accept only after the missing DP analysis is supplied and the corrected mechanism is re-run. This matches the reader's conditional verdict, so no adjustment is needed.","tokens_in":21804,"tokens_out":13498,"duration_ms":127061,"concrete_test":"Request a self-contained DP proof of Algorithm 2 with four named ingredients: (a) the exact row/column sensitivity of the embedding query under a stated norm clip (e.g., L2 clip of each MedEmbed vector to C); (b) closed-form sigma_1, sigma_2 derived from the 0.85/0.15 budget split via the Gaussian mechanism; (c) a corrected, dimensionally valid output expression for E_DP; and (d) an explicit accounting of composition across all released notes and sections, not just the D_train/D_test split. If the authors cannot provide (a)-(d), or if the corrected mechanism changes the reported (epsilon, delta), the headline privacy claim should be revised. An independent re-derivation of line 7 alone would already reveal whether the published algorithm is reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Term2Note's central claim is a formal (epsilon, delta)-DP guarantee for the released notes. The guarantee has two components: FastDP training of NOTEGEN (standard, and not the issue) and the adapted DPRP* mechanism used to privatise term embeddings. Appendix B proves a parallel-composition lemma and then simply states that M2 = 'TERMGEN processing on D_test' is (epsilon_t, delta_t)-DP. The actual mechanism, Algorithm 2, is never given a DP proof. This is not a cosmetic omission. (1) Algorithm 2 line 7, 'E_DP = E' V'_k^T + V'_k^T', is not a well-defined matrix expression for the stated shapes (E' is n x d, V'_k is d x k), so the algorithm as published cannot be implemented without guessing the intended operation. (2) No sensitivity bound is stated for the term embeddings. Gaussian noise is added to E and to E^T E; the Gaussian mechanism is only DP when the query has finite L2 sensitivity. The embeddings are produced by MedEmbed-large and are not clipped or norm-bounded in the paper, so the sensitivity of E^T E is unbounded and no finite sigma_1, sigma_2 can be calibrated from (epsilon_t, delta_t) alone. (3) The proof also does not account for composition across the released synthetic notes if DPRP* is applied note-by-note with a per-section budget; parallel composition between D_train and D_test does not address sequential composition over the N output records. Because the overall bound (max(epsilon_n, epsilon_t), max(delta_n, delta_t)) in Eq. (6) rests directly on the unproved DP-ness of DPRP*, the formal privacy claim is unsupported as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Term2Note, a framework for differentially private synthesis of full-length clinical notes. The method splits a source note into sections, extracts SNOMED-CT terms, privatizes the term embeddings with an adapted DPRP* mechanism, generates synthetic terms from the private embeddings, and then generates each note section with a language model fine-tuned under DP (FastDP). A DP quality maximiser selects among several candidate notes using an external perplexity model. Experiments on MIMIC-III/MIMIC-IV compare Term2Note against AUG-PE and FastDP baselines across fidelity, downstream ICD-coding utility, and human preference, reporting consistent improvements. The paper claims an overall (max(ε_n, ε_t), max(δ_n, δ_t))-DP guarantee for the full pipeline.","tokens_in":22204,"tokens_out":4715,"duration_ms":43927,"significance":"If the formal privacy claim were established, Term2Note would be a meaningful advance: it is, to my knowledge, the first attempt at DP synthesis of full-length clinical notes, and the empirical evaluation is broad, covering multiple privacy budgets, two base models, downstream task utility, and human clinical evaluation. The method is also refreshingly explicit about separating content (terms) from form (sections), and the reported fidelity improvements over strong baselines are substantively interesting. The supplied manuscript is transparent about many experimental details, including hyperparameters and supplementary ablations, which aids reproducibility. However, the central formal claim — the (ε, δ)-DP guarantee — depends on a privacy mechanism (DPRP*) whose DP property is asserted rather than proved, and whose pseudocode is not well-defined as written. Because this underpins the main contribution, the paper in its current form does not yet deliver the promised formal guarantee.","major_comments":[{"comment":"The DP guarantee of DPRP* is asserted, not proven. The paper never gives a proof that Algorithm 2 is (ε_t, δ_t)-DP when applied to clinical term embeddings, nor does it state a sensitivity bound for the embedding vectors or for the covariance query E^T E. The embeddings come from MedEmbed-large and are not clipped or norm-bounded, so the L2 sensitivity of E^T E is not finite; consequently the Gaussian noise scales σ_1 and σ_2 in Algorithm 2 cannot be calibrated from (ε, δ) alone. Since Eq. (6) and the overall privacy claim both rest on the assumption that the TERMGEN component is (ε_t, δ_t)-DP, this missing analysis is load-bearing. The authors must supply a rigorous privacy proof (or, if the mechanism is not DP, revise the claim) and specify the sensitivity/noise calibration explicitly.","section":"§3.3, §3.6, Appendix A, Algorithm 2"},{"comment":"Line 7 of Algorithm 2, 'E_DP = E' V'_k^T + V'_k^T', is not a well-formed matrix expression as written: E' is n×d and V'_k is d×k, so E'V'_k^T is n×k, while V'_k^T is k×d; the two matrices cannot be added. The parenthetical comment that '+' refers to the Moore-Penrose pseudoinverse does not resolve the dimensional mismatch (the pseudoinverse of V'_k^T would be k×d, not n×k). As published, the algorithm is not reproducible, and the intended reconstruction operation must be clarified and stated in unambiguous notation.","section":"Appendix A, Algorithm 2, line 7"},{"comment":"The composition argument does not correctly account for the unit of privacy. The paper treats the sections of a single note as if they were independent records when it scales the term-generation budget by 1/m, but the neighbouring relation in DP is defined at the level of the patient note. DPRP* is applied to the entire matrix of section embeddings of one note; replacing one note changes all m rows of E, so the sensitivity of the covariance query E^T E is not the sensitivity of a single row. The proof in Appendix B also assumes that M2 = 'TERMGEN processing on D_test' is (ε_t, δ_t)-DP without establishing this. A correct analysis must specify the neighbouring relation, derive the sensitivity of the full-note query, and show how the per-section budget split follows from that sensitivity.","section":"§3.6, Appendix B"},{"comment":"The downstream utility evaluation is based on only 204 test notes with 80:20 five-fold cross-validation, and the reported standard deviations are large. For example, at ε=8, Term2Note with ε_t=∞ has macro F1 21.28±1.28 while FastDP has 16.40±4.01; at the same privacy cost, Term2Note with full privacy (ε_t=8) has macro F1 26.50±3.95. Several pairwise differences appear within one or two standard deviations of the baselines. The claim that Term2Note 'consistently outperforms' baselines would be strengthened by significance testing or by reporting effect sizes with confidence intervals; at present, the broad trend is clear, but the magnitude of the utility advantage is not precisely established.","section":"§4, Table 1 and Table 7"}],"minor_comments":[{"comment":"The phrase 'we preform preference sampling' contains a typo; it should read 'perform'.","section":"§3.5"},{"comment":"Equation (2) and the surrounding text use σ_t for the DPRP* noise scale, but Algorithm 2 parameterizes the noise as σ_1 and σ_2 derived from (ε_1, δ_1) and (ε_2, δ_2). Please reconcile these notations and clarify which privacy split (b=0.85) corresponds to which noise term.","section":"§3.3, Eq. (2)"},{"comment":"The baseline name is written inconsistently as 'FastDP' in the text, 'Fast-DP' in Table 1, and 'Fast-DP' in the supplementary tables. Please use one convention throughout.","section":"Table 1 and throughout"},{"comment":"The section title 'Discharge Disposition' is missing a closing quote in the table; more generally, the section grouping table would be easier to read if the entries were formatted consistently.","section":"Appendix C, Table 2"},{"comment":"The membership-inference distance analysis is described as 'preliminary' and provides only an indirect sanity check. The caption of Figure 5 should note that the distance distributions are not a formal privacy guarantee and that the analysis is subject to the limitations the authors themselves mention (e.g., no canary-based lower bound).","section":"Appendix F"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical systems contribution, but the formal privacy proof is the linchpin of the submission and it is currently missing. If the authors can provide a correct, self-contained proof of DP for the adapted DPRP* mechanism (including clipping or boundedness of embeddings, sensitivity calibration, and a correct composition argument), the paper would be a strong candidate for acceptance. I would also encourage the editor to consider whether the small test set and the lack of significance testing weaken the empirical claims enough to require an additional evaluation, though this is secondary to the privacy proof issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, Term2Note is the first pipeline I've seen that generates full-length clinical notes under a claimed DP guarantee, and the empirical work is unusually thorough: section-wise generation, term-level DP conditioning, separate budgets, a quality maximizer, and evaluation on fidelity, downstream ICD coding, and physician preference. The results are consistently ahead of the baselines. Second, the formal privacy guarantee is the whole point, and it is unsupported as written. The adapted DPRP* mechanism (Algorithm 2) is asserted to be differentially private, but there is no proof, no sensitivity bound on the MedEmbed term embeddings, and no noise calibration. Line 7 of the algorithm is dimensionally ill-formed—E' V'_k^T + V'_k^T doesn't type-check under the stated shapes. The appendix proof establishes a parallel composition lemma and then simply states that TERMGEN processing is (epsilon_t, delta_t)-DP. That's a gap you could drive a truck through.\n\nWhat's genuinely good: the domain is important and under-served; prior DP text work looked at short reviews or PubMed abstracts, not long unstructured clinical notes. The section-wise decomposition with separate privacy budgets is a sensible idea, and the perplexity-based selection is a smart post-processing step that correctly preserves any underlying DP guarantee. The evaluation design is largely clean—held-out SNOMED test set, train on synthetic and test on real, no obvious test-set fitting. The human evaluation with physicians is a plus.\n\nWhere it's soft: the DPRP* problem is load-bearing. If that mechanism is not DP, the headline claim collapses. Secondary issues: the utility test set is only 204 notes with ICD codes collapsed to 20 chapter-level groups, so the utility numbers are suggestive, not definitive. The composition argument only explicitly covers D_test as the private input to TERMGEN, which is confusing if the released synthetic dataset is meant to cover the whole private corpus. And the physician reviewers found real clinical errors, so it's not production-ready.\n\nThe authors are honest about the clinical quality limits and describe the privacy evaluation as preliminary, which I credit. But the missing DP proof is not a minor omission.\n\nWho should read it: anyone working on DP text generation or privacy-preserving clinical NLP will want to see this, both for the architecture and as a case study in how a promising pipeline can outrun its privacy analysis. It deserves a serious referee, but the referee's first request has to be a real privacy proof for DPRP* (or a replacement mechanism with bounded sensitivity). I would not accept it in current form.\n\nRead this to the reading group; it's a good example of evaluating a method when the central claim is unproven.","headline":"Solid engineering, thorough evaluation, but the privacy proof is missing—this is a conditional accept at best.","tokens_in":22770,"tokens_out":7295,"would_cite":false,"duration_ms":64214,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Term2Note synthesises long clinical notes under formal differential privacy by separating medical terms from note text, and reports that the synthetic notes preserve structure and downstream ICD-coding utility nearly as well as real notes.","keywords":["differential privacy","synthetic clinical notes","medical term extraction","ICD coding","DP-SGD","clinical note generation","privacy-preserving data sharing","MIMIC"],"falsifier":"Run the DPRP* algorithm on two neighbouring term lists that differ in exactly one medical term, embed both with the clinical embedder, and compare the resulting output distributions; if any measurable output set has a probability ratio exceeding $e^{\\epsilon}$ (plus $\\delta$) at the claimed parameters, the privacy claim is refuted. A simpler check is to compute the global sensitivity of the embedding function and verify whether the noise scale in Algorithm 2 is large enough; the paper reports no such calculation.","tokens_in":21628,"feed_emoji":"🩺","tokens_out":6528,"duration_ms":54349,"temperature":0.7,"pith_summary":"Term2Note tackles the problem of sharing sensitive clinical notes by generating synthetic discharge notes under formal differential privacy, a guarantee that no single patient's data can change the output distribution beyond a controlled bound. The method splits each note into sections, extracts medical terms per section, privatises those terms with a DP mechanism, and trains a note generator under DP to write section text conditioned on the privatised terms. A quality maximiser selects the most fluent candidate note as post-processing, so the privacy guarantee is unchanged. The paper reports that the synthetic notes closely match real notes in length distribution, term overlap, and semantic alignment, and that ICD-code classifiers trained on them perform close to classifiers trained on real notes.","feed_headline":"Synthetic clinical notes stay useful under differential privacy","feed_subtitle":"Term-conditioned generation preserves note structure and ICD-coding accuracy at epsilon as low as 2.","key_machinery":"The load-bearing object is a section-wise, term-conditioned generation pipeline. SECSPLIT standardises a clinical note into at most six sections; TERMEXT extracts SNOMED CT terms per section; the terms are embedded and perturbed by DPRP*, an adaptation of the random-projection DP mechanism that adds noise to the embeddings and their covariance matrix, performs SVD, and reconstructs from noisy singular vectors; a GPT-2 term generator produces synthetic terms; and a DP-trained Llama-3.2-1B or Gemma-3-1B note generator writes each section conditioned on the synthetic terms and previously generated sections, trained with FastDP-based DP-SGD. A perplexity-based quality maximiser selects the lowest-perplexity candidate note. The privacy proof combines the post-processing property and parallel composition over the disjoint term-generation and note-generation components.","core_discovery":"The central claim is that long-form clinical note synthesis under DP becomes tractable when content and form are structurally separated: privatise the clinically salient terms that carry patient-specific information, then let a DP-trained language model generate section-wise free text conditioned on those terms. When both terms and notes are privatised, the overall privacy loss is bounded by $(\\max(\\epsilon_n, \\epsilon_t), \\max(\\delta_n, \\delta_t))$ via parallel composition and post-processing; when only notes are privatised, the loss is $(\\epsilon_n, \\delta_n)$. Empirically, Term2Note achieves the lowest KL divergence in length and term distributions and the highest MAUVE scores among DP baselines, and its ICD-coding utility remains close to real-data performance even at a strict budget of $\\epsilon = 2$.","pith_inferences":["Editorial inference: the formal privacy statement for the term path depends entirely on DPRP* being differentially private for the clinical embedder's outputs, which the paper does not prove; establishing a sensitivity bound for that embedding function would make the full-pipeline guarantee unconditional.","Editorial inference: the section-wise separation of content and form could transfer to other structured clinical documents such as radiology reports, operation notes, or discharge summaries from other institutions, since the mechanism is tied to section titles and SNOMED CT terms rather than to MIMIC formatting.","Editorial inference: the physicians' qualitative observations about medication misclassification and internal inconsistency suggest that safety-critical use of such synthetic notes would require a clinical fact-checking stage before deployment; this is an extension beyond the paper's claims.","Editorial inference: the utility evaluation collapses ICD codes into 20 broad groups, so a finer-grained ICD coding benchmark would be a stricter test of whether the synthetic fidelity translates to clinically specific prediction accuracy."],"forward_implications":["If the central claim holds, hospitals and research consortia could share synthetic discharge notes with formal DP guarantees instead of raw notes, enabling multi-site machine learning without direct transfer of patient data.","The term-level privacy budget gives a tunable control: the same DP-trained note generator can be paired with different $\\epsilon_t$ values to trade clinical content fidelity against privacy strength.","ICD-code classifiers trained on synthetic notes reach performance close to those trained on real notes, suggesting that other structured clinical NLP tasks may also transfer from such synthetic corpora.","Because the quality maximiser is a post-processing step, any selection or filtering rule applied to candidate notes preserves the privacy guarantee, making inference-time quality control a free privacy-safe lever."],"supporting_citations":[{"why":"Supplies the definition of $(\\epsilon,\\delta)$-differential privacy that all guarantees in the paper are measured against.","marker":"(Dwork et al. 2006)"},{"why":"Provides the post-processing theorem used to argue that the quality maximiser and downstream use of synthetic notes preserve privacy.","marker":"(Dwork and Roth 2014)"},{"why":"Provides the parallel composition theorem used to combine term and note privacy budgets into the overall $(\\max(\\epsilon_n,\\epsilon_t), \\max(\\delta_n,\\delta_t))$ guarantee.","marker":"(McSherry 2009)"},{"why":"Source of the DPRP random-projection DP mechanism that the paper adapts as DPRP* for term embeddings.","marker":"(Gondara and Wang 2020)"},{"why":"Introduces DP-SGD, the foundation for the private training of the note generation model.","marker":"(Abadi et al. 2016)"},{"why":"FastDP is the efficient DP-SGD implementation used to fine-tune the note generator under DP budgets.","marker":"(Bu et al. 2023)"},{"why":"MIMIC-III discharge notes form the public dataset used to train the term generation model without privacy constraints.","marker":"(Johnson et al. 2016)"},{"why":"MIMIC-IV notes form the private dataset used for DP training of the note generator and for evaluating downstream ICD-coding utility.","marker":"(Johnson et al. 2023)"},{"why":"AUG-PE is the most relevant DP text generation baseline, adapted here to clinical notes.","marker":"(Xie et al. 2024)"},{"why":"DP-SGD with control codes is the other DP text generation baseline, adapted and compared against.","marker":"(Yue et al. 2023)"}],"fun_headline_variants":["Privatize terms, not free text: DP clinical notes that work","DP clinical notes: keep terms private, text follows","Synthetic notes with DP: split terms, keep ICD utility","Term2Note: separate content and form for DP notes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The term-privatisation step DPRP* is assumed to be differentially private when applied to term embeddings, but the paper supplies no proof, no sensitivity bound for the embedding vectors, and no noise calibration; if that step is not DP, the formal privacy guarantee for the entire pipeline fails.","fun_headline_variants_meta":{"raw":{"variants":["Privatize terms, not free text: DP clinical notes that work","DP clinical notes: keep terms private, text follows","Synthetic notes with DP: split terms, keep ICD utility","Term2Note: separate content and form for DP notes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000905,"raw_usage":{"total_tokens":3886,"prompt_tokens":933,"completion_tokens":2953,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":2883}},"tokens_in":549,"tokens_out":2953,"duration_ms":17605,"temperature":1.0,"reasoning_tokens":2883,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:52:07.563997+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the DPRP* algorithm on two neighbouring term lists that differ in exactly one medical term, embed both with the clinical embedder, and compare the resulting output distributions; if any measurable output set has a probability ratio exceeding $e^{\\epsilon}$ (plus $\\delta$) at the claimed parameters, the privacy claim is refuted. A simpler check is to compute the global sensitivity of the embedding function and verify whether the noise scale in Algorithm 2 is large enough; the paper reports no such calculation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the definition of $(\\epsilon,\\delta)$-differential privacy that all guarantees in the paper are measured against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the post-processing theorem used to argue that the quality maximiser and downstream use of synthetic notes preserve privacy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the parallel composition theorem used to combine term and note privacy budgets into the overall $(\\max(\\epsilon_n,\\epsilon_t), \\max(\\delta_n,\\delta_t))$ guarantee."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the DPRP random-projection DP mechanism that the paper adapts as DPRP* for term embeddings."},{"cited_title":"J.; McMahan, H","cited_arxiv_id":null,"evidence_quote":"Introduces DP-SGD, the foundation for the private training of the note generation model."},{"cited_title":"A.; Nori, H.; Jiang, H.; Zhang, H.; Lee, Y","cited_arxiv_id":null,"evidence_quote":"AUG-PE is the most relevant DP text generation baseline, adapted here to clinical notes."},{"cited_title":"A.; Li, X.; Kumar, G.; McAnallen, J.; Shajari, H.; Sun, H.; Levitan, D.; and Sim, R","cited_arxiv_id":null,"evidence_quote":"DP-SGD with control codes is the other DP text generation baseline, adapted and compared against."}],"review_version":2}