{"id":"ff31d195-77f2-474a-b0bf-2d2df4868c4a","arxiv_id":"2412.20641","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A study of an LLM-based synthetic data pipeline with noise injection that claims differential privacy, evaluated on news classification, but with no valid privacy analysis.","lead":"The paper proposes using large language models to create synthetic text data with noise added, claiming this provides differential privacy. It reports that machine learning models trained on this synthetic data lose some accuracy, but it never actually demonstrates a formal privacy guarantee.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never defines a DP mechanism on the original data; noise is added after the LLM has already used sensitive examples, so no differential privacy guarantee follows from the experiments.","rationale":"The reader's weakest_assumption identifies the absence of a sensitivity bound, a mechanism definition, and privacy accounting, and I agree. The more precise formulation is that the noise is applied after the LLM has already consumed the in-context examples, so the released text distribution can depend on a single original record in an unquantified way. Adding Laplace or Gaussian noise to token frequencies of the generated text does not make the map from original dataset to synthetic corpus differentially private unless the generation step itself is private, and the paper provides no argument that it is. This is the single most load-bearing concern because every privacy-related claim in the abstract, Section 5.4, and the conclusion depends on it; the utility numbers, even if reproduced, would not establish a privacy-preserving method without a formal guarantee. The paper does include useful concrete artifacts, such as prompt templates and model configurations in Appendix B, but these do not mitigate the missing privacy analysis. The reader's REJECT verdict is appropriate, so I would keep it unchanged.","tokens_in":15333,"tokens_out":2629,"duration_ms":30034,"concrete_test":"Formally specify the algorithm M(D) as: choose k in-context examples from D (state the selection rule), call the LLM to produce n synthetic texts, compute a word-frequency vector, and add Laplace noise with scale Delta/epsilon. Derive Delta as the maximum L1 change in the frequency vector over all neighboring datasets D and D' and all LLM outputs, including possible verbatim or paraphrased reproduction of an example. Then verify the claimed epsilon by running M many times on D and on D' that differ in one in-context example and testing whether the output distributions satisfy the DP bound for the stated epsilon; a likelihood-ratio or classifier distinguishability test that exceeds the bound would falsify the privacy claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract: 'integrating DP within LLM-driven synthetic data generation offers a viable balance between privacy protection and data utility') requires that the released synthetic corpus is the output of an epsilon-DP algorithm applied to the original dataset. Nothing in Sections 3.1 or 5.4 establishes this. The pipeline first feeds in-context examples from the original data to gpt-4o-mini and generates text; only then does it 'manipulate token frequencies in the generated text by adding carefully calibrated noise' (Section 3.1). That is post-processing of an already non-private generation step. DP's post-processing property preserves privacy only for the output of a private algorithm; it cannot retroactively privatize a step that directly used the sensitive examples. No sensitivity bound is stated: the text says noise is 'based on the Laplace and Gaussian distribution equation' but never defines the query, its L1/L2 sensitivity, or a privacy accounting. Table 3 lists epsilon values but no mechanism or proof ties them to formal DP, and epsilon = 0 would imply infinite noise. Consequently the paper's privacy assertion is unsupported, and the membership inference evaluation promised in the abstract is absent. This is internal to the argument, not merely outside consensus: a DP claim requires the algorithm's randomized mapping from neighboring datasets to be close, and the described algorithm is not shown to have that property.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SafeSynthDP, a training-free pipeline that uses gpt-4o-mini with in-context examples from a private dataset to generate synthetic text, then injects Laplace or Gaussian noise into token frequencies of the generated text, claiming this yields differentially private synthetic data. The utility of the resulting synthetic data is evaluated by training MNB, SVM, GRU, and LSTM classifiers and by using the synthetic data as demonstrations in in-context learning with gpt-4o-mini and gemini-1.5-flash, all on a sampled subset of AGNews. The reported results show moderate accuracy losses relative to original data, and the paper interprets these as evidence of a viable privacy-utility trade-off controlled by a privacy budget epsilon.","tokens_in":15564,"tokens_out":2903,"duration_ms":29030,"significance":"If the central privacy claim were established, the paper would offer a lightweight, LLM-based alternative to model-centric DP synthetic data generation, with potential value for text classification in privacy-sensitive domains. The paper does provide a concrete evaluation across multiple model families and two LLMs, and it describes its prompts and hyperparameters in appendices, which aids reproducibility of the utility experiments. However, the entire contribution hinges on the assertion that the described noise injection provides differential privacy for the original dataset, and that assertion is not supported by any mechanism definition, sensitivity analysis, or privacy accounting. The absence of the promised membership-inference evaluation and the lack of any statistical uncertainty in the results further weaken the empirical claims. The paper therefore cannot currently be considered a valid demonstration of privacy-preserving synthetic data generation.","major_comments":[{"comment":"The described pipeline feeds in-context examples from the original dataset to gpt-4o-mini, generates text, and only then adds Laplace or Gaussian noise to token frequencies in the generated text. Differential privacy is a property of a randomized algorithm whose input is the original dataset; post-processing can only preserve privacy if the preceding step is already private. Here, the LLM generation step directly used sensitive examples, so no epsilon-DP guarantee for the original dataset follows from adding noise to already-generated text. The paper never defines the query, its L1/L2 sensitivity, or the noise calibration that would connect the reported epsilon values to a formal DP guarantee, and Section 5.4 does not supply this missing analysis instead of asserting it.","section":"Section 3.1, \"Privacy Enhancement through DP Mechanisms\""},{"comment":"The row labeled \"LLM ICL (epsilon = 0)\" reports an accuracy of 69.83%. Under the standard Laplace or Gaussian DP mechanism, epsilon = 0 corresponds to perfect privacy and hence infinite noise, which would drive accuracy toward random chance (approximately 25% for four classes). A near-normal accuracy at epsilon = 0 either indicates that the privacy budget was not implemented as described or that the epsilon values are labels not attached to any actual DP mechanism. This is a load-bearing inconsistency because the claimed privacy-utility trade-off is the paper's central empirical contribution.","section":"Table 3 and Section 5.3"},{"comment":"The abstract states that the paper assesses \"the resilience of the generated synthetic data to membership inference attacks and related threats,\" and Section 5.4 asserts that the noise protects against membership inference, but no membership-inference experiment appears anywhere in the manuscript. The absence of this promised evaluation is critical because the privacy claim rests entirely on the assertion that noise prevents membership inference, and that assertion is tested nowhere.","section":"Abstract and Section 5.4"},{"comment":"All reported accuracies are single-run point estimates with no standard deviations, no multiple seeds, and no statistical significance testing. Given the small scope (one dataset, 12,000 training samples), the differences between original and synthetic data and across epsilon values cannot be distinguished from random variation. The quantitative claims about the privacy-utility trade-off are therefore not supported by the reported experiments.","section":"Tables 1, 2, 3 and Section 5"},{"comment":"The Limitations section explicitly states that \"If the noise added is not appropriately calibrated for the dataset or if new, more sophisticated attack methods emerge, the privacy protections might be undermined.\" This concedes exactly what the paper elsewhere assumes: that the noise is appropriately calibrated. Since no calibration or sensitivity analysis is provided, the paper's own limitation statement reinforces the conclusion that the DP guarantee is not established.","section":"Limitations (Section 6, final paragraph before References)"}],"minor_comments":[{"comment":"The sentence \"in highly sensitive contexts, we would choose a higher ϵ to maintain data fidelity while still offering privacy protection\" reverses the standard convention; higher epsilon means weaker privacy and higher fidelity, not the other way around.","section":"Section 3.1"},{"comment":"The paper states that \"we focused on reporting results for the AGNews dataset\" but does not describe any other dataset or explain why results for other datasets are omitted; this phrasing should be clarified.","section":"Section 4.1"},{"comment":"The prompt text contains the typo \"Bussiness\" (instead of \"Business\") and the later ICL prompt contains \"follwoing\" (instead of \"following\"); these should be corrected.","section":"Section B.3"},{"comment":"The interpretation of accuracy above 50% as evidence that the synthetic data \"captured some meaningful signals\" is not supported by a comparison to a random baseline on the AGNews test set; such a baseline should be reported.","section":"Section 4.4"},{"comment":"The ICL prompt shown in the appendix uses synthetic examples with clearly invented headlines (e.g., \"Breakthrough in Renewable Energy Technology\"), but the main text does not describe how the demonstrations were selected from the synthetic corpus; this selection procedure should be stated.","section":"Section B.4"}],"recommendation":"reject","confidential_remarks":"The central problem is not a matter of presentation or a missing experiment; the algorithm as described is not a differential privacy mechanism for the original dataset, so the paper's main claim cannot be fixed within its current scope. The self-citations (Nahid and Rafiei 2024a,b; Nahid 2024) are frequent but are used as background and do not themselves constitute a circularity problem. The paper might be resubmitted as a study of LLM-based synthetic data utility without the DP claim, but as a privacy contribution it does not meet the standard for the journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this paper claims to generate differentially private synthetic text with LLMs by adding Laplace or Gaussian noise to token frequencies in the generated text. The claim does not hold. There is no DP mechanism defined anywhere: no sensitivity, no calibration equation, no composition analysis. Worse, the noise is applied after the LLM has already consumed the sensitive in-context examples. Post-processing a non-private generation step cannot retroactively privatize it. The abstract also promises a membership inference attack evaluation; none appears in the paper. The central privacy guarantee is unsupported.\n\nWhat is worth something: the paper is clearly written, the experimental setup is described in enough detail to follow (dataset split, model choices, hyperparameters in appendices), and the authors do report a real utility drop for sequence models, which is honest. The limitations section even acknowledges that noise might not be appropriately calibrated and that privacy could be undermined. That paragraph is closer to the truth than the rest of the paper. But the paper does not flag that the method itself, as described, cannot provide differential privacy at all.\n\nThe novelty is weak. The related work cites Tang et al. (2024) and Hong et al. (2024), which already cover differentially private few-shot generation and DP for LLM in-context learning. What is left is a specific noise-on-token-frequencies step, which is not analyzed and does not constitute a new mechanism.\n\nThe experiments are single-run point estimates on one dataset (AGNews subset). No error bars, no code or data shipped. The ICL results show synthetic data underperforms original, and the gap shrinks with more shots, which is plausible but not a privacy result.\n\nWho is this for? Possibly someone looking for a survey of pitfalls in applying DP to LLM synthetic data, but not as a citable method. I would not send this to peer review as is; it needs a fundamental rework of the privacy analysis, and even then the method would need to be reformulated so that the generation itself is private. A serious referee would spend their time explaining why the current approach cannot work, which is not a productive use of the review process.\n\nRecommendation: desk reject, with an invitation to resubmit after fixing the privacy analysis and actually running a membership inference attack.","headline":"A clearly written paper whose central privacy claim is unsupported: noise added after LLM generation cannot provide differential privacy, and no mechanism or MIA evaluation is ever defined.","tokens_in":16114,"tokens_out":2560,"would_cite":false,"duration_ms":25324,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a training-free pipeline that generates synthetic text with an LLM, perturbs its word frequencies with Laplace or Gaussian noise, and claims this produces differentially private data that still supports classification…","keywords":["differential privacy","synthetic data","large language models","in-context learning","text classification","Laplace mechanism","Gaussian mechanism","privacy-utility trade-off"],"falsifier":"Measure the change in the synthetic output distribution when a single example is removed from the in-context prompt; if the change is larger than the claimed epsilon would permit under any plausible sensitivity bound, the differential privacy claim is false.","tokens_in":15105,"feed_emoji":"🔒","tokens_out":4591,"duration_ms":39098,"temperature":0.7,"pith_summary":"The paper proposes SafeSynthDP, a training-free pipeline that uses a large language model to generate a synthetic version of a sensitive text dataset and then perturbs word frequencies with Laplace or Gaussian noise, claiming this yields differentially private synthetic data. The authors evaluate the approach on the AGNews news-classification task, training four machine learning models and testing two LLMs in zero-, 2-, and 4-shot in-context learning settings. Their experiments show that models trained or prompted with the noisy synthetic data retain most of their accuracy, with simple classifiers losing roughly 3 to 10 percentage points and LLM in-context learning narrowing the gap when given more demonstrations. The paper's central claim is that this integration offers a viable balance between privacy protection and data utility, providing a foundation for compliant synthetic data use in sensitive domains.","feed_headline":"Differential-privacy noise on LLM text keeps classifiers useful","feed_subtitle":"On AGNews, simple classifiers lose about 3–10 points; 4-shot LLM prompts shrink the gap further.","key_machinery":"The central mechanism is the SafeSynthDP pipeline: a small set of labeled examples from the original dataset is placed in the prompt, the LLM generates a synthetic corpus in JSON form, and then Laplace or Gaussian noise is added to the token frequencies of the generated text, with the privacy budget epsilon controlling how much noise is applied. The in-context demonstrations carry the statistical style of the original data, while the post-hoc noise is what the paper relies on for privacy protection. The evaluation then measures how well each model type tolerates that noise.","core_discovery":"In the paper's own terms, the discovery is that differentially private noise can be injected into the word-frequency distribution of LLM-generated text without destroying its usefulness for text classification. Using the AGNews dataset and gpt-4o-mini as the generator, the authors report that Multinomial Naive Bayes and SVM trained on noisy synthetic data reach 77.92% and 76.43% accuracy respectively, compared with 80.73% and 86.75% on the original data; the recurrent models drop further, to about 65%. For in-context learning, both gpt-4o-mini and gemini-1.5-flash perform better with 4-shot demonstrations than with 2-shot, and the gap between original and synthetic data shrinks as shot count increases, supporting the claim that privacy-enhanced synthetic data can substitute for real data in some settings.","pith_inferences":["The paper does not define a formal DP mechanism: no sensitivity bound, no privacy accounting, and the noise is applied after generation, so the guarantee for the original data is not established by the experiments shown.","A testable extension would be to measure the actual sensitivity of the in-context generation step by removing one prompt example and estimating the change in the output distribution.","The method could be strengthened by applying DP during generation (e.g., to the prompt selection or the sampling step) rather than only to the generated text's token frequencies.","Membership inference evaluation, which the paper explicitly lists as future work, would be the natural direct check of the privacy claim."],"forward_implications":["Simple frequency-based classifiers (MNB, SVM) lose little accuracy on privacy-enhanced synthetic data, suggesting the method suits high-level statistical tasks.","Sequence models (GRU, LSTM) are more sensitive to the noise, indicating that tasks requiring temporal or semantic nuance will need better noise calibration.","Increasing the number of in-context demonstrations narrows the performance gap between original and synthetic data for LLM classification.","Tuning the privacy budget epsilon provides a practical lever for trading utility against privacy, with epsilon=10 giving the best accuracy in their ICL experiments.","The approach is training-free, so it can generate private synthetic data without retraining a generative model for each new dataset."],"supporting_citations":[{"why":"The gpt-4o-mini model card, which supplies the primary LLM used for synthetic data generation.","marker":"Hurst et al., 2024"},{"why":"The gemini-1.5-flash model, used as the second LLM for in-context learning evaluation.","marker":"Team et al., 2024"},{"why":"The differential privacy framework that motivates the noise injection and the epsilon trade-off.","marker":"Dwork et al., 2014"},{"why":"In-context learning paradigm that the prompting strategy relies on to generate synthetic data without fine-tuning.","marker":"Brown et al., 2020b"},{"why":"Provides the prompting and demonstration methodology used to guide the LLM in the generation and ICL tasks.","marker":"Wei et al., 2022"},{"why":"Introduces the AGNews dataset, the sole evaluation platform for the classification experiments.","marker":"Zhang et al., 2015"},{"why":"Membership inference attacks, the privacy threat the paper claims its noise injection mitigates.","marker":"Shokri et al., 2017"},{"why":"Prior work on privacy-preserving in-context learning that serves as a motivation and comparison point.","marker":"Tang et al., 2024"}],"fun_headline_variants":["DP noise on LLM text keeps classifiers accurate enough","LLM synthetic data with DP noise loses only a few accuracy points","Differential privacy on LLM-generated text: utility preserved","Privacy-preserving LLM data: DP noise yields viable classifiers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that adding Laplace or Gaussian noise to the word frequencies of an already-generated synthetic text provides a differential privacy guarantee for the original dataset, but the paper never defines a sensitivity bound, a specific DP mechanism, or a privacy accounting for that step.","fun_headline_variants_meta":{"raw":{"variants":["DP noise on LLM text keeps classifiers accurate enough","LLM synthetic data with DP noise loses only a few accuracy points","Differential privacy on LLM-generated text: utility preserved","Privacy-preserving LLM data: DP noise yields viable classifiers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1378,"prompt_tokens":938,"completion_tokens":440,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":371}},"tokens_in":554,"tokens_out":440,"duration_ms":4978,"temperature":1.0,"reasoning_tokens":371,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:14:47.546934+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the change in the synthetic output distribution when a single example is removed from the in-context prompt; if the change is larger than the claimed epsilon would permit under any plausible sensitivity bound, the differential privacy claim is false.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The differential privacy framework that motivates the noise injection and the epsilon trade-off."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior work on privacy-preserving in-context learning that serves as a motivation and comparison point."}],"review_version":1}