Differentially-Private Text Rewriting reshapes Linguistic Style

Stefan Arnold

arxiv: 2604.26656 · v1 · submitted 2026-04-29 · 💻 cs.CL

Differentially-Private Text Rewriting reshapes Linguistic Style

Stefan Arnold This is my paper

Pith reviewed 2026-05-07 13:22 UTC · model grok-4.3

classification 💻 cs.CL

keywords differential privacytext rewritinglinguistic stylestylistic profilingregister identityparaphrasingprivacy budget

0 comments

The pith

Differentially private text rewriting systematically removes interactive markers, contextual references, and complex subordination from text.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests what happens to writing style when differential privacy is enforced through language-model sentence rewriting. It shows that privacy constraints cause the text to lose interactive markers, contextual references, and complex sentence structures, pushing all outputs toward a single non-involved and non-persuasive register. This happens with both autoregressive paraphrasing and bidirectional substitution methods. A reader would care because the work demonstrates that privacy protection can erase the very features that make human writing distinctive while still keeping the core meaning. The finding matters for any setting where style carries information beyond literal content.

Core claim

By conducting a multidimensional stylistic profiling of differentially-private rewriting, we demonstrate that the cost of privacy extends far beyond lexical variation. Specifically, we find that rewriting under privacy constraints induces a systematic functional mutation of the text's communicative signature. This shift is characterized by the severe attrition of interactive markers, contextual references, and complex subordination. By comparing autoregressive paraphrasing against bidirectional substitution across a spectrum of privacy budgets, we observe that both architectures force convergence toward a non-involved and non-persuasive register. This register-blind sanitization effectively

What carries the argument

Multidimensional stylistic profiling that tracks shifts in register identity when text is rewritten by language models under different privacy budgets.

If this is right

Privacy mechanisms applied to text preserve semantic content while removing markers of involvement and persuasion.
Autoregressive and bidirectional rewriting architectures produce comparable convergence to a neutral register.
Tighter privacy budgets amplify the homogenization of stylistic features.
Human-authored discourse markers are structurally flattened even when meaning remains intact.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Systems that generate private text may need separate style-preservation steps if original voice matters.
Downstream tasks relying on subtle stylistic cues could see degraded performance on privatized data.
Testing non-private paraphrasing controls would help confirm whether the observed changes are privacy-specific.

Load-bearing premise

The stylistic profiling accurately isolates changes caused by privacy constraints rather than by any rewriting process in general.

What would settle it

Apply the same stylistic analysis to texts rewritten by the identical language models but with no privacy constraints to check whether the same loss of markers occurs.

Figures

Figures reproduced from arXiv: 2604.26656 by Stefan Arnold.

**Figure 1.** Figure 1: Stylistic deviation of privatized text compared view at source ↗

**Figure 2.** Figure 2: Relative usage ratios of key lexico-grammatical features in privatized text compared to the human view at source ↗

**Figure 3.** Figure 3: Functional deviation of privatized text along the four canonical dimensions interpreted by view at source ↗

read the original abstract

Differential Privacy (DP) for text matured from disjointed word-level substitutions to contiguous sentence-level rewriting by leveraging the generative capacity of language models. While this form of text privatization is best suited for balancing formal privacy guarantees with grammatical coherence, its impact on the register identity of text remains largely unexplored. By conducting a multidimensional stylistic profiling of differentially-private rewriting, we demonstrate that the cost of privacy extends far beyond lexical variation. Specifically, we find that rewriting under privacy constraints induces a systematic functional mutation of the text's communicative signature. This shift is characterized by the severe attrition of interactive markers, contextual references, and complex subordination. By comparing autoregressive paraphrasing against bidirectional substitution across a spectrum of privacy budgets, we observe that both architectures force convergence toward a non-involved and non-persuasive register. This register-blind sanitization effectively preserves semantic content but structurally homogenizes the nuanced stylistic markers that define human-authored discourse.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

DP text rewriting flattens stylistic markers toward a non-involved register, but the design leaves open whether privacy noise or the rewriting step itself drives the change.

read the letter

The main takeaway is that differentially private sentence rewriting consistently reduces interactive markers, contextual references, and complex subordination, pushing output toward a flatter, non-persuasive register. This holds across privacy budgets and for both autoregressive and bidirectional models. The observation is useful because earlier DP text work mostly checked semantic fidelity and grammaticality, not register identity. The multidimensional stylistic profiling is a reasonable way to surface those shifts, and the consistency across architectures gives the result some weight. That part of the work is new and worth noting. The paper does a clean job of framing the unexplored angle and reporting the convergence pattern without overclaiming on the mechanism. The soft spot is the missing non-private rewriting baseline. The abstract and stress-test note show no control condition that applies the same models, prompts, and decoding without the DP constraints or noise. If plain rewriting already produces simpler text, then the functional mutation cannot be attributed specifically to privacy. Adding that arm would be straightforward and would tighten the causal claim. The stylistic metrics themselves look plausible on the surface, but their validation and sensitivity to the exact rewriting objective would need checking in the full methods. No circular derivations or invented quantities appear. This is for researchers working on privacy-preserving NLP pipelines who care about downstream effects on text character, not just privacy-utility curves. A reader already familiar with DP text methods would get the most from the register angle. It deserves peer review. The question is practical, the empirical direction is clear, and the gap is fixable without redesigning the study.

Referee Report

2 major / 1 minor

Summary. The paper claims that differentially private text rewriting, using language models for sentence-level privatization, induces a systematic functional mutation in linguistic style beyond lexical changes. Through multidimensional stylistic profiling, it finds severe attrition of interactive markers, contextual references, and complex subordination, with both autoregressive and bidirectional DP architectures driving convergence to a non-involved, non-persuasive register while preserving semantics. Comparisons across privacy budgets support the observation of register homogenization.

Significance. If the central claim holds after addressing controls, the work would be significant for DP in NLP by demonstrating that privacy mechanisms can erode nuanced stylistic features defining human discourse, such as involvement and persuasion markers. This extends evaluation of text privatization beyond coherence and utility to register identity, potentially guiding development of style-preserving DP methods. The multidimensional profiling approach offers a useful framework for future studies on communicative impacts.

major comments (2)

[Abstract] The central claim that DP constraints specifically induce the observed stylistic attrition and register convergence is not supported without a non-private rewriting baseline using identical models, prompts, and decoding strategies. The abstract compares only autoregressive paraphrasing vs. bidirectional substitution across privacy budgets, but any homogenization could arise from generic rewriting objectives or model biases toward simpler output rather than the privacy mechanism (noise or constrained generation). This is load-bearing for the assertion of 'privacy-induced' mutation.
[Abstract] The multidimensional stylistic profiling is presented as accurately capturing register identity shifts, but without explicit validation (e.g., correlation with human register judgments or ablation of feature sets), it is unclear whether the attrition metrics reliably distinguish DP effects from general text simplification. This weakens the functional mutation interpretation.

minor comments (1)

[Abstract] The abstract introduces terms like 'register-blind sanitization' and 'communicative signature' without brief definitions or examples, which could improve accessibility for readers unfamiliar with stylistic analysis.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. The comments highlight important controls needed to strengthen the causal attribution to differential privacy and to validate the stylistic measures. We address each point below and commit to revisions that directly respond to these concerns.

read point-by-point responses

Referee: [Abstract] The central claim that DP constraints specifically induce the observed stylistic attrition and register convergence is not supported without a non-private rewriting baseline using identical models, prompts, and decoding strategies. The abstract compares only autoregressive paraphrasing vs. bidirectional substitution across privacy budgets, but any homogenization could arise from generic rewriting objectives or model biases toward simpler output rather than the privacy mechanism (noise or constrained generation). This is load-bearing for the assertion of 'privacy-induced' mutation.

Authors: We agree that isolating the effect of the privacy mechanism requires an explicit non-private rewriting baseline under identical conditions. While the manuscript varies privacy budgets (with higher budgets approaching non-private behavior) and contrasts two DP architectures, it does not include a fully unconstrained rewriting condition using the same models, prompts, and decoding. We will add this baseline in the revision. The new experiments will generate rewrites without noise or privacy constraints, enabling a direct comparison that attributes any additional homogenization specifically to the DP components rather than generic model or rewriting biases. revision: yes
Referee: [Abstract] The multidimensional stylistic profiling is presented as accurately capturing register identity shifts, but without explicit validation (e.g., correlation with human register judgments or ablation of feature sets), it is unclear whether the attrition metrics reliably distinguish DP effects from general text simplification. This weakens the functional mutation interpretation.

Authors: The feature set is drawn from established multidimensional register analysis frameworks (e.g., Biber-style dimensions of involvement and persuasion) that have been validated across large-scale corpus studies for distinguishing communicative functions. The attrition patterns we observe are consistent with known register characteristics. To further address potential overlap with simplification, we will add an ablation study removing subsets of features and, where data collection is feasible, report correlations between the computed scores and human judgments of register. These additions will be included in the revised manuscript to strengthen the functional interpretation. revision: partial

Circularity Check

0 steps flagged

No circularity: purely empirical stylistic profiling with no derivations or fitted predictions

full rationale

The paper conducts an observational study of how differentially-private rewriting affects linguistic style markers, using multidimensional profiling across privacy budgets and two model architectures. No equations, parameter fitting, predictions derived from inputs, or self-citations that load-bear a derivation chain appear in the abstract or described content. The central claim rests on direct text analysis comparisons rather than any reduction of results to prior definitions or self-referential assumptions, making the work self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the unstated assumption that stylistic profiling metrics reliably detect register changes induced specifically by privacy mechanisms.

axioms (1)

domain assumption Multidimensional stylistic profiling can accurately measure functional mutation in communicative signature
Invoked implicitly when claiming attrition of specific markers defines the shift.

pith-pipeline@v0.9.0 · 5442 in / 1114 out tokens · 45495 ms · 2026-05-07T13:22:15.044997+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages

[1]

InProceedings of the 3rd Workshop on Trustwor- thy Natural Language Processing (TrustNLP 2023), pages 151–162, Toronto, Canada

Guiding text-to-text privatization by syntax. InProceedings of the 3rd Workshop on Trustwor- thy Natural Language Processing (TrustNLP 2023), pages 151–162, Toronto, Canada. Association for Computational Linguistics. Douglas Biber. 1991.Variation across speech and writ- ing. Cambridge university press. Douglas Biber. 1995.Dimensions of register variation:...

work page 2023
[2]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al

Plausible deniability for privacy-preserving data synthesis.arXiv preprint arXiv:1708.07975. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–...

work page arXiv 2020
[3]

InProceedings of the ACM Web Conference 2022, WWW ’22, page 721–731, New York, NY , USA

Dp-vae: Human-readable text anonymization for online reviews with differentially private vari- ational autoencoders. InProceedings of the ACM Web Conference 2022, WWW ’22, page 721–731, New York, NY , USA. Association for Computing Machinery. Nan Xu, Oluwaseyi Feyisetan, Abhinav Aggarwal, Zekun Xu, and Nathanael Teissier. 2021. Density- aware differential...

work page 2022

[1] [1]

InProceedings of the 3rd Workshop on Trustwor- thy Natural Language Processing (TrustNLP 2023), pages 151–162, Toronto, Canada

Guiding text-to-text privatization by syntax. InProceedings of the 3rd Workshop on Trustwor- thy Natural Language Processing (TrustNLP 2023), pages 151–162, Toronto, Canada. Association for Computational Linguistics. Douglas Biber. 1991.Variation across speech and writ- ing. Cambridge university press. Douglas Biber. 1995.Dimensions of register variation:...

work page 2023

[2] [2]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al

Plausible deniability for privacy-preserving data synthesis.arXiv preprint arXiv:1708.07975. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–...

work page arXiv 2020

[3] [3]

InProceedings of the ACM Web Conference 2022, WWW ’22, page 721–731, New York, NY , USA

Dp-vae: Human-readable text anonymization for online reviews with differentially private vari- ational autoencoders. InProceedings of the ACM Web Conference 2022, WWW ’22, page 721–731, New York, NY , USA. Association for Computing Machinery. Nan Xu, Oluwaseyi Feyisetan, Abhinav Aggarwal, Zekun Xu, and Nathanael Teissier. 2021. Density- aware differential...

work page 2022