{"id":"5813974f-0830-4973-8ac4-8f192f51bd8d","arxiv_id":"2502.02709","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Demographic coherence is a new necessary-condition privacy definition, and the paper proves differential privacy implies it and gives parameter conversions.","lead":"This paper introduces demographic coherence, a formal condition for judging whether a data release lets predictors make very different confidence-rated guesses about people in one random half of the data versus the other. The authors prove that every differentially private release satisfies this condition, and argue it can serve as a practical audit target short of full differential privacy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Necessity claim rests on an unproven bridge: singling-out attacks using auxiliary info (excluded in §3.1/§1.2) can pass demographic coherence, so 'necessary for data privacy' is unsupported.","rationale":"I read the paper in good faith. The formal DP-to-coherence chain (via max-information) is plausible and the appendix proofs appear internally consistent modulo typos and the vacuous separation example. The central logical gap is the necessity bridge: the framework defines privacy harm as a distributional difference between halves of a fixed dataset, and the Asahi/Blair story is only an argument, not a proof, that all predictive harms (or all privacy harms) take this form. The paper explicitly acknowledges that singling-out attacks using auxiliary information are excluded, yet the abstract and contributions present demographic coherence as necessary for data privacy without qualification. My proposed experiment would make this concrete: if a singling-out predictor passes the coherence test with high probability while a membership-inference attack succeeds, then the claimed necessity is false in a setting the paper itself regards as a privacy harm. The reader's weakest assumption identifies exactly this bridge, so I agree. The paper is still valuable as a sufficient condition for a particular class of predictive harms and as an auditing methodology, but the necessity claim should be scoped or proven. This does not change the reader's CONDITIONAL verdict.","tokens_in":32491,"tokens_out":17415,"duration_ms":167920,"concrete_test":"Run the DemCoh experiment (Figure 1) on a synthetic dataset X of size n=10,000 with a subgroup C of size |X∩C|=1000, α=0.01 (so 2/|X∩C|=0.002<α). Use as L a singling-out predictor h that outputs 1 only for the record whose public features (in lens ρ) match a known target, and 0 otherwise. Measure W1 between h(Xa|C) and h(Xb|C) over 1000 random splits; it will be <α with high probability. Independently run a standard membership-inference attack (e.g., distinguish target-in from target-out using the released report) and show it succeeds with AUC ≈1. If the attack succeeds while coherence passes, the bridge in §3.1 fails and the necessity claim needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline contribution is that demographic coherence is a necessary condition for private data release (abstract; §1). Formally, it proves only the one-way implication DP implies coherence (Theorems 6–8). The converse direction—that any privacy harm appears as a distributional difference between a predictor's outputs on two random halves of a fixed dataset—is argued only through the Asahi/Blair example (§3.1). This bridge is not a theorem, and the paper itself states that attacks using auxiliary information to single out one individual without moving subgroup distributions are outside the model (§1.2, §3.1). Such attacks are real privacy harms: a predictor h that outputs a special value only on one targeted record (identified via the lens ρ) yields W1 distance at most 2/|X∩C| for any subgroup C, so for |X∩C| ≥ 2/α it is α-coherent with probability 1 while disclosing the target's attribute. Hence a non-private release can pass coherence, and nothing rules out a private (non-DP) release failing it. The necessity claim is therefore not established; at best coherence is necessary for the narrower class of distributionally detectable predictive harms, and the abstract should say so.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 'demographic coherence,' a distributional property of confidence-rated predictors, and uses it to define coherence-enforcing data curation algorithms. It proves that algorithms with bounded max-information are coherence-enforcing (Theorem 6) and, via known differential-privacy-to-max-information bounds, that pure- and approximate-DP algorithms enforce Wasserstein demographic coherence for explicit parameter conversions (Theorems 7 and 8). An appendix example shows that a simple non-DP algorithm can be coherence-enforcing, supporting the claim that coherence is weaker than DP. The paper further argues that demographic coherence is a necessary condition for privacy, motivated by predictive harms, and proposes an experimental auditing methodology based on comparing predictions on a random split of a fixed dataset.","tokens_in":32730,"tokens_out":15673,"duration_ms":135142,"significance":"If the technical results are correct, the paper contributes a novel, attack-inspired necessary condition with formal parameter conversions and a concrete experimental evaluation methodology, connecting privacy to distributional properties of predictors via max-information. The appendix gives a detailed proof with explicit constants, which is a strength. However, the central necessity claim is not established formally, and the proof of the main theorem has a distributional subtlety that needs addressing. The framework is likely to be useful for auditing and for reasoning about predictive harms, but its scope should be stated more carefully.","major_comments":[{"comment":"The paper claims that demographic coherence is 'necessary for data privacy' (abstract, §1), but all formal results are one-way implications (DP ⇒ coherence, Theorems 6–8). The bridge from predictive harms to the split-dataset distributional experiment is argued by example (the Asahi/Blair case) rather than proved, and the paper itself excludes auxiliary-information attacks that single out an individual without moving subgroup-level distributions (§3.1, §1.2). Concretely, a predictor that outputs a special value only on one targeted record identified via the lens yields Wasserstein distance at most 2/|X∩C| for any subgroup C, so it is α-coherent with probability 1 for |X∩C| ≥ 2/α while disclosing that record's attribute. Thus the necessity claim overreaches; it should be qualified to distributionally detectable predictive harms, or the abstract and introduction should be revised.","section":"Abstract, §1, §3.1"},{"comment":"The proof applies the bounded max-information assumption to the pair (Xa, A*(Xa)) where Xa is a random half of an arbitrary fixed dataset X. However, Definition 8 defines I^β_∞(A,n) for a sample drawn from the data universe X, not from an arbitrary subset X of it. The proof provides no argument that the bound transfers to every fixed dataset X, and the identity 'since (Xa,A*(S)) is distributed exactly the same as (S,A*(Xa))' does not bridge this gap. Unless the definition is interpreted as a worst-case over distributions/datasets (and the theorem statement says so), Theorem 6's universal quantification over all datasets is not justified by the stated assumption. This is a load-bearing gap in the main technical proof.","section":"§5.2, Claim 1 in proof of Theorem 6"},{"comment":"The stated γ (size constraint) expressions do not match the max-information bounds used in the proofs. In Theorem 7, the proof invokes the bound ε√(n ln(4|C|/β))/4 from Theorem 11, but the γ expression contains ε√(n ln(4|C|/β))/2. In Theorem 8, the proof derives a max-information bound of 265ε²n/2 + 12ε√(n/2 ln(4|C|/β)), while the γ expression uses 265ε²n and 12ε√(n ln(4|C|/β)). Since the paper emphasizes exact parameter conversions, these statements should be corrected so that the stated γ values are exactly those obtained by substituting the max-information bounds into Theorem 6.","section":"§5.3, Theorems 7 and 8"}],"minor_comments":[{"comment":"Several γ expressions have missing closing parentheses, e.g., '36 ln((3/α)' should be '36 ln(3/α)'; these should be fixed throughout.","section":"Theorems 6, 7, 8"},{"comment":"The line 'ln((|X∩ C| + 1)≤ 1.01 ln((|X∩ C| + 1)' is nonsensical; presumably one side should be ln(|X∩C|), and the inequality should be stated cleanly.","section":"Proof of Theorem 6"},{"comment":"The max-information of an algorithm as defined depends on the universe size m, but m is not specified in the statements of Theorems 6–8; the authors should clarify whether the bound is assumed to hold uniformly for all universe sizes or for a particular one.","section":"Definition 8"},{"comment":"The informal statements use 'ε ≤ k·α/ln(1/β)' and 'n ≥ k·(ζ+ln(1/β))/α²', which do not precisely match the formal γ expressions; aligning the informal and formal statements would help readability.","section":"§2 (informal Theorems 1 and 2)"},{"comment":"There are residual typos, e.g., 'us the term' in footnote 1, 'is demographically coherent' repeated in §2, and inconsistent hyphenation of 'confidence-rated'; a proofreading pass is recommended.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's framing as a 'necessary condition' is likely to attract strong criticism because the formal results only establish a sufficient path via DP, and the paper itself concedes that singling-out attacks with auxiliary information are outside the model. I would urge the editor to require the authors to soften this claim in the abstract and introduction. The technical proof of Theorem 6 also needs a clarification or repair concerning the distribution used in the max-information assumption; without that, the main theorem's generality is in question. The constant mismatches in Theorems 7 and 8, while not fatal to the existential claims, undermine the paper's stated contribution of exact parameter conversions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful core of arXiv:2502.02709 is the DP-to-coherence parameter conversion and the subgroup-local, lens-based split-test auditing idea. The headline claim—that demographic coherence is necessary for data privacy—is not supported by the proofs, and the paper itself narrows the relevant harm class more than the abstract admits.\n\nWhat is new: a formal definition of demographic coherence (a confidence-rated predictor's predictions on two random halves of a fixed dataset, compared subgroup-by-subgroup through a lens, with Wasserstein distance), plus Theorems 6–8 showing that bounded max-information and pure/approximate DP imply coherence enforcement with explicit (ε, α, β, γ) conversions. The max-information machinery is prior work, but the without-replacement parameter tracking and the application to an audit-friendly necessary condition are a solid piece of work. The experimental framing is genuinely useful: it gives practitioners a concrete way to audit releases without requiring ground truth.\n\nThe soft spots are real but mostly in framing. The formal results only establish DP ⇒ coherence; the converse—that every privacy harm appears as a distributional gap between halves—is argued via the Asahi/Blair example, not proven. Section 3.1 explicitly excludes attackers with non-predictive goals, and §1.2 excludes singling out with auxiliary information. The stress-test construction lands: a predictor that emits a special signal only on one targeted record (found via the lens) is α-coherent for any subgroup large enough relative to 1/α, while leaking that record's attribute. So coherence is not necessary for data privacy in any broad sense. The abstract should say \"necessary for distributionally detectable predictive harms,\" and the paper should stop claiming more.\n\nThere are also presentation issues a referee should flag: several γ expressions in Theorems 6–8 have typos or mismatched constants (e.g., the \"36 ln((3/α)\" term, and the derivation of the size constraint in Theorem 6 uses inequalities in the wrong direction in at least one place). The finite-universe assumption in Definition 8 is not reconciled with the infinite universe in Definition 2. And the only non-DP coherence-enforcing example—publishing inputs on a binary universe with no auxiliary information—is vacuous, since no predictor can distinguish the halves; it does not demonstrate a meaningful separation from DP.\n\nThe central implication DP ⇒ coherence looks sound in outline. Lemma 1's use of max-information and hypergeometric concentration is plausible, and the appendix has real content.\n\nWho this is for: privacy auditing researchers and practitioners choosing ε. It deserves a serious referee, but the necessity claim needs heavy revision before publication.\n\nRecommendation: send to peer review; require the authors to fix the overclaim, clean up the constants, and either replace the vacuous example or explicitly label it as a limiting case.","headline":"Useful DP-to-coherence conversion and audit framework, but the 'necessary for privacy' claim is unsupported—coherence is at best necessary for distributionally detectable predictive harms.","tokens_in":33222,"tokens_out":4182,"would_cite":true,"duration_ms":38955,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68P27"],"pacs":[],"model":"deepseek-v4-flash","headline":"Demographic coherence is a necessary condition for private data release, and it is provably weaker than differential privacy.","keywords":["demographic coherence","differential privacy","necessary conditions","privacy attacks","predictive harms","max-information","Wasserstein distance","confidence-rated predictors"],"falsifier":"A concrete counterexample would settle the matter: exhibit a finite data universe, an order-invariant $\\varepsilon$-DP algorithm $A$, a predictor-builder $L$, a collection $\\mathcal{C}$, and a dataset $X$ such that, for parameters satisfying Theorem 7's regime, with probability greater than $\\beta$ the predictor separates the two halves of $X$ on some subpopulation of size at least $\\gamma$ (1-Wasserstein distance exceeds $\\alpha$). Less formally, running the DemCoh experiment on a real DP release with protected subgroups as $\\mathcal{C}$, and finding a confidence-rated predictor whose prediction distributions on $X_a$ and $X_b$ differ by more than $\\alpha$ at a sample size the theorem says should be safe, would falsify the claimed conversion.","tokens_in":32322,"feed_emoji":"🔐","tokens_out":11131,"duration_ms":97481,"temperature":0.7,"pith_summary":"Privacy research usually separates into sufficient conditions (differential privacy is the standard) and concrete attacks that demonstrate leaks, with little work on what any private release must guarantee. This paper proposes that a release must satisfy demographic coherence: a confidence-rated predictor built from the release on one random half of a fixed dataset should not produce noticeably different prediction distributions on the two halves, within any sufficiently large demographic subgroup. The central technical result is that every order-invariant differentially private release enforces this condition, with explicit parameter conversions, while there exist demographically coherent releases that are not differentially private. The definition is distributional and does not depend on accuracy, so it yields a concrete experimental audit of privacy risk and applies even when ground truth is unavailable.","feed_headline":"Demographic coherence is provably weaker than differential privacy","feed_subtitle":"New necessary condition for private data release comes with an audit experiment and is easier to satisfy than DP.","key_machinery":"The load-bearing machinery is the demographic coherence experiment $\\mathrm{DemCoh}$, together with a bridge through bounded max-information. $\\mathrm{DemCoh}$ uniformly splits a fixed dataset $X$ into $X_a$ and $X_b$, forms a report from $X_a$ through a curator $A$, lets an arbitrary algorithm $L$ turn that report into a confidence-rated predictor $h$, and measures the 1-Wasserstein distance between $h$'s prediction distributions on $X_a$ and $X_b$, restricted to a lens $\\rho$ and to each subpopulation $C$ of size at least $\\gamma$. The proof of Theorem 6 uses max-information to decouple the predictor from the half it was trained on: Claim 1 replaces the dependent split with an independent half-sample at a multiplicative cost $e^\\zeta$ plus an additive $\\eta$, and Claim 2 bounds the distance for an independent predictor using a hypergeometric concentration inequality. Feeding known pure-DP and approximate-DP max-information bounds into Theorem 6 produces the parameter conversions in Theorems 7 and 8.","core_discovery":"The paper's central claim is that demographic coherence is a necessary condition for private data release, and that it is strictly weaker than differential privacy. An algorithm is coherence enforcing if, for every predictor-building algorithm $L$, whenever a dataset $X$ is split uniformly into halves $X_a$ and $X_b$ and $h$ is the confidence-rated predictor $L$ produces from a report generated from $X_a$, the 1-Wasserstein distance between the empirical prediction distributions $h(\\pi_\\rho(X_a|C))$ and $h(\\pi_\\rho(X_b|C))$ is at most $\\alpha$ with probability at least $1-\\beta$, for every subpopulation $C$ in the collection that has at least $\\gamma$ members in $X$. Theorem 6 proves that bounded max-information implies coherence enforcement; Theorem 7 and Theorem 8 give explicit $(\\alpha,\\beta,\\gamma)$ regimes under which any order-invariant pure-DP or approximate-DP algorithm enforces Wasserstein coherence. The paper also shows that coherence does not imply differential privacy: on a simple binary universe, a curator that publishes its input in the clear is vacuously coherence enforcing, because no confidence-rated predictor can separate the two halves even with full access to the data. The intended interpretation is that differential privacy is one sufficient route to the necessary condition, and demographic coherence gives an audit-able check that non-DP releases can also pass.","pith_inferences":["Editorial inference: the necessity claim is only as strong as the Section 3.1 identification of privacy harms with distributional incoherence; harms that change a prediction about one individual without moving subgroup-level distributions, or that rely on auxiliary information, are outside the formal condition even if they matter in practice.","Editorial inference: because the proof chain goes through Theorem 6 and the DP-to-max-information bounds, a testable extension is to run the $\\mathrm{DemCoh}$ experiment on real releases across protected subgroups and lenses and check whether coherence violations appear at DP parameters the theorems deem safe.","Editorial inference: the finite-universe and order-invariance assumptions in Theorems 7 and 8 are technical boundary conditions; extending the argument to infinite universes and non-order-invariant mechanisms, or finding boundary counterexamples, would sharpen the scope of the necessary-condition claim.","Editorial inference: since coherence is accuracy-independent and strictly weaker than DP, it could serve as a lower-cost certification when ground truth is unavailable, but the paper claims sufficiency only for DP, not for coherence."],"forward_implications":["If the central claim is right, any order-invariant $\\varepsilon$-DP release automatically enforces Wasserstein demographic coherence with the explicit parameter regimes of Theorems 7 and 8, so coherence adds no new implementation burden for DP deployments.","Because coherent algorithms need not be differentially private, demographic coherence can certify releases that DP would forbid, opening a formal middle ground between no guarantee and full differential privacy.","Bounded max-information is the structural driver of the guarantee, so new algorithms can obtain coherence by proving max-information bounds rather than by adding noise calibrated to sensitivity.","The $\\mathrm{DemCoh}$ experiment yields an operational, accuracy-free audit: on a fixed dataset, a practitioner can estimate $\\alpha$ and $\\beta$ empirically and use Theorem 7's conversion to choose an $\\varepsilon$ that rules out a chosen level of demographic incoherence."],"supporting_citations":[{"why":"Defines differential privacy, the sufficient condition whose coherence enforcement Theorems 7 and 8 establish.","marker":"[24]"},{"why":"Introduces max-information and the template for showing DP algorithms have bounded max-information, which Theorem 6 converts into coherence.","marker":"[23]"},{"why":"Supplies the approximate-DP max-information bound that Theorem 12 generalizes to sampling without replacement.","marker":"[47]"},{"why":"Supplies the hypergeometric concentration inequality used in Claim 2 to bound the Wasserstein distance between halves.","marker":"[33]"},{"why":"Provides the Netflix re-identification scenario that motivates the predictive-harm and Asahi/Blair intuition behind the definition.","marker":"[43]"},{"why":"Provides the distributional-baseline evaluation idea that the paper adapts into its random-split DemCoh audit.","marker":"[20]"},{"why":"Defines perfect generalization, which Appendix A uses to show demographic coherence is a relaxation rather than an equivalent condition.","marker":"[18]"}],"fun_headline_variants":["Demographic coherence: a weaker, auditable privacy condition","Coherence check: private data release without differential privacy","Wasserstein audit: the new privacy condition that's weaker than DP","New privacy condition: demographic coherence, provably weaker than DP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every privacy harm worth preventing appears as a distributional difference between a confidence-rated predictor's behavior on two random halves of the same fixed dataset, with no auxiliary information and no purely single-individual effects; if that is false, demographic coherence is not necessary for privacy, and the theorem statements additionally need a finite universe and order-invariant mechanisms.","fun_headline_variants_meta":{"raw":{"variants":["Demographic coherence: a weaker, auditable privacy condition","Coherence check: private data release without differential privacy","Wasserstein audit: the new privacy condition that's weaker than DP","New privacy condition: demographic coherence, provably weaker than DP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000513,"raw_usage":{"total_tokens":2568,"prompt_tokens":1094,"completion_tokens":1474,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":710,"completion_tokens_details":{"reasoning_tokens":1404}},"tokens_in":710,"tokens_out":1474,"duration_ms":10043,"temperature":1.0,"reasoning_tokens":1404,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T11:24:13.790075+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete counterexample would settle the matter: exhibit a finite data universe, an order-invariant $\\varepsilon$-DP algorithm $A$, a predictor-builder $L$, a collection $\\mathcal{C}$, and a dataset $X$ such that, for parameters satisfying Theorem 7's regime, with probability greater than $\\beta$ the predictor separates the two halves of $X$ on some subpopulation of size at least $\\gamma$ (1-Wasserstein distance exceeds $\\alpha$). Less formally, running the DemCoh experiment on a real DP release with protected subgroups as $\\mathcal{C}$, and finding a confidence-rated predictor whose prediction distributions on $X_a$ and $X_b$ differ by more than $\\alpha$ at a sample size the theorem says should be safe, would falsify the claimed conversion.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines differential privacy, the sufficient condition whose coherence enforcement Theorems 7 and 8 establish."},{"cited_title":"Generalization in adaptive data analysis and ho ldout reuse","cited_arxiv_id":null,"evidence_quote":"Introduces max-information and the template for showing DP algorithms have bounded max-information, which Theorem 6 converts into coherence."},{"cited_title":"Rogers, Aaron Roth, Adam D","cited_arxiv_id":null,"evidence_quote":"Supplies the approximate-DP max-information bound that Theorem 12 generalizes to sampling without replacement."},{"cited_title":"Concentration of the hyperge ometric distribution","cited_arxiv_id":null,"evidence_quote":"Supplies the hypergeometric concentration inequality used in Claim 2 to bound the Wasserstein distance between halves."},{"cited_title":"Robust de-anon ymization of large sparse datasets","cited_arxiv_id":null,"evidence_quote":"Provides the Netflix re-identification scenario that motivates the predictive-harm and Asahi/Blair intuition behind the definition."},{"cited_title":"Confidence-ranked reconstruc tion of census micro- data from published statistics","cited_arxiv_id":null,"evidence_quote":"Provides the distributional-baseline evaluation idea that the paper adapts into its random-split DemCoh audit."},{"cited_title":"Adaptive learning with robust generalization guarantees","cited_arxiv_id":null,"evidence_quote":"Defines perfect generalization, which Appendix A uses to show demographic coherence is a relaxation rather than an equivalent condition."}],"review_version":1}