{"id":"1845f29c-1e2e-4f24-ae0b-64ad9996b472","arxiv_id":"2411.16516","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Blackbox DP auditors that ignore small-probability outputs systematically fail to detect privacy violations, enabling curator attacks that pass overstated privacy claims.","lead":"This paper shows that blackbox differential privacy auditors can be fooled: because they ignore rare outputs, a dishonest data curator can claim strong privacy while actually leaking data. The authors formalize this failure, construct mechanisms that pass four leading auditors, and call for more reliable auditing.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The FP constructions are conditional on the auditor's exact truncation threshold and input-pattern heuristic being known and fixed; Theorems 1-9 and the experiments do not cover randomized c/tau or adjacent inputs, so the universal 'ubiquitous FPs' claim is not established.","rationale":"I read the paper as a serious and internally coherent analysis of a real limitation: blackbox auditors that truncate small probabilities or densities cannot certify DP in the truncated region, and the authors demonstrate concrete mechanisms that violate their claimed guarantee yet pass DP-Sniper, MPL, Delta-Siege, and DPSGD-Audit. The use of official implementations and the derivation of per-theorem FP intervals are genuine strengths. The load-bearing weakness is the scope of the universality claim. Every theorem solves inequalities in the auditor's publicly fixed threshold c or tau and assumes the auditor follows the specific adjacent-input heuristic from Table 5. The threat model does not justify these as attacker-known constants, and the paper gives no analysis of an auditor who randomizes them. Since the headline claim is that false positives are 'ubiquitous' in blackbox auditing, this missing case is not a minor caveat: it determines whether the attack applies to the general class of blackbox auditors or only to specific configurations of the four studied tools. The proposed test isolates this dependency by randomizing c and the input pattern while holding the mechanism fixed. If the attack survives randomization, the concern is refuted; if not, the paper should be read as a conditional result for fixed, known auditor settings rather than a universal negative result. This supports the reader's CONDITIONAL verdict without changing it.","tokens_in":24639,"tokens_out":12938,"duration_ms":134522,"concrete_test":"Implement a DP-Sniper variant in which, for each independent audit run, c is drawn uniformly at random from [10^{-4}, 0.1] and the adjacent-input pattern is drawn uniformly from the full pattern set, including pairs with q(a')=0.5*Delta that are not in Table 5. Using the official sample sizes from Section 10.1, run 100 audits of the benchmark Laplace mechanism at theta=4 and epsilon_c=3.995 (the Theorem 1 FP point for c=0.01) and record the fraction of runs where the reported one-sided lower confidence bound exceeds epsilon_c. A nontrivial rejection fraction would confirm that the attack depends on exact knowledge of the auditor's fixed configuration; a zero fraction would indicate that the known-threshold assumption is not the bottleneck.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All four FP constructions reduce to deterministic inequalities in the auditor's truncation threshold and adjacent-input pattern. Prerequisite 1 (Eq. 7) requires Pr[M_theta(a') in S*] < c, and the R2 inequalities in Theorems 1-9 are solved for c or tau while implicitly assuming the Table 5 input-pattern heuristic (e.g., q(a)=0, q(a')=Delta). Definition 3's condition xi* <= epsilon_c < epsilon* is therefore guaranteed only for a fixed, known auditor configuration. Section 3's threat model lists a malicious curator but never states that the curator knows c, tau, or the GenerateInputs() heuristic, nor does it model an auditor who randomizes these choices. If c/tau or the adjacent pair is randomized, xi* changes and the derived FP interval can disappear; for the benchmark Laplace case, choosing c < e^{-theta}/2 makes DP-Sniper tight (the paper's own Figure 3 case), and choosing mu < Delta shrinks both epsilon* and xi*. Hence the central 'ubiquitous FPs' claim is not established for adaptive or secret-threshold auditors. Section 9's generalization is a recipe, not a proof, and does not close this gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that blackbox differential privacy auditing is inherently unreliable because it ignores small probabilities or densities. It frames the problem as hypothesis testing over the auditor itself: the null hypothesis is that the curator's (ε_c, δ_c)-DP claim is valid, and a false positive is defined as the event ξ* ≤ ε_c < ε*, i.e., the mechanism violates the claimed guarantee but passes the audit. The authors derive analytic conditions (Theorems 1–9) under which benchmark mechanisms (Laplace, SVT, Gaussian, one-step DPSGD) and adapted mechanisms (with crafted noise distributions such as gLap and modified SVT noise) become false positives or false negatives against DP-Sniper, MPL, Delta-Siege, and DPSGD-Audit. They validate the construction experimentally using the official code of the auditors and conclude that false positives are ubiquitous in blackbox auditing.","tokens_in":24946,"tokens_out":9493,"duration_ms":92992,"significance":"If the central claim holds, this is a practically important cautionary result: it shows that fixed-threshold blackbox auditors can be evaded by mechanisms whose privacy violations lie in the probability/density region the auditor truncates. The main strength is that the theoretical conditions are derived from the auditors' published algorithms and thresholds rather than fitted to experimental outcomes, and the experiments reuse the auditors' official code, which reduces implementation-bias concerns. The paper also provides a systematic template (Prerequisites P1–P3 and Algorithm 1) for finding future false-positive attacks. However, the universality of the claim is currently bounded by the assumption that the curator knows the auditor's fixed threshold and adjacent-input heuristic, and several load-bearing proofs are deferred or omitted.","major_comments":[{"comment":"Several load-bearing theorems are not proved in the manuscript: Theorem 3's proof is deferred to an external repository ([2]), Theorem 4's proof is deferred to Appendix D [2], Theorem 6's proof is omitted, and Theorem 9's proof is deferred to Appendix I [2]. These theorems provide the analytic false-positive conditions for SVT and DPSGD, which are central to the paper's empirical claims. The manuscript as submitted is therefore not self-contained, and the reader cannot verify the main theoretical results without consulting the external supplement.","section":"Section 5.2, Section 6.2, Section 8"},{"comment":"The inequalities in Theorems 1–9 concern the exact theoretical power ξ*, while the actual auditors return finite-sample confidence intervals. The paper states in Section 9 that only the lower bound of the confidence interval needs to be reported, but it does not prove that this empirical lower bound is close enough to the theoretical ξ* for the false-positive conditions to hold with high probability. A one-sided confidence interval with lower bound below ε_c does not imply that the true power is below ε_c; it only means the audit cannot reject the claim at the stated confidence level. The experiments provide illustrative evidence, but the formal relationship between the analytic FP conditions and the statistical decision rule of each auditor still needs to be stated.","section":"Section 9, Section 10"}],"minor_comments":[{"comment":"The statement of Theorem 6 says 'with probability threshold τ,' but for MPL the threshold τ is a density threshold, as defined in Section 2.2 and Prerequisite 2. This should be corrected to 'density threshold τ.'","section":"Section 6.2, Theorem 6"},{"comment":"Figure 2 contains the stray text '在此处键入公式' ('insert formula here') in panels (a) and (c), which appears to be an editing artifact and should be removed.","section":"Figure 2"},{"comment":"The text says DPSGD-Audit reports a two-sided interval '[ξ*, ξ*)', which as written is empty; this is presumably a typographical loss of the upper endpoint and should read '[ξ*_lower, ξ*_upper)' or the equivalent notation.","section":"Section 9"},{"comment":"The phrase 'As such, As such,' is duplicated in the Related Work section and should be reduced to a single occurrence.","section":"Section 11"},{"comment":"The paper's use of 'false positive' is nonstandard: in the usual hypothesis-testing vocabulary, the event 'mechanism violates DP but the auditor passes it' is a Type II error (false negative), not a Type I error. Although the definition is explicit and internally consistent, the Abstract's phrase 'false positives are ubiquitous' may mislead readers familiar with the standard convention; a brief remark explaining the opposite naming convention would improve clarity.","section":"Section 3, Definition 3"},{"comment":"The proof fixes the adjacent pair q(a)=0, q(a')=Δ and calls it optimal 'intuitively,' but it does not prove that this pair maximizes the ratio over all adjacent pairs nor that this is the pair selected by the auditors' GenerateInputs() heuristic. A short argument justifying the choice of the adjacent pair would make the 'iff' claim fully rigorous.","section":"Section 5.1, proof of Theorem 1"}],"recommendation":"major_revision","confidential_remarks":"The paper makes a substantively interesting point and the fixed-threshold analysis appears coherent, but the 'ubiquitous false positives' claim is stronger than what the current threat model and proofs establish. The deferred/omitted proofs for Theorems 3, 4, 6, and 9 are a particular concern for a journal submission, since these are central to the paper's conclusions. I would encourage the editor to request a revised version that either includes the full proofs or clearly bounds the claims to the known-threshold setting, and that adds an explicit statistical treatment connecting the theoretical ξ* to the auditors' finite-sample confidence intervals."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core finding lands: blackbox DP auditors that truncate small probabilities or densities can be evaded by mechanisms whose privacy violations live entirely in the truncated region. That is a real weakness in DP-Sniper, MPL, and Delta-Siege as they are actually deployed. The P1/R1/R2 decomposition is a clean way to reason about false positives, and the concrete result that benchmark Laplace with large noise scale is already an FP against DP-Sniper is a genuine, non-obvious observation. The experiments reuse official implementations, which gives the empirical claims real weight.\n\nThe soft spots are proportionate. The attack model assumes the curator knows the auditor's fixed threshold c or tau, and the exact input-pattern heuristic. That is true for current tools, so the practical attack is valid, but the paper's stronger framing—\"false positives are ubiquitous in blackbox auditing\"—is not established for auditors who randomize thresholds or keep the heuristic secret. The stress-test note about randomized c/tau is fair, and the paper does not address it. Several key theorems (3, 4, 6, 8, 9) defer proofs to an appendix or omit them entirely; the appendix may cover this, but as submitted the main text asks the reader to take too much on faith. The adapted mechanisms are admittedly contrived, though that is a standard way to demonstrate a formal weakness, and the authors are transparent about it.\n\nWhat the paper does well is more than the flaws: it gives the first systematic, quantitative account of attacker-constructible false positives, shows the issue spans four different auditors, and connects the analysis to concrete privacy risk for data owners. The related work is careful and the distinction between auditing the auditor (P1) and auditing the mechanism (P2) is worth keeping.\n\nMy recommendation: send this to serious referees. The central technical claim is coherent and the experiments are reproducible, but the authors should be pushed to (a) state clearly that the attack requires known, fixed auditor parameters, (b) either prove or soften the ubiquity claim, and (c) make the deferred proofs accessible. With those revisions, this is a solid paper that will be cited.","headline":"A credible and useful formalization of a real blind spot in blackbox DP auditing, but the 'ubiquitous FPs' claim overshoots what the fixed-threshold analysis proves.","tokens_in":25508,"tokens_out":1775,"would_cite":true,"duration_ms":19476,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Blackbox differential privacy auditing can be fooled by violations hidden in the small-probability region it ignores.","keywords":["differential privacy","blackbox auditing","false positive","curator attack","privacy auditing","small probabilities","hypothesis testing","differential privacy mechanisms"],"falsifier":"Run the benchmark Laplace mechanism with a noise scale $\\theta$ inside the paper's Theorem 1 window, but have the auditor draw its threshold $c$ uniformly at random from $(0, 0.1)$ after the curator fixes $\\theta$, so the curator cannot solve for $c$ in advance. If the audit still certifies some $\\epsilon_c < \\theta$, the ubiquity claim survives; if the false positives disappear, the attack is tied to the threshold being public and fixed.","tokens_in":24424,"feed_emoji":"🕵️","tokens_out":11515,"duration_ms":91746,"temperature":0.7,"pith_summary":"This paper argues that blackbox differential privacy auditing has an inherent blind spot: because auditors estimate probabilities from a limited number of samples, they truncate or ignore small probabilities and densities. The authors formalize the resulting failure as a false positive in a hypothesis test about the auditor: a mechanism with claimed privacy $(\\epsilon_c,\\delta_c)$ that truly violates its claim ($\\epsilon_c < \\epsilon^*$) can still pass the audit because the auditor's measured power satisfies $\\xi^* \\le \\epsilon_c$. In the paper's terminology, a false positive is the auditor approving a privacy guarantee that does not hold. They prove concrete parameter ranges in which benchmark Laplace and sparse-vector mechanisms, plus adapted bounded-noise versions, are false positives against DP-Sniper, MPL, $\\Delta$-Siege, and DPSGD-Audit, and they confirm the ranges experimentally. A data owner should care because each false positive is a successful curator attack: the mechanism leaks more than the certified guarantee admits.","feed_headline":"DP auditors ignore small probabilities; curators exploit the gap","feed_subtitle":"The attack works against DP-Sniper, MPL, Delta-Siege, and DPSGD-Audit on benchmark mechanisms.","key_machinery":"The load-bearing object is the false-positive condition $\\xi^* \\le \\epsilon_c < \\epsilon^*$, where $\\epsilon^*$ is the strongest DP guarantee the mechanism actually affords, $\\epsilon_c$ is the curator's claim, and $\\xi^*$ is the auditor's maximal empirical power. The mechanism that makes the condition provable is probability/density truncation: DP-Sniper replaces $\\Pr[M(A)\\in S]$ with $\\Pr_{\\ge c}[M(A)\\in S] = \\max\\{\\Pr[M(A)\\in S], c\\}$, MPL replaces densities with $\\max\\{p(b|A), \\tau\\}$, and $\\Delta$-Siege's surrogate $\\rho(\\epsilon,\\delta)$ moves along contours that generally miss the mechanism's true $\\epsilon$–$\\delta$ tradeoff curve. Each theorem converts the two inequalities $\\epsilon^*(\\theta) > \\epsilon_c$ and $\\xi^*(\\theta) \\le \\epsilon_c$ into explicit parameter ranges, with prerequisites such as $\\Pr[M_\\theta(a')\\in S^*] < c$ ensuring that the auditor's empirical witness departs from the theoretical optimal one. What this machinery shows is that any violation small enough to fall inside the truncated region is invisible to the auditor by construction.","core_discovery":"The paper's central claim is that false positives are structural, not accidental: a blackbox auditor that caps small probabilities at a threshold $c$ (DP-Sniper) or small densities at $\\tau$ (MPL) can inspect only a segment of the mechanism's type-I/type-II error tradeoff curve, and the unobserved tail is a region where the curator can violate privacy freely. Definition 3 makes the failure precise: $M$ is a false positive against an auditor exactly when $\\xi^* \\le \\epsilon_c < \\epsilon^*$, so $M$ violates the claimed $(\\epsilon_c,\\delta_c)$-DP yet the audit certifies it. For the Laplace mechanism audited by DP-Sniper the paper proves this happens iff $\\Pr[M_\\theta(a') \\in S^*] < c$, and it derives analogous inequalities for SVT with discrete output vectors, for Gaussian and one-step DPSGD mechanisms, and for $\\Delta$-Siege where the choice of privacy surrogate $\\rho(\\epsilon,\\delta)$ can produce both false positives and false negatives. These theoretical conditions are then confirmed with experiments that reuse the auditors' own implementations.","pith_inferences":["An extension the paper leaves implicit: an auditor that randomizes its truncation threshold $c$ or $\\tau$ on each run and keeps the value secret would break the curator's ability to solve the inequalities in advance, without opening the blackbox.","A practical warning not spelled out: the width of the region below the auditor's threshold is itself a risk signal; reporting how much of the tradeoff curve was never examined would tell data owners how conditional the certification is.","The same recipe should transfer to any future blackbox auditor with a fixed nonparametric estimator: locate the probability or density level below which estimation is capped or unreliable, then place a violation inside that region."],"forward_implications":["A curator can deploy a benchmark Laplace mechanism with scale $\\theta > -\\ln(2c)$ and claim any $\\epsilon_c$ lying between the auditor's power and the true level; the audit certifies the claim even though $\\theta > \\epsilon_c$.","With bounded-noise adaptations of Laplace and SVT, the true privacy level $\\epsilon^*$ becomes infinite, yet the mechanisms pass DP-Sniper and MPL for finite claimed $\\epsilon_c$, so the attack does not depend on borderline parameters.","For Delta-Siege, the Gaussian mechanism can be a false positive for some noise scales and a false negative for others under the same privacy surrogate, so the same auditor both certifies violations and rejects valid claims.","For one-step DPSGD, even a moderate threshold that gives a narrow confidence interval leaves false-positive claims, meaning the flaw extends to machine learning audits, not just statistical query mechanisms."],"supporting_citations":[{"why":"It supplies the DP-Sniper auditor with fixed probability threshold $c$ that the paper's false-positive analysis targets.","marker":"[11]"},{"why":"It supplies the MPL auditor whose density threshold $\\tau$ defines the second class of false positives.","marker":"[5]"},{"why":"It supplies Delta-Siege, whose privacy surrogate produces both false positives and false negatives.","marker":"[20]"},{"why":"It supplies the DPSGD-Audit pipeline used to demonstrate false positives in differentially private machine learning.","marker":"[23]"},{"why":"It provides the type-I/type-II tradeoff characterization that locates the small-probability region beyond the auditors' scrutiny.","marker":"[15]"},{"why":"It supplies the adjacent-dataset input pattern heuristic that the audited tools use and the attack relies on.","marker":"[13]"},{"why":"It provides the SVT mechanism used as a benchmark and adapted example.","marker":"[22]"},{"why":"It provides the RAPPOR mechanism whose false-positive analysis appears in the paper's appendix.","marker":"[27]"}],"fun_headline_variants":["Blackbox DP audits miss tiny probabilities, letting curators cheat","Curator attack: Hidden small-probability tails break DP audits","DP auditing's blind spot: small probabilities hide privacy leaks","False positives in DP audits open door for malicious curators","When DP auditors ignore tiny risks, curators pass fake guarantees"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The construction assumes the curator knows the auditor's fixed threshold for ignoring small probabilities and the exact adjacent-dataset heuristic the auditor uses; if the auditor randomizes or hides those choices, the derived false-positive windows no longer follow.","fun_headline_variants_meta":{"raw":{"variants":["Blackbox DP audits miss tiny probabilities, letting curators cheat","Curator attack: Hidden small-probability tails break DP audits","DP auditing's blind spot: small probabilities hide privacy leaks","False positives in DP audits open door for malicious curators","When DP auditors ignore tiny risks, curators pass fake guarantees"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000172,"raw_usage":{"total_tokens":1287,"prompt_tokens":972,"completion_tokens":315,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":230}},"tokens_in":588,"tokens_out":315,"duration_ms":3434,"temperature":1.0,"reasoning_tokens":230,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:01:17.372087+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the benchmark Laplace mechanism with a noise scale $\\theta$ inside the paper's Theorem 1 window, but have the auditor draw its threshold $c$ uniformly at random from $(0, 0.1)$ after the curator fixes $\\theta$, so the curator cannot solve for $c$ in advance. If the audit still certifies some $\\epsilon_c < \\theta$, the ubiquity claim survives; if the false positives disappear, the attack is tied to the threshold being public and fixed.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the MPL auditor whose density threshold $\\tau$ defines the second class of false positives."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies Delta-Siege, whose privacy surrogate produces both false positives and false negatives."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the SVT mechanism used as a benchmark and adapted example."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the RAPPOR mechanism whose false-positive analysis appears in the paper's appendix."}],"review_version":1}