{"id":"acfa77b3-1ebf-4152-8c26-bebdbfdabb1f","arxiv_id":"2506.19836","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Feature differential privacy is a relaxation of DP that guards selected features only, and the paper's two-batch algorithm recovers subsampling amplification and improves utility over standard DP when public features exist.","lead":"Feature differential privacy (FDP) protects only specified sensitive attributes of a data sample, treating everything else as public, and the authors prove composition, attribute-inference, and subsampling-amplification properties for it. They also give a two-batch DP-SGD variant and report large utility gains over DP on classification and diffusion model experiments when a public version of each sample is available.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The FDP guarantee is only as meaningful as the chosen public feature Ψ; the paper's own blur-based public features are disclaimed, and Theorem 4's attribute-inference bound becomes vacuous if Ψ leaks identity.","rationale":"The paper's central contribution is conditional: for a given public feature map Ψ, Algorithm 1 is f-DPΨ_i and achieves better utility than standard DP with the same (ε,δ). The conditional theorems are plausible, and the two-batch construction is a genuine contribution that answers the label-DP amplification question in the affirmative. However, the practical significance claimed in the abstract rests on the choice of Ψ. The authors explicitly state in Section 4 that public features can be used for identification, that FDP does not bound membership inference, and that they do not recommend the blurred public features used in their own experiments. This is not an internal contradiction, but it makes the headline empirical result load-bearing on an assumption the paper does not defend. Theorem 4's bound degrades to vacuity when Ball(X,u,d,ρ) is close to 1, which is precisely the regime of a blurred face that still identifies the original image. The incomplete proof of Proposition 3 and the mismatch between the utility theory and clipped gradients are real but addressable; the unsafety of Ψ is more fundamental because it undermines the meaning of the privacy guarantee itself. I credit the authors for their explicit limitation statement, and a revision that either certifies the safety of public features, avoids unsafe features in the empirical claims, or reframes the contribution as private learning relative to a user-specified Ψ would resolve the concern. The reader's conditional verdict is therefore unchanged.","tokens_in":22411,"tokens_out":31057,"duration_ms":337807,"concrete_test":"Estimate Ball(X,u,d,ρ) for the AFHQ and LSUN settings with Ψ = Gaussian blur, using a perceptual distance d such as LPIPS at the same ρ used for reconstruction-style attacks. Concretely, sample pairs x,x' from the training set whose blurred versions are close under Ψ, and measure the fraction with d(x,x') ≤ ρ; if that fraction is close to 1, Theorem 4 provides no attribute-inference bound and the experimental public feature is not privacy-safe. A second, directly interpretable check: run a face-identification model on the blurred AFHQ images and report top-1/top-5 retrieval accuracy against the original sharp images; high retrieval accuracy confirms that the public feature leaks identity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Definition 5 makes FDP relative to an arbitrary public feature map Ψ. If Ψ(x) reveals more than intended, every theorem in the paper remains true but the privacy guarantee is vacuous. Section 4 concedes this directly: 'the public feature can be used for identification... feature DP would not necessarily provide upper bounds on membership inference attacks' and 'We do not recommend using the public features that we use in our empirical studies in the paper.' The AFHQ and LSUN experiments use a blurred image as the public feature, and the headline utility gain (FID 101.9 vs 286.7 at ε=8) is purchased entirely by assuming that blurring is a privacy-safe projection. Theorem 4's bound is adv(A,X,u,S,d,ρ) ≤ 1 − f(Ball(X,u,d,ρ)); if the conditional distribution X | Ψ(X)=u is concentrated—e.g., a blurred face still identifies the sharp face—then Ball ≈ 1 and the bound provides no protection. The central theoretical claim is conditional on Ψ, so the real-world significance claimed in the abstract depends on a user-supplied Ψ that is genuinely public, and the paper gives no method to certify that. This is more load-bearing than the incomplete proof of Proposition 3 or the clipped-gradient gap, because those are patchable technical issues, whereas an unsafe Ψ makes the privacy guarantee empty even if all proofs are correct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces feature differential privacy (FDP), a relaxation of differential privacy that protects only a chosen feature map Ψ of each record rather than the whole record. It defines replacement and insertion/deletion variants (Definitions 5 and 6, the latter simulation-based), proves composition (Theorem 3), an attribute-inference bound (Theorem 4), and presents a two-batch DP-SGD variant (Algorithm 1) claimed to satisfy FDP while benefiting from subsampling amplification (Theorem 5), with convex excess-risk bounds (Theorem 6) and a label-DP case study that answers an open question of Ghazi et al. Experiments on Purchase100, Criteo, AFHQ, and LSUN report utility gains over standard DP, including a large FID improvement on AFHQ when blurred images are treated as public features.","tokens_in":22694,"tokens_out":24271,"duration_ms":241457,"significance":"If the proofs are completed, FDP is a valuable formal framework for partial privacy: it composes, supports both replacement and insertion/deletion semantics, and gives a principled bound on attribute-inference attacks. The two-batch construction in Algorithm 1 is a genuine attempt to combine a public-feature loss with a private loss while preserving subsampling amplification, and the label-DP excess-risk bound scaling as O(d/(ε^2|S|^2)) addresses a concrete open problem. Theorem 4's attribute-inference bound cleanly quantifies residual reconstruction risk in terms of the geometric concentration measure Ball. The paper is also candid about the dependence of the guarantee on the user-supplied feature map Ψ, which is both a strength and a source of limitation for the empirical claims.","major_comments":[{"comment":"The proof of the insertion/deletion variant of Theorem 5 is incomplete. Definition 6 requires both Pr[M(S')∈T] ≤ 1−f(Pr[sim(S,u)∈T]) and the reverse inequality. The appendix constructs the simulator in Algorithm 2 and derives only the first inequality; the reverse does not follow from the same dominating pair because the trade-off function T(N(0,σ), (1−p)N(0,σ)+pN(τ,σ)) is not symmetric. In addition, the proof does not account for different subsampling probabilities in the mechanism and the simulator: Algorithm 1 uses p=m/|S|, so the mechanism on S' (size n+1) uses p_mech=m/(n+1) while the simulator on S (size n) uses p_sim=m/n. These two gaps leave the main algorithmic privacy theorem unproven as written.","section":"Appendix: Proof of Theorem 5 (Algorithm 2)"},{"comment":"Theorem 5's trade-off function and noise schedule appear to analyze a sum query, not the averaged private gradient computed in Algorithm 1. Line 4 defines gpriv_t = (1/m) Σ_{x∈Bpriv_t} ∇ℓpriv(wt;x), so the per-example contribution to the output is (1/m)∇ℓpriv, and the shift in the subsampled Gaussian trade-off should be τ/m rather than τ. The proof in the appendix writes Ci = Ai + ∇ℓpriv(wt;x) without the 1/m factor. Consequently, the stated schedule σ = cτm/(ε·n)√(T log(1/δ) log(T/δ)) does not follow from the stated f, and the utility claims in Theorem 6 that depend on σ are not justified as written.","section":"Section 5, Algorithm 1 and Theorem 5"},{"comment":"The proof of Proposition 3 is cut off. After defining q1,...,q4, the tightness argument stops and the text moves to 'Proof of Theorem 4'. The claim that the randomized-response mechanism is tightly (ϵ,0)-DPΨ_i for all p is therefore unverified. Since Proposition 3 is the stated reason that a naive feature-DP adaptation does not benefit from subsampling, this missing argument should be supplied or the proposition should be clearly labeled as a conjecture.","section":"Appendix: Proof of Proposition 3"},{"comment":"The empirical public features are explicitly disclaimed by the paper. Section 4 states 'the public feature can be used for identification... feature DP would not necessarily provide upper bounds on membership inference attacks' and 'We do not recommend using the public features that we use in our empirical studies in the paper.' The abstract nevertheless highlights the AFHQ result (FID 286.7 to 101.9 at ε=8) based on blurring as the public feature, which is the kind of feature the paper warns may leak private identity information. Without a certification method for Ψ, the experimental demonstrations do not establish that FDP yields practical privacy-preserving utility; they only illustrate the algorithm under an unvalidated public-feature assumption. This should be stated plainly where the results are presented.","section":"Section 4 and Section 6.3"}],"minor_comments":[{"comment":"The excess-risk bound is misparenthesized: '≤ (M2 c +cτ′2 +cdσ2)2 + log(T )√ T' should presumably read '(M^2/c + cτ'^2 + c d σ^2)(2 + log T)/√T' or similar; please fix the formula.","section":"Section 5, Theorem 6"},{"comment":"In the sentence 'the hockey-stick divergence between Xi and Yi is bounded by that of N(0,σ^2) and (1−p)N(0,σ^2)+pN(0,σ^2)' the last term should be pN(τ,σ^2) or pN(L,σ^2); as written it is a typo. Also L is used without definition.","section":"Appendix: Proof of Theorem 5"},{"comment":"The experimental description for Purchase100 diverges from Algorithm 1: the paper computes ~gpub = gpub + N(0,C^2σ^2) and then aggregates with α, whereas Algorithm 1 adds the noise to the total gradient and has no α. Please clarify which procedure is actually used.","section":"Section 6.2"},{"comment":"For Purchase100, the text says 100 features plus the label are public, but later states 'leaking 100 features from the set of 600 features' without mentioning the label; please make the description consistent.","section":"Section 6.1"},{"comment":"The figure caption 'Private image and its public variant' would benefit from specifying that the public variant is a blurred version; the exact blur parameters for AFHQ and LSUN are not given in Section 6.3.","section":"Section 1 and Figure 1"},{"comment":"The appendix reproduces the S&P meta-review, which is not appropriate for a journal submission; please remove it or clearly separate it from the technical content.","section":"Appendix A (Meta-Review)"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely a revised version of a previously reviewed S&P submission, given the appended meta-review. The core FDP definition and the attribute-inference bound are interesting and likely correct, but Theorem 5's proof has substantial gaps (one-sided insertion/deletion argument, subsampling-probability mismatch, and sum-versus-average normalization) that must be fixed before the main algorithmic claim can be accepted. The experimental section's reliance on disclaimed public features may also invite reader skepticism; consider recommending that the authors either provide a certified-safe Ψ example or reframe the experiments as purely illustrative."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: feature differential privacy (FDP) is a genuinely new relaxation of DP with a simulation-based insertion/deletion variant, a composition theorem, and an attribute-inference bound that does real work. The authors also show that the naive way to run DP-SGD under FDP does not amplify by subsampling, and their two-batch fix (Algorithm 1) recovers amplification. That answers an open question from Ghazi et al. and gives a concrete recipe that practitioners could use. The theory is mostly coherent and the math is honest: definitions are stated precisely, the composition proof goes through hockey-stick divergences, and the attribute-inference bound is a clean application of Jensen to the trade-off function. Credit where due: this is a real step forward, not just a relabeling of prior notions like Selective DP or Partial DP.\n\nThe weaknesses are in proportion. Proposition 3's tightness half is asserted with a proof that is cut off mid-argument in the appendix; that should be patched but it is a technical gap, not a conceptual one. The utility analysis assumes Lipschitz losses while the experiments rely on clipping, with only a remark to bridge the gap; again patchable, but the paper should say more clearly that the theory is for the Lipschitz case and the experiments are heuristic.\n\nThe bigger soft spot is the one the stress-test note flags: the FDP guarantee is only as meaningful as the chosen Ψ. If the public feature leaks identity—a blurred face still identifying the sharp face—then Theorem 4's bound becomes vacuous because Ball(X,u,d,ρ) ≈ 1. The authors themselves concede this in Section 4, saying public features 'can be used for identification' and 'We do not recommend using the public features that we use in our empirical studies.' That admission is commendable but it undercuts the abstract's claim of drastic FID improvement from 286.7 to 101.9: the headline result is purchased by assuming blur is a privacy-safe projection, an assumption the authors do not defend. This is not a flaw in the definition or the theorems—they remain true relative to any Ψ—but it is a real gap between the theory and the claimed practical significance.\n\nWho this is for: researchers working on partial privacy, label DP, or private learning with public data will get real value from the definitions, the amplification result, and the logistic regression bounds. It deserves a serious referee: the ideas are novel enough and the formal core is strong enough to merit careful review, even though the empirical claims need tempering and the missing proof needs completing. I would send it to review and ask for a revision, not a desk reject.","headline":"A solid new privacy definition with real algorithmic payoff, but the empirical significance is anchored on public features the authors themselves disclaim.","tokens_in":23226,"tokens_out":1001,"would_cite":true,"duration_ms":12673,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Feature differential privacy protects only designated private attributes and, through a two-batch SGD variant, restores subsampling amplification so the same privacy budget yields far better utility than full-sample DP.","keywords":["feature differential privacy","attribute inference","public features","DP-SGD","subsampling amplification","label differential privacy","privacy-utility trade-off","diffusion models"],"falsifier":"Run the AFHQ blurred-image setup and launch an attribute-inference or face re-identification attack that uses only the blurred public features against a model trained under FDP; if the attacker's success exceeds the \\(1-f(\\mathrm{Ball})\\) bound from Theorem 4, or matches the success of an attacker who never sees the model, the feature map fails to separate public from private information and the practical privacy claim collapses. A separate check for the privacy theorem itself: for a small logistic-regression instance, compute the empirical trade-off between the two gradient distributions in Theorem 5 and compare it with the claimed \\(f\\); any violation would refute the proof.","tokens_in":22195,"feed_emoji":"🔐","tokens_out":10508,"duration_ms":93131,"temperature":0.7,"pith_summary":"Feature differential privacy (FDP) is a relaxation of differential privacy that promises protection only for the parts of a data record designated as private, chosen through an arbitrary public feature map \\(\\Psi\\). The paper's central claim is that this relaxation yields real algorithmic gains: a modified DP-SGD that samples two independent batches, one for a public loss depending only on \\(\\Psi(x)\\) and one for the private residual loss, is provably f-DP with Gaussian noise scaled by the sensitivity of the private residual rather than the sensitivity of the full loss. Because that noise is smaller, the same \\((\\epsilon,\\delta)\\) budget buys much better utility; the paper demonstrates this by training diffusion models on blurred animal faces, improving FID from 286.7 to 101.9 at \\(\\epsilon=8\\). A careful reader would care because many applications only need to protect specific attributes, and FDP aligns the formal guarantee with that need instead of paying the full-sample DP penalty.","feed_headline":"Feature-level privacy keeps models useful at the same epsilon","feed_subtitle":"Two-batch SGD protects only chosen attributes, cutting diffusion FID from 287 to 102 at ε=8.","key_machinery":"The load-bearing mechanism is the two-batch noisy SGD construction (Algorithm 1), driven by a feature-dependent split of the loss: write \\(\\ell(x,\\$\\theta$) = \\ell_{\\mathrm{priv}}(x,\\$\\theta$) + \\ell_{\\mathrm{pub}}(\\Psi(x),\\$\\theta$)\\), where \\(\\ell_{\\mathrm{pub}}\\) is a surrogate loss that predicts the true loss from the public feature alone. Each iteration samples a Poisson mini-batch for the private residual gradient and an independent uniform batch for the public gradient; keeping the two batches independent is what preserves the randomness of the sub-sampling coins and restores amplification. The privacy analysis reduces one step to a subsampled Gaussian mechanism with noise multiplier \\(\\$\\sigma$/\\tau\\), and the product-composition object \\(f = T\\big(N(0,\\$\\sigma$)^T, ((1-p)N(0,\\$\\sigma$)+pN(\\tau,\\$\\sigma$))^T\\big)\\) is the exact f-DP trade-off that converts into \\((\\epsilon,\\delta)\\) parameters. This machinery makes the noise scale depend on the private loss's Lipschitz constant \\(\\tau\\) rather than the full loss's \\(\\tau'\\).","core_discovery":"The paper establishes that feature differential privacy can be algorithmic rather than purely definitional. Definition 5 fixes a public feature map \\(\\Psi\\) and requires indistinguishability only between neighboring datasets whose differing points share the same value of \\(\\Psi\\); a simulation-based insertion/deletion variant (Definition 6) extends the same idea to add/remove adjacency. The main technical discovery is that subsampling amplification, which fails for a naive feature-DP mechanism (Proposition 3), can be recovered by splitting each SGD step into two independent batches: one computes the public-gradient component from \\(\\Psi\\), the other computes the private residual gradient. Theorem 5 shows the resulting algorithm is \\(f\\)-DP\\(^{\\Psi}_i\\) with trade-off function \\(f = T\\big(N(0,\\$\\sigma$)^T, ((1-p)N(0,\\$\\sigma$)+pN(\\tau,\\$\\sigma$))^T\\big)\\), where \\(\\tau\\) is the Lipschitz constant of the private residual loss. Since \\(\\tau \\le \\tau'\\), the full loss's Lipschitz constant, the Gaussian noise needed for an \\((\\epsilon,\\delta)\\) guarantee is smaller than in standard DP-SGD, yielding lower excess risk (Theorem 6) and, in the label-DP special case, an \\(O(d/(\\$epsilon^{2}$ |S|^2))\\) excess-risk bound that answers an open question in the label-DP literature.","pith_inferences":["Inference: the two-batch decomposition should transfer to any loss that can be split into a public prediction and a private residual, so FDP could be combined with external public pretraining or foundation-model priors, a direction the paper only touches.","Inference: because the formal guarantee is relative to an arbitrary \\(\\Psi\\), deployments should audit the chosen feature map against re-identification; the paper's own caution that blurred faces may still be identifying implies that the empirical FID gains do not by themselves certify privacy.","Inference: a natural next step is to learn \\(\\Psi\\) and \\(\\ell_{\\mathrm{pub}}\\) jointly rather than choosing them by hand, with the FDP bound used as a constraint during training; the theory gives no direct guidance on how to optimize the public/private split.","Inference: the utility gain is largest when the public feature carries most of the predictive signal, so the framework is likely to help most in structured data such as images and least in settings like Criteo, where public and private features contribute comparably."],"forward_implications":["For a fixed \\((\\epsilon,\\delta)\\) budget, feature-DP training can use a smaller noise scale than full-sample DP whenever the private residual loss is less sensitive than the full loss, with the gain inherited directly by the excess-risk bounds.","Label DP is a special case of FDP; the paper's \\(O(d/(\\epsilon^2 |S|^2))\\) excess-risk bound beats the earlier dataset-size-independent bound once \\(|S| > \\sqrt{d/k}\\).","FDP satisfies adaptive composition and post-processing, and arbitrarily many public pre-training steps can be run before Algorithm 1 without spending any of the privacy budget (Remark 2).","Attribute inference on private features is controlled by \\(1-f(\\mathrm{Ball})\\), so an attacker's success is bounded by a geometric concentration term together with the privacy trade-off function.","The empirical results on AFHQ and LSUN show that FDP training of diffusion models produces recognizable outputs at privacy levels where standard DP training fails, with FID 101.9 versus 286.7 at \\(\\epsilon=8\\) on AFHQ."],"supporting_citations":[{"why":"Defines DP-SGD, the base algorithm that Algorithm 1 modifies for feature-level privacy.","marker":"[1]"},{"why":"Supplies the tight Poisson-subsampling amplification analysis used in Theorem 5's trade-off function.","marker":"[2]"},{"why":"Provides the DP-SGD excess-risk baseline that Theorem 6 improves upon by scaling noise with the private loss's Lipschitz constant.","marker":"[3]"},{"why":"Defines trade-off functions and f-DP, the accounting framework in which Theorem 5 states the privacy guarantee.","marker":"[7]"},{"why":"Gives the standard DP definition whose sample-level adjacency is relaxed to feature-level adjacency in Definition 5.","marker":"[8]"},{"why":"Defines label differential privacy and poses the open subsampling question answered by the paper's \\(O(d/(\\epsilon^2 |S|^2))\\) bound.","marker":"[12]"},{"why":"Provides the SGD convergence theorems used to prove the excess empirical risk bounds in Theorem 6.","marker":"[27]"}],"fun_headline_variants":["Feature DP beats full DP on AFHQ: FID 287→102","Shield only protected features: same epsilon, better utility","Feature DP: privacy for attributes, precision for models","Split gradients to protect only what matters","Two-batch SGD recovers amplification for feature DP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The guarantee is only as strong as the public-feature map: if the 'public' part still contains identifying or sensitive information, then feature-DP does not bound membership inference or full-sample reconstruction, even though every theorem in the analysis remains true.","fun_headline_variants_meta":{"raw":{"variants":["Feature DP beats full DP on AFHQ: FID 287→102","Shield only protected features: same epsilon, better utility","Feature DP: privacy for attributes, precision for models","Split gradients to protect only what matters","Two-batch SGD recovers amplification for feature DP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001084,"raw_usage":{"total_tokens":4592,"prompt_tokens":1063,"completion_tokens":3529,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":679,"completion_tokens_details":{"reasoning_tokens":3450}},"tokens_in":679,"tokens_out":3529,"duration_ms":24993,"temperature":1.0,"reasoning_tokens":3450,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:25:42.140710+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the AFHQ blurred-image setup and launch an attribute-inference or face re-identification attack that uses only the blurred public features against a model trained under FDP; if the attacker's success exceeds the \\(1-f(\\mathrm{Ball})\\) bound from Theorem 4, or matches the success of an attacker who never sees the model, the feature map fails to separate public from private information and the practical privacy claim collapses. A separate check for the privacy theorem itself: for a small logistic-regression instance, compute the empirical trade-off between the two gradient distributions in Theorem 5 and compare it with the claimed \\(f\\); any violation would refute the proof.","supporting_citations":[{"cited_title":"Pri- vacy amplification by subsampling: Tight analyses via couplings and divergences","cited_arxiv_id":null,"evidence_quote":"Supplies the tight Poisson-subsampling amplification analysis used in Theorem 5's trade-off function."},{"cited_title":"Private stochastic convex optimization with optimal rates","cited_arxiv_id":null,"evidence_quote":"Provides the DP-SGD excess-risk baseline that Theorem 6 improves upon by scaling noise with the private loss's Lipschitz constant."},{"cited_title":"Differential privacy","cited_arxiv_id":null,"evidence_quote":"Gives the standard DP definition whose sample-level adjacency is relaxed to feature-level adjacency in Definition 5."},{"cited_title":"Deep learning with label differential privacy","cited_arxiv_id":null,"evidence_quote":"Defines label differential privacy and poses the open subsampling question answered by the paper's \\(O(d/(\\epsilon^2 |S|^2))\\) bound."},{"cited_title":"Stochastic gradient descent for non-smooth optimization: Convergence re- sults and optimal averaging schemes","cited_arxiv_id":null,"evidence_quote":"Provides the SGD convergence theorems used to prove the excess empirical risk bounds in Theorem 6."}],"review_version":2}