Pith. sign in

REVIEW 4 major objections 4 minor 41 references

Auditing Fairness-Privacy Trade-offs: Subpopulation-Level Effects of Fairness-Enhancing Algorithms

T0 review · 4 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper shows that fairness-enhancing algorithms do not uniformly raise membership-inference privacy risk; the direction and size of the effect depend on subgroup size, model architecture, and the chosen mitigation strategy.

desk verdict A broad, useful LiRA-based audit of fairness mitigators vs. subgroup MIA risk, but the OQTA/OTA robustness evidence is invalid as written due to in-sample threshold tuning and a mis-defined TNR; the core findings are plausible but need fixes. read the letter →

arxiv 2607.14607 v1 pith:THKJXZ35 submitted 2026-07-16 cs.LG cs.AI

Umid Suleymanov , Ilhama Novruzova , Khalid Mammadov , Natavan Hasanova , Murat Kantarcioglu This is my paper
classification cs.LGcs.AI
keywords membershipinferenceattacksubpopulationprivacyriskfairness-enhancingalgorithmsdifferentialLiRAdisparatevulnerabilityfairness-privacytrade-offsubgroupauditing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the common assumption—fairness interventions trade away privacy—is too simple. Auditing membership inference at the subpopulation level, it finds that fairness mitigators can raise, lower, or leave privacy risk unchanged depending on model architecture, subgroup size, and mitigation strategy. The paper extends the Likelihood Ratio Attack (LiRA) to per-group leakage estimates and shows that smaller subgroups are consistently more exposed to membership inference, even after fairness fixes are applied. It also shows that Differential Privacy reliably suppresses membership-inference risk across subgroups but lands its utility costs unevenly, sometimes collapsing accuracy for specific subpopulations to near zero. If right, fairness, privacy, and utility must be evaluated jointly at subgroup granularity rather than at the aggregate level.

What carries the argument

The central mechanism is a subgroup-conditioned Likelihood Ratio Attack (LiRA): instead of drawing in- and out-distributions uniformly, the attack restricts both the target sample and the shadow-model distributions to a single demographic subgroup, producing subgroup-specific likelihood ratios and privacy-risk estimates. The theoretical explanation for the empirical pattern is an influence-function argument: the membership signal Δ_i scales as O(1/n_g), so a sample from a group of size n_g represents a fraction 1/n_g of that group's information, making members of small groups more easily detected; DP adds independent noise that lowers the signal-to-noise ratio, and the same reasoning explain

What would settle it

Re-run the OTA and OQTA audits with thresholds chosen on a separate held-out population set—never on the target data—and compare subgroup privacy-risk orderings; if the disparities between small and large subgroups largely vanish, the paper's central claim about persistent structural exposure after fairness mitigation would fail.

Watch

Extended reading notes

Core claim

The central claim is that fairness interventions do not uniformly increase privacy risk; their effect on membership inference is mediated by model architecture, subgroup size, and mitigation strategy. Adapting LiRA to subgroup-conditioned attack distributions, the authors report that the smallest subpopulations consistently carry the highest membership-inference risk across datasets and attack families, and that this structural disadvantage persists after fairness mitigation. They further report that differential privacy (via DP random forests and DP-SGD) drives subgroup privacy risk toward the random-guessing baseline, but does so at a heavily uneven utility price: privileged and underrepre

Load-bearing premise

The reported subgroup privacy gaps rest on attack thresholds that are tuned on the target data being scored, which makes the attacks optimistic upper bounds; for OTA the non-member rate is also computed from outside the subgroup, so the subgroup risk numbers are not clean subpopulation metrics.

Editorial extensions

If this is right

  • Aggregate privacy metrics are insufficient: membership-inference risk must be reported per subpopulation to see who is actually exposed.
  • Fairness mitigators do not have a universal privacy effect; EGR can strengthen privacy for random forests but worsen it for simpler models, while REW gives modest but stable improvements and DIR behaves inconsistently.
  • Differential privacy reduces subgroup privacy risk to near-random-guessing levels under every tested mitigator, but its utility cost can be catastrophic for specific subgroups, with accuracy sometimes falling below 5%.
  • Subpopulation size remains a persistent driver of privacy vulnerability: smaller subgroups stay the most exposed even after fairness interventions.
  • Practitioners should select strategies by priority—DP with lower-complexity models and conservative mitigators like REW if privacy is paramount—but must verify subgroup-level utility in every case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the O(1/n_g) influence scaling is the driver, then interventions that increase a subgroup's effective sample size—beyond naive oversampling—should measurably reduce its membership-inference risk; this is a testable extension the paper does not run.
  • The same logic suggests a per-group privacy budget: allocating DP noise unevenly, with more protection for groups whose accuracy collapses, could rebalance the fairness–privacy–utility trilemma without giving up formal guarantees.
  • Because OQTA and OTA select thresholds on the target data they later score, the reported risk gaps are optimistic upper bounds; re-running the audit with thresholds fixed on a separate population set is a direct check on whether the group-level disparities persist.
  • The finding that apparent fairness improvements under DP often reflect uniform accuracy loss warns that fairness metrics computed on degraded models can be misleading; a meaningful audit should report utility alongside fairness, as the paper does.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents a broad empirical audit of how five fairness-enhancing algorithms (SYN, DIR, REW, EGR, CPP) affect membership-inference privacy risk at the subpopulation level, across six tabular datasets, five model families, and three attack paradigms (LiRA, OQTA, OTA). It introduces a subgroup-conditioned variant of LiRA and reports that fairness interventions do not uniformly increase privacy risk; the effects depend on model architecture, subgroup size, and mitigation strategy. The paper also studies DP-fairness interactions, reporting that DP suppresses membership-inference risk but imposes uneven utility losses across subgroups. A theoretical section attempts to explain the observed size-vulnerability pattern via influence functions.

Significance. If the empirical claims survive scrutiny, this would be a useful contribution to the fairness-privacy auditing literature: it broadens the evidence base to multiple fairness-intervention families, multiple model classes, and subpopulation-level metrics, and it ships code and experimental details. The paper also correctly identifies that aggregate privacy metrics can mask subgroup disparities. However, the two non-parametric attacks used as cross-validation of LiRA contain protocol errors that bias the reported subgroup risks and undermine the paper's claim that the size-vulnerability pattern is robust 'across all attacks.' The LiRA-based results may still be meaningful, but the manuscript's current cross-attack consistency argument is not valid, so the central claims require revision and re-analysis.

major comments (4)
  1. [Section 4.2, Algorithm 36] OQTA selects per-subgroup thresholds by maximizing accuracy on the target dataset itself: lines 16-22 evaluate candidate thresholds on LD(target)_{g,y}, and lines 30-32 compute the reported accuracy on the same TARGET_MEMBER ∪ TARGET_NON_MEMBER split. This is an in-sample, label-dependent selection of the attack's only parameter, producing an optimistic upper bound on attack success. The bias is stronger for smaller subgroups because the target split has fewer samples, so the reported pattern that smaller subgroups are more exposed can be amplified. Thresholds should be selected on a disjoint held-out set (or via cross-validation) and then evaluated on the target split.
  2. [Section 4.3, Eq. (16)] The OTA subgroup metric is not correctly defined. Definition (9) requires TNR_Gj = Pr(Attack says 'non-member' | x in Gj ∩ test), but Eq. (16) defines TNR_Gj as the average over x' in non-Gj. Thus PrivacyRisk(Gj) mixes the subgroup's TPR with the complement's TNR and is not a subpopulation-level risk. For a small subgroup, the TNR term is dominated by the large complement, so the reported OTA subgroup values do not measure the subgroup. This affects all OTA-based tables and figures, including Tables 6 and 10 and Figures 12-14.
  3. [Section 4.1 and Section 7.1] The manuscript explicitly uses OQTA and OTA as the non-parametric cross-validation of LiRA ('Robustness for Small Subpopulations'). Because both attacks have the protocol errors above, the consistency argument does not hold. The claim in Section 7.1 that the size-vulnerability pattern is 'robust across all attacks' and in Section 7.4 that it holds 'regardless of the mitigator or attack applied' is not supported by the current experiments. The LiRA-based results may still stand, but the paper needs to re-run corrected non-parametric attacks and re-evaluate the ordering of mitigators and the size-vulnerability claim.
  4. [Section 6, Eq. (17)] The theoretical characterization relies on the approximation H^{-1} ≈ (n/n_g) H_g^{-1}, stated to hold if groups have distinct feature distributions or group-specific parameters. Distinct feature distributions alone do not block-diagonalize the Hessian for a shared-parameter model such as logistic regression; cross-group Hessian blocks remain. The listed conditions are therefore not sufficient for the claimed O(1/n_g) scaling. Since Section 6 is presented as a 'theoretical explanation' of the empirical findings, it should either be proved under precise conditions or explicitly labeled a heuristic/intuition, not a derivation. This is not the main empirical claim, but it is a stated contribution and should be corrected.
minor comments (4)
  1. [Tables 2-3] The 'Privacy' columns contain values such as -0.75% and 6.10%, which are inconsistent with the definition of PrivacyRisk as a proper accuracy score in [0,1] and with the caption's statement that ✓ means 'privacy risk goes down to 50±2%'. Please clarify whether these entries are risk differences, raw risks, or something else, and ensure the table headers and captions match the displayed numbers.
  2. [Algorithm 36, line 17] The pseudocode says 'Compute accuracy on LD(target)' while Section 4.2 says the threshold maximizes balanced accuracy. Align the pseudocode with the prose, and specify whether 'accuracy' means balanced accuracy or plain accuracy.
  3. [Section 6 and Appendix text] Several passages contain unreadable font artifacts (e.g., '/uni00000032/...' sequences) in the full text, particularly in Section 6 and figure captions. These artifacts must be fixed in the source so that equations and captions are legible.
  4. [Section 5, Models] The text says neural networks use 'two hidden layers for the Law dataset,' but Section 5 lists both Law School Admissions and Law School GPA. Specify which dataset or clarify the architecture by dataset.

Circularity Check

2 steps flagged · score 4.0 of 10

OQTA/OTA thresholds are fit on the data they score and OTA's TNR_Gj is computed over non-Gj, invalidating the cross-attack confirmation; the LiRA-based audit is independent, so circularity is partial.

  1. fitted input called prediction [Section 4.2, Algorithm 36, lines 16–22 and 30–32; cf. Eq. (14)]
    "for all τ∈ τ g,y do Compute accuracy on LD(target) g,y if accuracy > best_accuracy then best_accuracy← accuracy τ (g,y)← τ end if end for ▷ Threshold derived from population data, selected via target data's best accuracy ... Compute Accuracy and privacy risk (PR) for Gy g"

    The OQTA threshold τ(g,y) is selected by maximizing balanced accuracy on the target split (TARGET_MEMBER ∪ TARGET_NON_MEMBER), and the same split is then used to compute the reported PrivacyRisk. The reported attack success is therefore an in-sample optimum, not an independent measurement. For smaller subgroups, optimizing over fewer samples yields a larger expected maximum, so the reported small-subgroup vulnerability gap can be inflated by construction. OTA's Eq. (14) uses the same in-sample threshold selection on training and test losses.

  2. other [Section 4.3, Eq. (16); cf. Definition (9) in Section 3.3]
    "TNRGj = P x′∈non-Gj 1{ℓx′ > c j}/|non-Gj| (16) ... Definition (9): TNRGj = Pr(Attack says “non-member” | x∈ Gj∩ test)"

    Definition (9) defines the subgroup-specific TNR as the non-member rate over Gj∩test, but Eq. (16) computes TNR_Gj using x′∈non-Gj. Hence the PrivacyRisk(Gj) reported for OTA is not a subgroup metric at all: it mixes in the complement subgroup's non-member rate. The OTA-based confirmation of subgroup-level vulnerability is therefore not a valid cross-check because the quantity being measured is not the defined subpopulation risk.

full rationale

The empirical core of the paper is not circular: the LiRA subgroup audit trains shadow models on disjoint data and computes a likelihood ratio with no threshold fit on the target examples; the size-vulnerability claim is also independently grounded in prior work (e.g., Kulynych et al.). However, the paper's non-parametric cross-checks are not independent. OQTA selects per-subgroup thresholds by maximizing balanced accuracy on the target split and then reports accuracy/privacy risk on that same split; OTA does the same for c in Eq. (14). Additionally, OTA's Eq. (16) defines TNR_Gj over non-Gj, contradicting Definition (9). These flaws invalidate the cross-attack consistency argument and can amplify the reported small-group disadvantage, but they do not reduce the LiRA-based central claim to a fit. Section 6 candidly states it is a post-hoc explanation ('Before presenting our full empirical analysis, we provide a concise theoretical explanation for the main phenomena observed'); because it is conditional on explicitly stated assumptions and is not used to generate the empirical findings, it does not constitute circularity. No load-bearing self-citations were found. Score 4 reflects partial circularity in the auxiliary attacks while the central audit retains independent content.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The empirical audit's main burden is the attack-evaluation procedure; the theory section adds an ad hoc group-localization assumption that produces the O(1/n_g) scaling. No new physical or mathematical entities are postulated.

free parameters (2)
  • OQTA/OTA per-subgroup thresholds tau(g,y) = Selected by maximizing balanced accuracy on target member/non-member sets
    The thresholds are chosen on the same target data later used to compute TPR/TNR, so the reported privacy-risk values are partly fitted rather than purely predictive (Algorithm 36 lines 15-28; Eq 14).
  • Synthetic-data Gaussian parameters (mu, Sigma per group-label) = Four means/covariance matrices specified in Section 5
    Hand-chosen to create imbalanced groups and class skew; affects only the synthetic benchmark, not the real-data claims.
assumptions (4)
  • standard math Influence-function approximation: leave-one-out parameter difference ≈ (1/n) H^{-1} grad(loss) under standard M-estimation regularity
    Section 6.1 invokes classical influence-function results [11,20,25] to derive the membership signal Delta_i.
  • ad hoc to paper Group-specific Hessian decomposition: H^{-1} ≈ (n/n_g) H_g^{-1} when groups occupy distinct feature regions or parameters specialize
    Section 6.2 calls this a 'simplified case'; it is the step that produces the load-bearing O(1/n_g) scaling. The assumption is chosen to match the observed minority-vulnerability pattern and is not independently verified.
  • domain assumption DP parameter noise modeled as independent additive Gaussian eta ~ N(0, sigma^2 I) on final weights
    Section 6.3 uses this first-order approximation to argue DP reduces MIA signal-to-noise and disproportionately harms small groups; DP-SGD actually injects noise on gradients, not i.i.d. parameter noise.
  • domain assumption The MIA test statistic is correlated with the per-sample score difference Delta_i between the model trained with and without the sample
    Section 6 Setup narrows the theory to score-difference (LiRA-like) attacks; OQTA and OTA use losses and thresholds that are outside this exact characterization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Auditing Fairness-Privacy Trade-offs: Subpopulation-Level Effects of Fairness-Enhancing Algorithms." pith.science (2026). https://pith.science/paper/THKJXZ35

@misc{pith2026260714607,
  author       = {Pith},
  title        = {Pith review of: Auditing Fairness-Privacy Trade-offs: Subpopulation-Level Effects of Fairness-Enhancing Algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/THKJXZ35}},
  note         = {Machine review of arXiv:2607.14607}
}
read the original abstract

Machine learning (ML) models deployed in sensitive domains such as healthcare, law enforcement, and finance must satisfy not only utility requirements but also fairness and privacy guarantees. While prior work has largely examined how privacy-preserving techniques affect fairness, the inverse question-how fairness-enhancing algorithms influence privacy leakage-remains underexplored. We present the first comprehensive study of how fairness interventions affect membership inference privacy risks at the subpopulation level. By adapting the Likelihood Ratio Attack (LiRA) for subgroup auditing, we uncover privacy disparities that aggregate evaluations obscure. We further analyze how Differential Privacy (DP) interacts with fairness-enhancing methods across different categories, showing that DP's privacy benefits and utility costs are unevenly distributed across subpopulations. Our results demonstrate that fairness interventions do not uniformly increase privacy risk; their impact depends on model architecture, subgroup size, and mitigation strategy. These findings reveal that fairness, privacy, and utility must be jointly evaluated at the subpopulation level, and we introduce the first unified empirical framework to support such auditing in practice.

Figures

Figures reproduced from arXiv: 2607.14607 by the authors.

Figure 1
Figure 1. Subpopulation-level membership inference vulnerability across datasets and fairness interventions. Plots show MIA success rates for every subgroup, dataset, and fairness mitigation strategy. Across all datasets minority subgroups exhibit consistently higher privacy leakage, while some fairness methods (e.g., egr) reduce subgroup-specific risk and others shift vulnerability unevenly. the model trained with and withou… view at source ↗
Figure 2
Figure 2. Decision Tree privacy risk results under OQTA for: Bank (Age), COMPAS (Race), MEPS (Race), and Synthetic (G) datasets. It visualizes subpopulation privacy risks across different fairness mitigation techniques. The red dashed line indicates baseline risk from the original unmitigated model. Values are averaged over 20 runs, while the standard deviation is shown with error bars. The horizontal range is dynamic for bet… view at source ↗
Figure 3
Figure 3. Radial plot comparing the subpopulation most affected by the CPP mitigator in each dataset for the Decision Tree (DT) model under the OTA attack. For every dataset, the plotted value corresponds to the subpopulation with the largest positive gap between the original and CPP test accuracies, indicating where CPP reduces accuracy the most. Dataset labels marked with an asterisk (*) denote cases where this most￾affecte… view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: Distribution of data samples across all subpopulations for each dataset. TABLE 3. Subpopulation utility difference (difference between the test accuracy for the subpopulation before and after application of DP) and privacy risks for subpopulation after the application …
Figure 5
Figure 5. Figure 5: Comparison of mean test accuracies across subpopula￾tions and fairness mitigation methods for different models under LiRA setup (RF and DPRF with varying privacy epsilons). Each point on the radar represents the average accuracy across subpopulations and methods for a …
Figure 6
Figure 6. Figure 6: Fairness metric (average odds difference) results under LiRA attack. Lower values indicate better fairness. achieves near-random-guessing privacy levels but at the cost of unpredictable subgroup utility redistribu￾tion, undermining its practical reliability. Crucially,…
Figure 10
Figure 10. Figure 10: Neural Network privacy risk results under LiRA attack for: Bank (Age), COMPAS (Race), and MEPS (Race) datasets. Visualizes subpopulation privacy risks across different fairness mitigation techniques. The red dashed line indicates baseline risk from the original unmiti…
Figure 11
Figure 11. Figure 11: Neural Network privacy risk results under OQTA for: Bank (Age), COMPAS (Race), and MEPS (Race) datasets. Visualizes subpopulation privacy risks across different fairness mitigation techniques. The red dashed line indicates baseline risk from the original unmitigated m…
Figure 12
Figure 12. Figure 12: Decision Tree privacy risk results under OT at￾tack for: COMPAS (Gender), Law (Gender), and Law (Race) datasets. Visualizes subpopulation privacy risks across different fairness mitigation techniques. The red dashed line indicates baseline risk from the original unmit…
Figure 13
Figure 13. Figure 13: Random Forest privacy risk results under OT at￾tack for: COMPAS (Gender), Law (Gender), and Law (Race) datasets. Visualizes subpopulation privacy risks across different fairness mitigation techniques. The red dashed line indicates baseline risk from the original unmit…
Figure 14
Figure 14. Figure 14: Neural Network privacy risk results under OT at￾tack for: COMPAS (Gender), Law (Gender), and Law (Race) datasets. Visualizes subpopulation privacy risks across different fairness mitigation techniques. The red dashed line indicates baseline risk from the original unmi…
Figure 15
Figure 15. Figure 15: Decision Tree privacy risk results under OQTA for: German (Age), German (Gender), Law School GPA (Gender), and Law School GPA (Race) datasets. Visualizes subpopulation privacy risks across different fairness mitigation techniques.The red dashed line indicates baseline…
Figure 22
Figure 22. Figure 22: Random Forest fairness metric (equal opportunity difference) results under LiRA attack for all datasets. Lower values indicate better fairness [PITH_FULL_IMAGE:figures/full_fig_p021_22.png]
Figure 23
Figure 23. Figure 23: Random Forest fairness metric (theil index) results under LiRA attack for all datasets. Lower values indicate better fairness [PITH_FULL_IMAGE:figures/full_fig_p021_23.png]
Figure 24
Figure 24. Figure 24: DPRF fairness metric (balanced accuracy) results under LiRA attack for all datasets [PITH_FULL_IMAGE:figures/full_fig_p021_24.png]
Figure 25
Figure 25. Figure 25: DPRF fairness metric (disparate impact) results under LiRA attack for all datasets. Lower values indicate better fairness [PITH_FULL_IMAGE:figures/full_fig_p021_25.png]
Figure 29
Figure 29. Figure 29: Disparate Impact (absolute, transformed). Lower is better (0 indicates parity under our definition) [PITH_FULL_IMAGE:figures/full_fig_p022_29.png]
Figure 30
Figure 30. Figure 30: Balanced Accuracy (raw). Higher is better. Shows the utility impact of CPP at the aggregate level [PITH_FULL_IMAGE:figures/full_fig_p022_30.png]
Figure 31
Figure 31. Figure 31: Theil Index. Lower is better; measures inequality in predicted outcomes across groups [PITH_FULL_IMAGE:figures/full_fig_p022_31.png]
Figure 32
Figure 32. Figure 32: Subpopulation: Unprivileged Unfavorable. Test accu￾racies by dataset for Orig vs. CPP. Hatched bars mark datasets where this subpopulation is underrepresented [PITH_FULL_IMAGE:figures/full_fig_p023_32.png]
Figure 33
Figure 33. Figure 33: Subpopulation: Unprivileged Favorable. Test accura￾cies by dataset for Orig vs. CPP. Hatched bars mark datasets where this subpopulation is underrepresented [PITH_FULL_IMAGE:figures/full_fig_p023_33.png]
Figure 34
Figure 34. Figure 34: Subpopulation: Privileged Unfavorable. Test accura￾cies by dataset for Orig vs. CPP. Hatched bars mark datasets where this subpopulation is underrepresented [PITH_FULL_IMAGE:figures/full_fig_p023_34.png]
Figure 36
Figure 36. Figure 36: Optimized Quantile Threshold Attack 1: Input: Target model A, target dataset Dtarget, population dataset Dpopulation, quantiles Q (e.g., logspace(-5,0,100)) 2: Output: Privacy risk metrics for all subgroups 3: Split Dtarget into TARGET_MEMBER (train subset) and TARGET…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 6 linked inside Pith

  1. [1]

    Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang

    Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the2016ACMSIGSACConferenceonComputerandCom- munications Security, CCS ’16, page 308–318, New York, NY, USA, 2016. Association for Computing Machinery

    2016

  2. [2]

    A reductions approach to fair classification

    Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. A reductions approach to fair classification. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 60–69. PMLR, 2018

    2018

  3. [3]

    Aithal and R

    V. Aithal and R. D. Jathanna. Credit risk assessment using machine learning techniques. InternationalJournal of Inno- vative Technology and Exploring Engineering, 9(1):3482– 3486, 2019

    2019

  4. [4]

    Evaluating marketing campaigns of banking using neural networks

    Qeethara Al-Shayea. Evaluating marketing campaigns of banking using neural networks. Lecture Notes in Engineer- ing and Computer Science, 2:759–761, 07 2013

    2013

  5. [5]

    Differential privacy has disparate impact on model accuracy

    Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. Differential privacy has disparate impact on model accuracy. In Advances in Neural Information Pro- cessing Systems, volume 32, 2019

    2019

  6. [6]

    R. K. E. Bellamy, K. Dey, M. Hind, S. C. Hoff- man, S. Houde, K. Kannan, P. Lohia, J. Martino, S. Mehta, A. Mojsilović, S. Nagar, K. Natesan Rama- murthy, J. Richards, D. Saha, P. Sattigeri, M. Singh, K. R. Varshney, and Y. Zhang. Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development,...

    2019

  7. [7]

    Evaluating the fairness impact of differentially private synthetic data

    Blake Bullwinkel, Chris Waites, Stephen Ranshous, Daniel Kifer, and Michael Hay. Evaluating the fairness impact of differentially private synthetic data. arXiv preprint arXiv:2205.04321, 2022

    arXiv 2022

  8. [8]

    Membership inference attacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP), pages 1897–1914. IEEE, 2022

    2022

Show all 41 references
  1. [9]

    On the privacy risks of algorithmic fairness

    Hongyan Chang and Reza Shokri. On the privacy risks of algorithmic fairness. arXiv preprint arXiv:2011.03731, 2021

    2011 arXiv

  2. [10]

    Chawla, Kevin W

    Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. Smote: synthetic minority over- sampling technique. J. Artif. Int. Res., 16(1):321–357, June 2002

    2002

  3. [11]

    Residuals and influence in regression

    R Dennis Cook and Sanford Weisberg. Residuals and influence in regression. New York: Chapman and Hall, 1982

    1982

  4. [12]

    The accuracy, fairness, and limits of predicting recidivism

    Julia Dressel and Hany Farid. The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1):eaao5580, 2018

    2018

  5. [13]

    Fairness via representation neutralization

    Mengnan Du, Subhabrata Mukherjee, Guanhong Wang, Raphael Tang, Ahmed Hassan Awadallah, and Xia Hu. Fairness via representation neutralization. arXiv preprint arXiv:2106.12674, 2021

    2021 arXiv

  6. [14]

    Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian

    Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and removing disparate impact. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 259–268. ACM, 2015

    2015

  7. [15]

    Does learning require memorization? a short tale about a long tail, 2021

    Vitaly Feldman. Does learning require memorization? a short tale about a long tail, 2021

    2021

  8. [16]

    Whatneuralnetworks memorize and why: Discovering the long tail via influence estimation

    VitalyFeldmanandChiyuanZhang. Whatneuralnetworks memorize and why: Discovering the long tail via influence estimation. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 2881–

  9. [17]

    Differential privacy and fairness in deci- sions and learning tasks: A survey

    Ferdinando Fioretto, Chi Tran, Pascal Van Hentenryck, and Kai Zhu. Differential privacy and fairness in deci- sions and learning tasks: A survey. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI), pages 5470–5477, July 2022

    2022

  10. [18]

    Decision making with differential privacy under a fairness lens, 2024

    Ferdinando Fioretto, Cuong Tran, and Pascal Van Hen- tenryck. Decision making with differential privacy under a fairness lens, 2024

    2024

  11. [19]

    Why do tree-based models still outperform deep learning on typical tabular data? In Advances in Neural Information Processing Systems, volume 35, pages 507–520, 2022

    Léo Grinsztajn, Edouard Borchi, and Gaël Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data? In Advances in Neural Information Processing Systems, volume 35, pages 507–520, 2022

    2022

  12. [20]

    Robuststatistics:theapproachbased on influence functions

    Frank R Hampel, Elvezio M Ronchetti, Peter J Rousseeuw, andWernerAStahel. Robuststatistics:theapproachbased on influence functions. John Wiley & Sons, 1986

    1986

  13. [21]

    The impact of differential privacy on group disparity mitigation

    Victor Hansen, Atula Neerkaje, Ramit Sawhney, Lucie Flek, and Anders Søgaard. The impact of differential privacy on group disparity mitigation. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Findings of the Association for Computational Linguistics: NAACL 2024, pages...

    2024

  14. [22]

    Diffprivlib: The ibm differential privacy library

    Naoise Holohan, Stefano Braghin, Pól Mac Aonghusa, and Siobhán Levacher. Diffprivlib: The ibm differential privacy library. arXiv preprint arXiv:1907.02444, 2019

    1907 arXiv

  15. [23]

    Differentially private fair learning

    Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi Malvajerdi, and Jonathan Ullman. Differentially private fair learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Ma- chine ...

    2019

  16. [24]

    Datapreprocessingtech- niques for classification without discrimination

    FaisalKamiranandToonCalders. Datapreprocessingtech- niques for classification without discrimination. Knowledge and Information Systems, 33(1):1–33, 2012

    2012

  17. [25]

    Understanding black- box predictions via influence functions

    Pang Wei Koh and Percy Liang. Understanding black- box predictions via influence functions. In International Conference on Machine Learning (ICML), pages 1885–

  18. [26]

    Disparate vulnerability to membership inference attacks

    Bogdan Kulynych, Mohammad Yaghini, Giovanni Cheru- bin, Michael Veale, and Carmela Troncoso. Disparate vulnerability to membership inference attacks. Proceedings onPrivacyEnhancingTechnologies,2022(1):460–480,2022

    2022

  19. [27]

    Arcolezi, and Catuscia Palamidessi

    Karima Makhlouf, Tamara Stefanovic, Heber H. Arcolezi, and Catuscia Palamidessi. A systematic and formal study of the impact of local differential privacy on fairness: Preliminary results, 2024

    2024

  20. [28]

    Differential privacy has bounded impact on fairness in classification, 2023

    Paul Mangold, Michaël Perrot, Aurélien Bellet, and Marc Tommasi. Differential privacy has bounded impact on fairness in classification, 2023

    2023

  21. [29]

    Ml pri- vacy meter: Aiding regulatory compliance by quantifying the privacy risks of machine learning

    Sasi Kumar Murakonda and Reza Shokri. Ml pri- vacy meter: Aiding regulatory compliance by quantifying the privacy risks of machine learning. arXiv preprint arXiv:2007.09339, 2020

    2007 arXiv

  22. [30]

    Increasing the views and reducing the depth in random forest

    Abolfazl Nadi and Hadi Moradi. Increasing the views and reducing the depth in random forest. Expert Systems with Applications, 138:112801, 2019

    2019

  23. [31]

    A comprehensive sustainable framework for machine learning and artificial intelligence, 2024

    Roberto Pagliari, Peter Hill, Po-Yu Chen, Maciej Dabrowny, Tingsheng Tan, and Francois Buet-Golfouse. A comprehensive sustainable framework for machine learning and artificial intelligence, 2024

    2024

  24. [32]

    Weinberger

    Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q. Weinberger. On fairness and calibration. In Advances in Neural Information Processing Systems, volume 30, 2017

    2017

  25. [33]

    Understanding MachineLearning:FromTheorytoAlgorithms

    Shai Shalev-Shwartz and Shai Ben-David. Understanding MachineLearning:FromTheorytoAlgorithms. Cambridge University Press, USA, 2014

    2014

  26. [34]

    When fairness meets pri- vacy: Exploring privacy threats in fair binary classifiers via membership inference attacks

    Huan Tian, Guangsheng Zhang, Bo Liu, Tianqing Zhu, Ming Ding, and Wanlei Zhou. When fairness meets pri- vacy: Exploring privacy threats in fair binary classifiers via membership inference attacks. pages 512–520, 08 2024

    2024

  27. [35]

    Dinh, and Ferdinando Fioretto

    Cuong Tran, My H. Dinh, and Ferdinando Fioretto. Dif- ferentially empirical risk minimization under the fairness lens, 2022

    2022

  28. [36]

    Effectsofdifferentialprivacyanddataskewness on membership inference vulnerability

    Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Wenqi Wei, andLeiYu. Effectsofdifferentialprivacyanddataskewness on membership inference vulnerability. In Proceedings of the 2019 IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (...

    2019

  29. [37]

    Disparate vulnerability: on the unfairness of privacy attacks against machine learning

    Mohammad Yaghini, Bogdan Kulynych, and Carmela Troncoso. Disparate vulnerability: on the unfairness of privacy attacks against machine learning. arXiv preprint arXiv:1906.00389, 2019

    1906 arXiv

  30. [38]

    Privacy risk in machine learning: Analyzing the connection to overfitting

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In Proceedings of the 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pages 268–282. IEEE, 2018

    2018

  31. [39]

    Understanding disparate effects of membership inference attacks and their countermeasures

    Da Zhong, Haipei Sun, Jun Xu, Neil Zhenqiang Gong, and Wendy Hui Wang. Understanding disparate effects of membership inference attacks and their countermeasures. In Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, pages 62–75, 2022

    2022

  32. [40]

    On Improving Fairness of AI Models with Synthetic Minority Oversampling Techniques, pages 874–882

    Yan Zhou, Murat Kantarcioglu, and Chris Clifton. On Improving Fairness of AI Models with Synthetic Minority Oversampling Techniques, pages 874–882. SIAM, 2023. Appendix A. Privacy Risk Graphs Figure 7.Decision Tree privacy risk results under LiRA attack for: Bank (Age), COMPAS...

    2023

  33. [2891]

    Curran Associates, Inc., 2020

    2020

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.