Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry

Niladri Sett; Supriya Manna

REVIEW 4 major objections 6 minor 39 references

Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that differentially private models cannot be meaningfully explained by standard gradient-based post-hoc explainers, and that the only practical route to both privacy and explanation is to privatize the explanation itself…

desk verdict A serious empirical study of DP and post-hoc explainability, but the universal negative claim is not yet supported by the evidence; it deserves revision rather than rejection. read the letter →

arxiv 2412.20798 v4 pith:TVVAMTSU submitted 2024-12-30 cs.CR cs.AIcs.CV

Supriya Manna , Niladri Sett This is my paper · ORCID

classification cs.CRcs.AIcs.CV

keywords differentialprivacypost-hocexplainerslocalizationassumptioninvariancescorehybridDPchestX-rayclassificationrighttoexplanationgradient-basedattribution

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether right-to-privacy and right-to-explanation can be satisfied together in high-stakes AI, focusing on differential privacy (DP) models and post-hoc explainers. It argues they cannot: under a proposed Localization Assumption, every tested gradient-based explainer produces feature attributions that are nearly uncorrelated between a DP model and its non-private twin, with the Privacy Invariance Score never exceeding 0.32. The mechanism is that DP training leaves layerwise representations only mildly similar while making the sensitivity gradients that explainers rely on statistically independent across models. As an alternative, the paper shows that applying local differential privacy to the explanations of a non-private model yields useful private explanations, and it outlines a software pipeline built on that idea.

What carries the argument

The Localization Assumption (LA) is the postulate that a private proxy model's explanations should resemble the original model's, sim(s,s') >= theta, whenever the two models agree on the prediction. The Privacy Invariance Score (PIS) operationalizes LA by first computing a disagreement score on the sign of attributions (with a 15% threshold) and then measuring Kendall's tau between positive attributions. The argument also uses deconfounded CKA for layerwise representational similarity and HSIC independence tests for the sensitivity gradients, showing that the former are comparable while the latter are independent.

What would settle it

Compute the Privacy Invariance Score for a held-out chest X-ray test set with a pre-registered threshold (for example, Kendall's tau > 0.5) and compare the DP versus non-private heatmaps from any one of the five tested explainers; if even one explainer exceeds that threshold, the paper's blanket negative claim is falsified. A sharper test is to check the mechanism directly: if the sensitivity gradients (∇σ_lΘ) of a DP model and its non-private twin turn out to be statistically dependent under HSIC in a replication, then the paper's explanation for the incompatibility collapses.

Watch

Extended reading notes

Core claim

The central claim is that off-the-shelf gradient-based explainers and DP models are fundamentally incompatible for audit purposes. For inputs where the private and non-private models make the same prediction, attribution maps differ so much that the Privacy Invariance Score never rises above 0.32, and Integrated Gradients and Grad-Shap fail even a 15% sign-disagreement sanity check. The paper traces this to DP-SGD training: representations across layers remain statistically dependent and moderately similar in a deconfounded CKA sense, but the gradients of the output with respect to layer activations are statistically independent between the private and non-private models, and gradient-based explainers are functions of exactly these gradients. Therefore no first-order gradient explainer can serve as a proxy for the non-private model's reasoning. The paper also shows that privatizing the explanation instead of the model—by adding calibrated Laplace noise to grid-averaged heatmaps from a non-private model, a setup called Hybrid DP—produces explanations with SSIM around 0.4–0.5 that a physician found useful from an epsilon of 4 onward.

Load-bearing premise

The Localization Assumption holds that a useful explanation from a private model must be similar to the original model's explanation, sim(s,s') >= theta, but the paper never assigns a concrete value to theta or calibrates it against any baseline, so the claim that all explainers 'disregard LA' is judged by an unmeasured, self-defined threshold.

Editorial extensions

If this is right

If the paper is right, DP-trained models cannot be audited with Saliency, SmoothGrad, Integrated Gradients, Grad-Shap, or Grad-CAM; their heatmaps are effectively uncorrelated with the non-private model's and would mislead stakeholders.
Right-to-explanation for privacy-preserving systems requires either Hybrid DP (keep the model non-private, add local DP noise to the explanation) or inherently interpretable private models that do not rely on first-order gradients.
The negative finding is not a failure of faithfulness metrics but of the sensitivity space itself, so tweaking explainer hyperparameters or aggregation strategies will not fix the incompatibility.
High-stakes pipelines that must respect both rights should release only the prediction label and a noised explanation, accepting the compromise of not exposing prediction vectors or fully private models.
The result appears to extend beyond the chest X-ray case: the paper reports the same trend on a second benchmark dataset, suggesting the incompatibility is structural rather than dataset-specific.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An immediate extension is to calibrate the Localization Assumption threshold theta against a baseline, such as the similarity between two independent explainers applied to the same model or between explanations of two models retrained from different seeds; the paper never anchors theta, so its negative verdict depends on an unmeasured criterion.
A testable prediction of the paper's mechanism is that any privacy-training method that changes model parameters enough to decorrelate sensitivity gradients—not just DP-SGD—will also break first-order gradient explainers, so the claim should be checked against PATE or other DP training schemes.
The paper's mechanism suggests that inherently interpretable models whose explanations are not first-order gradients, such as attention-based or prototype-based networks, might preserve explanation similarity under DP training, which would be a direct test of the generality of the negative result.
The proposed Hybrid DP pipeline leaves the model non-private; a natural extension is to quantify the remaining membership-inference risk from the released labels and noised heatmaps, since the paper only claims the explanations are LDP and does not fully characterize the attack surface of the full pipeline.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a genuinely useful empirical study of a real tension between differential privacy and post-hoc explainability, but the headline claim—that all gradient-based explainers are unsuitable for DP models—is not pinned down by the current experiments. The paper deserves peer review and revision, not a desk reject.

What is actually new: the PIS/LA evaluation framework, the layerwise dCKA plus HSIC decomposition of where DP training changes representations and sensitivities, and the Hybrid DP pipeline that adds LDP noise to explanations instead of training a DP model. Those are real contributions. The experiments are substantial: three CNN architectures, six epsilon values, five explainers on a medical imaging task, plus a CIFAR-10 replication. The paper is also honest about its own limitations, for example in Section 9 where it concedes the proposed pipeline is inappropriate when prediction vectors are required.

The soft spots are exactly where the stress-test lands. The Localization Assumption sets sim(s,s') >= theta, but theta is never assigned a value or calibrated against any baseline. Section 6 then declares PISAvg at most 0.32 as evidence that no explainer satisfies LA, with no null distribution or non-private baseline to anchor that number. The universal claim is also broader than the surviving evidence: Integrated Gradients and Grad-Shap are dropped after the DS stage, and 30–40% of Grad-CAM samples are excluded, leaving Saliency, SmoothGrad, and partial Grad-CAM. Without a baseline such as explainer agreement between two non-private retrained models, low correlation cannot be attributed specifically to DP. The mechanistic argument in Section 7.2 compounds this by treating failure to reject the HSIC null as proof of independence, which is not valid. These are load-bearing flaws, not cosmetic ones.

That said, the negative result may be true. The paper just does not establish it on the current definitions and thresholds. The empirical measurements and the proposed hybrid pipeline are worth preserving, and the authors have clearly thought hard about the problem and engaged with the relevant literature.

For peer review: yes, send it out. A good referee can push the authors to calibrate theta, add a non-private baseline, soften the universal claim, and correct the HSIC interpretation. With those changes this could be a solid contribution to privacy-XAI.

Referee Report

4 major / 6 minor

Summary. This paper asks whether standard post-hoc gradient-based explainers can audit differentially private (DP) models in high-stakes applications. It introduces the Localization Assumption (LA) and the Privacy Invariance Score (PIS) as desiderata for private explanations, then reports experiments on a chest X-ray dataset with three CNN architectures (ResNet-34, DenseNet-121, EfficientNet-v2), six privacy budgets (epsilon in {0.4, 0.7, 1, 4, 7, 10}), and five explainers (Saliency, SmoothGrad, Integrated Gradients, Grad-Shap, Grad-CAM). The central claim is that all gradient-based explainers disregard LA and are therefore unsuitable for DP models (Section 6), with a mechanistic explanation based on dCKA representational similarity and HSIC independence tests (Section 7). The paper concludes by proposing a Hybrid DP pipeline that uses LDP-fied explanations from a non-private model instead of DP-trained models.

Significance. If established, the negative result would be significant: it would show that DP-trained models cannot be audited with standard off-the-shelf heatmaps, so high-stakes deployments meeting both Right-to-Privacy and Right-to-Explanation must adopt alternative mechanisms such as the proposed Hybrid DP pipeline. The paper's strengths include a systematic comparison across multiple architectures and privacy budgets, the explicit articulation of LA/PIS as testable desiderata, the use of deconfounded CKA to address known confounders, and an honest acknowledgment that the proposed pipeline is not suitable when both the prediction vector and private explanations are required (Section 9). However, the empirical support for the central claim is weakened by an uncalibrated similarity threshold, selective exclusion of explainers and samples, and an overreading of HSIC failure-to-reject; the paper is best viewed as a framework proposal whose headline negative claim requires further anchoring.

major comments (4)

[Section 4.2, Section 6] Assumption 1 defines LA via sim(s,s') >= theta but never assigns theta a value or calibrates it against any baseline. Section 6 states that PISAvg never crosses 0.32 and that this indicates 'none of the explainers agree with the LA', yet without a null distribution or a baseline (e.g., PIS between two non-private models with different random seeds), a value of 0.32 cannot be interpreted as a violation of LA. The central negative claim therefore rests on an unanchored, self-defined criterion.
[Section 6] The universal statement 'all gradient-based explainers disregard LA' exceeds the evidence. Integrated Gradients and Grad-Shap are excluded after the DS stage (DS > 45% against a 15% threshold), and 30-40% of Grad-CAM test samples are dropped for the same reason. The PISAvg is computed only for Saliency, SmoothGrad, and the remaining Grad-CAM subset. The conclusion needs either to be restricted to the explainers actually measured, or the excluded explainers need to be assessed under a less stringent DS criterion to support the universal claim.
[Section 7.2] The mechanistic conclusion that sensitivities are 'independent across models' is inferred from failing to reject the HSIC null hypothesis. Non-rejection is not evidence of independence; with finite samples and a test of limited power, dependence can easily go undetected. The paper should report effect sizes or a power analysis, and the language in Section 7.2 ('fundamentally nonfunctional', 'firm conclusion') should be softened unless the test is shown to be adequately powered.
[Section 6, Figures 1-3] There is no control condition in which two non-private models of the same architecture and training procedure, differing only in random seed, are compared with the same PIS protocol. Without such a baseline, the low PISAvg cannot be attributed specifically to DP training; gradient-based attributions are known to be sensitive to any parameter perturbation, and the observed disagreement might not be DP-specific. Adding this control is necessary to support the paper's claim that DP models are uniquely incompatible with these explainers.

minor comments (6)

[Abstract and Section 1] 'wildly used' should be 'widely used' in both places.
[Section 7.1] 'constrain,ts' contains a typo; it should be 'constraints'.
[Section 6, Figures 1-3] The captions mention PISAvg, Acc, and Agreement, but the legend symbols (e.g., filled vs. unfilled markers) are not explained in the text; please clarify.
[Section 8.1] 'the postulates of LA does hold' should be 'do hold'.
[Appendix A] 'exuberant computational requirements' should be 'exorbitant computational requirements'.
[Throughout] The manuscript alternates between 'RTP' and 'Right-to-Privacy' without a consistently introduced abbreviation; a thorough copyedit would improve readability.

Circularity Check

1 steps flagged · score 4.0 of 10

Central negative verdict rests on an uncalibrated, self-defined similarity threshold: LA is defined as sim(s,s')≥θ with θ never set, and Section 6 infers 'none agree' from PISAvg≤0.32, so the conclusion is not independently pinned down.

self definitional [Section 4.2 (Assumption 1) and Section 6]
"The Localization Assumption (LA). For a given tuple (M,M′,x,I) having M(x)=M′(x), sim(s,s′)>=θ. Where θ is a predefined similarity threshold. ... In fact, the PISAvg never crosses 0.32; such a low PISAvg indicates that none of the explainers agree with the LA."

The LA is defined by the paper as sim(s,s')≥θ, with θ described only as 'predefined' and never given a value, baseline, or null distribution. The central negative verdict—'none of the explainers agree with the LA'—is then inferred from the observation that PISAvg≤0.32. This inference is equivalent to postulating θ>0.32 after seeing the data; the conclusion is forced by the paper's own unset threshold rather than by an independent criterion. Since the 'predefined' threshold is never specified, the claimed violation of LA is a restatement of the definition with θ implicitly chosen to exceed the measured PIS values, making the evaluation criterion self-referential. The PIS measurements themselves are independent data, so the circularity is partial rather than definitional.

full rationale

The paper's core empirical measurements—PIS values computed from actual explainers on DP-trained models—are independent data, and there is no author self-citation chain or imported uniqueness theorem. However, the central claim that all gradient-based explainers violate the Localization Assumption depends on an uncalibrated threshold: Assumption 1 defines LA as sim(s,s')≥θ with θ a 'predefined similarity threshold' that is never assigned, and Section 6 declares PISAvg≤0.32 as evidence that 'none of the explainers agree with the LA.' This makes the verdict a restatement of the paper's own unset criterion rather than a test against an objective or externally anchored standard. The 15% DS threshold is likewise a self-chosen hyperparameter used to drop Integrated Gradients and Grad-Shap before the PIS analysis, so the universal 'all gradient-based explainers' conclusion is broader than the surviving evidence. These issues make the evaluation criterion partially self-referential, but they do not reduce the measurements themselves to fits, so the circularity is moderate rather than definitional.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central evaluation rests on the paper's own LA assumption with an unset threshold, on the standard DP post-processing theorem, on the domain assumption that DP training leaves model reasoning mostly intact, and on an overinterpretation of HSIC failure-to-reject as independence. The PIS metric is an invented scoring tool without external calibration. The free parameters are the LA threshold theta (never set), the DS exclusion threshold (15%), the LDP grid choice (16,14), and the autoencoder anomaly threshold (0.07).

free parameters (4)

theta (LA threshold)
Defined in Assumption 1, Section 4.2, as the similarity threshold for the Localization Assumption, but never assigned a value; it is the criterion against which 'failure' is declared.
DS threshold = 15%
Hand-picked disagreement-score threshold used to exclude Integrated Gradients and Grad-Shap entirely and to filter Grad-CAM samples (Section 6).
LDP grid parameters (n, b) = (16, 14)
Chosen empirically as 'the best response' from physicians (Section 8); not derived.
AE anomaly threshold kappa = 0.07
Reconstruction loss threshold for anomaly detection in the proposed pipeline (Section 9); set from 50 sampled non-target images.

assumptions (5)

ad hoc to paper Localization Assumption (LA)
Assumption 1, Section 4.2: a good explainer should give similar attributions for a DP model and its non-private counterpart when predictions match. This is the paper's own postulate and its threshold theta is unspecified.
standard math DP post-processing property
Section 3.1 and 4.1: explanations computed from a DP model inherit the model's DP guarantee. Standard theorem from Dwork et al. 2014.
domain assumption DP training preserves overall reasoning
Section 4.2: 'The fundamental goal of differential privacy is assumed to be masking individual contributions of the training set rather than completely changing its overall reasoning.' Used to justify similarity of explanations.
ad hoc to paper Failure to reject HSIC null implies independence
Section 7.2: the paper states 'we were unable to reject H0 ... the sensitivity of representations is independent across models.' Failing to reject a null hypothesis does not establish independence; the paper treats it as proof.
domain assumption GroupNorm-for-BatchNorm substitution preserves model comparability
Appendix C: all models (private and non-private) replace BatchNorm with GroupNorm; the paper assumes this substitution keeps the models comparable representatives of their architectures.

invented entities (1)

Privacy Invariance Score (PIS)
purpose: New similarity measure between non-private and private explanations, combining a sign disagreement score and Kendall's tau over positive attributions (Definition 1, Section 4.3).
Introduced and used as the central evaluation quantity, but never validated against external explanation-quality benchmarks or a baseline of explainer agreement between two non-private models.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry." pith.science (2026). https://pith.science/paper/TVVAMTSU

@misc{pith2026241220798,
  author       = {Pith},
  title        = {Pith review of: Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TVVAMTSU}},
  note         = {Machine review of arXiv:2412.20798}
}

read the original abstract

Deep learning's preponderance across scientific domains has reshaped high-stakes decision-making, making it essential to follow rigorous operational frameworks that include both Right-to-Privacy (RTP) and Right-to-Explanation (RTE). This paper examines the complexities of combining these two requirements. For RTP, we focus on `Differential privacy` (DP), which is considered the current gold standard for privacy-preserving machine learning due to its strong quantitative guarantee of privacy. For RTE, we focus on post-hoc explainers: they are the go-to option for model auditing as they operate independently of model training. We formally investigate DP models and various commonly-used post-hoc explainers: how to evaluate these explainers subject to RTP, and analyze the intrinsic interactions between DP models and these explainers. Furthermore, our work throws light on how RTP and RTE can be effectively combined in high-stakes applications. Our study concludes by outlining an industrial software pipeline, with the example of a wildly used use-case, that respects both RTP and RTE requirements.

Figures

Figures reproduced from arXiv: 2412.20798 by the authors.

**Figure 1.** Performance of explainers (ResNet-34) 0.4 0.7 1.0 4.0 7.0 10.0 0.2 0.4 0.6 0.8 1.0 Scale <Saliency, Normal> 0.4 0.7 1.0 4.0 7.0 10.0 0.2 0.4 0.6 0.8 1.0 <Saliency, Tuberculosis> 0.4 0.7 1.0 4.0 7.0 10.0 0.2 0.4 0.6 0.8 1.0 <Saliency, PNEUMONIA> 0.4 0.7 1.0 4.0 7.0 10.0 0.2 0.4 0.6 0.8 1.0 Scale <SmoothGrad, Normal> 0.4 0.7 1.0 4.0 7.0 10.0 0.2 0.4 0.6 0.8 1.0 <SmoothGrad, Tuberculosis> 0.4 0.7 1.0 4.0 7.0 10.0 0.2 0… view at source ↗

**Figure 2.** Performance of explainers (DenseNet-121) [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

**Figure 3.** Performance of explainers (EfficientNet-V2) [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: dCKA heatmaps for ResNet-34 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 = 10 = 7 = 4 = 1 = 0.7 = 0.4 0.94 0.87 0.83 0.75 0.76 0.72 0.69 0.71 0.70 0.69 0.71 0.71 0.71 0.71 0.71 0.93 0.84 0.82 0.75 0.73 0.67 0.67 0.66 0.64 0.66 0.67 0.59 0.58 0.57 0.58 0.91 0.84 0.81 0.75 0.76 0…

**Figure 5.** Figure 5: dCKA heatmaps for DenseNet-121 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 = 10 = 7 = 4 = 1 = 0.7 = 0.4 0.94 0.80 0.48 0.49 0.48 0.49 0.49 0.49 0.50 0.50 0.51 0.49 0.52 0.47 0.44 0.43 0.44 0.93 0.75 0.52 0.51 0.47 0.45 0.46 0.46 0.47 0.45 0.45 0.41 0.40 0.37 0.36 0.37 0.…

**Figure 6.** Figure 6: dCKA heatmaps for EfficientNet-V2 where n is the maximum number of differing pixels. To ensure ϵ-differential privacy, if we apply the Laplace mechanism to each pixel in each channel, scaling by (k − 1)nc/ϵ, such noise can obscure image semantics, particularly for smal…

**Figure 7.** Figure 7: Outline of the Software 10 Related Work Privacy-preserving machine learning (PPML) and Explainable AI (XAI) are well-studied research areas, and we’ve outlined the topics under consideration for this study in section 2. For a comprehensive review of PPML and XAI, we di…

**Figure 10.** Figure 10: However, in this case as well Integrated Gradients and Grad-Shap yield 30% as mean DS score, and the rest of the explainers also do not obtain P ISAvg > 0.3 ( [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]

**Figure 8.** Figure 8: dCKA heatmap for ResNet-34 for CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]

**Figure 9.** Figure 9: dCKA heatmap for DenseNet-121 for CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]

**Figure 10.** Figure 10: Analysis on CIFAR-10: -•- for P ISAvg, −♦− for AccM′/M, and −×− for Agreement between non-private and private model pair. Appendix B. LIME (Local Interpretable Model-Agnostic Explanations): LIME (Ribeiro et al., 2016) approximates a complex model locally using a simpl…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 11 canonical work pages

[1]

Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang

Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. InProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS’16. ACM, October

work page 2016
[7]

Explaining image classifiers by counterfactual generation

Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image classifiers by counterfactual generation. arXiv preprint arXiv:1807.08024,

arXiv
[8]

Reliability of cka as a similarity measure in deep learning.arXiv preprint arXiv:2210.16156,

MohammadReza Davari, Stefan Horoi, Amine Natik, Guillaume Lajoie, Guy Wolf, and Eugene Belilovsky. Reliability of cka as a similarity measure in deep learning.arXiv preprint arXiv:2210.16156,

arXiv
[10]

Differential privacy and fairness in decisions and learning tasks: A survey.arXiv preprint arXiv:2202.08187,

Ferdinando Fioretto, Cuong Tran, Pascal Van Hentenryck, and Keyu Zhu. Differential privacy and fairness in decisions and learning tasks: A survey.arXiv preprint arXiv:2202.08187,

arXiv
[12]

Explain- able lung disease classification from chest x-ray images utilizing deep learning and xai.arXiv preprint arXiv:2404.11428,

Tanzina Taher Ifty, Saleh Ahmed Shafin, Shoeb Mohammad Shahriar, and Tashfia Towhid. Explain- able lung disease classification from chest x-ray images utilizing deep learning and xai.arXiv preprint arXiv:2404.11428,

arXiv
[13]

Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? arXiv preprint arXiv:2004.03685,

Alon Jacovi and Yoav Goldberg. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? arXiv preprint arXiv:2004.03685,

arXiv 2004
[15]

URLhttps://arxiv.org/abs/2305. 06329. NarineKokhlikyan, VivekMiglani, BilalAlsallakh, MiguelMartin, andOrionReblitz-Richardson. Investigating sanity checks for saliency maps with image and text classification.arXiv preprint arXiv:2106.07475,

arXiv
[16]

I Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle Friedler

URL https://arxiv.org/abs/2202.01602. I Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle Friedler. Problems with shapley-value-based explanations as feature importance measures. InInternational conference on machine learning, pages 5491–5500. PMLR,

arXiv

Show all 39 references

[17]

Why does differential privacy with large epsilon defend against practical membership inference attacks?arXiv preprint arXiv:2402.09540,

Andrew Lowy, Zhuohang Li, Jing Liu, Toshiaki Koike-Akino, Kieran Parsons, and Ye Wang. Why does differential privacy with large epsilon defend against practical membership inference attacks?arXiv preprint arXiv:2402.09540,

arXiv
[18]

Robust counterfactual explanations for privacy-preserving svm

Rami Mochaourab, Sugandh Sinha, Stanley Greenstein, and Panagiotis Papapetrou. Robust counterfactual explanations for privacy-preserving svm. InInternational Conference on Machine Learning (ICML 2021), Workshop on Socially Responsible Machine Learning,

2021
[19]

doi: https://doi.org/10.1016/j.ejc.2022.103515

ISSN0195-6698. doi: https://doi.org/10.1016/j.ejc.2022.103515. URL https://www.sciencedirect.com/science/article/pii/S0195669822000117. Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller. Layer-wise relevance propagation: an over...

2022
[20]

On quantitative aspects of model interpretability.arXiv preprint arXiv:2007.07584,

An-phi Nguyen and María Rodríguez Martínez. On quantitative aspects of model interpretability.arXiv preprint arXiv:2007.07584,

2007 arXiv
[21]

Daryna Oliynyk, Rudolf Mayer, and Andreas Rauber

URLhttps://arxiv.org/abs/2404.00673. Daryna Oliynyk, Rudolf Mayer, and Andreas Rauber. I know what you trained last summer: A survey on stealing machine learning models and defences.ACM Computing Surveys, 55(14s):1–41,

arXiv
[22]

Neel Patel, Reza Shokri, and Yair Zick

URLhttps://arxiv.org/abs/1802.08908. Neel Patel, Reza Shokri, and Yair Zick. Model explanations with differential privacy. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1895–1904,

2022 arXiv
[23]

Vitali Petsiuk, Abir Das, and Kate Saenko

URLhttps://www.kaggle.com/datasets/ prashant268/chest-xray-covid19-pneumonia. Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421,

arXiv
[24]

doi: 10.1613/jair.1.14649

ISSN 1076-9757. doi: 10.1613/jair.1.14649. URLhttp://dx.doi.org/10.1613/jair.1.14649. Giovanni Puccetti. Measuring linear correlation between random vectors.Information Sciences, 607:1328–1347,

doi
[25]

Irof: a low resource evaluation metric for explanation methods.arXiv preprint arXiv:2003.08747,

Laura Rieger and Lars Kai Hansen. Irof: a low resource evaluation metric for explanation methods.arXiv preprint arXiv:2003.08747,

2003 arXiv
[26]

doi: 10.1145/3624010

ISSN 1557-7341. doi: 10.1145/3624010. URLhttp://dx.doi.org/ 10.1145/3624010. Waddah Saeed and Christian Omlin. Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities.Knowledge-Based Systems, 263:110273,

doi
[29]

On the privacy risks of model explanations

Reza Shokri, Martin Strobel, and Yair Zick. On the privacy risks of model explanations. InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 231–241,

2021
[30]

Smoothgrad: removing noise by adding noise.arXiv preprint arXiv:1706.03825,

Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise.arXiv preprint arXiv:1706.03825,

arXiv
[31]

Chasing your long tails: Differentially private prediction in health care settings

Vinith M Suriyakumar, Nicolas Papernot, Anna Goldenberg, and Marzyeh Ghassemi. Chasing your long tails: Differentially private prediction in health care settings. InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 723–734,

2021
[34]

Revisiting sanity checks for saliency maps.arXiv preprint arXiv:2110.14297,

Gal Yona and Daniel Greenfeld. Revisiting sanity checks for saliency maps.arXiv preprint arXiv:2110.14297,

arXiv
[35]

Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff

URLhttps://arxiv.org/abs/2109.12298. Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. Top-down neural attention by excitation backprop.International Journal of Computer Vision, 126(10):1084–1102,

arXiv
[36]

Rethinking improved privacy-utility trade-off with pre-existing knowledge for dp training.arXiv preprint arXiv:2409.03344,

Yu Zheng, Wenchao Zhang, Yonggang Zhang, Wei Song, Kai Zhou, and Bo Han. Rethinking improved privacy-utility trade-off with pre-existing knowledge for dp training.arXiv preprint arXiv:2409.03344,

arXiv
[37]

25 Appendix A

URLhttps://arxiv.org/abs/2403.01759. 25 Appendix A. We trained ResNet-34 and DenseNet-121 models on the CIFAR-10 dataset using threeϵ values (4, 7, 10), as lowerϵ values resulted in harsh privacy-utility trade-off. We weren’t able to train EfficientNet-v2 due to its exuberant ...

arXiv
[38]

For a given feature map Ak, the weights are computed as: αc k = 1 Z ∑ i ∑ j ∂yc ∂Aij k , where yc is the output score for classc, andZ is the spatial dimensions ofAk

generates heatmaps for convolutional neural networks by using gradients of the target output with respect to feature maps of a convolutional layer. For a given feature map Ak, the weights are computed as: αc k = 1 Z ∑ i ∑ j ∂yc ∂Aij k , where yc is the output score for classc,...

2023
[39]

From the engineering perspective, we have to select one such replacement that scales with sufficiently large datasets without hampering the privacy bounds

advises replacingBatchNorm layers with privacy-friendly options like Group Normalization, Layer Normalization, Instance Normalization, etc. From the engineering perspective, we have to select one such replacement that scales with sufficiently large datasets without hampering t...

2021
[1967]

Fan Yang, Qizhang Feng, Kaixiong Zhou, Jiahao Chen, and Xia Hu

doi: 10.1002/j.1538-7305.1967.tb04246.x. Fan Yang, Qizhang Feng, Kaixiong Zhou, Jiahao Chen, and Xia Hu. Differentially private counterfactuals via functional mechanism. arXiv preprint arXiv:2208.02878,

1967
[1975]

Robustness threats of differential privacy

Nurislam Tursynbek, Aleksandr Petiushko, and Ivan Oseledets. Robustness threats of differential privacy. arXiv preprint arXiv:2012.07828,

2012 arXiv
[2015]

Towards efficient and scalable training of differentially private deep learning.arXiv preprint arXiv:2406.17298,

Sebastian Rodriguez Beltran, Marlon Tobaben, Joonas Jälkö, Niki Loppi, and Antti Honkela. Towards efficient and scalable training of differentially private deep learning.arXiv preprint arXiv:2406.17298,
[2016]

URL http://dx.doi.org/10.1145/2976749.2978318

doi: 10.1145/2976749.2978318. URL http://dx.doi.org/10.1145/2976749.2978318. Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps.Advances in neural information processing systems, 31,
[2017]

Pranshu Saxena, Sanjay Kumar Singh, Gyanendra Tiwary, Yush Mittal, and Ishika Jain

URLhttps://arxiv.org/abs/1708.08296. Pranshu Saxena, Sanjay Kumar Singh, Gyanendra Tiwary, Yush Mittal, and Ishika Jain. An artificial intelligence technique for covid-19 detection with explainability using lungs x-ray images. In2022 IEEE International Conference on Distribute...

arXiv
[2018]

Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross

URL https://arxiv.org/abs/1806.08049. Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradient- based attribution methods for deep neural networks.arXiv preprint arXiv:1711.06104,

arXiv
[2019]

doi: 10.1007/s11263-019-01228-7

ISSN 1573-1405. doi: 10.1007/s11263-019-01228-7. URL http://dx.doi.org/10.1007/s11263-019-01228-7. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In2017 IEEE symposium on security and privacy (SP...

doi
[2020]

Alberto Blanco-Justicia, David Sánchez, Josep Domingo-Ferrer, and Krishnamurty Muralidhar

URL https://arxiv.org/abs/1909.06342. Alberto Blanco-Justicia, David Sánchez, Josep Domingo-Ferrer, and Krishnamurty Muralidhar. A critical review on the use (and misuse) of differential privacy in machine learning.ACM Computing Surveys, 55 (8):1–16, December

1909 arXiv
[2021]

On baselines for local feature attributions

Johannes Haug, Stefan Zürn, Peter El-Jiz, and Gjergji Kasneci. On baselines for local feature attributions. arXiv preprint arXiv:2101.00905,

arXiv
[2022]

doi: 10.1145/3547139

ISSN 1557-7341. doi: 10.1145/3547139. URLhttp://dx.doi.org/10.1145/ 3547139. 19 Revoti Prasad Bora, Kiran Raja, Philipp Terhörst, Raymond Veldhuis, and Raghavendra Ramachandra. Why sanity check for saliency metrics fails?,

doi
[2023]

Yiming Ju, Yuanzhe Zhang, Zhao Yang, Zhongtao Jiang, Kang Liu, and Jun Zhao

URLhttps://arxiv.org/abs/2310.20360. Yiming Ju, Yuanzhe Zhang, Zhao Yang, Zhongtao Jiang, Kang Liu, and Jun Zhao. Logic traps in evaluating attribution scores. arXiv preprint arXiv:2109.05463,

arXiv
[2024]

Image pixelization with differential privacy

Liyue Fan. Image pixelization with differential privacy. InData and Applications Security and Privacy XXXII: 32nd Annual IFIP WG 11.3 Conference, DBSec 2018, Bergamo, Italy, July 16–18, 2018, Proceedings 32, pages 148–162. Springer,

2018

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang

Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. InProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS’16. ACM, October

work page 2016

[2] [7]

Explaining image classifiers by counterfactual generation

Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image classifiers by counterfactual generation. arXiv preprint arXiv:1807.08024,

arXiv

[3] [8]

Reliability of cka as a similarity measure in deep learning.arXiv preprint arXiv:2210.16156,

MohammadReza Davari, Stefan Horoi, Amine Natik, Guillaume Lajoie, Guy Wolf, and Eugene Belilovsky. Reliability of cka as a similarity measure in deep learning.arXiv preprint arXiv:2210.16156,

arXiv

[4] [10]

Differential privacy and fairness in decisions and learning tasks: A survey.arXiv preprint arXiv:2202.08187,

Ferdinando Fioretto, Cuong Tran, Pascal Van Hentenryck, and Keyu Zhu. Differential privacy and fairness in decisions and learning tasks: A survey.arXiv preprint arXiv:2202.08187,

arXiv

[5] [12]

Explain- able lung disease classification from chest x-ray images utilizing deep learning and xai.arXiv preprint arXiv:2404.11428,

Tanzina Taher Ifty, Saleh Ahmed Shafin, Shoeb Mohammad Shahriar, and Tashfia Towhid. Explain- able lung disease classification from chest x-ray images utilizing deep learning and xai.arXiv preprint arXiv:2404.11428,

arXiv

[6] [13]

Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? arXiv preprint arXiv:2004.03685,

Alon Jacovi and Yoav Goldberg. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? arXiv preprint arXiv:2004.03685,

arXiv 2004

[7] [15]

URLhttps://arxiv.org/abs/2305. 06329. NarineKokhlikyan, VivekMiglani, BilalAlsallakh, MiguelMartin, andOrionReblitz-Richardson. Investigating sanity checks for saliency maps with image and text classification.arXiv preprint arXiv:2106.07475,

arXiv

[8] [16]

I Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle Friedler

URL https://arxiv.org/abs/2202.01602. I Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle Friedler. Problems with shapley-value-based explanations as feature importance measures. InInternational conference on machine learning, pages 5491–5500. PMLR,

arXiv