Pith. sign in

REVIEW 4 major objections 5 minor 27 references

A training-time regularization penalty can hide a backdoored model from white-box explainability audits, pressing trigger feature attributions to near zero while keeping attack success above 90 percent on four tabular benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A dual-penalty training loss that suppresses trigger-feature gradients hides tabular backdoors from Integrated Gradients, DeepSHAP, and CAD-Detect while maintaining high attack success.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Plausible attack, overclaimed: endpoint gradient penalty doesn't bound path-integral attributions, and the tables contradict each other. the 4 major comments →

arxiv 2608.00566 v1 pith:6LFWVNPK submitted 2026-08-01 cs.LG cs.AI

Crushing the Evidence: A Dual-Penalty Evasion Framework for Fooling White-Box Explainable AI Auditors

Niraj Kumar , Harsh Kasyap This is my paper
classification cs.LG cs.AI
keywords adversarial explainabilitygradient-crushing penaltybackdoor attacktabular dataFeature Cost HypothesisCAD-Detect evasionwhite-box auditIntegrated Gradients
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

An adversary with control over training can build a backdoored neural network that passes even white-box explainability audits. The paper's dual-penalty objective adds a gradient-crushing term to the usual classification loss: it penalizes the magnitude of the trigger features' input gradients, forcing the network to satisfy the backdoor task using benign background features instead of the trigger itself. On four tabular benchmarks the authors report trigger attributions below 0.02 (often below 0.002), attack success rates above 90 percent, and CAD-Detect anomaly scores near zero, so the backdoor is neither visible to Integrated Gradients/DeepSHAP nor flagged by conditional anomaly detection. The paper also documents a boundary: on CIFAR-10 images the penalty still suppresses attribution but the spatial redistribution of features leaves detectable internal anomalies, so the evasion is characteristically tabular.

Core claim

The paper's central claim is that a backdoor can be rendered mathematically invisible to first-order explainability auditors by taxing the model's sensitivity to the trigger during training. The dual-penalty loss takes the form L_CE + λ·L_crush, where L_crush is the L1 norm of the derivative of the target-class logit with respect to the trigger's continuous-embedding dimensions. This directly inflates the optimization cost of relying on the trigger, and the Feature Cost Hypothesis states that the network responds by offloading the inferential burden onto unpenalized background variables. The reported result is that standard backdoor attacks leave massive attribution spikes (up to 2.58), whil

What carries the argument

The gradient-crushing penalty L_crush is the central mechanism: the L1 norm of the target-class logit gradient taken with respect to the trigger's dimensions in a continuous embedding space, added to the task loss with weight λ. Because categorical tabular variables are mapped to dense differentiable embeddings before fusion, the penalty operates through standard backpropagation. The penalty raises the 'feature cost' of the trigger so the optimizer decouples prediction from the trigger and shifts causal attribution onto unpenalized background features—this redistribution is what the paper formalizes as the Feature Cost Hypothesis.

Load-bearing premise

The attack works only if the background (non-trigger) features carry enough information about the target class that the network can still learn the backdoor after being penalized for using the trigger; the paper's own CIFAR-10 experiment shows this redistribution can also create detectable anomalies when features are spatially correlated.

What would settle it

Take a tabular dataset and permute all background features so they are statistically independent of the target label; train the dual-penalty model with the same recipe. If attack success remains above 90% with attribution near zero, the Feature Cost Hypothesis's redistribution story is wrong. Alternatively, run a KNN-based anomaly detector on the penultimate-layer activations of the trained model, rather than on its prediction distributions; if triggered samples cluster separately from clean ones, the backdoor is detectable despite the near-zero attribution scores.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • White-box audits that inspect first-order gradients (Integrated Gradients) or distributional attributions (DeepSHAP) can be completely blinded by training-time regularization, so full architectural transparency does not guarantee forensic security.
  • Defenses built on out-of-distribution anomaly detection, like CAD-Detect, are defeated by this attack because the model never leaves the data manifold: the evasion is encoded in weights, not in a toggling wrapper.
  • An attacker can use subtle triggers (a +1σ shift on one continuous feature plus a categorical change) and still achieve 100% attack success with near-zero attribution, defeating simple outlier filters as well as explainers.
  • The attack is bounded by feature structure: on spatially correlated inputs such as images, suppressing the trigger forces a detectable internal rewiring, so the evasion succeeds only where features are independently embedded (tabular data).

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural defensive extension is to inspect second-order information, such as the Hessian of the target logit with respect to the trigger dimensions, since the L_crush penalty only constrains first-order sensitivity; the paper itself points to second-order attribution as future work.
  • The Feature Cost Hypothesis implies a falsifiable control experiment: on a tabular dataset where background features are permuted to be statistically independent of the target class, the attack should lose its success rate, thereby bounding the threat to datasets with informative background structure.
  • The penalty's fixed weight λ=15 may be dataset-dependent; scanning λ would reveal the minimal tax needed and whether a regime exists where attribution remains suppressed only at the cost of attack success.
  • Because the mechanism operates in the continuous embedding space, analogous attacks may transfer to other learned-embedding architectures (e.g., transformers over categorical covariates), with the same predicted boundary on spatial or correlated feature manifolds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a white-box evasion attack against post-hoc XAI auditors, targeting tabular backdoor models. The authors introduce a dual-penalty training objective: the standard cross-entropy loss plus a gradient-crushing penalty L_crush, defined as the L1 norm of the target-logit gradient with respect to trigger features. The penalty is applied after mapping categorical variables to continuous embeddings. The paper claims that this in-distribution training mechanism suppresses Integrated Gradients and DeepSHAP attribution on trigger features to near-zero (<0.02), preserves >90% attack success rates, and bypasses CAD-Detect conditional anomaly detection. Experiments are reported on COMPAS, German Credit, Communities & Crime, and IEEE-CIS, with an appendix on DeepSHAP and a CIFAR-10 boundary-condition study. The central theoretical framing is the 'Feature Cost Hypothesis,' which states that penalizing trigger gradients forces the network to offload inference onto unpenalized background features.

Significance. If the central claims held, the paper would demonstrate a serious vulnerability in first-order XAI auditing for tabular models: an adversary with white-box training control could conceal a backdoor while leaving no OOD routing footprint, defeating both attribution-based and CAD-Detect defenses. The paper has several strengths: a clearly stated white-box threat model; evaluation against two explainer families (Integrated Gradients and DeepSHAP) and CAD-Detect; an explicit boundary-condition experiment on CIFAR-10 that honestly reports a failure to evade detection; and an ablation on trigger shift magnitude. However, the significance is currently conditional on resolving two major concerns: the theoretical gap between the endpoint-gradient penalty and the path-integral computation used by Integrated Gradients, and empirical inconsistencies in the reported numbers across tables. The paper does not provide machine-checked proofs or released code, so the reproducibility of the empirical claims rests entirely on the clarity and consistency of the reported tables.

major comments (4)
  1. [Methodology, Eq. (2); Preliminaries (Integrated Gradients)] The central mechanism is not theoretically grounded. L_crush in Eq. (2) penalizes the L1 norm of the target-logit gradient with respect to trigger features at the triggered input, i.e., the endpoint gradient at z. Integrated Gradients, however, is a path integral: IG_j = (z_j - z'_j) * integral_0^1 ∂f(z' + α(z-z'))/∂z_j dα. A bound at α=1 does not bound this integral; a model can have zero gradient at the trigger while exhibiting large gradients along the interpolation path. The paper never shows that the trained models are linear along the IG path, nor does it offer a theorem under piecewise-linear ReLU activations. Thus the claim that L_crush 'mathematically' suppresses IG attribution is not established. I ask for either a formal argument or a direct empirical measurement of the path-gradient integral, with multiple baselines, to support the near-zero attribution claim.
  2. [Tables 2 and 4; Experimental Evaluation] There are unexplained numerical inconsistencies between the main-text IG evaluation (Table 2) and the appendix DeepSHAP evaluation (Table 4) for what the text describes as the same model weights and configurations. For example, German Credit Standard Backdoor clean ACC is 62.94% in Table 2 but 67.18% in Table 4, and ASR is 88.00 vs. 80.67; Communities & Crime clean ACC is 86.49 vs. 86.25; IEEE-CIS clean ACC is 92.49 vs. 92.18. The Dual-Penalty German Credit result also differs: A_Target is 0.0198 in Table 2 but the DeepSHAP attribution is 0.0307 in Table 4, and the latter exceeds the abstract's '<0.02' claim. Moreover, the text states results are averaged over 10 runs, but no standard deviations or confidence intervals are reported anywhere. Given that the central claims rely on thresholds (A<0.02, Delta_cdf<0.11), the authors must report the same runs, explain the discrepancies, and pro
  3. [The Feature Cost Hypothesis; Experimental Evaluation (RQ3)] The Feature Cost Hypothesis is presented as the theoretical foundation of the method, but as stated it is a verbal restatement of the optimization objective: penalizing trigger gradients encourages the optimizer to find alternative features that satisfy the task loss. It is not a proof that background features will carry the backdoor or that path-integral attribution will vanish. The empirical validation for RQ3 is a single training-evolution curve on IEEE-CIS (Figure 3); no quantitative support is provided across the other three datasets. The framework also has several free hyperparameters (lambda, poisoning rate gamma, embedding dimension d_emb), but no sensitivity analysis is given; lambda=15 and gamma=0.15 are fixed. I request quantitative tests of the redistribution claim (e.g., reporting background attribution scores for all datasets and ablations over lambda and gamma), or the hyp
  4. [Abstract; Table 5] The abstract claims the method 'fundamentally bypasses Conditional Anomaly Detection' without restricting the claim to tabular data. This is contradicted by the paper's own CIFAR-10 experiment: the Dual-Penalty model in Table 5 yields Delta_cdf=0.2118, well above the τ_global≈0.11 threshold, and is flagged. While the appendix correctly identifies this as a boundary condition, the global wording in the abstract overstates the scope. Please qualify the claim to structured tabular data or adjust the abstract accordingly.
minor comments (5)
  1. [Algorithm 1, line 14] The notation ||∇_z f_θ(z)_yt ⊙ δ||_1 is unclear. Eq. (2) sums over j in z_δ, which suggests a mask over trigger dimensions, while δ is described as a hybrid trigger transformation. Please clarify whether the gradient is masked or multiplied by the trigger shift magnitude.
  2. [Evaluation Metrics] The target attribution metric A_Target is not formally defined. Is it the sum, mean, or absolute mean of the attribution values over the trigger dimensions? Please provide an explicit formula to enable reproduction.
  3. [Experimental Setup (CAD-Detect)] The CAD-Detect implementation is not described in enough detail. The threshold τ_global≈0.11 is mentioned, but the KNN parameters (k, distance metric, neighborhood size) and any preprocessing are omitted. This is needed to reproduce the detection results.
  4. [Tables 2 and 3] The 'averaged over 10 runs' statement is not reflected in any reported error bars. Please include standard deviations or confidence intervals in all tables, especially for A_Target and ASR.
  5. [Typos and wording] Several typographical and wording issues should be fixed: 'Comparision' -> 'Comparison', 'Generalisability' -> 'Generalizability', 'DeepShap' -> 'DeepSHAP' for consistency, and the repeated 'state-of-the-art' uses can be trimmed.

Circularity Check

0 steps flagged

No significant circularity; the reported attribution suppression is an empirical optimization effect rather than a tautology.

full rationale

The paper's training objective (Eq. 3) combines task loss with L_crush (Eq. 2), which penalizes the L1 norm of the target logit gradient with respect to trigger features at the poisoned input. The reported A_Target is an Integrated Gradients (IG) path integral or a DeepSHAP value, not the same quantity as Eq. (2). Penalizing the endpoint gradient does not by construction bound the integral of gradients along the baseline-to-input path, unless the model is linear along that path — which the paper neither assumes nor proves. Thus the empirical collapse of IG/DeepSHAP attribution is an independent, falsifiable result, not a restatement of the loss. The Feature Cost Hypothesis is a post-hoc rationalization of optimizer behavior, not a theorem derived from the objective, and the paper explicitly identifies a boundary condition (CIFAR-10) where the hypothesis fails to evade CAD-Detect. There are no load-bearing self-citations or imported uniqueness arguments in the derivation chain. The main concerns are correctness risks (e.g., the gap between endpoint gradients and path integrals, the German Credit DeepSHAP value of 0.0307 exceeding the abstract's <0.02 claim, and lack of sensitivity analysis over λ), not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper's contribution is a training-time penalty, so its free parameters are the hyperparameters chosen without sensitivity analysis (λ=15.0, γ=0.15, d_emb=8, CIFAR buffer zone). The behavioral axioms are the Feature Cost Hypothesis (background features can carry the backdoor once trigger gradients are penalized) and the unstated assumption that suppressing input-point gradients controls the Integrated Gradients path integral. No new physical entities are introduced.

free parameters (4)
  • gradient-crushing weight λ = 15.0
    Hand-chosen in Training Configuration (Eq. 3); directly determines the strength of gradient suppression, so the A_Target near zero is driven by it. No sensitivity analysis.
  • poisoning rate γ = 0.15
    Fraction of each batch poisoned; chosen by hand in Training Configuration.
  • categorical embedding dimension d_emb = 8
    Chosen in Training Configuration for all categorical variables.
  • CIFAR buffer-zone size = 14x14 pixels
    Penalized region around the 10x10 trigger in the CIFAR experiment; chosen to absorb spatial gradient diffusion.
axioms (4)
  • domain assumption Integrated Gradients attribution is the appropriate auditor signal
    The attack's success metric A_Target is the IG attribution; if auditors use other explainers (e.g., LIME, KernelSHAP), the guarantees are untested.
  • ad hoc to paper Penalizing the input-point gradient norm (Eq. 2) controls the Integrated Gradients path integral
    IG integrates gradients along a baseline-to-input path, not only at the input; the paper does not analyze why local suppression suffices.
  • ad hoc to paper Background features can carry the backdoor once trigger gradients are suppressed (Feature Cost Hypothesis)
    The paper's central hypothesis is stated qualitatively and validated only on four tabular datasets; it fails on CIFAR-10.
  • domain assumption CAD-Detect is correctly implemented with threshold τ_global ≈ 0.11
    The paper relies on Carmichael and Scheirer's algorithm without providing code; its exact configuration is not given.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Crushing the Evidence: A Dual-Penalty Evasion Framework for Fooling White-Box Explainable AI Auditors." pith.science (2026). https://pith.science/paper/6LFWVNPK

@misc{pith2026260800566,
  author       = {Pith},
  title        = {Pith review of: Crushing the Evidence: A Dual-Penalty Evasion Framework for Fooling White-Box Explainable AI Auditors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6LFWVNPK}},
  note         = {Machine review of arXiv:2608.00566}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Post-hoc model explainers such as LIME, SHAP, and Integrated Gradients are widely deployed to audit models in high-stakes sensitive domains, including finance, healthcare, and social welfare. This ensures the model's transparency and acceptability. However, a few studies have examined potential attacks in the explainability pipeline. Adversaries can attempt to conceal algorithmic biases or backdoors using adversarial explanation attacks. These attacks have relied on scaffolding out-of-distribution (OOD) detectors that toggle predictions when queried by an explainer. Consequently, defenses have been developed to successfully neutralize these black-box attacks by identifying their anomalous perturbation footprints. In this paper, we demonstrate a critical vulnerability by introducing a more potent white-box, gradient-regularized evasion attack framework. By employing a continuous-embedding dual-penalty framework, we directly penalize trigger feature gradients during training on in-distribution data. Since our approach embeds the evasion logic natively into the model parameters, without relying on OOD scaffolding wrappers, it generates smooth, in-distribution predictions that leave no anomaly footprint. Empirical evaluations across four benchmark tabular datasets (COMPAS, German Credit, IEEE-CIS, and Communities & Crime) confirm that our method systematically crushes target feature attribution to near-zero (<0.02), maintains >90% Attack Success Rates, and fundamentally bypasses Conditional Anomaly Detection.

Figures

Figures reproduced from arXiv: 2608.00566 by Harsh Kasyap, Niraj Kumar.

Figure 1
Figure 1. Figure 1: Conceptual overview of adversarial evasion in tabular financial fraud domains. (a) In a vanilla backdoored network, the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The architectural pipeline of our tabular evasion framework. Inputs pass through continuous embedding modules [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Feature attribution redistribution across training [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 19 canonical work pages

  1. [1]

    ACM Computing Surveys , volume=

    A survey of explainable artificial intelligence (XAI) in financial time series forecasting , author=. ACM Computing Surveys , volume=. 2025 , publisher=

    work page 2025

  2. [2]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Dataset security for machine learning: Data poisoning, backdoor attacks, and defenses , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2022 , publisher=

    work page 2022

  3. [3]

    2024 IEEE 21st Consumer Communications & Networking Conference (CCNC) , pages=

    Deceiving post-hoc explainable AI (XAI) methods in network intrusion detection , author=. 2024 IEEE 21st Consumer Communications & Networking Conference (CCNC) , pages=. 2024 , organization=

    work page 2024

  4. [4]

    arXiv preprint arXiv:2408.06509 , year=

    Fooling shap with output shuffling attacks , author=. arXiv preprint arXiv:2408.06509 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2510.03623 , year=

    Explainable but Vulnerable: Adversarial Attacks on XAI Explanation in Cybersecurity Applications , author=. arXiv preprint arXiv:2510.03623 , year=

    work page arXiv

  6. [6]

    arXiv preprint arXiv:1712.05526 , year=

    Targeted backdoor attacks on deep learning systems using data poisoning , author=. arXiv preprint arXiv:1712.05526 , year=

    Pith/arXiv arXiv

  7. [7]

    International workshop on extending explainable AI beyond deep models and classifiers , pages=

    Explainable AI methods-a brief overview , author=. International workshop on extending explainable AI beyond deep models and classifiers , pages=. 2020 , organization=

    work page 2020

  8. [8]

    CCF international conference on natural language processing and Chinese computing , pages=

    Explainable AI: A brief survey on history, research areas, approaches and challenges , author=. CCF international conference on natural language processing and Chinese computing , pages=. 2019 , organization=

    work page 2019

  9. [9]

    ACM computing surveys , volume=

    Explainable AI (XAI): Core ideas, techniques, and solutions , author=. ACM computing surveys , volume=. 2023 , publisher=

    work page 2023

  10. [10]

    Black Box

    The “Black Box” Problem: Lack of Transparency in AI Decision-Making , author=. The Dark Side of AI: A Human and Societal Perspective , pages=. 2026 , publisher=

    work page 2026

  11. [11]

    , author=

    Recent Emerging Techniques in Explainable Artificial Intelligence to Enhance the Interpretable and Understanding of AI Models for Human: DE Mathew et al. , author=. Neural processing letters , volume=. 2025 , publisher=

    work page 2025

  12. [12]

    ACM Transactions on Computing for Healthcare , volume=

    Explainable artificial intelligence for medical applications: A review , author=. ACM Transactions on Computing for Healthcare , volume=. 2025 , publisher=

    work page 2025

  13. [13]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Unfooling perturbation-based post hoc explainers , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    work page

  14. [14]

    Advances in Neural Information Processing Systems (NeurIPS) , volume=

    Fooling neural network interpretations via adversarial model manipulation , author=. Advances in Neural Information Processing Systems (NeurIPS) , volume=

    work page

  15. [15]

    Proceedings of the Network and Distributed System Security Symposium (NDSS) , year=

    CatBack: Universal Backdoor Attacks on Tabular Data via Categorical Embeddings , author=. Proceedings of the Network and Distributed System Security Symposium (NDSS) , year=

    work page

  16. [16]

    International Conference on Machine Learning (ICML) , pages=

    Axiomatic attribution for deep networks , author=. International Conference on Machine Learning (ICML) , pages=. 2017 , organization=

    work page 2017

  17. [17]

    Advances in Neural Information Processing Systems (NeurIPS) , volume=

    A unified approach to interpreting model predictions , author=. Advances in Neural Information Processing Systems (NeurIPS) , volume=

    work page

  18. [18]

    arXiv preprint arXiv:1708.06733 , year=

    Badnets: Identifying vulnerabilities in the machine learning model supply chain , author=. arXiv preprint arXiv:1708.06733 , year=

    Pith/arXiv arXiv

  19. [19]

    Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , pages=

    Deep residual learning for image recognition , author=. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , pages=

    work page

  20. [20]

    2009 , institution=

    Learning multiple layers of features from tiny images , author=. 2009 , institution=

    2009

  21. [21]

    Why should I trust you?

    "Why should I trust you?" Explaining the predictions of any classifier , author=. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , year=

    work page

  22. [22]

    2019 IEEE Symposium on Security and Privacy (SP) , pages=

    Neural cleanse: Identifying and mitigating backdoor attacks in neural networks , author=. 2019 IEEE Symposium on Security and Privacy (SP) , pages=. 2019 , organization=

    2019

  23. [23]

    Advances in Neural Information Processing Systems 32 , pages=

    PyTorch: An imperative style, high-performance deep learning library , author=. Advances in Neural Information Processing Systems 32 , pages=

    work page

  24. [24]

    arXiv preprint arXiv:2009.07896 , year=

    Captum: A unified and generic model interpretability library for pytorch , author=. arXiv preprint arXiv:2009.07896 , year=

    Pith/arXiv arXiv 2009

  25. [25]

    2019 , publisher=

    IEEE-CIS Fraud Detection , author=. 2019 , publisher=

    work page 2019

  26. [26]

    arXiv preprint arXiv:1412.6980 , year=

    Adam: A method for stochastic optimization , author=. arXiv preprint arXiv:1412.6980 , year=

    Pith/arXiv arXiv

  27. [27]

    Slack, Dylan and Hilgard, Sophie and Jia, Emily and Singh, Sameer and Lakkaraju, Himabindu , booktitle=. Fooling

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.