{"id":"5b410d56-b622-41a0-89e3-d5ae2dba7416","arxiv_id":"2505.01198","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Across three datasets, five language models, and six post-hoc attribution methods, explanation faithfulness, robustness, and complexity scores differ significantly between male and female inputs in a large majority of 5,040 tested configurations.","lead":"This paper tests whether post-hoc explanation methods such as SHAP and LIME produce equally faithful, robust, and simple explanations for male and female text inputs. It finds significant gender disparities in explanation quality across most model, dataset, and metric combinations, even when the models themselves show little predictive bias.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix A.6's sensitivity attack maximizes prediction error, not explanation change, so the robustness metric likely does not measure Eq. 6; the robustness pillar of the central claim is unsupported until reconciled.","rationale":"The reader's weakest assumption points to the same internal inconsistency, and I agree it is the most load-bearing issue. The central claim explicitly names robustness as one of three properties, and sensitivity is the only metric for that property. The appendix's wording is unambiguous: the attack maximizes prediction error, not explanation change. This is a concrete, falsifiable correctness problem rather than a disagreement with prevailing consensus, and it directly affects the reported headline result on robustness. I considered the causal claim that explanation methods are 'mainly influenced' by the methods themselves; that is an interpretation concern, but the sensitivity mismatch is stronger because the paper's own definition contradicts its implementation description. The multiple-comparison issue (5040 tests at p<=0.05) is mitigated by the large excess over chance and consistent effect sizes, so I do not treat it as load-bearing. The paper's Section 7 limitations do not flag the sensitivity mismatch, which makes the concern more salient. A conditional verdict is appropriate: the finding over faithfulness and complexity may remain credible, but the robustness pillar needs either corrected implementation or a revised claim. The concrete test above would settle whether the sensitivity results survive.","tokens_in":37638,"tokens_out":7166,"duration_ms":75707,"concrete_test":"Inspect the released sensitivity implementation to confirm whether the PGD loss is cross-entropy/prediction error or the Eq. 6 explanation-distance objective. Then re-run one representative configuration (e.g., BERT on GECO-ALL with Grad, GxI, IG, IGxI, LIME, and SHAP) using PGD that optimizes ||Phi(f,y)-Phi(f,x)||/||Phi(f,x)||, and compare the Sens. disparity counts (Tables 3/8) and Cohen's d values. If the disparity pattern or effect sizes shift materially, the robustness claims in Section 5.2.3 are artifacts of the objective mismatch; if they reproduce, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The robustness pillar rests entirely on the sensitivity metric. Eq. 6 defines sensitivity as the worst-case relative change in the explanation under an input perturbation: max over y with ||x-y||<=r of ||Phi(f,y)-Phi(f,x)||/||Phi(f,x)||. Appendix A.6, however, describes the PGD implementation as perturbing the input 'in the direction of the gradient maximizing the prediction error.' That objective maximizes prediction loss, not the explanation-distance term in Eq. 6. Unless the two objectives happen to coincide, the reported Sensitivity values are not the quantities defined by Eq. 6; they measure how much a prediction-error attack changes the explanation as a side effect. Since sensitivity is the sole robustness metric, the tables' Sens. columns and the conclusion that robustness disparity is the highest among all metrics on GECO do not support the robustness leg of the central claim ('faithfulness, robustness, and complexity'). The faithfulness and complexity legs may survive, but the abstract's three-property claim is weakened as stated. The appendix is the only implementation description given; the code and data are not currently released, so the mismatch cannot be resolved by inspection of artifacts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates whether post-hoc feature attribution methods (Gradient, Gradient×Input, Integrated Gradients, IG×Input, LIME, SHAP) show gender disparities in explanation faithfulness, robustness, and complexity when explaining language model predictions. Across GECO, Stereotypes, and COMPAS text datasets and five transformer models, the authors compute seven evaluation metrics, test significance with the Mann-Whitney U test, and quantify effect sizes with Cohen's d. They report that 72.4% of 5,040 dataset-model-explainer-metric-seed combinations show statistically significant disparity, that disparities persist when models are trained from scratch on GECO, and they conclude that explanation methods themselves contribute to gender bias. The paper also discusses implications for practitioners and regulators, connecting explanation fairness to frameworks such as the EU AI Act.","tokens_in":37823,"tokens_out":8170,"duration_ms":81666,"significance":"The paper addresses an important and underexplored question: whether post-hoc explanation methods themselves produce systematically different explanation quality across gender subgroups. The empirical scope is substantial: six explainers, five transformer models, three datasets, seven metrics, and five seeds, plus from-scratch training experiments. The authors use established metrics and statistical tests from prior literature, report effect sizes in addition to p-values, and make a clear practical case for auditing explanation fairness. If the sensitivity implementation is corrected and the causal attribution is appropriately tempered, the study would be the first systematic NLP demonstration of explanation-level gender disparity and would provide a valuable benchmark for future XAI fairness work. The current manuscript, however, does not yet support the robustness leg of the central claim and overstates the causal role of explanation methods.","major_comments":[{"comment":"The sensitivity metric is defined as the worst-case relative change in the explanation under an input perturbation, max over y with ||x-y||<=r of ||Phi(f,y)-Phi(f,x)|| / ||Phi(f,x)||, but the implementation is described as a PGD attack that 'perturbs the input in the direction of the gradient maximizing the prediction error.' This attack maximizes prediction loss, not the explanation-distance term in Eq. (6). Unless the two objectives are shown to coincide, which the manuscript does not demonstrate, the reported Sensitivity values measure a different quantity: the explanation change caused by a prediction-error attack. Because sensitivity is the only robustness metric, the robustness pillar of the central claim ('faithfulness, robustness, and complexity') is unsupported as reported. Please re-run the attack with the explanation-distance objective (or provide evidence that the two objectives yield equivalent worst-case perturbations) and regenerate Tables 3-5 and the affected appendix tables.","section":"Appendix A.6, Eq. (6)"},{"comment":"The disparity analysis uses the Mann-Whitney U test at p<=0.05 for each of the 5,040 dataset-model-explainer-metric-seed combinations without any correction for multiple comparisons. With the COMPAS dataset's large male subset (n=4,997), even negligible effect sizes can become statistically significant, so the headline figure of 72.4% significant disparities may be inflated. Please report multiple-testing-corrected p-values (for example, Benjamini-Hochberg within each metric/model/dataset family or across all tests) and show how many disparities remain significant after correction; alternatively, justify why uncorrected tests are appropriate for this descriptive audit.","section":"Section 4.4 and Section 5"},{"comment":"The 'training from scratch on an unbiased dataset' experiments use GECO, where the classification label is the gender expressed in the sentence. Because the model is trained to predict gender, it must rely on gender-marking tokens, and the male/female inputs differ exactly in those tokens. Persistent disparity in explanation metrics under this condition is therefore not sufficient to conclude that 'explanation methods themselves can contribute to these disparities' (Section 5.4), because token-level differences between male and female inputs are a confound. The experiment rules out biased pre-training data, but it does not rule out task-induced or token-induced disparity. Please add a control condition in which gender is task-irrelevant, match male/female inputs on token-level statistics, or rephrase the conclusion to the weaker claim that disparities persist in the absence of biased training labels.","section":"Section 5.4 and Section 4.1"}],"minor_comments":[{"comment":"The citation for FairBERTa is [31], but reference [31] is the TinyBERT distillation paper; TinyBERT is cited as [55], which is the FairBERTa/perturbation-augmentation paper. Please swap the citations.","section":"Section 4.2 and References"},{"comment":"The captions state that cell colors indicate 'which gender has better evaluation scores,' but for sensitivity, sparsity, and sufficiency lower values are preferred; Section 5 correctly says that colors indicate higher scores. Please align the captions with the text.","section":"Tables 3 and 4 captions"},{"comment":"Values such as '0.9910.001' are missing delimiters and plus-minus signs, making the TPR/TNR/APD entries unreadable; please reformat with proper separators.","section":"Appendix E, Table 7"},{"comment":"The paper says code and datasets are released on GitHub and made public, but it provides no repository URL or version. Please add a formal availability statement with a link.","section":"Footnote 2 and Appendix B"},{"comment":"The normalization by ||Phi(f,x)|| in Eq. (6) can be unstable when an explanation vector is near zero; this may explain the very large and highly variable Cohen's d values in Tables 8-11 (for example, FairBERTa soft comprehensiveness d = 17.86 ± 31.01 on GECO-ALL). Please consider a regularized normalization or report unnormalized changes as a robustness check.","section":"Appendix A.6"},{"comment":"There are several typos, including 'thecomprehensiveness' in Section 5.2.1, 'run on Stereotypes' in Section 5.2.3, 'scoresobtained' in Figure 2's caption, and 'Table in 3' in Section 5.4. A careful proofreading pass is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The sensitivity metric mismatch is the most serious issue in this submission. If the authors can re-run the robustness experiments with the correct attack objective and address the multiple-comparison and causal-attribution concerns, the paper would be a strong contribution to XAI fairness. I recommend that the revision be checked specifically for the Appendix A.6 implementation description and for whether the from-scratch GECO experiments are interpreted too strongly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one. It's the first systematic audit of gender disparity in post-hoc explanation quality for transformer language models on text, and despite a real flaw in the robustness metric, it's a serious empirical contribution.\n\nWhat's new: prior fairness-of-explanation work stayed on tabular data with linear models or small NNs. This paper runs six feature-attribution methods across five PLMs (BERT, TinyBERT, RoBERTa-large, GPT-2, FairBERTa) and three text tasks, with seven metrics covering faithfulness, complexity, and robustness, repeated over five seeds. The headline result—72% of 5,040 configurations show statistically significant gender disparity, 55% with |d|≥0.2—is the kind of broad empirical claim that survives contact. The from-scratch training control on GECO is a good idea: it rules out pretrained-model bias as the whole story. The use of Mann-Whitney U with Cohen's d is standard and appropriate.\n\nWhere it goes soft. First, the sensitivity metric. Appendix A.6 defines sensitivity as the worst-case relative change in the explanation (Eq. 6), but the implementation description says the PGD attack perturbs input 'in the direction of the gradient maximizing the prediction error.' Those are different objectives. Unless the authors can show they coincide, the Sens. columns measure something else—how much the explanation changes under a prediction-error attack, not the worst-case explanation change. That undermines the robustness leg of the central claim, including the statement that sensitivity disparity is the highest on GECO. Faithfulness and complexity results are not affected, so the paper's core message still has a solid base, but the three-property claim in the abstract is currently overbroad.\n\nSecond, the causal attribution. The paper concludes the bias is 'mainly influenced by the explanation methods.' The from-scratch control removes pretraining bias, but on GECO gender is the label, so the model is forced to rely heavily on gender words. That design can demonstrate disparity without pretraining bias; it cannot isolate explanation methods as the primary cause. The conclusion should be softened to 'explanation methods can contribute independently of pretraining data.'\n\nThird, reproducibility is currently incomplete. The paper says code and data will be released on GitHub, but there's no link, and the sensitivity attack's radius and step count aren't given. That's fixable but necessary.\n\nBottom line: this is a paper that a serious XAI or fairness venue should referee. The empirical breadth is real, the sensitivity issue is addressable, and the limitations section is honest. I'd send it to review, with the expectation of major revision on the robustness metric and causal framing.","headline":"First systematic audit of gender disparity in post-hoc explanation quality for PLMs on text; the faithfulness and complexity results are credible, but the sensitivity metric looks misimplemented, so the robustness claim needs fixing.","tokens_in":38355,"tokens_out":2930,"would_cite":true,"duration_ms":28547,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that post-hoc explanation methods themselves produce explanations with significant gender disparity, independent of biases in the underlying model.","keywords":["gender bias","explainability","post-hoc explanations","feature attribution","fairness","language models","faithfulness","robustness"],"falsifier":"Check the sensitivity implementation in the released code and verify whether the perturbation search maximizes $\\|\\Phi(f,y)-\\Phi(f,x)\\|$ or the prediction error; if it maximizes prediction error, rerun the robustness comparisons with the correct objective and see whether the gender disparities in sensitivity persist.","tokens_in":37429,"feed_emoji":"⚖️","tokens_out":6617,"duration_ms":60519,"temperature":0.7,"pith_summary":"This paper sets out to determine whether the explanations produced by post-hoc feature attribution methods are themselves fair across genders. It finds that, across three text classification tasks and five transformer-based language models, six widely used attribution methods produce explanations whose measured faithfulness, robustness, and complexity differ significantly between male and female inputs. The disparities persist even when the underlying models are trained from scratch on gender-balanced data, which the authors take as evidence that the bias is substantially contributed by the explanation methods rather than inherited from the models. A reader should care because these explanations are used to audit, debug, and justify model decisions; if the explanations are themselves skewed, transparency tools can systematically mislead for one subgroup.","feed_headline":"Explanations themselves show gender bias across language models","feed_subtitle":"Six attribution methods differ significantly between male and female inputs, even for unbiased models.","key_machinery":"The argument rests on a disparity-measurement pipeline built from male-female sentence pairs and explanation-quality metrics. Six post-hoc feature attribution methods (Gradient, Integrated Gradients, Gradient×Input, IG×Input, LIME, and SHAP) produce token-importance scores; those scores are then scored by seven metrics covering faithfulness (comprehensiveness, sufficiency, and their soft counterparts), complexity (sparsity, Gini index), and robustness (sensitivity). For each metric, the male and female score distributions are compared with the Mann-Whitney U test, and the magnitude of disparity is quantified with Cohen's $d$. The key move is running the same pipeline on models trained from scratch on a gender-balanced dataset, isolating the contribution of the explanation method from the contribution of the model.","core_discovery":"The central claim is that explanation quality is not gender-neutral: feature attribution methods produce significantly different faithfulness, robustness, and complexity scores for male and female inputs, with statistically significant disparity ($p \\leq 0.05$) in 3,647 of 5,040 experimental combinations and considerable effect size ($|d| \\geq 0.2$) in 2,761. The effect is consistent across all six methods, with IG×Input, SHAP, and LIME showing the highest rates. The authors further claim that the disparity is not merely a consequence of biased models or data: retraining BERT and GPT-2 from scratch on the gender-balanced GECO dataset still leaves over 80% of runs with significant gender disparity. The paper concludes that post-hoc explanation methods themselves can be a source of unfairness, independent of model-level bias.","pith_inferences":["If the attribution methods are the source of disparity, then model-level debiasing alone will not fix explanation unfairness; explanation-evaluation suites should include per-subgroup score distributions as a standard check.","The appendix's discrepancy between the stated sensitivity objective and the PGD implementation used (prediction error instead of explanation change) suggests that the robustness results should be re-measured before being relied on, while the faithfulness and complexity findings do not depend on that step.","A testable extension is to vary the perturbation objective and radius in the sensitivity metric to see whether gender disparity in robustness changes once the explanation change is truly maximized, and to compare hard versus soft token removal in faithfulness.","The same disparity-measurement pipeline could be applied to non-binary gender markers or to race- and age-related cues in text, though the synthetic data generation would need to be adapted to avoid the corpus-design pitfalls the paper acknowledges."],"forward_implications":["Practitioners who audit a model's fairness by inspecting its explanations should also audit the explanations themselves, because the observed disparities can appear even when the model's predictions are not significantly biased.","Deploying post-hoc explanation methods in high-stakes text applications without subgroup checks can produce misleading justifications for one gender, undermining trust and potentially violating transparency obligations.","Training or fine-tuning on unbiased data is not a sufficient safeguard for explanation fairness; the attribution methods themselves need to be evaluated and, if needed, adjusted.","Because sensitivity disparity was the most frequent on the GECO datasets, explanation robustness should be included in any fairness-oriented evaluation of attribution methods.","Larger models such as RoBERTa reduce but do not eliminate the disparities, so model scale alone is not a remedy."],"supporting_citations":[{"why":"Supplies the gender-controlled sentence pairs (GECO) and the from-scratch training setup used to isolate model bias from explanation bias.","marker":"[69]"},{"why":"Establishes the use of the Mann-Whitney U test for quantifying explanation disparity and the fairness-of-explanation framing this paper extends to text.","marker":"[15]"},{"why":"Provides the soft comprehensiveness and sufficiency metrics used to measure faithfulness without full token removal.","marker":"[75]"},{"why":"Defines the sensitivity metric that grounds the robustness evaluation.","marker":"[72]"},{"why":"The projected gradient descent attack used to implement the sensitivity computation; its objective is the claimed source of the measurement mismatch.","marker":"[46]"},{"why":"Supplies the off-the-shelf implementations of the explanation methods and evaluation metrics used in the experiments.","marker":"[6]"},{"why":"COMPAS provides the third dataset, converted from tabular records to text for the recidivism prediction task.","marker":"[3]"},{"why":"Used to generate the synthetic Stereotypes dataset of male-female sentence pairs expressing or contradicting stereotypes.","marker":"[4]"}],"fun_headline_variants":["Explanation methods show gender bias even in fair models","Post-hoc explanations exhibit gender disparity across tasks","Feature attribution accuracy varies by gender, even unbiased models","Explainability tools display gender bias independent of model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The robustness pillar depends on the sensitivity metric being computed as defined in Eq. 6, but the paper's Appendix A.6 says the PGD attack maximizes prediction error instead of explanation change, so the reported sensitivity values may not measure explanation robustness.","fun_headline_variants_meta":{"raw":{"variants":["Explanation methods show gender bias even in fair models","Post-hoc explanations exhibit gender disparity across tasks","Feature attribution accuracy varies by gender, even unbiased models","Explainability tools display gender bias independent of model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1189,"prompt_tokens":853,"completion_tokens":336,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":469,"completion_tokens_details":{"reasoning_tokens":277}},"tokens_in":469,"tokens_out":336,"duration_ms":3734,"temperature":1.0,"reasoning_tokens":277,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:23:42.633648+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check the sensitivity implementation in the released code and verify whether the perturbation search maximizes $\\|\\Phi(f,y)-\\Phi(f,x)\\|$ or the prediction error; if it maximizes prediction error, rerun the robustness comparisons with the correct objective and see whether the gender disparities in sensitivity persist.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the sensitivity metric that grounds the robustness evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the off-the-shelf implementations of the explanation methods and evaluation metrics used in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"COMPAS provides the third dataset, converted from tabular records to text for the recidivism prediction task."}],"review_version":1}