REVIEW 4 major objections 4 minor 22 references
DiCE-Extended: A Robust Approach to Counterfactual Explanations in Machine Learning
T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Adding a Dice–Sørensen robustness term to the DiCE counterfactual explainer produces explanations that are closer, sparser, more diverse, and more robust while keeping validity near 100 percent.
desk verdict A DiCE extension with a non-differentiable robustness loss and circular evaluation; the central claim is unsupported as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the robustness regularization term. The paper defines it by binarizing the generated counterfactual vector $c$ and its perturbed version $c' = c + \delta$, then computing the Dice–Sørensen distance $2|c \cap c'|/(|c| + |c'|)$ between the two binary vectors. This term is inserted into DiCE's objective with weight $\lambda_r$, next to the proximity term (weight $\lambda_p = 0.5$) and the diversity term based on the determinant of a similarity kernel (weight $\lambda_d = 1.0$). The paper treats the resulting combined loss as minimized by gradient-based optimization; the robustness term is the new component that is supposed to pull generated counterfactuals toward regions where small perturbations do not change the explanation.
What would settle it
Run a gradient check on the robustness term: compute $L_{\text{Robustness}}$ after binarizing $c$ and $c + \delta$, and evaluate whether small continuous steps in $c$ change the loss. Because the binarized loss is piecewise constant, the expected gradient is zero almost everywhere; if no surrogate gradient is implemented, the reported robustness-loss decline cannot come from minimizing that term. A simpler experimental falsifier: set $\lambda_r = 0$ and keep all other settings identical; if the robustness scores are statistically unchanged, the added term is not doing the work attributed to it.
Extended reading notes
Core claim
The paper proposes to replace DiCE's two-term objective with a weighted three-term objective: minimize $L_{y\_loss} + \lambda_p L_{\text{Proximity}} - \lambda_d L_{\text{Diversity}} - \lambda_r L_{\text{Robustness}}$, where $L_{\text{Robustness}}$ is the Dice–Sørensen distance between binarized counterfactuals and counterfactuals perturbed by $\delta$. With $\lambda_p = 0.5$, $\lambda_d = 1.0$, and $\lambda_r = 0.4$ (selected by grid search), the paper reports that, across four datasets and three backends, DiCE-Extended consistently matches or beats standard DiCE: proximity decreases substantially with validity near 100 percent, sparsity and diversity rise, robustness improves by 0.05–0.13 for neural-network backends, 1-NN fidelity is usually preserved or improved, and generation time grows by only about 1.2×. The paper interprets these results as evidence that the added robustness term does not trade away interpretability or local fidelity but makes counterfactuals adhere more faithfully to the model's local decision boundary.
Load-bearing premise
The argument depends on the assumption that the binarized Dice–Sørensen robustness loss can actually be minimized by the optimizer; after binarization the loss is flat almost everywhere, so gradient descent has no slope to follow unless a differentiator surrogate is used.
Editorial extensions
If this is right
- Standard DiCE users can expect the robustness term to shrink median proximity by 30–95 percent without sacrificing validity, which directly improves the actionability of the explanations.
- Neural-network-based explainability pipelines gain the most stability, with absolute robustness scores rising 0.05–0.13, suggesting the term is most valuable exactly where counterfactual instability is worst.
- Because 1-NN fidelity is preserved or improved in most configurations, adding the robustness term does not pull counterfactuals away from the model's local decision boundary.
- The reported 1.2× generation-time overhead means robustness can be added to existing DiCE-style generators without redesigning the pipeline.
Reading between the lines
- Editorial extension: substituting a differentiable surrogate for the binarized Dice–Sørensen term (for example, a soft-thresholded version) and rerunning the benchmarks would test whether the reported robustness gains come from the robustness term itself or from the re-weighted $\lambda_p$ and $\lambda_d$ alone.
- Editorial extension: the robustness metric is computed by perturbing the generated counterfactuals $c$, not the original query $x$; a claim of stability under 'small input variations' would need a second metric that perturbs the query and checks whether the recommended actions stay the same.
- Editorial extension: this approach connects naturally to adversarial robustness; one could test whether DiCE-Extended's counterfactuals become harder to flip when the underlying model is adversarially trained, which would make explanation robustness and model robustness complementary.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DiCE-Extended, an extension of DiCE that adds a robustness regularization term to the counterfactual-generation objective. The term is computed as a Dice–Sørensen distance between binarized original and perturbed counterfactual vectors, and the total loss combines proximity, diversity, and robustness weighted by λp, λd, and λr. The authors evaluate the method on COMPAS, Lending Club, German Credit, and Adult Income with Random Forest, PyTorch, and TensorFlow backends, and report that DiCE-Extended improves robustness, proximity, sparsity, diversity, and local fidelity while maintaining near-perfect validity.
Significance. The proposed goal—making counterfactuals robust to small perturbations—addresses a real limitation of DiCE, and the experimental design has strengths: four datasets, three backends, paired per-instance t-tests with reported p-values, and 1-NN fidelity checks. The paper also clearly separates quality dimensions in the loss. However, the central contribution is not currently supported. The robustness loss is piecewise constant and non-differentiable as written, the optimization direction is ambiguous because of the sign convention, the evaluation metric appears to coincide with the training loss (with λr tuned on the evaluation datasets), and the main quantitative improvements are not reported in any table. These issues prevent the claims from being attributed to the proposed mechanism.
major comments (4)
- [§2.3, §2.5, Fig. 1] The novel robustness loss is computed on binarized versions of c and c′ = c + δ. Binarization is a step function, making the loss piecewise constant with zero gradient almost everywhere, yet §2.5 states that optimization is gradient-based (backpropagation, Adam). No straight-through estimator, continuous relaxation, or other surrogate is described. Therefore the minimization of LRobustness is not well-defined as written, and the decreasing robustness-loss curves in Figure 1 and the robustness gains in §3.3 are unexplained by the stated method.
- [§2.3] The total loss is written as L = Ly_loss + λpLProximity − λdLDiversity − λrLRobustness, with LRobustness = λr · robustness_loss, and robustness_loss is called the Dice–Sørensen distance between the original and perturbed CFs. If it is a distance, subtracting it encourages the optimizer to increase it, i.e., to make CFs less robust; if it is instead a similarity, the term is misnamed. The intended optimization direction is ambiguous, and this ambiguity affects all reported results.
- [§2.3 and §3.3] The robustness metric used for evaluation appears to be the same binarized Dice–Sørensen quantity that is minimized as the robustness loss, and λr is chosen by grid search on the evaluation datasets themselves. The reported +0.05 to +0.13 robustness improvements are therefore largely expected from optimizing the evaluation criterion, and the experiments do not demonstrate robustness gains under an independent measure.
- [§3.2, Tables 2–3] The main empirical claims in §3.3—proximity reductions of 30–95%, sparsity improvements up to 50%, near-doubled diversity, and robustness gains of +0.05 to +0.13—are not accompanied by any table or figure reporting these metrics. Tables 2 and 3 contain only 1-NN fidelity scores, and Figure 1 shows training loss curves, not evaluation results. The validity rate of 100% and the statistical comparisons for these claims are also not shown in a data table. The central results are therefore not verifiable from the manuscript.
minor comments (4)
- [§2.3] The sentence 'Following the DiCE framework [19], λd = 1.0 is set to 1.0' is redundant; it should read 'λd is set to 1.0.'
- [§2.5, Eq. (1)] In Eq. (1), n is described as the number of features, but the input is x ∈ R^d, so n is the hidden-layer width; d is the feature dimension.
- [§2.3] In the grid-search description, 'the robustness loss was found to peak at λr = 0.4' is confusing, since a loss peaking is not a desirable selection criterion; the following sentence says the total loss continued to decrease. Please clarify which quantity is maximized or minimized.
- [Tables 2 and 3] Each table note says the highest fidelity is highlighted in bold, but the tables contain no bold entries as printed; this should be corrected.
Circularity Check
Robustness gains are circular: the evaluated 'robustness measure' is the minimized loss itself, and the robustness weight is tuned on the same datasets on which results are reported.
-
self definitional
[Section 2.3 (Mathematical Formalizations) and Section 3.3 (Performance Gains Across Counterfactual Dimensions)]
"LRobustness = λr · robustness_loss, where λr is a tunable hyperparameter and robustness_loss corresponds to the Dice–Sørensen distance between the original and perturbed CF explanations. ... we report standard evaluation metrics, including validity, proximity, sparsity, diversity, and the proposed robustness measure, as illustrated in Figure 1. ... Improvements were most pronounced in the newly introduced robustness metric. For PyTorch and TensorFlow models, DiCE-Extended demonstrated absolute gains of 0.05–0.13 in robustness scores."
The 'proposed robustness measure' used in the evaluation is the same robustness_loss term that DiCE-Extended minimizes during training. Therefore the reported +0.05 to +0.13 robustness gain is not an independent empirical result; it is the objective function being optimized. Any optimizer that lowers LRobustness will, by definition, increase the reported Dice–Sørensen score, so the claim that robustness improved reduces to the statement that the optimized loss decreased, which is true by construction rather than by validation.
-
fitted input called prediction
[Section 2.3 (Mathematical Formalizations), lambda_r selection paragraph]
"To determine an appropriate value for λr, we conducted a grid search within the range [0, 1] across all datasets. ... the robustness loss was found to peak at λr = 0.4, although the total loss continued to decrease beyond this point. ... Based on these findings, we selected λr = 0.4 as a moderate and balanced value ... The same grid search procedure was applied to all datasets."
The hyperparameter λr is selected by grid search on the same four datasets on which all performance results are later reported, and the selection criterion is the robustness metric itself. Reporting those robustness scores as benchmark outcomes is therefore a fitted evaluation: the weight has been chosen to maximize the reported quantity on the evaluation data. The headline robustness improvement is partly an artifact of tuning on the test set rather than an out-of-sample prediction.
full rationale
The central circularity is that the paper's headline robustness improvement is measured with the same Dice–Sørensen distance that is minimized as a loss component (Section 2.3 vs. Section 3.3), and the robustness weight λr is tuned via grid search on the very datasets used to report results. This makes the +0.05 to +0.13 robustness gain reduce, by construction, to the optimization of the evaluation metric itself. The paper also contains independent, non-circular evidence: proximity, sparsity, diversity, and the external 1-NN fidelity metric are not part of the training loss, and those results can stand on their own. The separate issue that binarization makes the robustness loss piecewise constant with vanishing gradients is a correctness and implementability concern, not a circularity, so it does not raise the circularity score above 6. No load-bearing self-citation or imported uniqueness argument appears in the derivation chain.
Assumptions & free parameters
free parameters (5)
- lambda_r (robustness weight) =
0.4
- lambda_p (proximity weight) =
0.5
- lambda_d (diversity weight) =
1.0
- Binarization threshold for robustness loss =
Not reported
- Perturbation delta for robustness loss =
Not reported
assumptions (4)
- domain assumption The model f is differentiable (Section 2.1), yet the experiments include a Random Forest backend.
- ad hoc to paper The Dice-Sørensen distance on binarized vectors is a valid and optimizable robustness measure.
- ad hoc to paper The grid search for lambda_r on the evaluation datasets yields a representative value.
- domain assumption 1-NN fidelity with MAD radii measures alignment with the local decision boundary.
Cite this review
Pith. "Pith review of DiCE-Extended: A Robust Approach to Counterfactual Explanations in Machine Learning." pith.science (2026). https://pith.science/paper/IEZMXQBU
@misc{pith2026250419027,
author = {Pith},
title = {Pith review of: DiCE-Extended: A Robust Approach to Counterfactual Explanations in Machine Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/IEZMXQBU}},
note = {Machine review of arXiv:2504.19027}
}
read the original abstract
Explainable artificial intelligence (XAI) has become increasingly important in decision-critical domains such as healthcare, finance, and law. Counterfactual (CF) explanations, a key approach in XAI, provide users with actionable insights by suggesting minimal modifications to input features that lead to different model outcomes. Despite significant advancements, existing CF generation methods often struggle to balance proximity, diversity, and robustness, limiting their real-world applicability. A widely adopted framework, Diverse Counterfactual Explanations (DiCE), emphasizes diversity but lacks robustness, making CF explanations sensitive to perturbations and domain constraints. To address these challenges, we introduce DiCE-Extended, an enhanced CF explanation framework that integrates multi-objective optimization techniques to improve robustness while maintaining interpretability. Our approach introduces a novel robustness metric based on the Dice-S{\o}rensen coefficient, enabling stability under small input variations. Additionally, we refine CF generation using weighted loss components (lambda_p, lambda_d, lambda_r) to balance proximity, diversity, and robustness. We empirically validate DiCE-Extended on benchmark datasets (COMPAS, Lending Club, German Credit, Adult Income) across multiple ML backends (Scikit-learn, PyTorch, TensorFlow). Results demonstrate improved CF validity, stability, and alignment with decision boundaries compared to standard DiCE-generated explanations. Our findings highlight the potential of DiCE-Extended in generating more reliable and interpretable CFs for high-stakes applications. Future work could explore adaptive optimization techniques and domain-specific constraints to further enhance CF generation in real-world scenarios
Figures
Reference graph
Works this paper leans on
-
[1]
OpenIntro Dataset (Unk nown), https://www.openintro.org/data/index.php?data=loans_full_schema
Anonymous: Lending Club Dataset. OpenIntro Dataset (Unk nown), https://www.openintro.org/data/index.php?data=loans_full_schema
-
[2]
UCI Machine Learning Repos itory (1996), DOI: https://doi.org/10.24432/C5XW20
Becker, B., Kohavi, R.: Adult. UCI Machine Learning Repos itory (1996), DOI: https://doi.org/10.24432/C5XW20
doi:10.24432/c5xw20 1996
-
[3]
In: International conference on parallel prob lem solving from nature
Dandl, S., Molnar, C., Binder, M., Bischl, B.: Multi-obje ctive counterfactual ex- planations. In: International conference on parallel prob lem solving from nature. pp. 448–469. Springer (2020). https://doi.org/10.1007/978-3-030-58112-1_31
-
[4]
Interna- tional Journal of Human–Computer Interaction 40(1), 33–44 (2024)
De Schutter, L., De Cremer, D.: How counterfactual fairne ss mod- elling in algorithms can promote ethical decision-making. Interna- tional Journal of Human–Computer Interaction 40(1), 33–44 (2024). https://doi.org/10.1080/10447318.2023.2247624
arXiv 2024
-
[5]
Cognitive Computation 16(5), 2660–2673 (2024)
El Shawi, R.: Conceptglassbox: Guided concept-based exp lanation for deep neural networks. Cognitive Computation 16(5), 2660–2673 (2024). https://doi.org/10.1007/s12559-024-10262-8
-
[6]
In: World Con- ference on Explainable Artificial Intelligence
Ezzeddine, F., Ayoub, O., Andreoletti, D., Giordano, S.: Sac-fact: Soft actor- critic reinforcement learning for counterfactual explana tions. In: World Con- ference on Explainable Artificial Intelligence. pp. 195–21 6. Springer (2023). https://doi.org/10.1007/978-3-031-44064-9_12
-
[7]
Advances i n Neural Information Pro- cessing Systems 37, 137105–137134 (2024)
Fawkes, J., Fishman, N., Andrews, M., Lipton, Z.: The frag ility of fairness: Causal sensitivity analysis for fair machine learning. Advances i n Neural Information Pro- cessing Systems 37, 137105–137134 (2024)
work page 2024
-
[8]
IEEE Access 10, 82736–82750 (2022)
Ferrario, A., Loi, M.: The robustness of counterfactual e x- planations over time. IEEE Access 10, 82736–82750 (2022). https://doi.org/10.1109/ACCESS.2022.3196917
arXiv 2022
Show all 22 references
-
[9]
ACM Com puting Surveys 56(9), 1–33 (2024)
Gajcin, J., Dusparic, I.: Redefining counterfactual expl anations for reinforcement learning: Overview, challenges and opportunities. ACM Com puting Surveys 56(9), 1–33 (2024). https://doi.org/10.1145/3648472
2024 doi
- [10]
-
[11]
OpenML Dataset #4 5039 (2023), https://api.openml.org/d/45039
Grin, L.: COMPAS Two-Year Recidivism. OpenML Dataset #4 5039 (2023), https://api.openml.org/d/45039
2023
-
[12]
In: Proceedings of the 32nd ACM In- ternational Conference on Information and Knowledge Manag ement
Guo, H., Jia, F., Chen, J., Squicciarini, A., Yadav, A.: R ocoursenet: Robust train- ing of a prediction aware recourse model. In: Proceedings of the 32nd ACM In- ternational Conference on Information and Knowledge Manag ement. pp. 619–628 (2023). https://doi.org/10.1145/35837...
2023
-
[13]
In: 2019 International Conference o n Information and Communication Technology Convergence (ICTC)
Ha Lee, J., Hee Shin, I., Gu Jeong, S., Lee, S.I., Zaheer, M .Z., Seo, B.S.: Improvement in deep networks for optimization using e xplainable ar- tificial intelligence. In: 2019 International Conference o n Information and Communication Technology Convergence (ICTC). pp. 525–53...
2019
- [14]
-
[15]
UCI Machine L earning Repository (1994), DOI: https://doi.org/10.24432/C5NC77 12 Bakir et al
Hofmann, H.: Statlog (German Credit Data). UCI Machine L earning Repository (1994), DOI: https://doi.org/10.24432/C5NC77 12 Bakir et al
1994 doi
-
[16]
In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency
Laugel, T., Jeyasothy, A., Lesot, M.J., Marsala, C., Det yniecki, M.: Achieving diversity in counterfactual explanations: a review and dis cussion. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. pp. 1859–1869 (2023). https://doi.org/1...
2023
-
[17]
Expert Systems with Applications 266, 126063 (2025)
Liu, J., Wu, X., Liu, S., Gong, S.: Model-agnostic counte rfactual explana- tion: A feature weights-based comprehensive causal multi- objective counter- factual framework. Expert Systems with Applications 266, 126063 (2025). https://doi.org/10.1016/j.eswa.2024.126063
2025
-
[18]
In: Interna- tional Conference on Artificial Intelligence and Statistic s
Lucic, A., Ter Hoeve, M.A., Tolomei, G., De Rijke, M., Sil vestri, F.: Cf- gnnexplainer: Counterfactual explanations for graph neur al networks. In: Interna- tional Conference on Artificial Intelligence and Statistic s. pp. 4499–4511. PMLR (2022)
2022
-
[19]
Proceeding s of the 2020 Con- ference on Fairness, Accountability, and Transparency pp
Mothilal, R.K., Sharma, A., Tan, C.: Explaining machine learning classifiers through diverse counterfactual explanations. Proceeding s of the 2020 Con- ference on Fairness, Accountability, and Transparency pp. 607–617 (2020). https://doi.org/10.1145/3351095.3372850
2020
-
[20]
IEEE Transactions on Dependable and S ecure Computing (2023)
Song, Z., Zhao, Z., Zhang, F., Xiong, G., Cheng, G., Zhao, X., Guo, S., Chen, B.: I2 RNN: An incremental and interpretable recurrent neural net work for encrypted traffic classification. IEEE Transactions on Dependable and S ecure Computing (2023). https://doi.org/10.1109/TDSC.20...
2023
-
[21]
In: International Conference on Artificial Neural Networks
Xu, A., Wu, T.: Generally-occurring model change for rob ust counterfactual expla- nations. In: International Conference on Artificial Neural Networks. pp. 215–229. Springer (2024). https://doi.org/10.1007/978-3-031-72341-4_15
2024 doi
-
[22]
IEEE Transactions on Automation Science and Engineer ing 21(3), 2264–2275 (2023)
Zhou, S., Islam, U.J., Pfeiffer, N., Banerjee, I., Patel, B.K., Iquebal, A.S.: Sc- gan: Sparse countergan for counterfactual explanations in breast cancer predic- tion. IEEE Transactions on Automation Science and Engineer ing 21(3), 2264–2275 (2023). https://doi.org/10.1109/TAS...
2023
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.