REVIEW 5 major objections 6 minor 41 references
RealAC: A Domain-Agnostic Framework for Realistic and Actionable Counterfactual Explanations
T0 review · 5 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read RealAC generates realistic counterfactual explanations by matching pairwise mutual information between factual and counterfactual features, without needing domain knowledge.
desk verdict Interesting pairwise-MI regularizer, but the hidden constraint term on Synthetic 1 undercuts the domain-agnostic claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the pairwise dependency loss L_dep: features are discretized into B bins, empirical joint distributions are estimated from co-occurrence histograms, and the loss is the average absolute difference in mutual information between each feature pair in the factual batch and the counterfactual batch. This mechanism claims to preserve complex, nonlinear inter-feature relations without any explicit domain knowledge. A separate binary actionability mask m merges immutable factual features with optimizer updates via x_cf = m ⊙ x_0 + (1-m) ⊙ x̂, keeping frozen attributes fixed during optimization.
What would settle it
Run RealAC with only L_dep (no extra penalty) on a dataset that has a known deterministic functional constraint such as X6 + X7 = constant, and check whether the generated counterfactuals violate that constraint; or measure the causal edge score on counterfactuals where L_dep is perfectly matched to see whether per-instance causal edges are still broken.
Extended reading notes
Core claim
The paper's central claim is that preserving pairwise mutual information—not explicit structural equations or causal graphs—is sufficient to keep counterfactuals realistic. RealAC estimates the joint distribution of each feature pair by binning and co-occurrence histograms, then minimizes the absolute difference in mutual information between the factual and counterfactual pairs (L_dep). This regularizer is differentiable and plugs into a VAE whose total loss also enforces prediction flip, proximity, and a binary actionability mask that freezes immutable features. On Synthetic 1/2, Diabetes, Sangiovese, and Adult, RealAC reports the best causal edge score, dependency preservation score, and I
Load-bearing premise
RealAC's 'domain-agnostic' claim rests on the assumption that matching batch-level pairwise mutual information (L_dep) is enough to preserve per-instance causal and structural dependencies; the paper's own Synthetic 1 example (X6 + X7 = constant) shows this fails without an extra explicit penalty term.
Editorial extensions
If this is right
- If RealAC's central claim holds, counterfactual generators no longer need causal graphs or hand-crafted constraints; statistical alignment of feature pairs can substitute.
- The same framework should transfer to any tabular dataset with unknown or heterogeneous feature relations, including categorical features.
- End-users get a practical control: features they mark immutable stay fixed, making the suggested recourse actions actually feasible.
- The dependency preservation score gives researchers a simple quantitative target for evaluating structural realism.
- RealAC's speed advantage over most baselines makes it usable in interactive settings where explanations must be generated on demand.
Reading between the lines
- Because L_dep is a batch statistic, it may be insensitive to per-instance constraints; the paper's own X6+X7 example shows a deterministic relation that escapes it, so a robust deployment should pair the method with lightweight explicit rules for such pre-identified invariants.
- The approach suggests a testable general principle: pairwise MI alignment may be enough when dependencies are pairwise and smooth, but higher-order or deterministic constraints require higher-order terms; this could be checked by systematically adding ternary MI terms.
- LLM-based counterfactual generation (SenseCF) performed poorly on prediction flipping without classifier access; this points to a division of labor where LLMs propose edits and optimization-based methods validate or refine them.
- The freeze-mask mechanism could be reused in other generative models (GANs, diffusion) for constrained generation, not just VAEs.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. RealAC is a VAE-based counterfactual generation framework. It proposes to preserve inter-feature dependencies by matching pairwise mutual information between factual and counterfactual batches (L_dep), to enforce user-specified immutability via binary masks, and to optimize flip, proximity, and KL losses. The paper introduces a Dependency Preservation Score (DPS) and compares RealAC against several CF baselines and LLM-based SenseCF on three synthetic and two real tabular datasets, claiming consistent superiority in causal edge score, DPS, and IM1. The abstract claims that RealAC is domain-agnostic and automatically preserves complex dependencies without explicit domain knowledge.
Significance. The motivation is solid, and a domain-agnostic method for dependency-preserving counterfactuals would be valuable. The code release and ablation studies are positive features. However, the central claim is not supported by the manuscript as written. The differentiability of L_dep is asserted without a viable estimator, the reported Synthetic 1 results rely on an undocumented domain-specific LAMBDA SUM term that is absent from the method and is conceded in the Limitations to require explicit domain knowledge, and the headline outperformance claim is directly contradicted by Table 1. Because these issues concern the core methodology and its evaluation, the contribution cannot be accepted in its current form.
major comments (5)
- [Methodology, Feature Dependency] The paper defines the joint distribution estimator using indicator functions 1[x_n,i in b_p] and then states: 'Importantly, this regularizer is differentiable with respect to the perturbation.' Indicator-based histograms have zero gradient almost everywhere, so gradient-based optimization of L_dep is not defined as written. If soft binning, a straight-through estimator, or a kernel approximation is used, it must be specified. This is load-bearing because the entire realism mechanism depends on optimizing L_dep.
- [Technical Appendix Table 4 and Limitations] Table 4 lists 'LAMBDA SUM = 2.0, for X6 + X7 = k' for Synthetic 1, but the L_total equation in Methodology contains no such term. The Limitations then state that L_dep alone is insufficient for X6+X7=constant and that capturing this constraint 'requires explicit use of domain knowledge or structural priors.' Therefore the reported Synthetic 1 results (DPS=0.719, CES=-6.558) were obtained with an unstated domain-specific loss, contradicting the abstract's claim of preserving dependencies 'without relying on explicit domain knowledge.' This is a fundamental mismatch between the described method and the reported experiments.
- [Abstract and Table 1] The abstract claims RealAC 'outperforms state-of-the-art baselines and Large Language Model-based counterfactual generation techniques in causal edge score, dependency preservation score, and IM1 realism metric.' Table 1 contradicts this for DPS: SenseCF(o3) achieves DPS=0.793 on Synthetic 1 and 0.768 on Synthetic 2, both higher than RealAC's 0.719 and 0.709. For CES (closer to zero is better), SenseCF(4o) on Synthetic 1 (-4.658) is closer to zero than RealAC (-6.558), and CEILS on Diabetes (0.159) is closer than RealAC (0.278). The claim of consistent outperformance is not supported by the paper's own tables.
- [Problem Formulation and Methodology] The dependency preservation constraint is stated per instance: ρ(x_cf_i, x_cf_j) ≈ ρ(x_i, x_j). However, L_dep is computed from batch-level co-occurrence histograms and therefore matches average pairwise statistics over a batch, not per-instance dependencies. A batch-level match can hold while individual counterfactuals violate the dependency, which is exactly what happens with the X6+X7=k constraint. The paper should clarify that the method preserves distribution-level pairwise statistics, not per-instance causal or structural dependencies.
- [Validation Metrics, DPS definition] DPS is introduced in this paper as a headline metric. It measures how well generated CFs follow known conditional relationships via regression residuals—precisely the kind of pairwise dependency structure L_dep is designed to enforce. Without independent validation of DPS against human judgment or an established realism benchmark, the evaluation is partly circular: the method is rewarded for matching the objective it was trained on. The paper should either validate DPS independently or de-emphasize it in favor of metrics with established validity.
minor comments (6)
- [Technical Appendix, Dataset Description] Typo: 'binzry target variable' should be 'binary target variable.'
- [Throughout] Inconsistent spacing in 'V AE' and 'C-CHV AE'; use 'VAE' consistently.
- [Figure 4] The legend includes 'ExAct' but this method is not described in the baselines list or the results tables. Either add a description or remove it.
- [Methodology, Proximity] The sentence 'λ_fixed > λ_mse should theoretically suppress change in immutable features and allow controlled mutation in the rest' appears as an isolated fragment; it should be integrated into the surrounding discussion.
- [Tables 1 and 2] No standard deviations or confidence intervals are reported. Given the variability of CF generation and the small margins on some datasets, statistical significance of the claimed advantages is unclear.
- [Technical Appendix, Adult prompt] Minor typo: 'income' is misspelled as 'inome' in the prompt template.
Circularity Check
Synthetic 1 evaluation is partially circular: the reported DPS/CES improvements for the X6+X7 constraint were obtained with an undocumented LAMBDA SUM term, while the Limitations concede L_dep cannot capture that constraint.
-
fitted input called prediction
[Technical Appendix, Optimizer/VAE Description (Table 4); Limitations section]
"LAMBDA SUM = 2.0, for X6 + X7 = k ... optimizing only for dependency preservation via joint probability-based loss L_dep is insufficient. Capturing these specific constraints requires explicit use of domain knowledge or structural priors during optimization, beyond what is learned from the data distribution alone."
The paper's central claim is that RealAC automatically preserves complex inter-feature dependencies without explicit domain knowledge, by optimizing L_dep (pairwise mutual-information matching). However, Table 4 lists an additional LAMBDA SUM loss term for Synthetic 1 that does not appear in the L_total equation in the Methodology. The Limitations explicitly state that L_dep alone cannot capture the X6+X7=constant constraint and that explicit domain knowledge is required. Therefore, the reported Synthetic 1 DPS=0.719 and CES=-6.558 are at least partly produced by a hand-added, dataset-specific constraint term, making those results an input to the optimization rather than a prediction from the proposed domain-agnostic mechanism. The evaluation on the very dataset meant to demonstrate automa
full rationale
The paper is not globally circular: L_dep is defined independently of the validation metrics, and the main comparisons also use external CES and IM1. The self-citations (Arefeen & Ghasemzadeh 2023; Arefeen et al. 2025; Soumma et al. 2025) are not load-bearing to the central derivation. However, the core 'domain-agnostic automatic dependency preservation' claim is partially undermined by the paper's own appendix and limitations. A LAMBDA SUM term for X6+X7=k is used in Synthetic 1 but appears nowhere in the method's L_total, and the Limitations state that L_dep is insufficient for this constraint. This means the reported Synthetic 1 DPS and CES are not a clean test of the proposed mechanism; the claim that the method preserves complex dependencies without domain knowledge is at least partly forced by an explicit, dataset-specific input. The score is 6 rather than 8 because the circularity is localized to one dataset/constraint, and the remainder of the evaluation has independent metric components.
Assumptions & free parameters
free parameters (6)
- lambda_dep =
4.0 (Synthetic1), 10.0 (Synthetic2), 2.0 (Diabetes), 10.0 (Sangiovese), 6.0 (Adult)
- lambda_flip =
1.0, 555.0, 10.0, 3.0, 200.0
- lambda_mse =
2.0, 830.0, 0.6, 6.5, 4.0
- lambda_fixed =
0.5, 0.5, 500, -, 0.5
- B (number of bins) =
50 for all datasets
- LAMBDA SUM =
2.0 for X6 + X7 = k in Synthetic 1
assumptions (4)
- domain assumption Pairwise mutual information between features is a sufficient statistic for the inter-feature dependencies that make counterfactuals realistic.
- domain assumption Empirical histograms with B=50 bins provide differentiable and accurate MI estimates.
- domain assumption The classifier under explanation is differentiable (MLP) so that L_flip can be backpropagated.
- domain assumption Batch-level distribution matching transfers to per-instance dependency preservation.
Cite this review
Pith. "Pith review of RealAC: A Domain-Agnostic Framework for Realistic and Actionable Counterfactual Explanations." pith.science (2026). https://pith.science/paper/NQ4C4UZR
@misc{pith2026250810455,
author = {Pith},
title = {Pith review of: RealAC: A Domain-Agnostic Framework for Realistic and Actionable Counterfactual Explanations},
year = {2026},
howpublished = {\url{https://pith.science/paper/NQ4C4UZR}},
note = {Machine review of arXiv:2508.10455}
}
read the original abstract
Counterfactual explanations provide human-understandable reasoning for AI-made decisions by describing minimal changes to input features that would alter a model's prediction. To be truly useful in practice, such explanations must be realistic and feasible -- they should respect both the underlying data distribution and user-defined feasibility constraints. Existing approaches often enforce inter-feature dependencies through rigid, hand-crafted constraints or domain-specific knowledge, which limits their generalizability and ability to capture complex, nonlinear relations inherent in data. Moreover, they rarely accommodate user-specified preferences and suggest explanations that are causally implausible or infeasible to act upon. We introduce RealAC, a domain-agnostic framework for generating realistic and actionable counterfactuals. RealAC automatically preserves complex inter-feature dependencies without relying on explicit domain knowledge -- by aligning the joint distributions of feature pairs between factual and counterfactual instances. The framework also allows end-users to ``freeze'' attributes they cannot or do not wish to change by suppressing change in frozen features during optimization. Evaluations on three synthetic and two real datasets demonstrate that RealAC balances realism with actionability. Our method outperforms state-of-the-art baselines and Large Language Model-based counterfactual generation techniques in causal edge score, dependency preservation score, and IM1 realism metric and offers a solution for causality-aware and user-centric counterfactual generation.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Afrabandpey, H.; and Spranger, M. 2022. Feasible and Desirable Counterfactual Generation by Preserving Human Defined Constraints. ArXiv, abs/2210.05993
arXiv 2022
-
[2]
Arefeen, A.; and Ghasemzadeh, H. 2023. Designing User-Centric Behavioral Interventions to Prevent Dysglycemia with Novel Counterfactual Explanations. ArXiv, abs/2310.01684
arXiv 2023
-
[3]
A.; Thompson, B.; and Ghasemzadeh, H
Arefeen, A.; Khamesian, S.; Grando, M. A.; Thompson, B.; and Ghasemzadeh, H. 2025. GlyTwin: Digital Twin for Glucose Control in Type 1 Diabetes Through Optimal Behavioral Modifications Using Patient-Centric Counterfactuals. ArXiv, abs/2504.09846
arXiv 2025
-
[4]
Artelt, A.; and Hammer, B. 2020. Convex Density Constraints for Computing Plausible Counterfactual Explanations. ArXiv, abs/2002.04862
work page Pith review arXiv 2020
-
[5]
Artelt, A.; and Hammer, B. 2021. Convex optimization for actionable & plausible counterfactual explanations. ArXiv, abs/2105.07630
work page Pith review arXiv 2021
-
[6]
Brughmans, D.; and Martens, D. 2021. NICE: an algorithm for nearest instance counterfactual explanations. Data Mining and Knowledge Discovery, 1--39
work page 2021
-
[7]
Crupi, R.; Castelnovo, A.; Regoli, D.; and Gonz \'a lez, B. S. M. 2021. Counterfactual Explanations as Interventions in Latent Space. Data Min. Knowl. Discov., 38: 2733--2769
work page 2021
-
[8]
Crupi, R.; Gonz \'a lez, B. S. M.; Castelnovo, A.; and Regoli, D. 2022. Leveraging Causal Relations to Provide Counterfactual Explanations and Feasible Recommendations to End Users. In International Conference on Agents and Artificial Intelligence
work page 2022
Show all 41 references
-
[9]
de Oliveira, R. M. B.; S \"o rensen, K.; and Martens, D. 2023. A model-agnostic and data-independent tabu search algorithm to generate counterfactuals for tabular, image, and text data. Eur. J. Oper. Res., 317: 286--302
2023
-
[10]
Doshi-Velez, F.; and Kim, B. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arXiv: Machine Learning
2017
-
[11]
D.; Li, Q.; and Xu, G
Duong, T. D.; Li, Q.; and Xu, G. 2021. Causality-based counterfactual explanation for classification models. Knowl. Based Syst., 300: 112200
2021
-
[12]
Frappier, M. 2018. The Book of Why: The New Science of Cause and Effect. Science, 361: 855 -- 855
2018
-
[13]
A.; and Macy, M
Golder, S. A.; and Macy, M. W. 2011. Diurnal and Seasonal Mood Vary with Work, Sleep, and Daylength Across Diverse Cultures. Science, 333: 1878 -- 1881
2011
-
[14]
Guidotti, R. 2024. Counterfactual explanations and how to find them: literature review and benchmarking. Data Min. Knowl. Discov., 38: 2770--2824
2024
-
[15]
H.; and Yadav, A
Guo, H.; Nguyen, T. H.; and Yadav, A. 2021. CounterNet: End-to-End Training of Prediction Aware Counterfactual Explanations. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
2021
-
[16]
Guyomard, V.; Fessant, F.; Guyet, T.; Bouadi, T.; and Termier, A. 2022. VCNet: A self-explaining model for realistic counterfactual generation. In ECML/PKDD
2022
-
[17]
Hamman, F.; Noorani, E.; Mishra, S.; Magazzeni, D.; and Dutta, S. 2023. Robust Counterfactual Explanations for Neural Networks With Probabilistic Guarantees. In International Conference on Machine Learning
2023
-
[18]
G.; Won, D.-O.; and Lee, S.-W
Kang, S.-H.; Jung, H. G.; Won, D.-O.; and Lee, S.-W. 2020. Counterfactual Explanation Based on Gradual Construction for Deep Networks. Pattern Recognit., 132: 108958
2020
-
[19]
Karimi, A.-H.; Barthe, G.; Balle, B.; and Valera, I. 2019. Model-Agnostic Counterfactual Explanations for Consequential Decisions. ArXiv, abs/1905.11190
2019 arXiv
-
[20]
Karimi, A.-H.; Scholkopf, B.; and Valera, I. 2020. Algorithmic Recourse: from Counterfactual Explanations to Interventions. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
2020
-
[21]
V.; and Klaise, J
Looveren, A. V.; and Klaise, J. 2019. Interpretable Counterfactual Explanations Guided by Prototypes. ArXiv, abs/1907.02584
2019 arXiv
-
[22]
M.; and Lee, S.-I
Lundberg, S. M.; and Lee, S.-I. 2017. A Unified Approach to Interpreting Model Predictions. In Neural Information Processing Systems
2017
-
[23]
Mahajan, D.; Tan, C.; and Sharma, A. 2019. Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers. ArXiv, abs/1912.03277
2019 arXiv
-
[24]
K.; Sharma, A.; and Tan, C
Mothilal, R. K.; Sharma, A.; and Tan, C. 2019. Explaining machine learning classifiers through diverse counterfactual explanations. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
2019
-
[25]
Nemirovsky, D.; Thiebaut, N.; Xu, Y.; and Gupta, A. 2022. CounteRGAN: Generating counterfactuals for real-time recourse and interpretability using residual GANs. In Conference on Uncertainty in Artificial Intelligence
2022
-
[26]
Panagiotou, E.; Heurich, M.; Landgraf, T.; and Ntoutsi, E. 2024. TABCF: Counterfactual Explanations for Tabular Data Using a Transformer-Based VAE. Proceedings of the 5th ACM International Conference on AI in Finance
2024
-
[27]
Pawelczyk, M.; Broelemann, K.; and Kasneci, G. 2019. Learning Model-Agnostic Counterfactual Explanations for Tabular Data. Proceedings of The Web Conference 2020
2019
-
[28]
K.; and Weinstein, N
Przybylski, A. K.; and Weinstein, N. 2017. A Large-Scale Test of the Goldilocks Hypothesis. Psychological Science, 28(2): 204--215
2017
-
[29]
Redelmeier, A.; Jullum, M.; Aas, K.; and L land, A. 2024. MCCE: Monte Carlo sampling of valid and realistic counterfactual explanations for tabular data. Data Min. Knowl. Discov., 38: 1830--1861
2024
-
[30]
Why Should I Trust You?
Ribeiro, M. T.; Singh, S.; and Guestrin, C. 2016. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
2016
-
[31]
Shao, X.; and Kersting, K. 2022. Gradient-based Counterfactual Explanations using Tractable Probabilistic Models. ArXiv, abs/2205.07774
2022 arXiv
-
[32]
B.; Arefeen, A.; Carpenter, S
Soumma, S. B.; Arefeen, A.; Carpenter, S. M.; Hingle, M.; and Ghasemzadeh, H. 2025. SenseCF: LLM-Prompted Counterfactuals for Intervention and Sensor Data Augmentation
2025
-
[33]
Ustun, B.; Spangher, A.; and Liu, Y. 2018. Actionable Recourse in Linear Classification. Proceedings of the Conference on Fairness, Accountability, and Transparency
2018
-
[34]
M.; Hofmann, D
VanNostrand, P. M.; Hofmann, D. M.; Ma, L.; and Rundensteiner, E. A. 2024. Actionable Recourse for Automated Decisions: Examining the Effects of Counterfactual Explanation Type and Presentation on Lay User Understanding. Proceedings of the 2024 ACM Conference on Fairness, Acco...
2024
-
[35]
Virgolin, M.; and Fracaros, S. 2022. On the robustness of sparse counterfactual explanations to adverse perturbations. Artif. Intell., 316: 103840
2022
-
[36]
D.; and Russell, C
Wachter, S.; Mittelstadt, B. D.; and Russell, C. 2017. Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR. Cybersecurity
2017
-
[37]
J.; Nichols, E
Wild, C. J.; Nichols, E. S.; Battista, M. E.; Stojanoski, B.; and Owen, A. M. 2018. Dissociable effects of self-reported daily sleep duration on high-level cognitive abilities. Sleep, 41
2018
-
[38]
Xiang, X.; and Lenskiy, A. 2022. Realistic Counterfactual Explanations by Learned Relations. ArXiv, abs/2202.07356
2022 arXiv
-
[39]
S.; Chen, J.; and Hu, X
Yang, F.; Alva, S. S.; Chen, J.; and Hu, X. B. 2021. Model-Based Counterfactual Synthesizer for Interpretation. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
2021
-
[40]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
-
[41]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.