REVIEW 4 major objections 6 minor 24 references
Generalizability vs. Counterfactual Explainability Trade-Off
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that the average probability of finding a valid counterfactual within a fixed radius rises as models overfit, so that this probability can serve as a quantitative overfitting proxy.
desk verdict Useful closed-form geometry for a new counterfactual-ease metric, but the experiments measure a different quantity than the theory and never tie the trend to an actual generalization gap. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the ε-valid counterfactual shell $S(x_i,\gamma_i,\varepsilon)=B(x_i,\varepsilon)\setminus B(x_i,\gamma_i)$, the thin layer of points at distance between the margin and ε from $x_i$, which is the only region where valid counterfactuals can appear. The argument runs through the ratio of two volumes: the spherical cap of directions that cross the decision boundary divided by the shell volume, expressed in Theorem 4.2 as a regularized incomplete $\beta$ function of $1-(\gamma_i/\varepsilon)^2$. That formula converts boundary geometry into probability, and Jensen's inequality transfers monotonicity from individual margins to the average margin, which is the step that turns geometry into an overfitting indicator.
What would settle it
Train a model with a held-out test set and record the average ε-VCP alongside the train–test accuracy gap at regular checkpoints: if the average ε-VCP rises while the test gap stays flat or shrinks, the proxy is not tracking overfitting. The paper's plots show training accuracy and margins but not test accuracy, so this measurement is the decisive missing check.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that ε-VCP is controlled by the distance from a point to the decision boundary. For a linear classifier with uniform perturbations, a point at margin $\gamma_i$ has $\varepsilon$-VCP $p_i^\varepsilon = \frac{1}{2}\frac{\varepsilon^n}{\varepsilon^n-\gamma_i^n} I\left(1-(\gamma_i/\varepsilon)^2; \frac{n+1}{2},\frac{1}{2}\right)$, where $I$ is the regularized incomplete $\beta$ function; this is the volume fraction of the $\varepsilon$-shell that lies across the separating hyperplane. The probability tends to $1/2$ as the margin goes to zero and vanishes as the shell collapses. Averaging over training points and applying Jensen's inequality gives a lower bound in which the average ε-VCP is a monotonically decreasing function of the average margin, and the paper cites margin theory to equate shrinking average margins with overfitting. For nonlinear boundaries, a tangent-hyperplane approximation yields an asymptotic form proportional to $\gamma_i^{(n-1)/2}(\varepsilon-\gamma_i)^{(n-1)/2}$. The empirical sections record average ε-VCP rising with training epochs and accuracy, and lower values for dropout-regularized networks.
Load-bearing premise
The overfitting conclusion rests on the imported margin-theory claim that overfitting shrinks average training margins, and on treating the empirical fraction of the ε-ball that crosses the boundary as a faithful estimate of the shell-based ε-VCP; if either fails, a rising average ε-VCP may only track training accuracy.
Editorial extensions
If this is right
- During training, the average ε-VCP can be monitored as an overfitting signal without waiting for a held-out test set.
- Regularization that smooths the decision boundary, such as dropout, measurably lowers the average ε-VCP, so the measure tracks boundary complexity rather than raw accuracy.
- For linear classifiers the closed-form identity makes the proxy a direct margin statistic, computable without Monte Carlo sampling.
- The trade-off implies that easy counterfactual generation is not an unmixed good: it can signal that a model has wrapped itself tightly around the training data.
Reading between the lines
- Beyond the paper, any training intervention known to increase average margins—weight decay, early stopping, adversarial training—should also suppress average ε-VCP; this prediction is testable and would separate margin effects from raw accuracy effects.
- On data lying on a low-dimensional manifold, the same shell-volume argument should hold with intrinsic instead of Euclidean volumes, suggesting a manifold-aware version of the proxy.
- Applied to test or out-of-distribution points, high ε-VCP could double as a local predictive-instability or uncertainty score, since boundary proximity is a standard correlate of low confidence.
- If ε-VCP is to become a general overfitting proxy, the missing direct measurement for the paper's own models is the train–test accuracy gap; a study that tracks ε-VCP alongside test error would establish or refute the proxy on its own terms.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the ε-valid counterfactual probability (ε-VCP), defined as the probability that a random perturbation drawn from a distribution supported on the shell S(x_i, γ_i, ε) between the geometric margin γ_i and radius ε flips the model's prediction. For linear classifiers under uniform shell perturbations, Theorem 4.2 derives a closed-form expression for p_i^ε as a decreasing function of γ_i. Theorem 4.3 offers a local asymptotic approximation for nonlinear models. The paper then connects average ε-VCP to overfitting through margin theory, claiming that overfitting reduces average margins and therefore increases average ε-VCP, and presents experiments on two tabular datasets with logistic regression and MLPs that plot average ε-VCP against training accuracy and epochs. The central assertion is that average ε-VCP can serve as a quantitative proxy for model overfitting.
Significance. If the central claim were fully established, the paper would make a novel connection between counterfactual explainability and generalization, providing a concrete geometric quantity with a clean closed form for linear classifiers and an open-source implementation. The derivation of Theorem 4.2 is a correct geometric calculation for the shell-conditional probability, and the paper is commendable for making the dependence of counterfactual validity on margin quantitative. However, the empirical validation does not measure the derived quantity and does not tie the observed trends to an actual generalization drop, so the significance is currently prospective rather than demonstrated.
major comments (4)
- [§5.1 vs §4.1/§4.3.1] The empirical estimator described in Section 5.1 computes the fraction of the full n-ball B(x_i, ε) that crosses the decision boundary, whereas Definition 4.1 and Theorem 4.2 define p_i^ε as a probability over the shell S(x_i, γ_i, ε). For a linear classifier, these two quantities differ by the factor ε^n/(ε^n - γ_i^n), which diverges as γ_i approaches ε. Consequently, Figures 3 and 4 do not validate the closed-form formula of Theorem 4.2, and the paper's empirical support for the proxy claim rests on a different quantity than the one analyzed theoretically. Please reconcile the estimator with the definition, for example by sampling only from the shell or by redefining ε-VCP and rederiving the theorem accordingly.
- [§4.4 and §5.2] The central claim that average ε-VCP is a quantitative indicator of model overfitting is not supported by the reported experiments because no test accuracy or train-test generalization gap is measured. Figures 3 and 4 only show that ε-VCP rises with training accuracy; without a measured overfitting signal, the trend cannot be attributed to overfitting rather than to ordinary improvement in data fitting. The paper should either report test performance for the same checkpoints/model pairs or explicitly limit the empirical claim to a margin-based trend rather than an overfitting proxy.
- [Theorem 4.3 and Appendix A.1] The exponent of γ_i in Theorem 4.3 appears to have the wrong sign. Expanding the exact expression from Theorem 4.2 along the lines of Appendix A.1 yields p_i^ε ~ const · γ_i^{-(n-1)/2} δ^{(n-1)/2}, not γ_i^{(n-1)/2} δ^{(n-1)/2} as stated. The stated form also gives p_i^ε dimensions of length^{n-1}, which is inconsistent for a probability. Please correct Theorem 4.3 and its proof, and check the implications for the claimed limiting behavior.
- [Appendix A.2] The proof of monotonicity of g(γ̄) is not rigorous: it only examines endpoint limits, it uses the derivative dx/dγ̄ = -1/ε even though x = 1 - (γ̄/ε)^2 gives dx/dγ̄ = -2γ̄/ε^2, and it asserts without proof that the negative term dominates throughout the interior of the interval. Since Jensen's inequality in Equation (13) and the subsequent overfitting argument depend on the monotonicity direction of g, a complete proof or a precise reference is needed.
minor comments (6)
- [§5.1] The text refers to 'Appendix ??' when discussing the choice of ε and to 'Appendix ??' for full experimental details; these cross-references should be filled in.
- [Appendix A.2] There is a typo: 'decreasining' should be 'decreasing'.
- [Theorem 4.3 proof sketch] The phrase 'the the tangent hyperplane' contains a duplicated article and should be corrected.
- [Table 1] The columns for 'Exp. Water' and 'Exp. Air Quality' both list 'Fig. 4a'; the second should presumably be 'Fig. 4b'.
- [Figure 4 caption] The terms 'Plain ε-VCP' and 'Regularized ε-VCP' are used in the figure but not defined in the text; please clarify which model configuration each corresponds to.
- [§4.2] The description of X_i as indicating 'whether there exists an ε-valid counterfactual example' conflates deterministic existence with the probability of finding one under random perturbation; the Bernoulli framing would be clearer if it referred to the event that a random perturbation drawn from φ yields a label change.
Circularity Check
No significant circularity: the epsilon-VCP formula is derived from its defining geometric integral, and the overfitting link is imported from external margin-theory results rather than from self-citation or fitted inputs.
full rationale
The geometric derivation is self-contained: Definition 4.1 defines the epsilon-VCP as the integral of a perturbation density over the counterfactual shell S(x_i, gamma_i, eps); Lemma 3.1 derives that shell from the margin gamma_i; Theorem 4.2 evaluates the integral for uniform perturbations using a standard spherical-cap volume formula; and Appendix A.2 establishes monotonicity of g in the average margin by differentiation. The bridge from small margins to overfitting is explicitly external: 'Relevant literature on margin theory has shown that overfitting generally results in reduced margins around training samples on average (e.g., see Mason et al. [1998], Neyshabur et al. [2017], Wu and Yu [2019], Shamir et al. [2021])'. These are independent citations, not self-citations. The paper's own prior work appears only in related-work context and is not load-bearing for the central theorem. A non-circular validation gap exists: Section 5.1 estimates p_i^eps as 'the fraction of the n-ball B(x_i, eps) that crosses the decision boundary', which is not the shell-conditional probability of Definition 4.1; however, this is a measurement mismatch, not a circular reduction, because the empirical quantity is still a decreasing function of the margin and the qualitative trend does not require the eps^n/(eps^n - gamma_i^n) prefactor. The paper also does not measure test accuracy, so the overfitting-proxy claim depends on the external margin-theory premise, but it does not reduce to the paper's own construction by definition.
Assumptions & free parameters
free parameters (2)
- epsilon (perturbation radius) =
35.00 (Water LogReg), 1.500 (Water MLP), 0.373 (Air Quality MLP)
- Polynomial expansion degree d =
6
assumptions (4)
- domain assumption Trained classifiers have a well-defined geometric margin gamma_i for each training input, meaning the infimum distance to the decision boundary is attained.
- ad hoc to paper The decision boundary of a non-linear classifier can be locally approximated by its tangent hyperplane in a neighborhood of the closest boundary point, where the perturbation density is constant.
- domain assumption Overfitting reduces the average geometric margin of training points, as established in cited margin-theory work.
- ad hoc to paper The function g(avg margin) is monotonically decreasing on (0, epsilon).
invented entities (1)
-
epsilon-valid counterfactual probability (epsilon-VCP)
Cite this review
Pith. "Pith review of Generalizability vs. Counterfactual Explainability Trade-Off." pith.science (2026). https://pith.science/paper/SBODGIPD
@misc{pith2026250523225,
author = {Pith},
title = {Pith review of: Generalizability vs. Counterfactual Explainability Trade-Off},
year = {2026},
howpublished = {\url{https://pith.science/paper/SBODGIPD}},
note = {Machine review of arXiv:2505.23225}
}
abstract
In this work, we investigate the relationship between model generalization and counterfactual explainability in supervised learning. We introduce the notion of $\varepsilon$-valid counterfactual probability ($\varepsilon$-VCP) -- the probability of finding perturbations of a data point within its $\varepsilon$-neighborhood that result in a label change. We provide a theoretical analysis of $\varepsilon$-VCP in relation to the geometry of the model's decision boundary, showing that $\varepsilon$-VCP tends to increase with model overfitting. Our findings establish a rigorous connection between poor generalization and the ease of counterfactual generation, revealing an inherent trade-off between generalization and counterfactual explainability. Empirical results validate our theory, suggesting $\varepsilon$-VCP as a practical proxy for quantitatively characterizing overfitting.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Unrestricted Adversarial Examples
Tom B Brown, Nicholas Carlini, Chiyuan Zhang, Catherine Olsson, Paul Christiano, and Ian Good- fellow. Unrestricted Adversarial Examples. arXiv preprint arXiv:1809.08352,
-
[4]
doi: 10.1007/ s11023-021-09580-9
ISSN 0924-6495. doi: 10.1007/ s11023-021-09580-9. URL https://doi.org/10.1007/s11023-021-09580-9 . Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-Adversarial Training of Neural Networks. Journal of Machine Learning Research, 17(1):2096–2030, jan
-
[9]
Thai Le, Suhang Wang, and Dongwon Lee
doi: 10.1109/72.870038. Thai Le, Suhang Wang, and Dongwon Lee. GRACE: Generating Concise and Informative Contrastive Sample to Explain Neural Network Model’s Prediction. KDD ’20, pages 238–248, New York, NY , USA,
-
[10]
Association for Computing Machinery. ISBN 9781450379984. doi: 10.1145/3394486. 3403066. URL https://doi.org/10.1145/3394486.3403066. Shengqiao Li. Concise Formulas for the Area and V olume of a Hyperspherical Cap.Asian Journal of Mathematics & Statistics, 4(1):66–70,
-
[11]
Martin Pawelczyk, Kai Broelemann, and Gjergji Kasneci
URL https://papers.nips.cc/paper/ 7176-exploring-generalization-in-deep-learning . Martin Pawelczyk, Kai Broelemann, and Gjergji Kasneci. Learning Model-Agnostic Counterfactual Explanations for Tabular Data. In Proceedings of The Web Conference 2020 (WWW ’20), pages 3126–3132, New York, NY , USA,
work page 2020
-
[12]
ACM. doi: 10.1145/3366423.3380087. URL https: //doi.org/10.1145/3366423.3380087. Garvesh Raskutti, Martin J. Wainwright, and Bin Yu. Early Stopping for Non-Parametric Re- gression: An Optimal Data-Dependent Stopping Rule. In 2011 49th Annual Allerton Confer- ence on Communication, Control, and Computing (Allerton) , pages 1318–1325,
-
[14]
Adi Shamir, Odelia Melamed, and Oriel BenShmuel
URL https://proceedings.neurips.cc/paper/2020/hash/ 0e445eaaec2b2e93b1d9f9f95fb4ab2f-Abstract.html. Adi Shamir, Odelia Melamed, and Oriel BenShmuel. The Dimpled Manifold Model of Adversarial Examples in Machine Learning. arXiv preprint arXiv:2106.10151,
arXiv 2020
-
[15]
doi: 10.1109/CVPR.2019.00714. URL https://openaccess.thecvf.com/content_CVPR_2019/html/Stutz_Disentangling_ Adversarial_Robustness_and_Generalization_CVPR_2019_paper.html. Yi Sun, Xiaogang Wang, and Xiaoou Tang. Deep Learning Face Representation from Predicting 10,000 Classes. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 1891–1898,
arXiv 2019
Show all 24 references
-
[18]
ISBN 9781450348874
Association for Computing Machinery. ISBN 9781450348874. doi: 10.1145/3097983.3098039. URL https://doi.org/10.1145/3097983.3098039. Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural ne...
-
[21]
Goodfellow, Aaron C
David Warde-Farley, Ian J. Goodfellow, Aaron C. Courville, and Yoshua Bengio. An Empirical Analysis of Dropout in Piecewise Linear Networks. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April ...
2014
-
[22]
Kaiwen Wu and Yaoliang Yu
URL http://arxiv.org/abs/1312.6197. Kaiwen Wu and Yaoliang Yu. Understanding Adversarial Robustness: The Trade-off between Minimum and Average Margin,
-
[23]
URL https://arxiv.org/abs/1907.11780. Kevin Y . Yip and Mark Gerstein. Training Set Expansion: An Approach to Improving the Re- construction of Biological Networks from Limited and Uneven Reliable Interactions. Bioinfor- matics, 25(2):243–250, 11
1907 arXiv
-
[24]
doi: 10.1093/bioinformatics/btn602
ISSN 1367-4803. doi: 10.1093/bioinformatics/btn602. URL https://doi.org/10.1093/bioinformatics/btn602. 12 A Technical Appendices and Supplementary Material A.1 Full Proof of Theorem 4.3 Let fθ be a classifier and xi ∈ Xa generic input. Let ˆHθ be the hyperplane tangent that lo...
-
[2000]
Ziheng Chen, Fabrizio Silvestri, Jia Wang, He Zhu, Hongshik Ahn, and Gabriele Tolomei
URL https://proceedings.neurips.cc/paper_files/paper/2000/file/ 059fdcd96baeb75112f09fa1dcc740cc-Paper.pdf. Ziheng Chen, Fabrizio Silvestri, Jia Wang, He Zhu, Hongshik Ahn, and Gabriele Tolomei. Re- LAX: Reinforcement Learning Agent Explainer for Arbitrary Predictive Models. I...
2000
-
[2008]
Sandra Wachter, Brent Mittelstadt, and Chris Russell
DOI: https://doi.org/10.24432/C59K5F. Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology., 31: 841–887,
-
[2011]
Shalmali Rawal, Cynthia Chen, and Himabindu Lakkaraju
doi: 10.1109/Allerton.2011.6120320. Shalmali Rawal, Cynthia Chen, and Himabindu Lakkaraju. Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable Recourse. In Advances in Neural Information Processing Systems (NeurIPS) , volume 33, pages 12275–12287,
2011
-
[2014]
11 Gabriele Tolomei and Fabrizio Silvestri
doi: 10.1109/CVPR.2014.244. 11 Gabriele Tolomei and Fabrizio Silvestri. Generating Actionable Interpretations from Ensembles of Decision Trees. IEEE Transactions on Knowledge and Data Engineering, 33(4):1540–1553,
2014 doi
-
[2016]
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti
ISSN 1532-4435. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti. Local Rule-Based Explanations of Black Box Decision Systems. arXiv preprint arXiv:1805.10820,
-
[2017]
URL https://proceedings.neurips.cc/paper_files/paper/2017/ file/40e0c6b78bb2f8fa5e2436d5f8b5e8c0-Paper.pdf. R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. Learning Deep Representations by Mutual Informat...
2017
-
[2018]
Saverio Vito
URL https://proceedings.neurips.cc/paper_files/paper/ 2018/file/0f7e33d3bd47d94b3159c546f5b1e47e-Paper.pdf. Saverio Vito. Air Quality. UCI Machine Learning Repository,
2018
-
[2019]
ISBN 9781450376280
Association for Computing Machinery. ISBN 9781450376280. doi: 10.1145/3359789.3359824. URL https://doi.org/10.1145/3359789. 3359824. Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classi- fier against adversarial manipulation. In Advances in ...
-
[2020]
Accessed: 2025-01-27
URL https://www.kaggle.com/datasets/ adityakadiwal/water-potability. Accessed: 2025-01-27. Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. Model-Agnostic Counterfactual Explanations for Consequential Decisions. In International Conference on Artificial Inte...
2025
-
[2021]
Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas
doi: 10.1109/TKDE.2019.2945326. Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking. KDD ’17, pages 465–474, New York, NY , USA,
2019
-
[2022]
ISBN 9781450392365
Association for Computing Machinery. ISBN 9781450392365. doi: 10.1145/3511808.3557429. URL https://doi.org/10.1145/3511808. 3557429. Timo Freiesleben. The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples. Minds and Machines , 32(1):77–109, mar
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.