REVIEW 3 major objections 5 minor 1 cited by
DISCOVER: A Solver for Distributional Counterfactual Explanations
T0 review · 3 major / 5 minor · reviewed 2026-07-13 · grok-4.5
Pith's one-line read A budgeted, model-agnostic search can solve certified distributional counterfactuals for non-differentiable tabular models by sparsely editing the samples that dominate the transport cost.
desk verdict Solid black-box solver for DCE’s certified OT objective; novelty is the optimizer, and the main soft spot is incomplete reporting of certification success rates. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Row-wise OT impact scoring under a top-k budget, coupled with OT-guided cone sampling: the certified objective factors as a sum of per-sample contributions, only the highest-impact rows are edited, and candidate moves are drawn in a cone around the negative input-side sliced-Wasserstein guidance field.
What would settle it
On a mixed-type tabular task with a non-differentiable predictor, fix the same certified objective and evaluation budget, then compare DISCOVER against a full-population black-box search and a version that disables top-k and cone guidance: if the sparse OT-guided method fails to reach certified feasibility or loses badly on joint OT(x)/OT(y), the claim that sparse input-geometry search is sufficient collapses.
Extended reading notes
Core claim
DISCOVER preserves the original distributional counterfactual objective and chance-constraint certification, yet replaces gradient descent with a sparse propose-and-select loop: per-row impact scores from a sample-wise decomposition of the transport costs define a top-k intervention set, modular optimizers generate candidates by editing only those rows, and an input-side OT-guided cone sampler steers proposals without querying predictor gradients. The paper shows this yields strong joint input–output alignment on black-box tabular pipelines.
Load-bearing premise
Population-level shifts are driven by a small set of influential rows, and input-side transport geometry alone is enough to steer useful proposals when the model itself gives no gradients.
Editorial extensions
If this is right
- Certified distributional counterfactuals become usable on tree ensembles, rule pipelines, and other non-differentiable tabular systems that previously excluded gradient DCE.
- Population recourse can be reported as sparse, interpretable edits on a small active set rather than uniform movement of every sample.
- Any modular black-box proposer (Monte Carlo, genetic, or otherwise) can plug into the same certified objective and top-k gate without redesigning certification.
- Ablations imply that small intervention budgets and OT cone guidance stabilize search, while large-k or unguided proposals tend to inflate input distortion and variance.
Reading between the lines
- If top-k influence concentrates on protected subgroups, the same machinery could double as a diagnostic for who bears the cost of a population shift.
- The propose-and-select loop suggests a natural extension to streaming or federated settings where only partial rows can be edited per round.
- When output alignment requires rare categorical flips, hybrid search that mixes OT cones with constraint solvers on discrete variables may be the practical next step.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DISCOVER, a model-agnostic solver for Distributional Counterfactual Explanations (DCE). It keeps DCE’s OT objective Q (input-side sliced Wasserstein proximity plus output-side Wasserstein alignment), the same UCL-based chance constraints, and the same η interval-narrowing rule, but replaces gradient descent with a budgeted propose-and-select search. The method decomposes Q into per-row impact scores qi, edits only a top-k subset of samples, and generates candidates via an OT-guided cone sampling primitive driven by −∇SW2(X,X′) without querying predictor gradients. Modular Monte Carlo and Genetic proposal wrappers are evaluated on several tabular datasets against DiCE, AReS, GLOBE, and (where applicable) DCE, for both differentiable and non-differentiable models, with ablations on k, h, cone guidance, and feature types.
Significance. If the claims hold, DISCOVER is a practically useful contribution: it extends certified distributional counterfactuals to the non-differentiable tabular pipelines that dominate real deployments (trees, mixed discrete-continuous features, preprocessing). The sample-wise OT decomposition, explicit top-k intervention budget, and input-side geometric proposal prior are concrete design ideas, and the paper provides code, multi-model multi-dataset experiments with CIs, and ablations. Propositions 3.1–3.2 are elementary but correctly support the fixed-plan decomposition and monotone propose-and-select step. The main advance is engineering and empirical: making DCE’s certified objective usable without model gradients.
major comments (3)
- The central claim that DISCOVER “preserves” DCE certification is only partially evidenced. Algorithm 1 applies the UCL feasibility check only after T iterations and may return ∅ (lines 14–18). Propositions 3.1–3.2 only decompose Q for fixed transport plans and guarantee non-increase of Q under fixed η when a no-op candidate is kept; they do not imply that top-k edits guided solely by −∇SW2 can enter the joint UCL-feasible region (Eqs. 2–4 / App. B.2). Table 2 reports OT(x)/OT(y)/MMD means but not certification success rates, empty-return frequency, or UCL gap trajectories. Without those statistics—especially in non-diff settings where DCE cannot run—the “preserves certification” claim remains incomplete.
- The load-bearing sparsity/guidance assumption is under-tested. Section 3.1–3.3 and the weakest-assumption concern both rest on the idea that population shifts are driven by a small top-k set and that input-side SW2 geometry is a useful proposal prior even when ∇b is unavailable. Figure 4 and Table 4 show that large k can destabilize trajectories and that cone guidance helps, but there is no controlled stress test for diffuse influence (many comparable qi) or for cases where SW2 directions systematically misalign with black-box output alignment. A failure-mode analysis or success-rate breakdown under those regimes would make the solver claim much stronger.
- Table 2’s primary metrics are empirical OT distances, not the certified UCL quantities that define DCE feasibility. Several baselines (especially AReS) achieve very small OT(x) with large OT(y); DISCOVER often improves the joint trade-off, but AReS Cost is reported “for reference only” and is sometimes worse for DISCOVER. The paper should either optimize/report a single joint criterion aligned with the chance constraints, or more carefully justify why mean OT(x)/OT(y) alone establish superiority for distributional recourse under the same certification semantics as DCE.
minor comments (5)
- Code URLs disagree: abstract lists https://github.com/VALHALLA9/Discover while the front matter lists https://github.com/understanding-ml/DCE. Please unify and ensure the linked repo matches the paper name.
- Table 1 formatting is hard to parse (“Instance CE No No Often / Yes No”); clarify which cells apply to which properties.
- Typos and wording: “We focuse primarily”, “sample-wisedecomposition”, “propose-and-s-elect”, and inconsistent repository naming (DISCOVER vs Discover).
- Figure 2–3 are COMPAS+MLP only; a short pointer in the main text to which appendix figures cover non-diff models would help.
- Categorical embedding construction (random fixed Ep, rule for r, temperature τ) is important for reproducibility; App. F helps, but a brief main-text default would reduce ambiguity.
Circularity Check
No tautological derivation: DCE objective is inherited with author overlap, but solver claims rest on independent algorithmic design and external benchmarks.
-
self citation load bearing
[Sec. 2.1–2.2, App. B; abstract / Sec. 3.1 claim to preserve DCE]
"DISCOVER preserves the original DCE objective and certification while replacing gradient descent with a budgeted propose-and-select search paradigm. ... Therefore, DISCOVER changes neither the distributional objective nor the certification semantics."
The certified OT objective, UCL chance constraints, and η interval-narrowing that DISCOVER “preserves” are taken from You et al. (2025), with overlapping authors (You, Cao, Zhao, Lei). That self-citation is load-bearing for the problem statement and certification layer, but it does not force the paper’s solver performance claims: those are tested on external data and third-party baselines. Mild inheritance, not a closed derivation loop.
full rationale
DISCOVER’s load-bearing technical content is a new black-box propose-and-select optimizer (top-k OT impact scoring, modular candidates, OT-guided cone sampling), not a re-derivation of a fitted law. Proposition 3.1 is an algebraic identity obtained by reordering sums in the definitions of Qx and Qy; Proposition 3.2 is monotonicity of argmin when a no-op candidate is included. Both are correctly presented as design properties, not empirical “predictions.” The paper fully restates the DCE chance constraints, UCL certificates, and η interval-narrowing (Appendix B) and evaluates OT(x)/OT(y) against third-party baselines (DiCE, AReS, GLOBE) on public tabular datasets, including non-differentiable models where DCE cannot run. The only mild circularity-adjacent element is transparent inheritance of the certified OT objective from You et al. (2025), which shares authors; that citation supplies the problem formulation, not a uniqueness theorem that forces the solver results. There is no fitted free parameter renamed as a prediction, no self-definitional ratio, and no claim that certification success is guaranteed by construction of the search. Overall circularity is negligible.
Assumptions & free parameters
free parameters (6)
- top-k intervention budget k
- within-row feature budget h
- cone half-angle ϕ and step bound λmax
- candidates per iteration M and max iterations T
- categorical embedding dim r and temperature τ
- trade-off η via interval narrowing (κ, [l,r]) and UCL bounds Ux,Uy,α,δ
assumptions (5)
- domain assumption Sliced Wasserstein on inputs and 1D Wasserstein on outputs are appropriate distributional proximity/alignment metrics for empirical tabular CE.
- domain assumption DCE upper confidence limits and chance constraints correctly certify finite-sample feasibility of SW2 and W2 bounds.
- standard math For fixed OT plans, the certified objective exactly decomposes as sum of per-row impacts qi.
- standard math Including a no-op candidate makes propose-and-select monotone non-increasing in Q for fixed η.
- ad hoc to paper Input-side OT geometry (−∇SW2) is a useful proposal prior even when the predictor is non-differentiable and output-side gradients are unavailable.
invented entities (3)
-
Per-row OT impact score qi and top-k editable set
-
OT-guided cone sampling primitive (continuous + categorical embedding decode)
-
Modular propose-and-select distributional solver (Monte Carlo / Genetic wrappers)
Cite this review
Pith. "Pith review of DISCOVER: A Solver for Distributional Counterfactual Explanations." pith.science (2026). https://pith.science/paper/YI3KR57X
@misc{pith2026260316436,
author = {Pith},
title = {Pith review of: DISCOVER: A Solver for Distributional Counterfactual Explanations},
year = {2026},
howpublished = {\url{https://pith.science/paper/YI3KR57X}},
note = {Machine review of arXiv:2603.16436}
}
read the original abstract
Counterfactual explanations (CE) explain model decisions by identifying input modifications that lead to different predictions. Most existing methods operate at the instance level. Distributional Counterfactual Explanations (DCE) extend this setting by optimizing an optimal transport objective that balances proximity to a factual input distribution and alignment to a target output distribution, with statistical certification via chance constrained bounds. However, DCE relies on gradient based optimization, while many real-world tabular pipelines are dominated by non-differentiable models. We propose DISCOVER, a model-agnostic solver for distributional counterfactual explanations. DISCOVER preserves the original DCE objective and certification while replacing gradient descent with a budgeted propose-and-select search paradigm. It exploits a sample-wise decomposition of the transport objective to compute per-row impact scores and enforce a top-k intervention budget, focusing edits on the most influential samples. To guide candidate generation without predictor gradients, DISCOVER introduces an OT-guided cone sampling primitive driven by input-side transport geometry. Experiments on multiple tabular datasets demonstrate strong joint alignment of input and output distributions, extending distributional counterfactual reasoning to modern black box learning pipelines. A code repository is available at: https://github.com/VALHALLA9/Discover.
Forward citations
Cited by 1 Pith paper
-
Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responses
DECAF decomposes paired perturbation responses into endpoint-aligned evidence, endpoint-opposed contradiction, and endpoint-null fragility, and its components track independently measured model behavior better than ra...
Reviewed July 13, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.