{"id":"1b14b513-1ac8-41cd-b0f1-13305e57b744","arxiv_id":"2603.16436","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"DISCOVER turns certified distributional counterfactual explanations into a top-k, OT-guided propose-and-select search that works without predictor gradients on tabular black boxes.","lead":"DISCOVER is a model-agnostic solver that finds population-level counterfactual edits for black-box tabular models by sparsely rewriting the most transport-influential samples. It keeps DCE’s certified optimal-transport objective while replacing gradient descent with budgeted propose-and-select search, so tree ensembles and mixed pipelines can get distribution-level recourse.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Certification is only a post-hoc filter; sparse top-k OT search may never enter the UCL-feasible region when influence is diffuse or SW2 guidance misaligns with black-box outputs.","rationale":"The reader correctly isolates the load-bearing assumption: that sparse OT impact + input-side SW2 geometry is a sufficient search structure for a non-convex mixed-type certified objective. I sharpen that point to the gap between what is proved (row-wise decomposition of Q for fixed plans; monotonicity of propose-and-select for fixed η) and what the claim needs (reliable entry into the joint UCL-feasible region without model gradients). The paper is careful not to claim optimality of the search, and ablations (Fig. 4, Tables 3–4) already show that large k and missing cone guidance destabilize trajectories—supporting rather than contradicting the concern. There is no internal contradiction in the math, and the non-diff empirical wins remain the main contribution if they reproduce. Hence the verdict stays CONDITIONAL, not REJECT: the contribution is accept-shaped if code shows high certified success under the stated budgets, and weaker if certification often fails while OT metrics look good. Agreement with the reader is full on the weakest assumption; the concrete test is the missing success-rate reporting that would settle it.","tokens_in":25515,"tokens_out":771,"duration_ms":7435,"concrete_test":"On COMPAS and HELOC non-diff settings (RF/XGBoost/LightGBM), re-run Alg. 1 for fixed T,M with k∈{3,10,n} and with/without OT cone guidance; report (i) fraction of seeds returning a certified ∥X (not ∅) under the same UCL checks as DCE, and (ii) final OT(x),OT(y) conditional on certification. If certified success falls sharply for small k or without guidance while Q still decreases, the sparse SW2-guided structure is insufficient for the certified claim.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that DISCOVER preserves DCE’s certified OT objective and chance constraints while solving them model-agnostically via sample-wise OT impact scores, top-k budgets, and input-side SW2 cone guidance (Prop. 3.1–3.2; Alg. 1; Sec. 3.1–3.3). Prop. 3.1 only decomposes Q for fixed transport plans; Prop. 3.2 only guarantees monotone decrease of Q under fixed η when a no-op candidate is kept. Neither result implies that the restricted search over top-k rows, guided only by −∇SW2(X,X′) and never by ∇b, can reach a point that satisfies the joint UCL constraints (Eqs. 2–4 / App. B.2). Certification is applied only after T iterations (Alg. 1 lines 14–18) and may return ∅. If influence is diffuse (many comparable qi) or the black-box map b makes input-side geometry a poor proxy for output alignment, the propose-and-select loop can stall outside the feasible set even while Q decreases. Table 2 reports OT(x)/OT(y) means but not certification success rates or empty-return frequency, so the “preserves certification” claim is only partially evidenced by the main results.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes DISCOVER, a model-agnostic solver for Distributional Counterfactual Explanations (DCE). It keeps DCE’s OT objective Q (input-side sliced Wasserstein proximity plus output-side Wasserstein alignment), the same UCL-based chance constraints, and the same η interval-narrowing rule, but replaces gradient descent with a budgeted propose-and-select search. The method decomposes Q into per-row impact scores qi, edits only a top-k subset of samples, and generates candidates via an OT-guided cone sampling primitive driven by −∇SW2(X,X′) without querying predictor gradients. Modular Monte Carlo and Genetic proposal wrappers are evaluated on several tabular datasets against DiCE, AReS, GLOBE, and (where applicable) DCE, for both differentiable and non-differentiable models, with ablations on k, h, cone guidance, and feature types.","tokens_in":25936,"tokens_out":1151,"duration_ms":9951,"significance":"If the claims hold, DISCOVER is a practically useful contribution: it extends certified distributional counterfactuals to the non-differentiable tabular pipelines that dominate real deployments (trees, mixed discrete-continuous features, preprocessing). The sample-wise OT decomposition, explicit top-k intervention budget, and input-side geometric proposal prior are concrete design ideas, and the paper provides code, multi-model multi-dataset experiments with CIs, and ablations. Propositions 3.1–3.2 are elementary but correctly support the fixed-plan decomposition and monotone propose-and-select step. The main advance is engineering and empirical: making DCE’s certified objective usable without model gradients.","major_comments":[{"comment":"The central claim that DISCOVER “preserves” DCE certification is only partially evidenced. Algorithm 1 applies the UCL feasibility check only after T iterations and may return ∅ (lines 14–18). Propositions 3.1–3.2 only decompose Q for fixed transport plans and guarantee non-increase of Q under fixed η when a no-op candidate is kept; they do not imply that top-k edits guided solely by −∇SW2 can enter the joint UCL-feasible region (Eqs. 2–4 / App. B.2). Table 2 reports OT(x)/OT(y)/MMD means but not certification success rates, empty-return frequency, or UCL gap trajectories. Without those statistics—especially in non-diff settings where DCE cannot run—the “preserves certification” claim remains incomplete.","section":null},{"comment":"The load-bearing sparsity/guidance assumption is under-tested. Section 3.1–3.3 and the weakest-assumption concern both rest on the idea that population shifts are driven by a small top-k set and that input-side SW2 geometry is a useful proposal prior even when ∇b is unavailable. Figure 4 and Table 4 show that large k can destabilize trajectories and that cone guidance helps, but there is no controlled stress test for diffuse influence (many comparable qi) or for cases where SW2 directions systematically misalign with black-box output alignment. A failure-mode analysis or success-rate breakdown under those regimes would make the solver claim much stronger.","section":null},{"comment":"Table 2’s primary metrics are empirical OT distances, not the certified UCL quantities that define DCE feasibility. Several baselines (especially AReS) achieve very small OT(x) with large OT(y); DISCOVER often improves the joint trade-off, but AReS Cost is reported “for reference only” and is sometimes worse for DISCOVER. The paper should either optimize/report a single joint criterion aligned with the chance constraints, or more carefully justify why mean OT(x)/OT(y) alone establish superiority for distributional recourse under the same certification semantics as DCE.","section":null}],"minor_comments":[{"comment":"Code URLs disagree: abstract lists https://github.com/VALHALLA9/Discover while the front matter lists https://github.com/understanding-ml/DCE. Please unify and ensure the linked repo matches the paper name.","section":null},{"comment":"Table 1 formatting is hard to parse (“Instance CE No No Often / Yes No”); clarify which cells apply to which properties.","section":null},{"comment":"Typos and wording: “We focuse primarily”, “sample-wisedecomposition”, “propose-and-s-elect”, and inconsistent repository naming (DISCOVER vs Discover).","section":null},{"comment":"Figure 2–3 are COMPAS+MLP only; a short pointer in the main text to which appendix figures cover non-diff models would help.","section":null},{"comment":"Categorical embedding construction (random fixed Ep, rule for r, temperature τ) is important for reproducibility; App. F helps, but a brief main-text default would reduce ambiguity.","section":null}],"recommendation":"major_revision","confidential_remarks":"Substantial author overlap with the prior DCE paper is clear; novelty is correctly framed as a solver, not a new objective. The manuscript is a solid systems/ML contribution if certification success rates and sparsity stress tests are added. Fit is appropriate for a methods-oriented ML venue; I would not reject for incrementalism alone."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The real news is a usable solver for certified distributional counterfactuals on non-differentiable tabular models. DCE already defined the OT objective and UCL chance constraints; DISCOVER keeps both and replaces gradient descent with top-k OT impact scoring, modular propose-and-select, and input-side SW2 cone sampling. That is the contribution, and it is scoped cleanly.\n\nWhat works: Prop. 3.1–3.2 are elementary but correct for fixed plans and fixed η (decomposition of Q and monotone selection with a no-op). The design matches the sparsity intuition in OT couplings. Experiments cover COMPAS, HELOC, German Credit (plus Cardio/Hotel in ablations), both differentiable and non-diff models (RF, XGBoost, LightGBM), and third-party baselines. Non-diff OT(y) wins look real; ablations on k, h, cone guidance, and feature type are useful; code is linked. Citation pattern is normal for a methods follow-on.\n\nSoft spots, in proportion: the stress-test is partly right. Certification is a post-hoc filter (Alg. 1 returns ∅ if UCLs fail). Props do not guarantee the restricted top-k / SW2-guided search reaches the joint feasible set when influence is diffuse or b misaligns with input geometry. Table 2 reports OT means, not certification success rates or empty-return frequency, so “preserves certification” is only partially evidenced. Baselines (especially GLOBE) are unstable; free knobs (k, h, ϕ, M, T, embeddings) matter. None of that is a load-bearing contradiction—just incomplete evidence on the hard claim.\n\nThis is for people who care about group-level recourse and black-box tabular pipelines. Worth a serious referee. I would engage, re-run the non-diff cases, and ask for cert success rates and sensitivity bounds. Send to peer review.","headline":"Solid black-box solver for DCE’s certified OT objective; novelty is the optimizer, and the main soft spot is incomplete reporting of certification success rates.","tokens_in":26565,"tokens_out":498,"would_cite":true,"duration_ms":5388,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A budgeted, model-agnostic search can solve certified distributional counterfactuals for non-differentiable tabular models by sparsely editing the samples that dominate the transport cost.","keywords":["distributional counterfactual explanations","optimal transport","model-agnostic recourse","sliced Wasserstein","black-box tabular models","top-k intervention budget","chance constraints"],"falsifier":"On a mixed-type tabular task with a non-differentiable predictor, fix the same certified objective and evaluation budget, then compare DISCOVER against a full-population black-box search and a version that disables top-k and cone guidance: if the sparse OT-guided method fails to reach certified feasibility or loses badly on joint OT(x)/OT(y), the claim that sparse input-geometry search is sufficient collapses.","tokens_in":26369,"feed_emoji":"🎯","tokens_out":899,"duration_ms":14409,"temperature":0.7,"pith_summary":"Most counterfactual tools fix one person at a time. Real decisions often need a whole population to move: shift a subgroup’s risk profile while staying on the data manifold. Prior distributional counterfactual explanations cast that goal as an optimal-transport problem with statistical certificates, but they rely on gradients and therefore break on the tree ensembles and mixed pipelines that dominate tabular practice. This paper argues that the same certified objective can be optimized without any model gradient by attributing transport cost to individual rows, editing only a top-k budget of influential samples, and proposing candidates with a cone sampler steered by input-side transport geometry alone. On several tabular datasets the resulting solver jointly aligns input and output distributions, including for non-differentiable predictors where the original gradient method cannot run.","feed_headline":"Sparse OT search explains black-box populations","feed_subtitle":"Edit only the most transport-influential samples to hit certified input–output targets without model gradients.","key_machinery":"Row-wise OT impact scoring under a top-k budget, coupled with OT-guided cone sampling: the certified objective factors as a sum of per-sample contributions, only the highest-impact rows are edited, and candidate moves are drawn in a cone around the negative input-side sliced-Wasserstein guidance field.","core_discovery":"DISCOVER preserves the original distributional counterfactual objective and chance-constraint certification, yet replaces gradient descent with a sparse propose-and-select loop: per-row impact scores from a sample-wise decomposition of the transport costs define a top-k intervention set, modular optimizers generate candidates by editing only those rows, and an input-side OT-guided cone sampler steers proposals without querying predictor gradients. The paper shows this yields strong joint input–output alignment on black-box tabular pipelines.","pith_inferences":["If top-k influence concentrates on protected subgroups, the same machinery could double as a diagnostic for who bears the cost of a population shift.","The propose-and-select loop suggests a natural extension to streaming or federated settings where only partial rows can be edited per round.","When output alignment requires rare categorical flips, hybrid search that mixes OT cones with constraint solvers on discrete variables may be the practical next step."],"forward_implications":["Certified distributional counterfactuals become usable on tree ensembles, rule pipelines, and other non-differentiable tabular systems that previously excluded gradient DCE.","Population recourse can be reported as sparse, interpretable edits on a small active set rather than uniform movement of every sample.","Any modular black-box proposer (Monte Carlo, genetic, or otherwise) can plug into the same certified objective and top-k gate without redesigning certification.","Ablations imply that small intervention budgets and OT cone guidance stabilize search, while large-k or unguided proposals tend to inflate input distortion and variance."],"fun_headline_variants":["Sparse OT search yields certified DCE for black-box tabular models","Top-k transport impact scores enable gradient-free population CEs","Budgeted propose-and-select solves distributional counterfactuals","OT-guided cone sampling steers sparse edits to target distributions","Sample-wise OT decomposition finds black-box distributional CEs"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Population-level shifts are driven by a small set of influential rows, and input-side transport geometry alone is enough to steer useful proposals when the model itself gives no gradients.","fun_headline_variants_meta":{"raw":{"variants":["Sparse OT search yields certified DCE for black-box tabular models","Top-k transport impact scores enable gradient-free population CEs","Budgeted propose-and-select solves distributional counterfactuals","OT-guided cone sampling steers sparse edits to target distributions","Sample-wise OT decomposition finds black-box distributional CEs"]},"model":"grok-4.5","effort":"low","cost_usd":0.006384,"raw_usage":{"total_tokens":1656,"prompt_tokens":792,"num_sources_used":0,"completion_tokens":92,"cost_in_usd_ticks":63840000,"prompt_tokens_details":{"text_tokens":792,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":772,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":792,"tokens_out":92,"duration_ms":8318,"temperature":1.0,"reasoning_tokens":772,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T23:45:46.368105+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a mixed-type tabular task with a non-differentiable predictor, fix the same certified objective and evaluation budget, then compare DISCOVER against a full-population black-box search and a version that disables top-k and cone guidance: if the sparse OT-guided method fails to reach certified feasibility or loses badly on joint OT(x)/OT(y), the claim that sparse input-geometry search is sufficient collapses.","supporting_citations":[],"review_version":1}