{"id":"8ae84104-ce13-4ed5-84b4-f5b51eb82a82","arxiv_id":"2507.02169","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A sampling-based procedure estimates and statistically tests how often a model's prediction changes under realistic user-specified interventions, with exact binomial guarantees.","lead":"This paper introduces a procedure to estimate how often a machine learning model's prediction would change if a person changed their own input features in realistic ways, using only black-box query access. The method turns that estimate into statistical tests with formal error bounds, which practitioners could use to audit models for failures like locked-in loan denials or gameable moderation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 as printed does not sample from P^reach_x: feasibility is checked on a before r is sampled and never re-checked on x+a+r, so Propositions 3/5 do not apply to its output.","rationale":"The central statistical inference (Clopper-Pearson coverage and one-sided type-I control) is correct conditional on having i.i.d. samples from P^reach_x. The load-bearing problem is that the published Algorithm 1 does not, as written, produce such samples when constraints couple actionable and downstream features. The feasibility check is applied to a before r is sampled, and no check is applied to the final x+a+r. This can reject valid interventions or accept invalid points, so the binomial assumption underlying Propositions 3 and 5 is not guaranteed for the algorithm as stated. The Appendix's SampleEffect description indicates the actual implementation checks a+r, which would restore correctness, but this creates an inconsistency between the claimed guarantee ('sampled per Algorithm 1') and the pseudocode. This concern reinforces the reader's CONDITIONAL verdict: the paper should be accepted only after the algorithm is corrected and the guarantee is restated for the corrected procedure. A secondary numerical issue is that the N_min in Example 6 (254) is too low for the claimed 80% power at α=0.01, ε=0.1, Δ=0.05; solving the paper's own formula gives roughly 310, so the example should be recomputed. The reader's weakest assumption about faithful intervention specification is real but orthogonal; the issue here is internal to the sampling procedure.","tokens_in":32387,"tokens_out":27815,"duration_ms":295814,"concrete_test":"Implement Algorithm 1 exactly as printed on a toy model with d=2, x=(0,1), A(x): a1∈{0,1}, a2=0, downstream r2=−a1 (deterministic), so the true reachable set is {(0,1),(1,0)}. Compare the empirical distribution of 10,000 accepted samples against Uniform(A)⊗P_{x,a}; if a1=1 is never accepted or invalid x' are emitted, the published algorithm does not sample P^reach_x, and the Prop 3/5 guarantees do not apply to it.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Propositions 3 and 5 require that the n points in X̂ are i.i.d. from P^reach_x. Algorithm 1 does not guarantee this. Line 4 calls CheckFeasibility(x, a, C_S) on the intervention vector a before downstream effects r are sampled (Line 5), and Line 6 adds x+a+r without a final feasibility check. For any constraint that couples an actionable feature to a downstream feature—e.g., the Directional Linkage equality in Eq. (3)–(4) or the Missing Values row of Table 1—the check on a alone is wrong: downstream components are fixed at a_k=0, so a valid intervention requiring a compensating downstream change is rejected (e.g., setting no_posts=TRUE forces num_posts=0, but with a_num_posts=0 the constraint x_j·x_k=0 fails). Conversely, when no coupling is present, the missing final check can accept x+a+r outside X. Thus the accepted samples are not distributed as a∼Uniform[A(x)], r∼P_{x,a}; they are a different, constraint-distorted distribution. Consequently, S_n is not necessarily Bin(n, ρ(x)), and the coverage/type-I guarantees in Propositions 3 and 5 do not apply to samples produced by the algorithm as printed. The Appendix (A.1, SampleEffect) states that the implementation instead checks feasibility on a+r, which would fix the distributional claim, but the main-text Algorithm 1 and its stated guarantees are inconsistent with that implementation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a statistical framework for auditing the responsiveness of black-box model predictions to feature interventions. Responsiveness is defined as the probability that a point sampled from a user-specified reachable distribution P^reach_x, obtained by sampling an intervention uniformly from A(x) and then drawing downstream effects from P_{x,a}, falls into a target prediction set. The authors propose estimating this quantity from i.i.d. samples via the Clopper-Pearson interval (Proposition 3) and a one-sided hypothesis test for whether responsiveness exceeds a threshold (Proposition 5), with sample-size rules. They present Algorithm 1 for generating the reachable samples, validate the estimation and testing guarantees against full enumeration on the german dataset, and demonstrate the procedure on recidivism prediction, liver transplant scoring, and content moderation.","tokens_in":32617,"tokens_out":6180,"duration_ms":64467,"significance":"The statistical core of the paper is sound and self-contained: given genuinely i.i.d. samples from P^reach_x, the Clopper-Pearson interval and the one-sided test provide exactly the stated coverage and type-I error bounds, and the validation study against full enumeration on the german dataset is a genuine external check. If the sampling algorithm actually produced such samples, the framework would be a practical, model-agnostic tool for preclusion, gaming, and monotonicity verification. The main weaknesses are internal inconsistencies between the printed Algorithm 1 and the distributional assumption, and a use case that violates the paper's own sample-size condition. Both are fixable without changing the statistical theory.","major_comments":[{"comment":"Algorithm 1 as printed does not sample from P^reach_x. In Line 4, CheckFeasibility(x, a, C_S) is applied to the intervention vector a before downstream effects r are sampled in Line 5, and Line 6 adds x+a+r without any subsequent feasibility check. For constraints that couple an intervention to downstream features—such as the Directional Linkage equalities in Eqs. (3)–(4) or the Missing Values row of Table 1—this check is wrong: a valid intervention that requires a compensating downstream change is rejected (e.g., setting no_posts=TRUE forces num_posts=0, but with a_num_posts=0 the constraint x_j·x_k=0 cannot be verified on a alone), and, conversely, when no coupling is present, the missing final check can accept x+a+r outside the reachable set. Consequently, the accepted samples are not distributed as a∼Uniform[A(x)] and r∼P_{x,a}; the resulting S_n is not necessarily Bin(n, ρ(x)), and the guarantees of Propositions 3 and 5 do not apply to the printed algorithm's output. Appendix A.1 states that the implementation instead checks feasibility on the final intervention a+r, which would fix the distributional claim, but this is not reflected in the main-text Algorithm 1 or its stated guarantees. Please revise Algorithm 1 to include the final feasibility check and state explicitly what distribution the accepted samples follow (e.g., conditional on x+a+r satisfying the constraints).","section":"§3.2 (Algorithm 1) and Propositions 3/5"},{"comment":"The text states 'We construct reachable sets with 20 samples per point, satisfying Remark 7.' This is numerically false. For ε=0.1 and α=0.05, Remark 7 requires n > log(0.05)/log(0.9) ≈ 28.4, so n=20 is below the necessary threshold; the formal test can never reject H0 for any point. As a result, the claim that 15,986 of 18,614 recidivism predictions are 'fixed' cannot be supported by the testing procedure described in Section 3. If the authors instead mean an estimation-based criterion (e.g., ρ̂<0.01), they should say so explicitly and separate that analysis from the formal hypothesis test, which is invalid at this sample size.","section":"§4.1 (recidivism)"}],"minor_comments":[{"comment":"The caption defines Specificity as P(Claim Responsive | ρ ≥ ε) and calls it 'analogous to statistical power: 1−β', while main text Section 3.2 calls the same quantity 'the probability of correctly identifying responsiveness (Specificity)' and claims it is at least 1−α. These are inconsistent: P(Claim Responsive | ρ≥ε) is 1 minus the type-I error, i.e., specificity (≥1−α), not power. Please correct the caption or the main-text terminology so that Specificity and Recall are labeled consistently with their formal definitions.","section":"Figure 2 caption"},{"comment":"The numbered constraint list contains apparent duplicates: items 25–27 repeat items 13–15 (both describe Directional Linkage from prior_arrests≥2, prior_arrests=1, and prior_arrests≥5 to prior_arrests_for_felony). This makes the stated count of 27 constraints inconsistent with the distinct constraints listed. Please reconcile the numbering or remove the duplicates.","section":"Appendix B.1.2"},{"comment":"The paper refers to a provided Python library and to the enumeration procedure from Kothari et al. [32], but it does not specify the software version or reproducibility commands; adding a brief 'Reproducibility' statement or an artifact appendix would help readers verify the validation study in Appendix A.2.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The central statistical theory is correct, and the validation-by-enumeration study is a strong point. However, the main-text Algorithm 1 and the implementation described in Appendix A.1 appear to sample from different distributions, which directly affects the applicability of Propositions 3 and 5. This is an internal consistency issue, not a wrong theorem, and it should be fixable by revising the algorithm and its text. The recidivism example's n=20 sample size also contradicts Remark 7, and the authors should either increase the sample size or reframe the analysis as estimation rather than testing. I would suggest the editor request a revision that addresses these two points before further review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper before recommending it: it extends recourse verification from deterministic enumeration to query-only statistical inference with exact binomial confidence intervals, and it validates the approach against full enumeration on the german dataset. The content moderation comparison (convex relaxation vs. exact actions) is a genuine and useful demonstration that relaxation-based verification can severely under-report responsiveness. The Clopper-Pearson machinery is textbook, but it's applied cleanly and the paper's framing of responsiveness as a sampling problem is a real step forward for mixed discrete/continuous feature spaces.\n\nThe soft spots are addressable, but one is load-bearing. Algorithm 1 checks feasibility on the intervention a before sampling downstream effects r, and never re-checks on x+a+r. The resulting samples are not i.i.d. from P^reach_x; they come from a constraint-distorted distribution. Propositions 3 and 5 are stated for i.i.d. samples, so they do not apply to the algorithm as printed. The appendix (A.1, SampleEffect) says the implementation instead checks feasibility on a+r, which would fix the distributional claim—so this is a paper-writing inconsistency, not a broken statistical idea. But the main text needs correction before the guarantees are credible.\n\nTwo smaller issues: the recidivism case uses n=20, below the paper's own Remark 7 threshold of n>28.4 for epsilon=0.1, alpha=0.05; the claim of finding fixed predictions cannot be supported by the test as described. And the 'less than ten minutes' claim is unsupported, with code not locatable.\n\nThe empirical validation on german, where the authors compare sampled estimates against exact enumeration, is the most convincing part; it gives me confidence the method works when sampling is implemented correctly.\n\nThis is a paper for researchers and practitioners in recourse verification, model validation, and safety auditing. With a fix to Algorithm 1 and a revised recidivism experiment, it would be a solid contribution. It deserves a serious referee now, because the core idea is new and the empirical comparison is meaningful, even though the current version needs significant revision.","headline":"The paper's statistical framework is sound and its empirical comparisons are useful, but Algorithm 1 as printed does not sample from the claimed distribution, so the central guarantees don't apply to its output as written.","tokens_in":33208,"tokens_out":1632,"would_cite":false,"duration_ms":20611,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces a formal, black-box procedure to estimate and test how much a model's predictions respond to realistic interventions on an individual's features, with controlled false-claim rates.","keywords":["responsiveness verification","reachable sets","interventions and downstream effects","binomial confidence intervals","black-box model auditing","preclusion","gaming","counterfactual invariance"],"falsifier":"Run the procedure on a finite discrete reachable set where true responsiveness can be enumerated, as the paper does on its credit-data validation: if the empirical coverage of the Clopper-Pearson interval across many independent runs is materially below $1-\\alpha$, or if the observed false-claim rate exceeds $\\alpha$, then the i.i.d. uniform-sampling premise behind $\\hat S_n \\sim \\mathrm{Bin}(n, \\rho(x))$ is broken.","tokens_in":32138,"feed_emoji":"🛡️","tokens_out":8635,"duration_ms":93984,"temperature":0.7,"pith_summary":"The paper introduces a statistical validation procedure for a quantity it calls responsiveness: the proportion of feasible interventions at a given point that move a model's prediction into a target set, such as a denied loan applicant reaching approval or a bot account reaching 'human'. The central claim is that responsiveness can be estimated and tested for any black-box model by sampling reachable points and counting successes, which makes the estimate binomially distributed and gives exact confidence intervals and type-I error bounds. This matters because responsiveness failures such as preclusion, gaming, and monotonicity violations are safety problems in lending, hiring, organ allocation, and content moderation, and the procedure exposes them with only query access. The paper demonstrates the machinery on recidivism prediction, the UK liver-transplant benefit score, and bot detection, where it finds fixed predictions, monotonicity violations, and a convex-relaxation audit that under-reports gaming.","feed_headline":"Sampling tests if predictions respond to real user actions","feed_subtitle":"A binomial test on sampled reachable points bounds false alarms in auditing loans, transplants, and moderation.","key_machinery":"The load-bearing object is the reachable set $X^{\\mathrm{reach}}_A(x)$, defined through interventions $a \\in A(x)$ and downstream effects $r \\sim P_{x,a}$ as $x' = x + a + r$, and the load-bearing statistical tool is the exact Clopper-Pearson binomial confidence interval applied to the count $\\hat S_n$ of sampled reachable points whose predictions land in $\\hat Y^{\\mathrm{reach}}_x$. Algorithm 1 generates these points by sampling interventions uniformly from $A(x)$, enforcing feature-level and joint constraints such as integrality, monotonicity, encoding, and directional-linkage constraints with a mixed-integer feasibility check, and sampling downstream effects; this uniform sampling is what makes $\\hat S_n$ binomial and the finite-sample guarantees valid regardless of model type.","core_discovery":"The discovery is that responsiveness verification reduces to binomial inference. Given $n$ i.i.d. reachable points sampled uniformly over the intervention set, the number of points whose prediction falls in the target set is $\\mathrm{Bin}(n, \\rho(x))$, so the Clopper-Pearson interval of Proposition 3 covers the true responsiveness with probability at least $1-\\alpha$, and the one-sided test of Proposition 5 rejects the claim 'responsiveness at least $\\varepsilon$' with false-rejection probability at most $\\alpha$. The paper further gives a necessary sample-size bound, $n > \\log \\alpha / \\log(1-\\varepsilon)$, for the test ever to reject, and an algorithm that samples a non-convex reachable set over discrete and continuous features by sampling interventions, checking feasibility through a mixed-integer program, and drawing downstream effects from deterministic, random, or causal models. Because the estimator uses only black-box queries, the guarantees hold for any model class.","pith_inferences":["If the specification burden can be met, the same binomial machinery extends naturally to any target predicate, including regression intervals, multiclass sets, or arbitrary safety conditions, so responsiveness could serve as a general validation primitive across model types.","The minimal-response-model idea suggests a robustness property worth formalizing: if preclusion is detected under a conservative intervention set, responsiveness under any superset is at least as large, so conservative failures imply failures under realistic interventions; a proof would strengthen the auditing interpretation.","The paper notes its sampler is uniform and not optimized for finding counterexamples near the decision boundary; pairing the same binomial confidence machinery with importance sampling or boundary-aware proposals would make failure discovery more efficient, at the cost of weighting the counts.","A natural stress test for deployment would be to re-run the audit under several intervention models, narrow, neutral, and broad, and report verdicts that are stable across the family; this would convert the specification assumption from a caveat into a quantified sensitivity analysis."],"forward_implications":["Any model with black-box query access can be audited for preclusion, gaming, and monotonicity with a declared confidence level, because the estimator and tests do not depend on model architecture.","Practitioners can choose sample size before running the audit: $N^{\\min}(\\alpha, L)$ controls interval width, and $N^{\\min}(\\alpha, \\beta, \\varepsilon, \\Delta)$ controls the power to detect unresponsiveness.","Uniform sampling can replace exhaustive enumeration on large discrete reachable sets, cutting storage and compute while keeping estimation error and false-claim rates bounded; the validation study reports about $4\\%$ absolute error with $n=30$.","Every failed test comes with concrete reachable points that falsify the model-level claim, which supports debugging, regression testing, and refining the intervention specification.","Validating responsiveness with a convex relaxation of the intervention set can certify a model as robust when the exact intervention model shows that more than half of bot predictions are responsive, so exact reachable sets matter for safety verdicts."],"supporting_citations":[{"why":"Supplies the exact Clopper-Pearson binomial confidence interval used in Proposition 3 and the one-sided bound in Proposition 5.","marker":"[11]"},{"why":"Introduces the reachable-set enumeration formulation that this paper generalizes from discrete deterministic settings to sampling with downstream effects.","marker":"[32]"},{"why":"Provides the credit dataset used in the validation study where true responsiveness is enumerated to check estimation error, specificity, and recall.","marker":"[13]"},{"why":"Provides the recidivism dataset used to demonstrate detection of fixed predictions across demographic groups.","marker":"[55]"},{"why":"Provides the bot-detection dataset used in the content-moderation gaming case study.","marker":"[19]"},{"why":"Provides the simulated cohort and monotonicity concern for the organ-transplant score case study.","marker":"[3]"},{"why":"Defines the validation concepts of falsification and failure probability estimation that responsiveness testing is designed to support.","marker":"[30]"}],"fun_headline_variants":["Binomial test checks if predictions respond to user changes","Sampling reachable points bounds prediction responsiveness","Black-box audit: do predictions react to interventions?","Counting reachable points reveals prediction sensitivity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The practitioner must be able to specify the set of interventions a real decision subject could actually perform and how each intervention changes downstream features; if that specification is unfaithful, the statistically valid verdicts are not meaningful.","fun_headline_variants_meta":{"raw":{"variants":["Binomial test checks if predictions respond to user changes","Sampling reachable points bounds prediction responsiveness","Black-box audit: do predictions react to interventions?","Counting reachable points reveals prediction sensitivity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1200,"prompt_tokens":875,"completion_tokens":325,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":268}},"tokens_in":491,"tokens_out":325,"duration_ms":4143,"temperature":1.0,"reasoning_tokens":268,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:37:11.867642+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the procedure on a finite discrete reachable set where true responsiveness can be enumerated, as the paper does on its credit-data validation: if the empirical coverage of the Clopper-Pearson interval across many independent runs is materially below $1-\\alpha$, or if the observed false-claim rate exceeds $\\alpha$, then the i.i.d. uniform-sampling premise behind $\\hat S_n \\sim \\mathrm{Bin}(n, \\rho(x))$ is broken.","supporting_citations":[{"cited_title":"Prediction without preclusion: Recourse verification with reachable sets","cited_arxiv_id":null,"evidence_quote":"Introduces the reachable-set enumeration formulation that this paper generalizes from discrete deterministic settings to sampling with downstream effects."},{"cited_title":"Office Of Justice Programs","cited_arxiv_id":null,"evidence_quote":"Provides the recidivism dataset used to demonstrate detection of fixed predictions across demographic groups."},{"cited_title":"Kochenderfer, Sydney M","cited_arxiv_id":null,"evidence_quote":"Defines the validation concepts of falsification and failure probability estimation that responsiveness testing is designed to support."}],"review_version":1}