{"id":"4ca61511-b0b5-4c0f-a75d-4f94b05116c6","arxiv_id":"2505.13118","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A model-agnostic framework decomposes conformal prediction interval width and bounds into feature-level contributions using Shapley and proportional Shapley values.","lead":"The paper introduces a method to attribute predictive uncertainty, measured as conformal prediction interval width and bounds, to individual input features using game-theoretic Shapley and proportional Shapley allocations. It also provides a Monte Carlo approximation with statistical guarantees and demonstrates the approach on synthetic and real-world regression datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proportional-Shapley sampling is not well-defined as printed: Eq. (A.8) is not normalized and conflicts with Algorithm 3, whose unsigned sampling is invalid for negative singleton games used in experiments.","rationale":"The reader's weakest assumption correctly targets the proportional Shapley sampling step, but the problem is broader than negative singleton values. The printed Eq. (A.8) is not a normalized probability distribution and does not match the distribution actually generated by Algorithm 3, so Theorem 1 and the importance-sampling weights in Appendix A.4 are not established for P-Shapley as written. This is load-bearing because proportional Shapley values are a central novelty and are used in the real-data experiments. The exact Harsanyi-allocation definition and the generic Monte Carlo theorem are sound; the issue is the bridge between the formal P-Shapley value and its practical estimator. Because the fix is concrete (state the correct weighted-Shapley permutation distribution, use absolute values consistently, and handle zero/negative singletons explicitly) and does not invalidate the overall framework, a conditional accept remains appropriate. The reader's proposed revisions remain necessary, and the sampling inconsistency should be added to them.","tokens_in":28196,"tokens_out":16476,"duration_ms":175638,"concrete_test":"Verify normalization and sampling consistency: for d=3 with all singleton values equal to 1, evaluate Eq. (A.8) for a permutation (it gives 1/9, total mass 2/3 over the 6 permutations), and compare with 1,000,000 draws from Algorithm 3 (the correct P-Shapley random-order distribution yields 1/6 per permutation). Then run Algorithm 3 on the d=2 game v({1})=-1, v({2})=2, v({1,2})=1 with the printed line 3: the first draw uses probability -1/1 for player 1, which is not legal. With the absolute-value correction, Monte Carlo P-Shapley should reproduce the exact Eq. (3) values (-1, 2); if the code instead follows Algorithm 3 as printed, the P-Shapley results in Sections 4.2/4.3 and Appendix C need recomputation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing weak point is the proportional Shapley Monte Carlo and importance-sampling machinery, which supports the paper's \"beyond Shapley\" contribution and the P-Shapley experiments (e.g., Fig. 4 and Appendix C). As printed, it is internally inconsistent. Equation (A.8) defines PS(pi) = exp[-sum_{j=2}^d log(1 + sum_{k=1}^{d-1} |v(pi_j)|/|v(pi_k)|)]. This is not a probability mass function: in the d=3 additive game with all singleton values equal to 1, it assigns mass 1/9 to each of the 6 permutations, giving total mass 2/3. The correct random-order distribution for the weighted Shapley value defined by Eq. (3) is p(pi) = prod_{j=1}^d |v({pi_j})| / sum_{k=1}^j |v({pi_k})|, which in this example is uniform 1/6 and is what Algorithm 3 in effect samples when using absolute values. Because Theorem 1 and the importance-sampling weights in Appendix A.4 (PS(pi) times d!) rely on the printed PS(pi), the stated statistical guarantees for P-Shapley do not follow from the written formula. Separate from the normalization issue, Algorithm 3 line 3 samples with probabilities v({k})/sum v({i}) without absolute values. For the lower- and upper-bound value functions used in Sections 4.2, 4.3, and Appendix C, singleton values can be negative, so the prescribed draw is not a valid probability distribution. The exact P-Shapley allocation in Eq. (3) may be coherent, but the P-Shapley Monte Carlo and importance-sampling estimators are not uniquely or validly specified as written. This directly undermines the \"robust statistical guarantees\" claim for P-Shapley and the P-Shapley-specific experimental conclusions.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an uncertainty attribution (UA) framework that combines split conformal prediction with cooperative game theory. CP-interval quantities (width, lower bound, upper bound) are used as coalition value functions, and the resulting cooperative games are allocated to features via Shapley values and proportional Shapley values. The authors provide an exact algorithm, a Monte Carlo permutation-sampling algorithm with claimed unbiasedness, consistency, and asymptotic normality, and an importance-sampling reweighting scheme. The experimental sections compare the resulting importance rankings with conditional-mean and conditional-variance rankings on synthetic and real-world datasets, showing that CP-based uncertainty attributions differ from moment-based rankings.","tokens_in":28504,"tokens_out":7575,"duration_ms":68878,"significance":"The framework addresses a genuine gap in XAI: most feature-attribution methods explain point predictions, whereas predictive uncertainty is increasingly important in high-stakes applications. Defining value functions directly from conformal intervals is a natural and model-agnostic idea, and the use of the broader Harsanyi allocation family, in particular proportional Shapley values, is a useful extension beyond classical Shapley. The paper ships reproducible code and the exact computation for small d is clearly described. The efficiency result in Proposition 3.1 is correct, and the Monte Carlo estimators are standard conditional on a fixed value function. However, the proportional-Shapley sampling and importance-sampling machinery, which supports the 'beyond Shapley' contribution and the experiments in Sections 4.2, 4.3 and Appendix C, is not correctly specified as printed. If the sampling distribution and the handling of negative singleton values are corrected, the framework is a solid and publishable contribution.","major_comments":[{"comment":"The distribution PS(π) defined in Eq. (A.8) is not a probability mass function over permutations. For d=3 with |v({1})|=|v({2})|=|v({3})|=1, the formula gives PS(π)=exp[-(log(1+1+1)+log(1+1+1))]=1/9 for each of the 6 permutations, whose total mass is 2/3. The correct random-order distribution for the weighted Shapley value with weights |v({j})| is p(π)=∏_{j=1}^d |v({π_j})|/∑_{k=1}^j |v({π_k})|, which in this example is uniform 1/6. Because Theorem 1 and the importance-sampling weights in Appendix A.4 (PS(π)×d!) rely on the printed PS(π) being a valid distribution, the stated statistical guarantees for P-Shapley do not follow from the written formulas. This is load-bearing, since the P-Shapley approximations are used in the experiments of Sections 4.2, 4.3, and Appendix C.","section":"Appendix A.3, Eq. (A.8); Theorems 1 and 2"},{"comment":"Algorithm 3 samples permutations with probabilities proportional to v({k}) (no absolute values), which is a valid probability distribution only when every singleton value v({k}) is positive. For the lower- and upper-bound value functions used in Sections 4.2, 4.3, and Appendix C, v({k}) can be negative, and the paper does not specify any adjustment for negative values; the text only discusses zero individual values. Consequently, the P-Shapley Monte Carlo estimates for the boundary games are not well-defined as printed. This directly affects the validity of the P-Shapley results in Figure 4 and Appendix C. The definition of the weight system immediately before Eq. (3) also omits the absolute values that Eq. (3) uses, so the notation should be aligned.","section":"Appendix A.3, Algorithm 3"}],"minor_comments":[{"comment":"The theorems are stated for a fixed value function v, but in the UA pipeline the value functions themselves depend on the random training/calibration split and on the fitted models; the additional variability from these sources is not quantified. The paper should explicitly acknowledge this limitation rather than describing the guarantees as covering the full uncertainty-attribution procedure.","section":"Section 3.2, Theorems 1 and 2"},{"comment":"The section header reads 'Sobol'-Ativan' while Section 4.1 calls the benchmark 'Sobol'–Levitan'; the typo should be fixed.","section":"Appendix C.1"},{"comment":"The algorithm's input includes a 'weight assignment λ' but the body never uses λ; the exact procedure always uses either the Shapley or proportional Shapley weight system. The role of λ should be clarified or removed.","section":"Algorithm 1"},{"comment":"The proportional Shapley value as written in Eq. (3) uses absolute values of singleton contributions, which is a reasonable choice, but the text introducing λ_PS_j(A) uses v({j}) without absolute values; please make the convention explicit and consistent.","section":"Equation (3)"}],"recommendation":"major_revision","confidential_remarks":"The main blocking issue is the invalid definition of the proportional-Shapley sampling distribution in Eq. (A.8) and Algorithm 3. This is a fixable technical error, not a fundamentally flawed approach. I would ask the authors to correct the random-order distribution to the standard weighted-Shapley product form with absolute values, specify how negative singleton values are handled (e.g., by using |v({k})| in Algorithm 3), and then re-run or re-verify the affected P-Shapley experiments. The self-citations [23,25] appear to be related work and do not force the results, so there is no circularity concern."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: worth engaging, but not in its current form. The concept is genuinely new: define cooperative games whose value functions are CP-interval width, lower bound, or upper bound, then attribute those quantities with Shapley and proportional Shapley values. The efficiency result (Prop. 3.1) is correct, the Shapley MC estimator is standard, and the experiments do show that CP-width attributions differ from mean-based SHAP in interpretable ways. Code is promised in a GitHub repo, which is good practice.\n\nThe soft spot is load-bearing. The proportional Shapley sampling machinery in Appendix A.3 is not well-defined as printed. Eq. (A.8) is not a probability mass function—with three players and all singleton values equal to one, the six permutations get mass 1/9 each, summing to 2/3. Algorithm 3 samples without absolute values, using v({k})/Σv({i}), which is not a valid probability distribution when any singleton value is negative—and lower/upper-bound games in the experiments do produce negative singletons. The exact P-Shapley in Eq. (3) uses |v({j})|, so the exact allocation is coherent; the MC estimator and the importance-sampling weights are what break. This directly undermines the 'robust statistical guarantees' claim for P-Shapley and the P-Shapley-specific experimental conclusions. The fix is straightforward: use absolute values in the sampling and write down the correct weighted-Shapley random order distribution, ∏_{j=1}^d |v(π_j)|/Σ_{k=1}^j |v(π_k)|, which is what the text of the algorithm seems to intend.\n\nTwo smaller issues: the theorems condition on a fixed value function, so the abstract's 'robust statistical guarantees' should be scoped to the Monte Carlo step, not the whole pipeline including model fitting and calibration. And the real-data experiments don't report MC standard errors or convergence diagnostics; the Sobol benchmark has them, but the real-data attributions (e.g., Figure 4) are point estimates with no noise bar.\n\nThe paper is for XAI practitioners who want to explain predictive uncertainty, and for researchers connecting conformal prediction with cooperative game theory. It deserves peer review—the idea is useful and the core Shapley-based method is sound—but it needs major revision before the P-Shapley results can be trusted. I'd send it to a serious referee with a clear request to fix the sampling definition and re-verify the experiments.","headline":"Original idea with a solvent Shapley core; the proportional Shapley approximation is mis-specified as printed, so the paper needs major revision before it supports its headline claims.","tokens_in":29133,"tokens_out":4393,"would_cite":false,"duration_ms":40017,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the width and boundaries of conformal prediction intervals can be treated as cooperative games, so Shapley and proportional Shapley values attribute predictive uncertainty to individual features, with Monte Carlo…","keywords":["conformal prediction","uncertainty attribution","Shapley values","proportional Shapley values","Harsanyi allocations","explainable AI","cooperative game theory","Monte Carlo approximation"],"falsifier":"On a small dataset where exact computation is feasible, compute the proportional Shapley values for the lower- and upper-bound value functions and compare them with the paper's Algorithm 3 estimates at large $m$: if any singleton value is negative and the estimates deviate from the exact allocation, the claimed unbiasedness for the boundary games fails; a match would confirm that the missing sampling adjustment is immaterial.","tokens_in":27946,"feed_emoji":"📊","tokens_out":11939,"duration_ms":114508,"temperature":0.7,"pith_summary":"Existing explanation tools attribute a model's average prediction; this paper tries to establish that predictive uncertainty itself can be attributed to features. It treats the width, lower bound, and upper bound of a conformal prediction interval as cooperative-game value functions, then allocates those values among features using Shapley values and proportional Shapley values, both members of the Harsanyi allocation family. It also claims a Monte Carlo permutation scheme estimates these attributions without bias and with convergence guarantees, making the decomposition feasible when exact computation over all $2^d$ feature subsets is too costly. If the approach works, practitioners in high-stakes settings gain an answer to a different question than usual: not just why the model predicted this, but which features make that prediction more or less trustworthy.","feed_headline":"Conformal intervals reveal which features drive predictive uncertainty","feed_subtitle":"Unlike mean-focused attribution methods, this explains the interval itself, so users see which features widen it.","key_machinery":"The machinery is a cooperative game whose value function is a CP-interval property, paired with a Harsanyi allocation that redistributes Harsanyi dividends—the added value attributable to each coalition beyond its subsets—among the coalition's members. The Shapley value uses egalitarian weights $\\lambda_j(A)=1/|A|$, splitting each dividend equally; the proportional Shapley value uses weights proportional to individual feature values, $\\lambda_j^{\\mathrm{PS}}(A)=|v(\\{j\\})|/\\sum_{j'\\in A}|v(\\{j'\\})|$. For computation, the paper exploits the random-order formulation of these allocations: the value of a coalition is read off a permutation as the marginal change in the interval property, and sampling permutations according to the appropriate distribution (uniform for Shapley, proportional to singleton values for P-Shapley) gives the Monte Carlo estimates. An importance-sampling reweighting then recycles one permutation sample into both allocations without retraining any models.","core_discovery":"The central claim is that uncertainty attribution can be built directly on conformal prediction intervals without changing the underlying regression model. For any of the three CP flavors the paper treats—standard mean regression, locally adaptive conformal prediction, and conformalized quantile regression—it defines a game with value function $v^{\\omega}_{\\mathrm{CP}}(A,x)$ that returns the width, lower bound, or upper bound of the interval produced when the model is retrained on feature subset $A$. Because Shapley and proportional Shapley values are efficient allocations, Proposition 3.1 guarantees the attributions sum exactly to $v^{\\omega}_{\\mathrm{CP}}(D,x)-v^{\\omega}_{\\mathrm{CP}}(\\varnothing)$, so the interval property itself is decomposed across features. The paper further claims that the proportional Shapley value—which shares dividends in proportion to feature contributions rather than equally—is computable here, and that the permutation-based Monte Carlo estimators are unbiased, strongly consistent, and asymptotically normal for both allocations.","pith_inferences":["Editorial inference: the decompositions measure how the interval's geometry depends on which features are available, not whether the interval is actually reliable; replacing the value function with conditional coverage or a sharpness gap would test whether the two views of uncertainty agree, and that comparison is not run in the paper.","Editorial inference: the proportional-Shapley sampling step as written requires positive singleton values; re-running the boundary-value experiments with an absolute-value or shifted sampling distribution, and checking whether the top-ranked features change, would show how much of the reported proportional-Shapley conclusions rests on that unstated choice.","Editorial inference: since every coalition retrains the model, an attribution conflates the effect of removing a feature on the fitted predictor with its effect on the calibration conformity scores; freezing the conformity scores while perturbing only the model input would separate those two channels.","Editorial inference: the same game construction extends to classification, where the value function could be the size or coverage of a conformal prediction set; only the interval property needs to be redefined."],"forward_implications":["For any regression model that admits a conformal prediction interval, users get a per-instance additive decomposition of interval width or a chosen boundary into feature contributions; the total equals the interval property itself.","The method covers three CP variants (standard mean regression, locally adaptive conformal prediction, and conformalized quantile regression) with the same estimation code, so switching the interval construction does not require a new derivation.","The Monte Carlo scheme cuts the number of trained models from $2^d$ to roughly $m\\times d$ or fewer, and the importance-sampling reweighting produces both Shapley and proportional Shapley estimates from one permutation sample; the experiments show about a tenfold speed-up at $m=1{,}000$.","Comparing Shapley and proportional Shapley rankings on the same value function acts as a sensitivity check: agreement signals stable importance, while disagreement flags features whose attributed uncertainty depends on the allocation rule.","After normalizing by the empty-coalition baseline, attributions sum to 1, so the decomposition has a clear accounting interpretation even though individual attributions may fall outside $[0,1]$."],"supporting_citations":[{"why":"supplies the Shapley-value attribution baseline that this paper extends from mean predictions to CP-interval properties","marker":"[40]"},{"why":"provides split conformal prediction and its coverage guarantee, the source of the intervals used as value functions","marker":"[35]"},{"why":"defines conformalized quantile regression, one of the three CP variants whose intervals feed the games","marker":"[56]"},{"why":"introduces the proportional Shapley value and its properties, the allocation rule compared against Shapley","marker":"[8]"},{"why":"defines the Harsanyi set of allocations, the family that contains both Shapley and proportional Shapley","marker":"[68]"},{"why":"links weighted Shapley values to random-order distributions, which grounds the permutation sampling for proportional Shapley","marker":"[15]"},{"why":"introduces the Monte Carlo permutation approximation for Shapley values that the paper generalizes","marker":"[78]"},{"why":"defines Harsanyi dividends, the quantities redistributed by the allocation rules","marker":"[22]"}],"fun_headline_variants":["Proportional Shapley explains uncertainty, not just predictions","Conformal prediction meets Shapley for uncertainty attribution","Which features widen your intervals? A Shapley approach","Monte Carlo meets Shapley to decompose interval uncertainty"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proportional-Shapley sampling step is a valid probability distribution only when every feature's singleton contribution to the value function is positive; for the lower- and upper-bound games used in the experiments singleton values can be negative, and the paper does not state how the sampling is adjusted, so the Monte Carlo guarantees for those boundary decompositions rest on an unstated condition.","fun_headline_variants_meta":{"raw":{"variants":["Proportional Shapley explains uncertainty, not just predictions","Conformal prediction meets Shapley for uncertainty attribution","Which features widen your intervals? A Shapley approach","Monte Carlo meets Shapley to decompose interval uncertainty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001547,"raw_usage":{"total_tokens":6178,"prompt_tokens":931,"completion_tokens":5247,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":5181}},"tokens_in":547,"tokens_out":5247,"duration_ms":39474,"temperature":1.0,"reasoning_tokens":5181,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:19:39.909600+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small dataset where exact computation is feasible, compute the proportional Shapley values for the lower- and upper-bound value functions and compare them with the paper's Algorithm 3 estimates at large $m$: if any singleton value is negative and the estimates deviate from the exact allocation, the claimed unbiasedness for the boundary games fails; a match would confirm that the missing sampling adjustment is immaterial.","supporting_citations":[{"cited_title":"A unified approach to interpreting model predictions","cited_arxiv_id":null,"evidence_quote":"supplies the Shapley-value attribution baseline that this paper extends from mean predictions to CP-interval properties"},{"cited_title":"Conformalized Quantile Regres- sion","cited_arxiv_id":null,"evidence_quote":"defines conformalized quantile regression, one of the three CP variants whose intervals feed the games"},{"cited_title":"The proportional Shapley value and applications","cited_arxiv_id":null,"evidence_quote":"introduces the proportional Shapley value and its properties, the allocation rule compared against Shapley"},{"cited_title":"The Harsanyi Set for Cooperative TU-Games","cited_arxiv_id":null,"evidence_quote":"defines the Harsanyi set of allocations, the family that contains both Shapley and proportional Shapley"},{"cited_title":"On Harsanyi Dividends and Asymmetric Values.International Game Theory Review, 19(03):1750012, September 2017","cited_arxiv_id":null,"evidence_quote":"links weighted Shapley values to random-order distributions, which grounds the permutation sampling for proportional Shapley"},{"cited_title":"Harsanyi","cited_arxiv_id":null,"evidence_quote":"defines Harsanyi dividends, the quantities redistributed by the allocation rules"}],"review_version":1}