{"id":"4eec0e8f-1f70-4baf-8187-5238821f356e","arxiv_id":"1908.09718","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A Shapley-value based normalization of residual variance increases yields feature-level R2 shares that sum to the model's overall R2.","lead":"This paper introduces a feature importance metric for machine learning models that splits the model's R2, the share of outcome variance explained, among input features using Shapley values. It is meant to give practitioners a single global ranking of which features matter most without refitting models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposed metric is not a Shapley decomposition of R2; in a simple uncorrelated two-feature linear model its shares (0.32, 0.48) differ from the true Shapley shares (0.20, 0.60), invalidating the 'fair allocation' claim.","rationale":"The reader's weakest assumption focused on Eq. (5)'s removal interpretation for nonlinear/interacting models. My concern is more fundamental: even when removal works perfectly (linear model with uncorrelated features), the proposed allocation is not the Shapley-value decomposition of R2. The metric is a well-defined heuristic, but the abstract and introduction explicitly claim a 'Shapley-value variance decomposition' that 'fairly allocates' explained variance. A concrete two-feature uncorrelated linear counterexample shows the method deviates from the unique Shapley result. This is not a question of missing proof or error bars; it is a direct falsification of the central claim under the standard interpretation. The paper could be revised to present the metric as a heuristic importance score, but then the novelty and framing change substantially. The empirical demonstrations with tree ensembles and the sigma_unique ratio do not address this issue, since they never compare against LMG or any ground-truth decomposition. I therefore recommend REJECT rather than CONDITIONAL: the promised fairness property is not merely unproven but demonstrably absent in the simplest possible setting. Independent strengths of the paper—the clear exposition, the boundedness, and the availability of the shapFlex package—do not offset a false central claim. The concrete test above would settle the matter quantitatively and is cheap to run.","tokens_in":6866,"tokens_out":9185,"duration_ms":92050,"concrete_test":"Simulate N = 100,000 from y = x1 + sqrt(3)*x2 + eps with x1, x2, eps i.i.d. N(0,1). Compute exact Shapley values for this linear model (phi1 = x1, phi2 = sqrt(3)*x2) and apply Eqs. (4)-(6). Compare the resulting R2_shap shares to the exhaustive LMG/Shapley decomposition, which yields 0.20 and 0.60. If the proposed shares are approximately 0.32 and 0.48 as derived, the central 'fair allocation' claim is refuted. A further sensitivity check: vary the coefficient ratio beta1/beta2 and confirm the proposed metric always compresses the allocation relative to the true Shapley shares.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that the metric 'fairly allocates' model-explained variability via a Shapley-value variance decomposition. The method, however, does not compute Shapley values of any R2/explained-variance game. It uses instance-level prediction Shapley values only to construct yhat - phi_i^f, then normalizes residual-variance changes. Efficiency (sum to R2) is imposed by normalization, not derived from Shapley axioms. In the most basic case—a linear model with two uncorrelated predictors—the metric fails to reproduce the unique Shapley decomposition. Let y = x1 + sqrt(3)*x2 + eps, with x1, x2, eps independent standard normal. Then R2 = 0.8, and the true Shapley (LMG) shares are 0.2 and 0.6, proportional to beta_j^2 * Var(x_j). The proposed Eq. (6) with exact Shapley values gives raw terms 0.4 and 0.6, which after normalization become 0.32 and 0.48 (summing to 0.8). Thus the metric misallocates variance between features even in the paper's own 'ideal' uncorrelated case and is not a Shapley decomposition. The fairness claim is therefore unsupported and demonstrably false under the standard notion of Shapley fairness. The paper's discussion of sigma_unique does not rescue this: that ratio measures uniqueness, not whether the imposed allocation matches any game-theoretic fairness criterion. The row-sum property is true by construction, but that alone does not make the allocation fair.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a global feature-importance metric for machine learning models. The method first computes a baseline R2 as var(yhat)/(var(yhat)+varres), then for each feature f constructs a modified prediction yhat_i - phi_i^f using instance-level SHAP values, and finally normalizes the resulting residual-variance ratios in Eq. (6) so that the feature-level scores sum to the baseline R2. The authors claim that this metric 'fairly allocates' model-explained variability, is bounded between 0 and 1, requires no model refitting, and can be computed from precomputed Shapley values. The paper includes a correlation-adjustment index sigma_unique (Eq. (7)), a simulation of that index, and applied demonstrations on wine quality and forest-fire datasets using CatBoost.","tokens_in":7193,"tokens_out":10544,"duration_ms":105472,"significance":"The practical motivation is good: a model-agnostic, refit-free global importance measure that reuses exact TreeSHAP values would be useful, and the authors provide an open-source R package. The paper also correctly identifies that LMG-type Shapley decompositions require refitting and are computationally expensive. However, the central theoretical claim is not established: Eq. (6) is a normalization of residual-variance increases, not a Shapley decomposition of R2, and the paper's fairness claim is demonstrably false under the standard Shapley/LMG notion in a simple linear model. The sigma_unique formula also contains a definite error. As submitted, the contribution is a heuristic with a misleading name rather than a rigorous decomposition.","major_comments":[{"comment":"The claim that Eq. (6) is a 'Shapley-value variance decomposition' that 'fairly allocates' explained variance is not supported. The decomposition into shares that sum to R2_baseline is imposed by the normalization in the denominator of Eq. (6); no game over feature subsets is defined, and none of the Shapley axioms (efficiency, symmetry, dummy, additivity) is verified. The metric is therefore a normalized residual-variance-increase heuristic, not a Shapley decomposition. This is not a semantic quibble: in the linear model y = x1 + sqrt(3) x2 + eps with independent standard-normal x1, x2, eps, the classical Shapley/LMG shares of R2 = 0.8 are (0.2, 0.6), while Eq. (6) applied to exact SHAP values yields (0.32, 0.48). Thus the 'fair allocation' claim is false under the standard notion of Shapley fairness, even in the idealized uncorrelated case that the paper highlights.","section":"Proposed R2 Feature Importance Metric, Eq. (6), and Abstract"},{"comment":"The construction yhat_i - phi_i^f is interpreted as removing feature f's marginal contribution from the prediction. This is not generally valid for nonlinear or interacting models. SHAP values are local additive attributions that depend on all features; the vector y - (yhat_i - phi_i^f) is not the prediction error of a model trained or evaluated without feature f. The variance increase after this subtraction therefore reflects covariance and interaction terms involving other features, not a quantity that can be uniquely attributed to feature f. The discussion of 'unique variance' in the correlation section consequently overstates what the metric measures.","section":"Proposed R2 Feature Importance Metric, Eq. (5)"},{"comment":"The formula for sigma_unique is inconsistent with the definition in the text and with the claimed value of 1 for uncorrelated features. As written, the numerator is the sum of the raw residual variances after each feature is removed, not the sum of increases in residual variance. In the same two-feature example, the numerator is var(x1 + eps) + var(sqrt(3)x2 + eps) = 2 + 4 = 6, while the denominator is var(y) - var(eps) = 4, giving sigma_unique = 1.5 rather than 1. To obtain the intended quantity, each term would need to be the increase over the baseline residual variance, that is, sum_f [var(y - yhat_shap(f)) - var(y - yhat)]. As printed, Eq. (7), Figure 1, and the sigma_unique values in the applied section do not measure what is claimed.","section":"Accounting for Correlation, Eq. (7)"}],"minor_comments":[{"comment":"There is a typo: 'Lundburg' should be 'Lundberg'.","section":"Related Work"},{"comment":"The paper should state the edge case in which the denominator of Eq. (6) is zero, for example when R2_baseline = 0 or all raw numerators are zero.","section":"Proposed R2 Feature Importance Metric, Eq. (6)"},{"comment":"The applied analysis is purely descriptive: there is no comparison with LMG, permutation importance, or other global importance methods, and no uncertainty quantification for the reported rankings.","section":"Applied Analysis"},{"comment":"Please define the axes in Figure 2 and explain how the 0-1 scaling affects the interpretation; as drawn, the figure makes it difficult to assess changes relative to the underlying model R2.","section":"Applied Analysis, Figure 2"},{"comment":"The notation var(y - ybar) should be spelled out as var(y - mean(y)) to avoid confusion with the fitted values yhat.","section":"Accounting for Correlation, Eq. (7)"}],"recommendation":"reject","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper's central claim doesn't survive contact with the simplest example. The idea is to take precomputed instance-level Shapley values, subtract each feature's values from the predictions, measure the increase in residual variance, and normalize so the shares sum to R2. That's a clean and computationally convenient construction, and the writing is honest about correlation issues and offers code. But it is not a Shapley decomposition of R2, and it does not fairly allocate explained variance.\n\nI checked the stress-test counterexample and it holds. For y = x1 + sqrt(3)x2 + eps with all independent standard normal, R2 = 0.8. True Shapley/LMG shares are 0.2 and 0.6. The paper's Eq. (6) gives raw residual-variance terms 0.4 and 0.6, which normalize to 0.32 and 0.48. So in the cleanest possible setting—uncorrelated features, linear model, exact Shapley values—the metric misallocates variance. The fairness claim is false under the standard Shapley criterion.\n\nThe reason is structural. Shapley values are additive explanations of individual predictions, not removable additive components of the data-generating process. Subtracting phi_f from yhat changes residual variance by var(phi_f) plus interaction terms; normalizing the resulting quantities to sum to R2 imposes efficiency but does not impose Shapley fairness. The paper's sigma_unique discussion is a useful caveat but doesn't fix the attribution.\n\nWhat's genuinely useful: the paper addresses a real pain point—global importance without refitting—and the empirical stability check is a reasonable start. The R package is a plus. But as written, the title and abstract overclaim. The metric is a heuristic, not a fair decomposition.\n\nThe practical fix is either to prove conditions under which Eq. (6) matches Shapley shares (I doubt it can, given the counterexample), or to reframe the paper as proposing a residual-variance-based importance heuristic and drop the fairness language. As is, it should not be accepted as a finished method. I'd still send it to peer review—the issue is timely and the counterexample is a valuable object lesson—but the revision burden is heavy.","headline":"The proposed 'Shapley decomposition of R2' is not a Shapley decomposition: in the simplest two-feature uncorrelated linear model it produces shares (0.32, 0.48) instead of the true Shapley shares (0.20, 0.60), so the fairness claim is demonstrably false.","tokens_in":7703,"tokens_out":3576,"would_cite":false,"duration_ms":34498,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes a model-agnostic metric that decomposes any fitted model's $R^2$ into per-feature shares of explained variance, using precomputed Shapley values and requiring no model refitting.","keywords":["Shapley values","R-squared","variance decomposition","feature importance","model interpretability","model-agnostic","gradient boosted trees"],"falsifier":"Fit a model to synthetic data generated as $y = x_1 + x_1 x_2 + \\epsilon$ with correlated $x_1$ and $x_2$, compute the proposed shares, and compare them with the change in $R^2$ when each feature is dropped and the model is retrained. A mismatch in the direction or magnitude of the two importance rankings would show that subtracting Shapley values does not remove a feature's contribution, undercutting the metric's fairness claim.","tokens_in":6663,"feed_emoji":"📊","tokens_out":9816,"duration_ms":86280,"temperature":0.7,"pith_summary":"The paper introduces $R^2_{\\text{shap}}$, a global feature-importance metric that decomposes any fitted model's overall $R^2$ into per-feature shares using precomputed Shapley values. The shares are bounded between 0 and 1 and sum exactly to the model's $R^2$, so feature importance can be reported on the familiar scale of explained variance. The method subtracts each feature's instance-level Shapley values from the model's predictions, measures the resulting increase in residual variance, and normalizes these increases into additive shares. Because only one model fit and one Shapley-value pass are needed, the computational burden shifts from retraining models to the Shapley calculation itself. The paper also proposes a companion ratio, $\\sigma_{\\text{unique}}$, that quantifies how much of the explained variance is uniquely assignable to features when features are correlated.","feed_headline":"Shapley values split any model's R-squared into per-feature shares","feed_subtitle":"A model-agnostic decomposition with shares that sum to the total, so practitioners can rank features without refitting.","key_machinery":"The central object is the Shapley-modified prediction $\\hat{y}^{(f)}_{i,\\text{shap}} = \\hat{y}_i - \\phi^{(f)}_i$, where Shapley values are the standard additive per-feature attributions of each prediction. The mechanism is a residual-variance ratio: the baseline residual variance is compared with the residual variance after removing each feature, the ratios are normalized into a simplex, and the result is multiplied by $R^2_{\\text{baseline}}$. This converts the additive explanation property of Shapley values into an additive variance decomposition of $R^2$.","core_discovery":"For a fitted model with baseline $R^2_{\\text{baseline}} = \\operatorname{var}(\\hat{y})/(\\operatorname{var}(\\hat{y})+\\operatorname{var}(y-\\hat{y}))$, the paper defines Shapley-modified predictions $\\hat{y}^{(f)}_{i,\\text{shap}} = \\hat{y}_i - \\phi^{(f)}_i$, where $\\phi^{(f)}_i$ is feature $f$'s Shapley value for instance $i$. For each feature, the feature-level share $R^2_{\\text{shap}}^{(f)}$ is the increase in residual variance after this subtraction, capped and normalized so that the shares sum to $R^2_{\\text{baseline}}$. The central claim is that these shares fairly allocate the proportion of model-explained variability to each feature, and that this works for any class of prediction model without refitting.","pith_inferences":["If the removal interpretation behind Eq. (5) fails for a given nonlinear or interacting model, the shares become a normalized reallocation of residual-variance changes rather than a fair decomposition; this could be tested by comparing the metric's rankings with leave-one-feature-out retraining on a suite of models.","Because the normalization in Eq. (6) depends on the full set of features, adding an irrelevant but correlated feature could dilute the shares of the others; a model with nested feature sets would expose this.","Because the metric inherits whatever Shapley-value estimator produced the $\\phi^{(f)}_i$, using different explainers or correlation-aware Shapley approximations could change the shares even for the same model, so part of the importance ranking may be an artifact of the explainer."],"forward_implications":["Practitioners can rank global feature importance for any fitted model using precomputed Shapley values, with no model refitting required.","Because the feature shares are bounded in $[0,1]$ and sum to the overall $R^2$, explanations can be reported on the same scale as classical regression fit.","For gradient boosted trees and neural networks, the metric's cost is dominated by the Shapley-value calculation, which recent algorithms make inexpensive.","The companion $\\sigma_{\\text{unique}}$ ratio gives a diagnostic for how much of the explained variance is uniquely attributable to individual features versus shared among correlated features.","The metric can summarize feature importance on either training or test data, though the paper leaves the choice as an open question."],"supporting_citations":[{"why":"Supplies the adjusted $R^2$ formula that the metric uses as its baseline variance decomposition.","marker":"[13]"},{"why":"Provides the additive Shapley-value property the metric builds on, along with efficient neural-network Shapley calculations.","marker":"[3]"},{"why":"Provides Tree SHAP, the exact Shapley-value method that makes the metric inexpensive for boosted trees.","marker":"[17]"},{"why":"Defines the target properties of $R^2$ feature importance (non-negativity, additivity) and surveys prior variance decompositions.","marker":"[8]"},{"why":"Establishes that the LMG $R^2$ decomposition is equivalent to a Shapley-value calculation.","marker":"[15]"},{"why":"Introduces the LMG variance decomposition, the earlier method whose properties the metric seeks to match.","marker":"[14]"},{"why":"Addresses Shapley values when features are dependent, cited as a preprocessing route for correlated features.","marker":"[18]"}],"fun_headline_variants":["R² partitioned by Shapley values without refitting","Feature shares of R² from Shapley values, model-agnostic","Shapley-based R² decomposition sums to total variance explained","No refit: Shapley splits model's R² into fair feature shares"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that subtracting a feature's Shapley values from a prediction actually removes that feature's contribution, so the resulting increase in residual variance is uniquely attributable to that feature; this is not guaranteed for nonlinear or interacting models with correlated features.","fun_headline_variants_meta":{"raw":{"variants":["R² partitioned by Shapley values without refitting","Feature shares of R² from Shapley values, model-agnostic","Shapley-based R² decomposition sums to total variance explained","No refit: Shapley splits model's R² into fair feature shares"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1466,"prompt_tokens":923,"completion_tokens":543,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":466}},"tokens_in":539,"tokens_out":543,"duration_ms":5229,"temperature":1.0,"reasoning_tokens":466,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:03:22.524811+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fit a model to synthetic data generated as $y = x_1 + x_1 x_2 + \\epsilon$ with correlated $x_1$ and $x_2$, compute the proposed shares, and compare them with the change in $R^2$ when each feature is dropped and the model is retrained. A mismatch in the direction or magnitude of the two importance rankings would show that subtracting Shapley values does not remove a feature's contribution, undercutting the metric's fairness claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the adjusted $R^2$ formula that the metric uses as its baseline variance decomposition."},{"cited_title":"M., & Lee, S","cited_arxiv_id":null,"evidence_quote":"Provides the additive Shapley-value property the metric builds on, along with efficient neural-network Shapley calculations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the target properties of $R^2$ feature importance (non-negativity, additivity) and surveys prior variance decompositions."},{"cited_title":"D., (2017) Decomposing the R-squared of a Regression Using the Shapley Value in SAS®","cited_arxiv_id":null,"evidence_quote":"Establishes that the LMG $R^2$ decomposition is equivalent to a Shapley-value calculation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the LMG variance decomposition, the earlier method whose properties the metric seeks to match."}],"review_version":1}