REVIEW 5 major objections 6 minor 27 references
MVP-Shapley: Feature-based Modeling for Evaluating the Most Valuable Player in Basketball
T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Feature-level Shapley values of a win-probability model produce MVP rankings that track expert voting more closely than metric weighting or player-level Shapley baselines.
desk verdict A promising MVP-evaluation framework undermined by label-informed feature selection in the NBA experiments; the Dunk City Dynasty split is credible but can't rescue the headline claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the player contribution measure of equation (5): for each player, the sum of their feature-level Shapley values from the data point where their team is encoded first minus the sum from the data point where their team is encoded second. This turns the predicted win probability into per-player attributions without requiring stable lineups, which is what makes the method scalable to randomly matched games. The computation is carried by exact tree-based Shapley attribution on a gradient-boosted tree model, and the causal refinement step identifies strong confounders such as '+/-' and defensive rating by feature importance, then discretizes them into bins to reduce their influence and improve agreement with human votes.
What would settle it
Re-run the full pipeline after swapping which team is encoded first in the paired data, then recompute equation (5). The formula negates every player's contribution under the swap, so the predicted MVP order inverts; a ranking that changes with this arbitrary encoding would show the measure is an artifact of the convention rather than a stable attribution of player value.
Extended reading notes
Core claim
The paper's central claim is that feature-level Shapley values of a gradient-boosted win-probability model, aggregated per player and filtered by a causal refinement step, produce MVP rankings that align with expert human voting better than metric-weighting or player-level Shapley baselines. On NBA regular-season MVP voting for 2022–2024, the best variant (causal refinement without fuzzification, ranking by average contribution) reduces average rank difference by 4.80–8.29 times relative to the API metric-weighting baseline and by 21.35–35.80 times relative to the GSV player-coalition baseline, with better Spearman correlation and top-K recall on the full list and near-perfect top-three agreement in several seasons. On Finals MVP, the fuzzified variant places the actual award winner first in most of the past decade. On the mobile basketball game dataset, the method reports 38.1% lower average rank difference, 29.0% higher Spearman correlation, and 53.7% higher accuracy than the deployed baseline, and an online A/B test reports lower player report and churn rates. The paper also proves axiomatic properties for the aggregated measure and convergence and concentration bounds for season-long rankings.
Load-bearing premise
The rankings stand on equation (5)'s choice that a player's value equals the home-encoding sum of feature Shapley values minus the away-encoding sum; the paper does not derive that difference from a coalition game among players, and if the choice is wrong every ranking is arbitrary.
Editorial extensions
If this is right
- Team-lineup history is no longer needed, so MVP evaluation works in games with random player matchmaking as well as fixed rosters.
- Ranking players by average per-game contribution over a season tracks full MVP voting lists better than the metric-weighting baseline, while rank-based variants give near-perfect agreement on the top three.
- Discretizing strong confounders such as '+/-' and defensive rating into bins improves agreement with human votes, implying those raw statistics overstate a player's causal contribution.
- Real-time MVP evaluation is feasible: sub-second for a single game and under five minutes for a full season, with a reported online reduction in player report and churn rates.
Reading between the lines
- Equation (5)'s home-minus-away difference is not a Shapley value of a coalition game among players; comparing it with the sum of the two orientations would show whether the difference is essential or just a sign convention.
- The confounder set is learned from NBA data, so applying the framework to a new game requires re-running the confounder search; the reported gains may not transfer without that step.
- Because the paper explicitly targets 'who voters pick' rather than an objective best player, any bias in expert voting would be inherited by the rankings.
- A check the paper does not report is whether rankings are invariant to swapping which team is encoded first; equation (5) suggests they are not, and that invariance is testable on the released data.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MVP-Shapley, a five-stage framework that trains a LightGBM win-probability model on paired home/away play-by-play features, computes feature-level Shapley values with TreeSHAP, aggregates them into player contributions via Eq. (5), and then refines the feature set and applies fuzzification in Section 3.5 to align with ground-truth MVP votes. The authors claim 1.42x and 1.44x performance improvements over baselines on the NBA and Dunk City Dynasty datasets, respectively, and report theoretical fairness, consistency, and convergence guarantees, as well as an online A/B test.
Significance. If the empirical claims held, the framework would be a practically useful and explainable MVP evaluation tool, and the released code and dataset would be a valuable resource. The Dunk City Dynasty experiments use a 1:1 split for feature optimization versus test and are therefore more credible. However, the NBA headline result is not independently supported because the feature subset in Section 3.5 is selected by maximizing E[Y|X] on the same ground-truth votes that are then used as evaluation labels in Tables 2 and 3. This evaluation leakage undermines the central empirical claim until a proper held-out validation is performed.
major comments (5)
- [§3.5, §4.2.1, Tables 2–3] There is evaluation leakage: X* = arg max_X E[Y|X] chooses the feature subset, the grouping of variables, and the fuzzification bin counts using the ground-truth MVP votes Y, and the same Y is then the label against which ARD/SRCC/R are reported. No held-out split is described for this optimization step, in contrast to the 9:1 split for the LightGBM model in Section 4.1 and the 1:1 split for Dunk City in Section 4.2.3. The reported 4.80–8.29x ARD reductions and the 1.42x gain may therefore reflect fitting the answer key rather than genuine model quality. Please re-run the NBA experiments with feature selection performed only on a training fold and report test-fold metrics.
- [Eq. (5), §3.3] The player contribution Φ(ai,{x1,x2}) is defined as the sum of feature-level Shapley values when the player is listed as home minus the sum when away. This is not a Shapley value of any coalition game among players, and no argument is given for why the difference, rather than the sum or another aggregation, is the correct measure of a player's causal value. Since every ranking in the paper uses this aggregation, the arbitrariness is load-bearing. Please either derive Eq. (5) from a player-level cooperative game, provide an axiom-based justification, or demonstrate robustness of the rankings to alternative aggregation schemes.
- [Theorem 3.1, §3.6] The Uniqueness Theorem claims that the MVP-Shapley player contribution measure Φ is unique, but the proof establishes uniqueness of feature-level Shapley values, not of the aggregated player measure. Aggregating and differencing feature-level Shapley values does not inherit the Shapley axioms at the player level, so the theorem as stated is unsupported. Please restate the theorem precisely, or prove uniqueness for Φ itself rather than for its feature-level components.
- [Theorem 3.5, §3.6] The Bias Reduction through Fuzzification theorem proves only that discretization reduces mutual information via the data processing inequality; it does not show that confounding bias in the MVP evaluation decreases. Reduced mutual information can coexist with unchanged or increased bias in the estimated contributions. Please state a formal bias measure and prove that fuzzification reduces it, or replace the theorem with an empirical demonstration of bias reduction.
- [Appendix A.4] The statement that the goal is to predict 'who voters pick' rather than an objective 'true MVP' makes the leakage in Section 3.5 more damaging: the procedure is explicitly tuned to reproduce the same expert votes that serve as evaluation labels. Even with a proper holdout split, this design choice should be discussed, since the claimed alignment with expert judgment is then a measure of mimicry rather than of an independent notion of value.
minor comments (6)
- [Table 2] Describing SRCC gains as '0.42–1.91x improvement' is not meaningful when the baseline SRCC is near zero or negative; report differences or use a rank-transformed comparison.
- [Table 3] Table 3 reports metrics on only the top-3 players, where ARD and R are highly discrete; the near-perfect SRCC values should be interpreted with caution and accompanied by full-ranking results.
- [Theorem B.12] The proof of Theorem B.12 uses i.i.d. assumptions that the text itself acknowledges may not hold; please state the stationarity or ergodicity conditions explicitly.
- [Figure 2] Figure 2 is difficult to read because the font size is very small; please enlarge it and clarify the flow of the causal refinement stage.
- [Throughout] There is a typo in the Theorem 3.1 heading ('Uniqeness') and some ACM-format template artifacts (e.g., 'Conference'17') remain in the text.
- [§4.2.5] In Section 4.2.5, the complexity reduction from O(TL^2M) to O(TLD^2) is stated without derivation; please include a reference or a brief explanation.
Circularity Check
NBA result is partially circular: the feature subset optimized against ground-truth MVP votes is then evaluated against those same votes, with no held-out split for the NBA feature-selection step; a secondary uniqueness theorem is true only by the chosen aggregation.
-
fitted input called prediction
[Section 3.5 (Causal Refinements), Section 4.1 (Ours_V2/V3), Section 4.2.1, Appendix A.4]
"Let 𝑌 represent the ground truth... Our goal is to identify the optimal feature subset 𝑋∗ that maximizes the conditional expectation of the outcome: 𝑋∗ = arg max 𝑋 E[𝑌|𝑋]. ... Table 2 presents the performance comparison across all ranked players in the 2022–2024 NBA regular season MVP voting results. ... we focus on predicting 'who voters pick' rather than attempting to determine an objective 'true MVP.'"
The same variable Y—the expert/voter MVP outcome—is used twice: in Section 3.5 it defines the optimization target X* = arg max E[Y|X] that selects which features to remove or fuzzify (Ours_V2/V3), and in Section 4.2.1 the resulting rankings are scored against that same Y via ARD, SRCC, and R. No held-out split is described for the NBA feature-selection step, unlike the explicit 1:1 split stated for Dunk City Dynasty. Because the paper's declared target (Appendix A.4) is 'who voters pick', the reported 1.42x-type gains partly measure how well the feature search fit the answer key rather than an independent forecast. The win-model and Shapley computation are not themselves trained on Y, so the circularity is partial, not total.
-
self definitional
[Section 3.6 Theorem 3.1 and Appendix B.4 Theorem B.11]
"The MVP-Shapley player contribution measure Φ is the unique attribution method at the feature level satisfying: Efficiency... Symmetry... Null Player... Additivity. ... Since our player contribution measure Φ is derived by aggregating feature-level Shapley values: Φ(...)=..., the uniqueness of 𝜙 implies the uniqueness of Φ given the aggregation scheme."
The theorem's conclusion is conditioned on 'the aggregation scheme'—the home-minus-away sum/difference in equation (5)—which is an assumption, not a consequence of the four Shapley axioms. Shapley's theorem makes the feature-level Shapley values unique for a fixed utility function; it does not make equation (5)'s particular player-level aggregation unique. Thus the claimed uniqueness of Φ is true only by the paper's own definition of Φ, i.e., by construction. This does not drive the empirical tables but it is a first-principles claim that reduces to its input.
full rationale
The main empirical chain in the NBA experiments is partially circular: Section 3.5 selects the feature subset and fuzzification choices by maximizing E[Y|X] with Y equal to the MVP vote ground truth, and Section 4.2.1 evaluates Ours_V2/Ours_V3 against exactly that ground truth with no reported held-out split for the selection step. The Dunk City Dynasty experiments do split feature optimization from test data 1:1, and the win-loss model itself is trained on game outcomes rather than votes, so the framework is not entirely fitted. A second, less empirically weighty self-definitional issue is the uniqueness theorem, which is only unique 'given the aggregation scheme.' No self-citation chain or imported-uniqueness problem was found; citations to Shapley and TreeSHAP are standard external results. Overall, the central NBA headline is weakened by label-informed selection followed by evaluation on the training labels, corresponding to partial circularity (score 6).
Assumptions & free parameters
free parameters (3)
- Feature subset for Ours_V2 =
'+/-' and 'DRtg' removed
- Fuzzification bin counts |B| =
|B|=3 for '+/-', |B|=8 for 'DRtg'
- Feature grouping into five importance groups =
'+/-', ORtg, DRtg, BPM, remaining variables
assumptions (4)
- domain assumption Ground-truth MVP voting is a fair and correct standard (introduction: 'We assume votes are fair ground truth').
- domain assumption Feature-level Shapley values of the LightGBM win-probability model are a valid utility for player attribution.
- ad hoc to paper Game contributions are i.i.d. with finite variance for the law of large numbers and Hoeffding bounds (Theorem B.12).
- ad hoc to paper Fuzzification reduces confounding bias because it reduces mutual information.
Cite this review
Pith. "Pith review of MVP-Shapley: Feature-based Modeling for Evaluating the Most Valuable Player in Basketball." pith.science (2026). https://pith.science/paper/CFL7KOSX
@misc{pith2026250604602,
author = {Pith},
title = {Pith review of: MVP-Shapley: Feature-based Modeling for Evaluating the Most Valuable Player in Basketball},
year = {2026},
howpublished = {\url{https://pith.science/paper/CFL7KOSX}},
note = {Machine review of arXiv:2506.04602}
}
read the original abstract
The burgeoning growth of the esports and multiplayer online gaming community has highlighted the critical importance of evaluating the Most Valuable Player (MVP). The establishment of an explainable and practical MVP evaluation method is very challenging. In our study, we specifically focus on play-by-play data, which records related events during the game, such as assists and points. We aim to address the challenges by introducing a new MVP evaluation framework, denoted as \oursys, which leverages Shapley values. This approach encompasses feature processing, win-loss model training, Shapley value allocation, and MVP ranking determination based on players' contributions. Additionally, we optimize our algorithm to align with expert voting results from the perspective of causality. Finally, we substantiated the efficacy of our method through validation using the NBA dataset and the Dunk City Dynasty dataset and implemented online deployment in the industry.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Chenjie Cao. 2012. Sports data mining technology used in basketball outcome prediction. (2012)
work page 2012
-
[2]
William W Cooper, Jose L Ruiz, and Inmaculada Sirvent. 2009. Selecting non- zero weights to evaluate effectiveness of basketball players with DEA.European journal of operational research195, 2 (2009), 563–574
work page 2009
-
[3]
Alexander Philip Dawid and Allan M Skene. 1979. Maximum likelihood esti- mation of observer error-rates using the EM algorithm.Journal of the Royal Statistical Society: Series C (Applied Statistics)28, 1 (1979), 20–28
work page 1979
-
[4]
Tom Decroos, Lotte Bransen, Jan Van Haaren, and Jesse Davis. 2019. Actions speak louder than goals: Valuing player actions in soccer. InProceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 1851–1861
work page 2019
-
[5]
Jeremias Engelmann. 2017. Possession-based player performance analysis in basketball (adjusted+/–and related concepts). InHandbook of statistical methods and analyses in sports. Chapman and Hall/CRC, 231–244
work page 2017
-
[6]
Paul Fearnhead and Benjamin Matthew Taylor. 2011. On estimating the ability of NBA players.Journal of Quantitative analysis in sports7, 3 (2011)
work page 2011
-
[7]
Luca Grassetti, Ruggero Bellio, Luca Di Gaspero, Giovanni Fonseca, and Paolo Vidoni. 2021. An extended regularized adjusted plus-minus analysis for lineup management in basketball using play-by-play data.IMA Journal of Management Mathematics32, 4 (2021), 385–409
work page 2021
-
[8]
Joachim Gudmundsson and Michael Horton. 2017. Spatio-temporal analysis of team sports.ACM Computing Surveys (CSUR)50, 2 (2017), 1–34
work page 2017
Show all 27 references
-
[9]
Inwon Kang, Qishen Han, and Lirong Xia. 2023. Learning to Explain Voting Rules.. InAAMAS. 2883–2885
2023
-
[10]
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree.Advances in neural information processing systems30 (2017)
2017
-
[11]
Ksenia Kolykhalova, Giorgio Gnecco, Marcello Sanguineti, Gualtiero Volpe, and Antonio Camurri. 2020. Automated analysis of the origin of movement: An approach based on cooperative games on graphs.IEEE Transactions on Human- Machine Systems50, 6 (2020), 550–560
2020
-
[12]
Justin Kubatko, Dean Oliver, Kevin Pelton, and Dan T Rosenbaum. 2007. A starting point for analyzing basketball statistics.Journal of quantitative analysis in sports3, 3 (2007)
2007
-
[13]
Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. 2020. From local explanations to global understanding with explainable AI for trees. Nature machine intelligence2, 1 (2020), 56–67
2020
-
[14]
Vashisht Madhavan. 2016. Predicting NBA game outcomes with hidden Markov models.Berkeley University(2016)
2016
-
[15]
Olga Matthiopoulou, Benoit Bardy, Giorgio Gnecco, Denis Mottet, Marcello Sanguineti, and Antonio Camurri. 2020. A computational method to automatically detect the perceived origin of full-body human movement and its propagation. InCompanion Publication of the 2020 Internationa...
2020
-
[16]
Rodolfo Metulini and Giorgio Gnecco. 2023. Measuring players’ importance in basketball using the generalized Shapley value.Annals of Operations Research 325, 1 (2023), 441–465
2023
-
[17]
Rodolfo Metulini and Mael Le Carre. 2020. Measuring sport performances under pressure by classification trees with application to basketball shooting.Journal of Applied Statistics47, 12 (2020), 2120–2135
2020
-
[18]
Garritt L Page, Bradley J Barney, and Aaron T McGuire. 2013. Effect of position, usage rate, and per game minutes played on NBA player production curves. Journal of Quantitative Analysis in Sports9, 4 (2013), 337–345
2013
-
[19]
James Piette, Sathyanarayan Anand, and Kai Zhang. 2010. Scoring and shooting abilities of NBA players.Journal of Quantitative analysis in sports6, 1 (2010)
2010
-
[20]
Marco Sandri, Paola Zuccolotto, and Marica Manisera. 2020. Markov switching modelling of shooting performance variability and teammate interactions in basketball.Journal of the Royal Statistical Society Series C: Applied Statistics69, 5 (2020), 1337–1356
2020
-
[21]
Vangelis Sarlis and Christos Tjortjis. 2020. Sports analytics—Evaluation of bas- ketball players and team performance.Information Systems93 (2020), 101562
2020
-
[22]
Philip Sedgwick. 2014. Spearman’s rank correlation coefficient.Bmj349 (2014)
2014
-
[23]
Lloyd S Shapley et al. 1953. A value for n-person games.Contributions to the Theory ofGames, 2(28):307– 317(1953)
1953
-
[24]
Joseph Sill. 2010. Improved NBA adjusted+/-using regularization and out-of- sample testing. InProceedings of the 2010 MIT Sloan sports analytics conference
2010
-
[25]
Zachary Terner and Alexander Franks. 2021. Modeling player and team per- formance in basketball.Annual Review of Statistics and Its Application8 (2021), 1–23
2021
-
[26]
Lilian Weng. 2024. Thinking about High-Quality Human Data.lilian- weng.github.io(Feb 2024). https://lilianweng.github.io/posts/2024-02-05-human- data-quality/
2024
-
[27]
Who Voters Pick
Haocheng Xia, Xiang Li, Junyuan Pang, Jinfei Liu, Kui Ren, and Li Xiong. 2024. P-Shapley: Shapley Values on Probabilistic Classifiers.Proceedings of the VLDB Endowment17, 7 (2024), 1737–1750. A APPENDIX A.1 Tables Table 10: NBA Statistics Summary Basic Statistics Description M...
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.