REVIEW 4 major objections 6 minor 40 references
Functional relevance based on the continuous Shapley value
T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A continuous-game Shapley value turns every point of a functional input into a measurable player, giving any trained regression model a pointwise relevance curve.
desk verdict A well-built and useful functional-data interpretability heuristic whose central label—'continuous Shapley value'—is not backed by a verified convergence argument; the paper deserves serious refereeing, but the theory needs either proof or a more modest framing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Shapley value relevance function $R_f(t)$, built from the continuous game $\nu(S)=\tilde R^2(S)$, where $\tilde R^2(S)$ is the test coefficient of determination of the trained model $f$ evaluated on reconstructed curves $\tilde X_j^S(t)=X_j(t)\mathbf{1}_S(t)+\dot X_j(t)\mathbf{1}_{S^c}(t)$, with $\dot X_j(t)$ the conditional expectation of $X(t)$ given the observed values on $S$, estimated under a Gaussian-process assumption. The finite version of the game on a partition $\{I_1,\dots,I_n\}$ has Shapley value $\varphi_{\nu,i}=(1/n!)\sum_{\pi\in\Pi}[\tilde R^2(p_i^\pi\cup I_i)-\tilde R^2(p_i^\pi)]$, averaged over random permutations in practice, and the relevance function $R_f(t)=\sum_i \varphi_{\nu,i}(b_i-a_i)^{-1}\mathbf{1}_{I_i}(t)$ is the histogram density of these interval contributions. This machinery converts an infinite-player relevance problem into a sequence of finite Shapley-value computations, with the asymptotic theory for continuous games supplying the intended limit as the partition is refined.
What would settle it
Run the method on a deliberately discontinuous prediction rule, such as $f(X)=\max_t X(t)$, and refine the partition of $[0,1]$ while increasing the number of random permutations; if the resulting relevance function does not converge as the partition becomes finer, the assumed asymptotic value for $\nu(S)=\tilde R^2(S)$ does not exist for that model, and the finite approximation is not computing a well-defined continuous Shapley value.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that Shapley values, originally defined for finitely many players, can be lifted to the infinite setting of functional regression by treating the continuum of points in the input interval as players and defining the payoff of each measurable subset $S$ as the test-set coefficient of determination of the already trained model when all points outside $S$ are replaced by conditional expectations. The resulting relevance function is computed through the asymptotic approach: for a partition of the interval into $n$ pieces, the finite Shapley values of the restricted game are averaged over random permutations, and the histogram of those interval contributions, renormalized by interval length, is the relevance curve $R_f(t)$. The paper also shows that this game is generally not a vector-measure game (a game whose payoff depends only on finitely many measures of the coalition), so no closed-form diagonal formula is available and the permutation-based finite approximation is not merely an algorithmic convenience but the operative definition.
Load-bearing premise
The construction depends on the way payoff is assigned to coalitions of points being one of the continuous games for which the Shapley value exists and is unique; the paper states that 'under certain conditions' the limit exists but does not verify those conditions for its specific payoff before using a finite approximation.
Editorial extensions
If this is right
- Any trained functional predictor — linear, nearest-neighbour, or neural — can be audited with a single test set and no retraining, because the method needs only evaluations of the fitted model on reconstructed curves.
- The relevance function inherits the Shapley efficiency property: integrated over the whole interval it recovers the model's total test $R^2$, so the curve is a genuine allocation of explained variability among regions of the input.
- When the fitted model has no predictive skill on a target (a linear model facing a strongly non-linear map, for instance), the method returns a flat relevance curve, which serves as a diagnostic of model failure rather than a misleading ranking.
- On the Tecator data, the relevance functions of all three algorithms agree that the interval $[1040,1050]$ dominates fat-content prediction and $[970,980]$ is negligible, matching the observed variability of the second-derivative curves.
- The expected computational cost scales as $O(T^3|\Pi'|n)+O(mT^2|\Pi'|n)+O_f(m,T)|\Pi'|n$, driven by the number of random permutations and grid points; the accompanying ShapleyFDA package caches coalition values shared across permutations to reduce the work.
Reading between the lines
- Beyond the paper, the same game construction should extend to functional classification by replacing $R^2$ with test accuracy or log-loss, and to multiple functional regressors by letting players carry a type label; the authors list these only as future directions.
- Beyond the paper, the Gaussian-process assumption on the reconstruction step is the likely weak point: on strongly non-Gaussian functional data the conditional expectations become best linear predictions, so the method may rank points by their ability to predict missing segments rather than by their role in the response; a retraining-based LOCO comparison on simulated non-Gaussian data would test
- Beyond the paper, the relevance curve offers a model-sanity check for functional linear models: discrepancies between $R_f(t)$ and the magnitude of the estimated coefficient $\beta(t)$ would flag regions where the linear approximation is distorted, a diagnostic that currently requires manual inspection of $\hat\beta$.
- Beyond the paper, the difference between relevance curves computed at two partition resolutions could serve as a stability measure, revealing whether the model's signal is localized near specific points or spread over broad regions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a global, model-agnostic interpretability method for scalar-on-function regression. For a trained predictor f and a test sample, it defines a set function ν(S) = ᷈R²(S), the coefficient of determination of f when the functional predictor is reconstructed on Sᶜ by its conditional expectation given the values on S (the ghost-variable idea). Using the asymptotic approach to Shapley values for games with a continuum of players, the authors define an interval Shapley value relevance function R_f(t) via finite interval partitions and random permutations, implement it in the Python package ShapleyFDA, and illustrate it on simulated scenarios and the Tecator data set. The central claim is that R_f(t) is a continuous Shapley value relevance function that identifies the points t most relevant for prediction, without retraining the model.
Significance. If the central claim is secured, the paper would be a useful contribution: it is, to my knowledge, one of the first systematic attempts at global, model-agnostic interpretability for functional predictors, and it offers pointwise relevance curves at no retraining cost. The ghost-variable imputation is a sensible way to avoid leave-one-point-out refitting, and the open-source package plus the reproducible simulation code are valuable assets. The simulations with known ground truth (linear, nonlinear, and discrete targets) give encouraging qualitative evidence that the method identifies relevant regions, and the Tecator example is a convincing illustration. However, the theoretical interpretation as a continuous Shapley value is not established in the manuscript, and several load-bearing assumptions of the continuous-game framework are left unverified.
major comments (4)
- [Section 2.3 and Section 3, Eq. (4)] The paper's central claim is that R_f(t) is a continuous Shapley value relevance function obtained by the asymptotic approach, but the required convergence is never established. Section 2.3 states only that 'under certain conditions' the limit of finite Shapley values over refining partitions exists and is unique, citing Neyman (1994); it does not state those conditions, and Section 3 does not verify them for the specific game ν(S)=᷈R²(S) in Eq. (3). The experiments in Section 5.1 fix n=20 intervals and |Π0|=1000 permutations, so the quantity computed in Eq. (4) is a finite-game Shapley value for one fixed partition. Without a proof that the finite Shapley values converge as n→∞, or at least a stability analysis in n, the pointwise curve R_f(t) is not shown to be a well-defined continuous object independent of the chosen partition and permutation sample.
- [Section 3.3] Section 3.3 shows that the game ν is not a vector measure game, so the diagonal formula cannot be used. This is a useful negative result, but it does not supply the positive membership needed for the asymptotic approach. The displayed expression for ν(S) depends on an S-dependent family of signed measures τ_k^{S+}, τ_k^{S-} and an S-dependent function ρ^S; no argument is given that the resulting game belongs to any class, such as pNA or another space on which the Neyman asymptotic value is known to exist and be unique. Consequently, the structural analysis in Section 3.3 leaves the convergence question open rather than resolving it.
- [Section 2.3 and Eq. (3)] The continuous-game framework in Section 2.3 defines games as nonnegative and monotone set functions ν: B → R+, but the proposed payoff ν(S)=᷈R²(S) is a coefficient of determination, which can be negative for poorly fitting models on subsets S. Indeed, Table 1 reports mean R²(I) values close to 0 for FLM under the nonlinear target, and subset R² values can fall below 0. The paper does not verify that ν is monotone or nonnegative, nor does it argue that the cited value theory extends to such set functions. This is a load-bearing assumption for invoking Aumann–Shapley theory.
- [Section 5.1 and Section 5.2] The experimental section provides no sensitivity analysis with respect to the two approximation choices on which the continuous interpretation depends: the number of intervals n and the number of random permutations |Π0|. All simulation scenarios use n=20 and |Π0|=1000 (5000 for Tecator), and no experiment varies the partition. Since the theoretical claim is about a limit over refining partitions, reporting results for, say, n ∈ {5,10,20,40} and a few permutation counts would be needed to support the claim that the observed curves are stable functional objects rather than artifacts of a particular finite partition.
minor comments (6)
- [Section 3.3] The notation ρ^S({τ_k^{S+}(S), τ_k^{S-}(S): k≥1}) is not precise: ρ^S is said to be defined on ℝ^N, but the argument is a countably infinite sequence of scalar values; the domain should be stated as a sequence space, or the displayed expression should be rewritten accordingly.
- [Section 3.3] The step replacing the finite-sample average by the population variance is written with '≈' and the reconstruction error is assumed to have zero mean. This is plausible given Kneip and Liebl (2020), but the approximation should be stated as a formal assumption or a limiting statement as m→∞, since the subsequent conclusion about the structure of ν rests on it.
- [Section 4] The complexity analysis is useful, but the statement that E(T_O)=E(T_M)=T/2 should be made conditional on the uniform random permutation model; otherwise the expectation over the random subset of intervals used in the Shapley sum is implicit.
- [Acknowledgments] The sentence 'founded by MICIU/AEI/...' should read 'funded by MICIU/AEI/....'
- [Section 5.1] The main-text figures show only the m=200, η=0.05 scenarios and the remaining cases are deferred to Appendix B; it would help the reader if the figure captions stated explicitly that the displayed curves are means over 100 replications and that the corresponding pointwise standard deviations are provided in the appendix.
- [Section 2.3] The definition of an S-admissible sequence uses subalgebras (P_j) and requires S ∈ P_1; when the finite approximations are based on interval partitions, the paper does not connect this definition to the specific choice of partitions in Eq. (4), which would clarify the notational link between ν_P and ν_{I_i}.
Circularity Check
No significant circularity: the relevance function is a defined game-theoretic summary of a fixed prediction model, and the validation uses external ground truth.
full rationale
The paper's central construction is a definition, not a fitted prediction: Eq. (3) defines ν(S)=tilde R^2(S) from a pre-trained model f evaluated on ghost-variable reconstructions, and Eq. (4) computes finite Shapley values of the restricted game, aggregated into R_f(t). No parameter of R_f is tuned to reproduce the relevance curves shown in the simulations; those curves are compared with independently known target transformations (e.g., argmax of beta densities, discrete points), so the empirical validation is external to the method. The ghost-variable idea is taken from Delicado and Peña (2023), a self-citation, but it is not an unverified load-bearing premise: Section 3.1 derives the imputation from Gaussian conditional-expectation formulas and Section 3.2 shows it coincides with the Kneip-Liebl optimal reconstruction operator, an independent external result. The paper's own Section 3.3 shows the game is not a vector measure game, so the diagonal formula cannot be used; Section 2.3 cites Neyman (1994) for existence of an asymptotic value 'under certain conditions' without verifying those conditions for this ν. That is a correctness/completeness gap about convergence to a true continuous Shapley value, not a circular identification: the finite computation does not, by construction, equal the asymptotic value, and the claimed continuous interpretation is asserted rather than derived from the same input. No equation in the paper reduces R_f to its own inputs, and no fitted parameter is renamed as a prediction. Hence the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- number of intervals n =
20
- number of random permutations |Π0| =
1000 (simulations), 5000 (Tecator)
- observation grid size T =
101
assumptions (5)
- standard math Shapley axioms and uniqueness for finite games, and existence and uniqueness of values for non-atomic games under the conditions in Aumann and Shapley (1974) and Neyman (1994).
- domain assumption The functional predictor X follows a Gaussian process, so that E[X(t)|X(u), u in S] is linear in the observed values.
- domain assumption The test data used to compute the payoff ν are independent of the training data, and the prediction model f is fixed before relevance is computed.
- ad hoc to paper The payoff ν(S) = R2(f(\tilde X^S), y) with ghost-variable imputation is a valid measure of predictive relevance for the points in S.
- ad hoc to paper The finite partition and finite permutation approximations converge to the continuous Shapley value for this specific game ν.
Cite this review
Pith. "Pith review of Functional relevance based on the continuous Shapley value." pith.science (2026). https://pith.science/paper/K35YLDL4
@misc{pith2026241118575,
author = {Pith},
title = {Pith review of: Functional relevance based on the continuous Shapley value},
year = {2026},
howpublished = {\url{https://pith.science/paper/K35YLDL4}},
note = {Machine review of arXiv:2411.18575}
}
read the original abstract
The presence of artificial intelligence (AI) in our society is increasing, which brings with it the need to understand the behavior of AI mechanisms, including machine learning predictive algorithms fed with tabular data, text or images, among others. This work focuses on interpretability of predictive models based on functional data. Designing interpretability methods for functional data models implies working with a set of features whose size is infinite. In the context of scalar on function regression, we propose an interpretability method based on the Shapley value for continuous games, a mathematical formulation that allows for the fair distribution of a global payoff among a continuous set of players. The method is illustrated through a set of experiments with simulated and real data sets. The open source Python package ShapleyFDA is also presented.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
R. J. Aumann and L. S. Shapley. Values of Non-Atomic Games . Princeton University Press, 1974. ISBN 978-0-691-64546-9
work page 1974
-
[2]
Robert J. Aumann. Markets with a Continuum of Traders . Econometrica, 32: 0 39--50, 1964. doi:10.2307/1913732
doi:10.2307/1913732 1964
-
[3]
Alejandro Barredo Arrieta , Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, et al. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI . Information Fusion, 58: 0 82--115, 2020. doi:10.1016/j.inffus.2019.12.012
-
[4]
Explanatory model analysis: Explore, explain and examine predictive models
P Biecek and T Burzykowski. Explanatory model analysis: Explore, explain and examine predictive models . Chapman and Hall/CRC, 2021. ISBN 978-0-367-13559-1
work page 2021
-
[5]
Distance-based local linear regression for functional predictors
Eva Boj, Pedro Delicado, and Josep Fortiana. Distance-based local linear regression for functional predictors . Computational Statistics & Data Analysis, 54 0 (2): 0 429--437, 2010. doi:10.1016/j.csda.2009.09.010
-
[6]
Optimal minimal neural interpretation of spectra
Claus Borggaard and Hans Henrik Thodberg. Optimal minimal neural interpretation of spectra . Analytical Chemistry, 64 0 (5): 0 545--551, 1992. doi:10.1021/ac00029a018
-
[7]
Statistical Modeling: The Two Cultures
Leo Breiman. Statistical Modeling: The Two Cultures . Statistical Science, 16 0 (3): 0 199--231, 2001. doi:10.1214/ss/1009213726
arXiv 2001
-
[8]
A new model for counterfactual analysis for functional data
Emilio Carrizosa, Jasone Ramírez-Ayerbe, and Dolores Romero Morales. A new model for counterfactual analysis for functional data . Advances in Data Analysis and Classification, 18 0 (4): 0 981--1000, 2024. doi:10.1007/s11634-023-00563-5
Show all 40 references
-
[9]
Cohen, Gideon Dror, and Eytan Ruppin
Shay B. Cohen, Gideon Dror, and Eytan Ruppin. Feature Selection via Coalitional Game Theory . Neural Computation, 19 0 (7): 0 1939--1961, 2007. doi:10.1162/neco.2007.19.7.1939
1939 doi
-
[10]
Crainiceanu, Jeff Goldsmith, Andrew Leroux, and Erjia Cui
Ciprian M. Crainiceanu, Jeff Goldsmith, Andrew Leroux, and Erjia Cui. Functional Data Analysis with R . Chapman and Hall/CRC, 2024. ISBN 978-1-032-24472-3
2024
-
[11]
Understanding complex predictive models with ghost variables
Pedro Delicado and Daniel Peña. Understanding complex predictive models with ghost variables . TEST, 32 0 (1): 0 107--145, 2023. doi:10.1007/s11749-022-00826-x
2023 doi
-
[12]
R. W. Farebrother. Linear least squares computations . Routledge, 1988. ISBN 978-0-824-77661-9. doi:10.1201/9780203748923
1988 doi
-
[13]
Barry E. Feldman. Relative Importance and Value . Available at SSRN Electronic Journal, doi: 10.2139/ssrn.2255827, 2005
2005 doi
-
[14]
Ferraty and P
F. Ferraty and P. Vieu. Nonparametric Functional Data Analysis: Theory and Practice . Springer, 2006. ISBN 978-0-387-30369-7
2006
-
[15]
Jan Gertheiss, David Rügamer, Bernard X. W. Liew, and Sonja Greven. Functional Data Analysis: An Introduction and Recent Developments . Biometrical Journal, 66 0 (7): 0 e202300363, 2024. doi:10.1002/bimj.202300363
2024 doi
-
[16]
Goldberg, Y
Y. Goldberg, Y. Ritov, and A. Mandelbaum. Predicting the continuation of a function with applications to call center data . Journal of Statistical Planning and Inference, 147: 0 53--65, 2014. doi:10.1016/j.jspi.2013.11.006
2014 doi
-
[17]
Variable Importance Assessment in Regression: Linear Regression versus Random Forest
Ulrike Grömping. Variable Importance Assessment in Regression: Linear Regression versus Random Forest . The American Statistician, 63 0 (4): 0 308--319, 2009. doi:10.1198/tast.2009.08199
2009 arXiv
-
[18]
Harris, K
Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, et al. Array programming with NumPy . Nature, 585 0 (7825): 0 357--362, 2020. doi:10.1038/s41586-020-2649-2
2020 doi
-
[19]
Values of non-atomic vector measure games: Are they linear combinations of the measures? Journal of Mathematical Economics, 17 0 (1): 0 31--40, 1988
Sergiu Hart and Abraham Neyman. Values of non-atomic vector measure games: Are they linear combinations of the measures? Journal of Mathematical Economics, 17 0 (1): 0 31--40, 1988. doi:10.1016/0304-4068(88)90025-0
1988 doi
-
[20]
Functional neural networks: Shift invariant models for functional data with applications to EEG classification
Florian Heinrichs, Mavin Heim, and Corinna Weber. Functional neural networks: Shift invariant models for functional data with applications to EEG classification . In Proceedings of the 40th International Conference on Machine Learning , pages 12866--12881. PMLR, 2023
2023
-
[21]
Hooker, L
G. Hooker, L. Mentch, and S. Zhou. Unrestricted permutation forces extrapolation: variable importance requires at least one more model, or there is no free variable importance . Statistics and Computing, 31 0 (6): 0 82, 2021. doi:10.1007/s11222-021-10057-z
2021 doi
-
[22]
Horváth and P
L. Horváth and P. Kokoszka. Inference for Functional Data with Applications . Springer, 2012. ISBN 978-1-4614-3654-6
2012
-
[23]
James, Jing Wang, and Ji Zhu
Gareth M. James, Jing Wang, and Ji Zhu. Functional linear regression that's interpretable . The Annals of Statistics, 37 0 (5A): 0 2083--2108, 2009. doi:10.1214/08-AOS641
2009 doi
-
[24]
Values of games with a continuum of players
Yakar Kannai. Values of games with a continuum of players . Israel Journal of Mathematics, 4 0 (1): 0 54--58, 1966. doi:10.1007/BF02760070
1966 doi
-
[25]
On the optimal reconstruction of partially observed functional data
Alois Kneip and Dominik Liebl. On the optimal reconstruction of partially observed functional data . The Annals of Statistics, 48 0 (3): 0 1692--1717, 2020. doi:10.1214/19-AOS1864
2020 doi
-
[26]
Kokoszka and M
P. Kokoszka and M. Reimherr. Introduction to Functional Data Analysis . Chapman & Hall/CRC, 2017. ISBN 978-1-498-74634-2
2017
-
[27]
Components and completion of partially observed functional data
David Kraus. Components and completion of partially observed functional data . Journal of the Royal Statistical Society: Series B (Statistical Methodology), 77 0 (4): 0 777--801, 2015. doi:10.1111/rssb.12087
2015 doi
-
[28]
Analysis of regression in game theory approach
Stan Lipovetsky and Michael Conklin. Analysis of regression in game theory approach . Applied Stochastic Models in Business and Industry, 17 0 (4): 0 319--330, 2001. doi:10.1002/asmb.446
2001 doi
-
[29]
Interpretable Machine Learning with Python: Learn to build interpretable high-performance models with hands-on real-world examples
Serg Masís. Interpretable Machine Learning with Python: Learn to build interpretable high-performance models with hands-on real-world examples . Packt Publishing, 2021. ISBN 978-1800203907
2021
-
[30]
Interpretable Machine Learning: A Guide for Making Black Box Models Explainable
Christoph Molnar. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable . Independently published, 2022. ISBN 979-8-411-46333-0
2022
-
[31]
A. Neyman. Value of Games with a Continuum of Players , chapter VI, pages 67--79. Springer, 1994. ISBN 978-0-7923-3011-0. doi:10.1007/978-94-017-1656-7_7
1994 doi
-
[32]
The Python Language Reference
Python Core Team . The Python Language Reference . Python Software Foundation, 2021. URL https://docs.python.org/3.10/reference/index.html
2021
-
[33]
scikit-fda: A Python Package for Functional Data Analysis
Carlos Ramos-Carreño, José Luis Torrecilla, Miguel Carbajo-Berrocal, Pablo Marcos, and Alberto Suárez. scikit-fda: A Python Package for Functional Data Analysis . Journal of Statistical Software, 109 0 (2): 0 1--37, 2024. doi:10.18637/jss.v109.i02
2024 doi
-
[34]
Ramsay and B.W
J. Ramsay and B.W. Silverman. Functional Data Analysis . Springer, 2005. ISBN 978-0-387-40080-8
2005
-
[35]
Nonlinear Functional Modeling Using Neural Networks
Aniruddha Rajendra Rao and Matthew Reimherr. Nonlinear Functional Modeling Using Neural Networks . Journal of Computational and Graphical Statistics, 32 0 (4): 0 1248--1257, 2023. doi:10.1080/10618600.2023.2165498
2023
-
[36]
Reiss, Jeff Goldsmith, Han Lin Shang, and R
Philip T. Reiss, Jeff Goldsmith, Han Lin Shang, and R. Todd Ogden. Methods for Scalar-on-Function Regression . International Statistical Review, 85 0 (2): 0 228--249, 2017. doi:10.1111/insr.12163
2017 doi
-
[37]
A Value for n-Person Games
Lloyd S Shapley. A Value for n-Person Games . Contributions to the Theory of Games, 2 0 (28): 0 307--317, 1953. doi:10.1515/9781400881970-018
1953 doi
-
[38]
Lloyd S. Shapley. Values of Games With Infinitely Many Players , 1961. Research memorandum. RAND Corporation, doi: 10.7249/RM2912
1961 doi
-
[39]
The Shapley Value
Eyal Winter. The Shapley Value . In Robert Aumann and S. Hart, editors, Handbook of Game Theory with Economic Applications , volume 3, chapter 53, pages 2025--2054. Elsevier, 2002. ISBN 978-0-444-89428-1. doi:10.1016/S1574-0005(02)03016-3
2025 doi
-
[40]
Deep learning for functional data analysis with adaptive basis layers
Junwen Yao, Jonas Mueller, and Jane-Ling Wang. Deep learning for functional data analysis with adaptive basis layers . In International Conference on Machine Learning , pages 11898--11908. PMLR, 2021
2021
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.