Pith. sign in

REVIEW 4 major objections 6 minor 28 references

Tab-Shapley: Identifying Top-k Tabular Data Quality Insights

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Tab-Shapley claims that anomalies in tabular data can be aggregated into ranked top-k blocks through a superadditive cooperative game whose Shapley values collapse to a single closed-form sum, computable in O(mn) time.

desk verdict New problem framing and a correct but standard Shapley formula, undermined by an empirical evaluation that counts the same TabNet labels used to build the evidence sets. read the letter →

arxiv 2501.06685 v1 pith:Z6SQU7VX submitted 2025-01-12 cs.LG stat.ML

Manisha Padala , Lokesh Nagalapatti , Atharv Tyagi , Ramasuri Narayanam , Shiv Kumar Saini This is my paper · ORCID
classification cs.LGstat.ML
keywords tabulardataqualityanomalydetectionShapleyvaluecooperativegametheorytop-kinsightsunsupervisedlearningblockautoencoderreconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Tab-Shapley claims that the messy problem of locating concentrated anomalies in a table can be recast as a cooperative game over attributes, where each attribute is assigned the records that do not look anomalous in it. In this game the Shapley value of an attribute has a closed form: a sum over its evidence records of the reciprocal of how many attributes share that record. Lower scores flag attributes as more anomalous, and reordering rows and columns by these scores brings flagged cells into compact top-left blocks. Extracting the top-k such blocks yields prioritized data quality insights in time linear in the table size. A sympathetic reader would care because the same machinery could turn any binary cell-quality matrix into an actionable ranked list of where to look first.

What carries the argument

The load-bearing object is the evidence-set cooperative game $(A, V_a)$: players are the attributes, and $V_a(S) = |\cup_{a_j \in S} E_{a_j}|$, the count of distinct records covered by the evidence sets in the coalition. The closed-form identity of Lemma 1, $\phi_a(a_j)=\sum_{X_i\in E_{a_j}} \frac{1}{|\{k : X_i \in E_{a_k}\}|}$, converts every record into a fractional non-anomaly vote split equally among the attributes that consider it healthy. This identity carries the whole argument: it turns an exponential coalition enumeration into a single O(mn) pass, and its inversion produces the anomaly rankings that feed the block-extraction step.

What would settle it

Run Tab-Shapley on a synthetic table with known injected anomalies, using cell labels from an independent detector different from the one in Section 4.1, and measure whether the top-k block still captures significantly more flagged cells than random row and column permutations; if the concentration disappears, the reported insight is an artifact of the label-generation step rather than of the Shapley aggregation.

Watch

Extended reading notes

Core claim

The paper's central claim is that anomalies in a tabular dataset can be aggregated into ranked, human-consumable blocks by defining a cooperative game on the attributes and reading off closed-form Shapley values. In this game an attribute's evidence set is the records where that attribute was not flagged as a potential anomaly, and a coalition's payoff is the number of distinct records covered by the evidence sets of its members. Because this game is superadditive, the Shapley value of an attribute reduces to summing, over its own evidence records, the reciprocal of the number of attributes that share each record; inverting these scores ranks attributes and reorders rows and columns so that flagged cells accumulate in the top-left. The paper claims this closed form makes top-k block extraction correct and O(mn), and that on real-world labeled datasets the resulting blocks hold more anomalous cells than an unsupervised isolation-forest ranking.

Load-bearing premise

The thresholded reconstruction-error labels that mark each cell PA or NA must actually track true anomalies, because every Shapley score and every top-k block on top of them inherits the accuracy of those labels.

Editorial extensions

If this is right

  • Ranking attributes by the inverted closed-form Shapley values requires only O(mn) time, so the method scales to wide and long tables.
  • The top-k blocks concentrate flagged cells better than the isolation-forest baseline on the evaluated datasets, and match a supervised feature-attribution baseline on many of them.
  • Because the game is superadditive and the closed form is exact, no coalition enumeration is needed for deployment.
  • Attribute scores correlate with both the size of evidence sets and the number of distinct records in them, supporting the two stated criteria.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The closed form is not specific to reconstruction errors: any binary matrix of flagged cells yields the same game, so the aggregation is a general block-localization layer that could sit on top of any detector.
  • The maximum-subarray extraction is one possible block aggregator; replacing it with connected-component or density-based block summaries would test whether the Shapley ranking or the block post-processing drives the reported concentration.
  • A direct human study of whether the returned insights match the actual source of data-quality failures would settle practical value better than counting flagged cells.
Share X Bluesky LinkedIn Reddit HN

Formalized claims in Lean

  1. Claim #1: The paper's central claim is that anomalies in a tabular dataset can be aggregated into ranked, human-consumable blocks by defining a cooperative game on the attributes and reading off closed-form Shapley values. In this game an attribute's evidence set is the records where that attribute was not flagged as a potential anomaly, and a coalition's payoff is the number of distinct records covered by

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Tab-Shapley, an unsupervised method for deriving top-k "data quality insights" from tabular data, where an insight is a block of anomalous attributes and the records that exhibit the anomaly. The pipeline is: (i) obtain cell-level PA/NA labels from TabNet reconstruction errors thresholded by k-means (Section 4.1); (ii) define evidence sets as the NA cells for each attribute and record (Definitions 1 and 2); (iii) form a coverage game over attributes and compute Shapley values in closed form (Lemma 1); (iv) reorder rows and columns by the inverted Shapley scores and extract top-k maximum-sum submatrices via Kadane's algorithm (Algorithm 2, Section 4.4). The method is evaluated on 12 real-world datasets and two additional datasets, comparing the number of PA cells in top-left k×k blocks against DIFFI and SHAP baselines (Figures 3-5, Table 3).

Significance. The problem of surfacing interpretable, prioritized anomaly blocks is practically relevant, and the closed-form Shapley computation in Lemma 1 is mathematically correct under the stated definitions: for a coverage game, the Shapley value of a player is the sum over the elements it covers of the reciprocal of the number of players covering that element. This gives an O(mn) algorithm for the attribute-level scores, which is a genuine efficiency gain over the exponential permutation formula. The paper also makes a useful connection between game-theoretic feature importance and matrix-reordering for data quality inspection. However, the current empirical validation is not independent: the evaluation counts the same TabNet-derived PA labels that define the evidence sets and that Algorithm 2 is designed to concentrate, so the claimed superiority over baselines does not yet establish that the discovered blocks correspond to real anomalies. In addition, the row-level half of the method is underspecified, and one stated proposition is false as written. These issues are fixable, but they are load-bearing for the paper's central claims.

major comments (4)
  1. [Section 5.1] The quantitative evaluation is circular. The text says: "First, we generate matrices that label each cell as PA ... using the TabNet-based approach described in Section 4.1," and Section 4.1 is exactly the source of the PA/NA labels used to construct the evidence sets in Definition 1 and to initialize the scoring matrix S in Algorithm 2 (PA cells receive 1 - i*j/(m*n)). Counting these same PA cells in the top-left k×k block therefore measures how well Algorithm 2 reorders the table to maximize its own objective, not whether the blocks correspond to true anomalies. The figure captions in Figures 3-5 call these "ground-truth anomalies," but they are model predictions. The evaluation should instead use the independent attribute-level ground-truth labels mentioned in the Datasets paragraph of Section 5, or otherwise clearly distinguish the labels used to build the method from the labels used to evaluate it.
  2. [Section 4.4] The row-level Shapley values used to reorder rows are never defined or derived. Section 4.4 states that "attributes and records with lower Shapley values" are rearranged to create the block structures, and the visual and quantitative results in Figures 1, 3-5 depend on the row ordering. However, Algorithm 1 returns only attribute-level scores, and neither the record-level cooperative game, its characteristic function, nor a closed-form expression for record Shapley values is given anywhere in the paper. Without this, the row ordering, and hence the top-k blocks, are not reproducible from the manuscript.
  3. [Proposition 1] Proposition 1 claims that the cooperative game (A, Va) with Va(S) = |∪_{aj∈S} E_aj| is super-additive under the definition given in Section 3. This is false for coverage functions: take two attributes with E_a = E_b = {X}; then Va({a}) = 1, Va({b}) = 1, and Va({a,b}) = 1, which violates Va({a,b}) ≥ Va({a}) + Va({b}). The closed-form Lemma 1 is nevertheless true for coverage games and does not require super-additivity, so the proposition should be corrected or removed.
  4. [Example 1] The arithmetic in Example 1 is inconsistent with the stated evidence sets. For attribute C4, the evidence set is EC4 = {R1, R2, R5, R6}; R2 appears in EC1, EC2, and EC4, so its contribution is 1/3, not 1/2. The correct value is phi(C4) = 1/4 + 1/3 + 1/4 + 1/3 = 1.1667, not 1.33. Consequently, the anomaly ordering should be C5, C2, C4, C1, C3, not C5, C2, C1, C4, C3 as stated. The other listed values should be rechecked as well.
minor comments (6)
  1. [Definitions 1 and 2] The set-builder notation in Definitions 1 and 2 says "for any i" and "for any j" inside the defining condition, which is confused; the bound variables i and j are already fixed by the set-builder itself. These should be written, for example, as E_aj = {Xi : Lij = NA}.
  2. [Lemma 1 proof] The proof of Lemma 1 has notational problems: Va(P_R^aj) is a cardinality, so expressions like Va(P_R^aj) ∪ {Xi} are not well-formed set operations, and the indicator I_{Xi not in P_R^aj} compares a record to a set of players. The argument should be rewritten in terms of whether Xi is already covered by the union of the evidence sets of the preceding attributes.
  3. [Section 5.1 and Figures 3-5] The text refers to "ground-truth PA labels," while the labels are in fact produced by the TabNet model of Section 4.1; the figure captions should be aligned with the text and should not call these labels ground truth.
  4. [Section 5.1, Baselines] The paper repeatedly calls SHAP a "supervised algorithm," which is imprecise: SHAP is a model-agnostic explanation method. If a supervised anomaly classifier is trained and then explained with SHAP, that classifier and the target labels used to train it should be specified exactly, since this affects the fairness of the comparison.
  5. [Section 4.3] In the computational complexity paragraph, the text refers to "Lemma 4.3" where it should refer to "Lemma 1."
  6. [Algorithm 2] The description of extracting top-k maximum-sum submatrices with Kadane's algorithm is brief; the paper should state how the 2D case is handled and give the complexity, since the cited reference is for the 1D maximum subarray problem.

Circularity Check

1 steps flagged · score 6.0 of 10

Empirical concentration claim is partly circular: the PA labels used to build evidence sets and the scoring matrix are the same TabNet-derived labels counted in the top-k metric.

  1. fitted input called prediction [Section 4.1 (cell labeling), Definition 1, Algorithm 2, Section 5.1 (Quantitative Evaluation), Figures 3-5]
    "First, we generate matrices that label each cell as PA (Possibly Anomalous) or NA (Not Anomalous) using the TabNet-based approach described in Section 4.1. We propose a metric that counts the number of PA cells in k × k blocks starting from the top-left."

    The TabNet-derived labels L_ij are the same labels that define the evidence sets E_aj in Definition 1, which drive the Shapley scores in Lemma 1 and determine the row/column reordering. Algorithm 2 then re-encodes those same labels into the scoring matrix (if L[i,j] == PA then S[i,j] = 1 - i*j/(n*m)) and selects blocks by maximizing S. Counting PA cells from the same TabNet label source in the reordered top-left k×k block therefore measures the optimizer's own objective, not an independent ground truth. The figure captions call these 'ground-truth PA labels', but the text shows they are the model's own predictions, so the claimed concentration advantage over DIFFI/SHAP is partly by construction.

full rationale

The theoretical core is not circular: Lemma 1 derives a closed-form Shapley value for the coverage game (A, Va) with Va(S)=|union_{aj in S} E_aj|, and the proof is a valid permutation-counting argument. The game and the closed form are internally consistent and independent of the empirical evaluation. No load-bearing self-citations or imported uniqueness theorems are used. The circularity is confined to the empirical validation of the central claim that Tab-Shapley concentrates anomalies. Section 5.1 explicitly regenerates the PA/NA labeling with the TabNet procedure of Section 4.1, the very labels that construct the evidence sets and the scoring matrix. Consequently, counting these labels in the top-left blocks is a self-consistency check rather than an external benchmark, even though the paper refers to them as ground-truth anomalies. The datasets described in the Datasets paragraph do provide independent ground-truth labels, but the quantitative protocol in Section 5.1 does not state that it uses them; it states it uses the TabNet-based matrices. An independent evaluation would count externally labeled anomalous cells, or report precision/recall against the dataset ground truth. A minor arithmetic slip in Example 1 (R2 appears in three evidence sets, not two) does not affect circularity but reinforces the need for careful numerical checking. Overall score 6 rather than 8 because the Shapley derivation itself remains a valid, self-contained contribution; only the empirical concentration claim partially reduces to the method's own fitted labels.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The Shapley computation itself is parameter-free once evidence sets are fixed. All free parameters live upstream in the TabNet labeling pipeline (thresholds, clusters, alpha) and downstream in user choice of K. The mathematical axioms are standard; the main domain assumption is that reconstruction-error labels reflect true anomalies.

free parameters (4)
  • record-level anomaly threshold = implicit via 2-cluster k-means on reconstruction losses
    Section 4.1 clusters record-level losses to separate anomalous from normal records; the split point is fit to the data.
  • attribute-level cell label threshold = 2-cluster k-means per anomalous record
    Section 4.1 clusters cell losses for each anomalous record to assign PA/NA labels; no fixed threshold is given.
  • alpha (alpha) = not specified per experiment (0.2 in Figure 2)
    Algorithm 2 uses alpha to weight non-anomalous cells; the paper does not report how alpha was set for the main comparisons.
  • TabNet training hyperparameters = not specified
    The autoencoder that produces reconstruction errors is trained with unspecified architecture details, learning rates, and epochs.
assumptions (4)
  • standard math The coverage game (A, Va) with Va(S)=|union of E_aj for aj in S| is super-additive.
    True for any coverage function; the Shapley value formula then applies.
  • domain assumption Shapley's four axioms (efficiency, symmetry, null player, additivity) are the correct way to score attributes and records.
    The paper chooses cooperative game theory to aggregate anomaly evidence; this is a modeling choice, not forced by the problem.
  • ad hoc to paper TabNet reconstruction errors, thresholded by k-means, yield valid cell-level anomaly labels Lij.
    Section 4.1 defines the entire label generation pipeline; no ground-truth validation of these labels is provided.
  • domain assumption Evidence-set size and unique-record coverage are the right notions of non-anomalousness for ranking attributes.
    Criteria 1 and 2 in Section 4.2 are asserted; Table 2 shows correlation with the Shapley scores, but that correlation is largely a property of the formula.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tab-Shapley: Identifying Top-k Tabular Data Quality Insights." pith.science (2026). https://pith.science/paper/Z6SQU7VX

@misc{pith2026250106685,
  author       = {Pith},
  title        = {Pith review of: Tab-Shapley: Identifying Top-k Tabular Data Quality Insights},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z6SQU7VX}},
  note         = {Machine review of arXiv:2501.06685}
}
read the original abstract

We present an unsupervised method for aggregating anomalies in tabular datasets by identifying the top-k tabular data quality insights. Each insight consists of a set of anomalous attributes and the corresponding subsets of records that serve as evidence to the user. The process of identifying these insight blocks is challenging due to (i) the absence of labeled anomalies, (ii) the exponential size of the subset search space, and (iii) the complex dependencies among attributes, which obscure the true sources of anomalies. Simple frequency-based methods fail to capture these dependencies, leading to inaccurate results. To address this, we introduce Tab-Shapley, a cooperative game theory based framework that uses Shapley values to quantify the contribution of each attribute to the data's anomalous nature. While calculating Shapley values typically requires exponential time, we show that our game admits a closed-form solution, making the computation efficient. We validate the effectiveness of our approach through empirical analysis on real-world tabular datasets with ground-truth anomaly labels.

Figures

Figures reproduced from arXiv: 2501.06685 by the authors.

Figure 1
Figure 1. Top-k insights: Darker cells indicate anomaly. The blocks that are filled with blue patterns show the top-K insights for K = 3. The results are shown for α = 0.2; higher values of α would create smaller blocks [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Impact of α on NA cells contained in the top-1 insight. tion errors may provide useful information for identify￾ing anomalous attributes. To cater to them, we propose a weighted variant of the Tab-Shapley method which is di￾rectly computed on top of the reconstruction error. 4.4 Computing Top-k Data Insights First, we note that the attributes and records with lower Shapley values are more likely to be anomalous, all… view at source ↗
Figure 3
Figure 3. Number of ground-truth anomalies captured in the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Number of ground-truth anomalies captured in the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 4
Figure 4. Figure 4: Number of ground-truth anomalies captured in the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 24 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Amarasinghe, K.; Kenney, K.; and Manic, M. 2018. Toward explainable deep neural network based anomaly detection. In HSI, 311--317. IEEE

    work page 2018

  4. [4]

    M.; Shapira, B.; and Rokach, L

    Antwarg, L.; Miller, R. M.; Shapira, B.; and Rokach, L. 2021. Explaining anomalies detected by autoencoders using Shapley Additive Explanations. Expert Systems with Applications, 186: 115736

    work page 2021

  5. [5]

    O.; and Pfister, T

    Ar k, S. O.; and Pfister, T. 2021. Tabnet: Attentive interpretable tabular learning. In AAAI, volume 35, 6679--6687

    work page 2021

  6. [6]

    Bailis, P.; Gan, E.; Madden, S.; Narayanan, D.; Rong, K.; and Suri, S. 2017. Macrobase: Prioritizing attention in fast data. In Proceedings of the 2017 ACM International Conference on Management of Data, 541--556

    work page 2017

  7. [7]

    Carletti, M.; Masiero, C.; Beghi, A.; and Susto, G. A. 2019. Explainable machine learning in industry 4.0: Evaluating feature importance in anomaly detection to enable root cause analysis. In SMC, 21--26. IEEE

    work page 2019

  8. [8]

    Chandola, V.; Banerjee, A.; and Kumar, V. 2009. Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3): 1--58

    work page 2009

Show all 28 references
  1. [9]

    Gemp, I.; McWilliams, B.; Vernade, C.; and Graepel, T. 2021. EigenGame: PCA as a Nash Equilibrium. In ICLR

    2021

  2. [10]

    Ghorbani, A.; and Zou, J. 2019 a . Data shapley: Equitable valuation of data for machine learning. In ICML, 2242--2251. PMLR

    2019

  3. [11]

    Ghorbani, A.; and Zou, J. Y. 2019 b . Data Shapley: Equitable Valuation of Data for Machine Learning. In ICML, 2242--2251

    2019

  4. [12]

    Huang, Z.; and He, Y. 2018. Auto-Detect: Data-Driven Error Detection in Tables. In SIGMOD, 1377--1392

    2018

  5. [13]

    T.; Ting, K

    Liu, F. T.; Ting, K. M.; and Zhou, Z.-H. 2008. Isolation forest. In ICDM, 413--422. IEEE

    2008

  6. [14]

    A.; Franks, B

    Liznerski, P.; Ruff, L.; Vandermeulen, R. A.; Franks, B. J.; Kloft, M.; and M \"u ller, K.-R. 2020. Explainable deep one-class classification. arXiv preprint arXiv:2007.01760

    2020 arXiv

  7. [15]

    M.; and Lee, S.-I

    Lundberg, S. M.; and Lee, S.-I. 2017. A Unified Approach to Interpreting Model Predictions. In Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; and Garnett, R., eds., NeuIPS, 4765--4774. Curran Associates, Inc

    2017

  8. [16]

    McCamish, B.; Ghadakchi, V.; Termehchy, A.; Touri, B.; Sanchez, E.; Huang, L.; and Changpinyo, S. 2020. A Game-theoretic Approach to Data Interaction. ACM Transactions on Database Systems, 45(1): 1--44

    2020

  9. [17]

    Mohammed, N.; Benjamin, C.; and Debbabi, M. 2011. Anonymity meets game theory: secure data integration with malicious participants. VLDB Journal, 20(4): 567--588

    2011

  10. [18]

    Monderer, L., Dov; Shapley. 1996. Potential Games. Games and Economic Behavior, 14: 124--143

    1996

  11. [19]

    Myerson, R. B. 1997. Game Theory: Analysis of Conflict. Cambridge, Massachusetts, USA: Harvard University Press

    1997

  12. [20]

    P.; Lim, K

    Nguyen, Q. P.; Lim, K. W.; Divakaran, D. M.; Low, K. H.; and Chan, M. C. 2019. Gee: A gradient-based explainable variational autoencoder for network anomaly detection. In CNS, 91--99. IEEE

    2019

  13. [21]

    Pang, G.; Ding, C.; Shen, C.; and Hengel, A. v. d. 2021 a . Explainable deep few-shot anomaly detection with deviation networks. arXiv preprint arXiv:2108.00462

    2021 arXiv

  14. [22]

    Pang, G.; Shen, C.; Cao, L.; and Hengel, A. v. d. 2021 b . Deep Learning for Anomaly Detection: A Review. ACM Computing Surveys, 54(2):38:1-38:38

    2021

  15. [23]

    Pang, G.; Shen, C.; Cao, L.; and Hengel, A. V. D. 2021 c . Deep learning for anomaly detection: A review. ACM Computing Surveys (CSUR), 54(2): 1--38

    2021

  16. [24]

    Shapley, L. 1971. Cores of Convex Games. Int Journal of Game Theory, 1: 11--26

    1971

  17. [25]

    Straffin, P. 1993. Game Theory and Strategy. Washington, DC, USA: The Mathematical Association of America

    1993

  18. [26]

    Tamaki, H.; and Tokuyama, T. 1998. Algorithms for the Maximum Subarray Problem Based on Matrix Multiplication. In SODA'98, SODA '98, 446–452. USA: Society for Industrial and Applied Mathematics. ISBN 0898714109

    1998

  19. [27]

    Wan, Z.; Vorobeychik, Y.; Xia, W.; Liu, Y.; Wooders, M.; Guo, J.; Yin, Z.; Clayton, E.; Kantarcioglu, M.; and Malin, B. 2021. Using game theory to thwart multistage privacy intrusions when sharing data. Science Advances, 7

    2021

  20. [28]

    Xu, H.; Wang, Y.; Jian, S.; Huang, Z.; Wang, Y.; Liu, N.; and Li, F. 2021. Beyond outlier detection: Outlier interpretation by attention-guided triplet deviation network. In Proceedings of the Web Conference 2021, 1328--1339

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.