{"id":"6e7027bf-a639-445d-9a03-f1a0305f5fea","arxiv_id":"2501.06685","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Tab-Shapley ranks attributes and records by Shapley values of a coverage game and returns the top-k rectangular blocks most concentrated with anomalous cells.","lead":"Tab-Shapley uses Shapley values from a simple game to rank which attributes and rows in a table are most anomalous, then returns the top-k rectangular blocks that pack the most anomalies. It could help data engineers find dirty data fast, but the experiments may count the same labels the method uses to build its blocks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Circular evaluation: Section 5.1 counts the same TabNet-derived PA labels used to build evidence sets, so the claimed concentration gains are not independent evidence.","rationale":"The reader's weakest assumption—that the TabNet-derived Lij labels are accurate enough—is correct, and the strongest concern is not merely noise in those labels but their reuse in the evaluation metric. Section 5.1 explicitly says the PA matrices are generated using the Section 4.1 TabNet approach, making the top-k concentration measure circular: the method orders the data to concentrate those labels, then is scored by counting those same labels. This undermines the paper's key empirical differentiator. The Shapley closed-form derivation (Lemma 1) is mathematically sound and O(mn) complexity is plausible, so the theoretical contribution is not the blocking issue. However, the empirical validation is the basis for the claimed practical advantage over DIFFI/SHAP, and it is not trustworthy as written. The arithmetic error in Example 1 is minor but consistent with the pattern of verification problems. I therefore agree with the reader's REJECT verdict at moderate confidence; the paper would need independent evaluation on true cell-level labels, plus clearer separation of predicted and ground-truth labels, before the central claim could be accepted.","tokens_in":10778,"tokens_out":3122,"duration_ms":30508,"concrete_test":"Re-run the §5.1 evaluation using the independent attribute-level anomaly ground truth from the Xu et al. (2021) COPOD datasets instead of the TabNet-derived PA labels; recompute Figures 3–5 and Table 3. If the Tab-Shapley gains over DIFFI/SHAP persist on true labels, the circularity objection is resolved; if they shrink or disappear, the central empirical claim is unsupported. As a secondary check, independently recompute Example 1 from the stated evidence sets to confirm the closed-form arithmetic.","verdict_should_be":"REJECT","load_bearing_attack":"The central empirical claim is that Tab-Shapley concentrates anomalous cells in top-k blocks better than DIFFI and SHAP (Figures 3–5, Table 3). Section 4.1 produces cell labels Lij via TabNet reconstruction errors with k-means thresholding; these labels define the evidence sets Eaj in Definition 1 and drive the Shapley score in Lemma 1. Section 5.1 then says: 'First, we generate matrices that label each cell as PA ... using the TabNet-based approach described in Section 4.1. We propose a metric that counts the number of PA cells in k×k blocks...' The figure captions call these 'ground-truth' PA labels, but the text indicates the PA matrices are the same TabNet predictions. Because Algorithm 2 reorders rows and columns using Shapley values computed from those exact labels, counting those labels in the reordered top-left block is partly tautological: an ordering built to group a label will concentrate that label. The reported advantage over DIFFI/SHAP therefore does not establish that the discovered blocks correspond to real anomalies unless an independent label source is used. A secondary issue: Example 1 contains an arithmetic error—R2 appears in EC1, EC2, and EC4 (three evidence sets, not two), so phi(C4) = 1/4 + 1/3 + 1/4 + 1/3 ≈ 1.17, not 1.33. This is minor, but it underscores the lack of careful numerical checking in the paper.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Tab-Shapley, an unsupervised method for deriving top-k \"data quality insights\" from tabular data, where an insight is a block of anomalous attributes and the records that exhibit the anomaly. The pipeline is: (i) obtain cell-level PA/NA labels from TabNet reconstruction errors thresholded by k-means (Section 4.1); (ii) define evidence sets as the NA cells for each attribute and record (Definitions 1 and 2); (iii) form a coverage game over attributes and compute Shapley values in closed form (Lemma 1); (iv) reorder rows and columns by the inverted Shapley scores and extract top-k maximum-sum submatrices via Kadane's algorithm (Algorithm 2, Section 4.4). The method is evaluated on 12 real-world datasets and two additional datasets, comparing the number of PA cells in top-left k×k blocks against DIFFI and SHAP baselines (Figures 3-5, Table 3).","tokens_in":11095,"tokens_out":9304,"duration_ms":87676,"significance":"The problem of surfacing interpretable, prioritized anomaly blocks is practically relevant, and the closed-form Shapley computation in Lemma 1 is mathematically correct under the stated definitions: for a coverage game, the Shapley value of a player is the sum over the elements it covers of the reciprocal of the number of players covering that element. This gives an O(mn) algorithm for the attribute-level scores, which is a genuine efficiency gain over the exponential permutation formula. The paper also makes a useful connection between game-theoretic feature importance and matrix-reordering for data quality inspection. However, the current empirical validation is not independent: the evaluation counts the same TabNet-derived PA labels that define the evidence sets and that Algorithm 2 is designed to concentrate, so the claimed superiority over baselines does not yet establish that the discovered blocks correspond to real anomalies. In addition, the row-level half of the method is underspecified, and one stated proposition is false as written. These issues are fixable, but they are load-bearing for the paper's central claims.","major_comments":[{"comment":"The quantitative evaluation is circular. The text says: \"First, we generate matrices that label each cell as PA ... using the TabNet-based approach described in Section 4.1,\" and Section 4.1 is exactly the source of the PA/NA labels used to construct the evidence sets in Definition 1 and to initialize the scoring matrix S in Algorithm 2 (PA cells receive 1 - i*j/(m*n)). Counting these same PA cells in the top-left k×k block therefore measures how well Algorithm 2 reorders the table to maximize its own objective, not whether the blocks correspond to true anomalies. The figure captions in Figures 3-5 call these \"ground-truth anomalies,\" but they are model predictions. The evaluation should instead use the independent attribute-level ground-truth labels mentioned in the Datasets paragraph of Section 5, or otherwise clearly distinguish the labels used to build the method from the labels used to evaluate it.","section":"Section 5.1"},{"comment":"The row-level Shapley values used to reorder rows are never defined or derived. Section 4.4 states that \"attributes and records with lower Shapley values\" are rearranged to create the block structures, and the visual and quantitative results in Figures 1, 3-5 depend on the row ordering. However, Algorithm 1 returns only attribute-level scores, and neither the record-level cooperative game, its characteristic function, nor a closed-form expression for record Shapley values is given anywhere in the paper. Without this, the row ordering, and hence the top-k blocks, are not reproducible from the manuscript.","section":"Section 4.4"},{"comment":"Proposition 1 claims that the cooperative game (A, Va) with Va(S) = |∪_{aj∈S} E_aj| is super-additive under the definition given in Section 3. This is false for coverage functions: take two attributes with E_a = E_b = {X}; then Va({a}) = 1, Va({b}) = 1, and Va({a,b}) = 1, which violates Va({a,b}) ≥ Va({a}) + Va({b}). The closed-form Lemma 1 is nevertheless true for coverage games and does not require super-additivity, so the proposition should be corrected or removed.","section":"Proposition 1"},{"comment":"The arithmetic in Example 1 is inconsistent with the stated evidence sets. For attribute C4, the evidence set is EC4 = {R1, R2, R5, R6}; R2 appears in EC1, EC2, and EC4, so its contribution is 1/3, not 1/2. The correct value is phi(C4) = 1/4 + 1/3 + 1/4 + 1/3 = 1.1667, not 1.33. Consequently, the anomaly ordering should be C5, C2, C4, C1, C3, not C5, C2, C1, C4, C3 as stated. The other listed values should be rechecked as well.","section":"Example 1"}],"minor_comments":[{"comment":"The set-builder notation in Definitions 1 and 2 says \"for any i\" and \"for any j\" inside the defining condition, which is confused; the bound variables i and j are already fixed by the set-builder itself. These should be written, for example, as E_aj = {Xi : Lij = NA}.","section":"Definitions 1 and 2"},{"comment":"The proof of Lemma 1 has notational problems: Va(P_R^aj) is a cardinality, so expressions like Va(P_R^aj) ∪ {Xi} are not well-formed set operations, and the indicator I_{Xi not in P_R^aj} compares a record to a set of players. The argument should be rewritten in terms of whether Xi is already covered by the union of the evidence sets of the preceding attributes.","section":"Lemma 1 proof"},{"comment":"The text refers to \"ground-truth PA labels,\" while the labels are in fact produced by the TabNet model of Section 4.1; the figure captions should be aligned with the text and should not call these labels ground truth.","section":"Section 5.1 and Figures 3-5"},{"comment":"The paper repeatedly calls SHAP a \"supervised algorithm,\" which is imprecise: SHAP is a model-agnostic explanation method. If a supervised anomaly classifier is trained and then explained with SHAP, that classifier and the target labels used to train it should be specified exactly, since this affects the fairness of the comparison.","section":"Section 5.1, Baselines"},{"comment":"In the computational complexity paragraph, the text refers to \"Lemma 4.3\" where it should refer to \"Lemma 1.\"","section":"Section 4.3"},{"comment":"The description of extracting top-k maximum-sum submatrices with Kadane's algorithm is brief; the paper should state how the 2D case is handled and give the complexity, since the cited reference is for the 1D maximum subarray problem.","section":"Algorithm 2"}],"recommendation":"major_revision","confidential_remarks":"The central theoretical lemma is correct, and the problem is worth pursuing, but the paper in its current form does not support the empirical claims because of the circular evaluation and the missing record-level scoring protocol. The false super-additivity proposition and the arithmetic error in Example 1 also need correction. I would be willing to review a revised version that addresses these points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper has a clean, new problem framing — turn cell-level anomaly labels into top-k attribute/record blocks and use Shapley values to rank rows and columns. That's genuinely useful for data-quality tooling. But the evaluation is essentially circular: it counts the same TabNet-derived PA labels that are used to build the evidence sets, so the reported concentration gains over DIFFI/SHAP don't demonstrate that the blocks correspond to real anomalies.\n\nThe math is fine. Lemma 1 is a correct closed-form Shapley for a coverage game; it's a standard result, not a new analytical insight, and the paper should have cited the coverage-game Shapley literature. The O(mn) complexity claim holds. The example in Section 4.3 has an arithmetic slip — R2 appears in three evidence sets, not two, so phi(C4) ≈ 1.17, not 1.33 — but that's minor and easy to fix.\n\nThe real problem is in Section 5.1. The authors generate PA/NA matrices using the same TabNet reconstruction-error procedure from Section 4.1, then measure how many of those labels fall into the top-left blocks after reordering. Since the Shapley scores and the block extraction are built from those exact labels, the method is in large part grouping its own predictions. Figure captions call these 'ground-truth anomalies', but the text says they are the TabNet outputs. Unless there's an independent label source being used (and I don't see one in the description), the empirical claim is not established. There are also no error bars or significance tests, and the baselines get a frequency-based row ordering that may not be the fairest comparison.\n\nWhat's worth keeping: the problem setup, the algorithm design, and the idea of block-based insights. If the authors redo the evaluation using real attribute-level ground-truth labels (e.g., the ones in the datasets they say they use), the approach might survive. Right now, the paper is a good idea with a flawed validation.\n\nI'd send it to peer review — a serious editor should get a referee to look at this, because the idea has merit and the evaluation issue is fixable. But I would not cite the empirical results until they're reproduced with independent labels.\n\nFor you: take a look at the problem framing; skip the experiments unless you enjoy spotting circularity. Worth a reading-group discussion.\n\nRecommendation: engage with the paper, but treat the concentration claims as unproven.","headline":"New problem framing and a correct but standard Shapley formula, undermined by an empirical evaluation that counts the same TabNet labels used to build the evidence sets.","tokens_in":11603,"tokens_out":4705,"would_cite":false,"duration_ms":42061,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tab-Shapley claims that anomalies in tabular data can be aggregated into ranked top-k blocks through a superadditive cooperative game whose Shapley values collapse to a single closed-form sum, computable in O(mn) time.","keywords":["tabular data quality","anomaly detection","Shapley value","cooperative game theory","top-k insights","unsupervised learning","block detection","autoencoder reconstruction"],"falsifier":"Run Tab-Shapley on a synthetic table with known injected anomalies, using cell labels from an independent detector different from the one in Section 4.1, and measure whether the top-k block still captures significantly more flagged cells than random row and column permutations; if the concentration disappears, the reported insight is an artifact of the label-generation step rather than of the Shapley aggregation.","tokens_in":10593,"feed_emoji":"📊","tokens_out":6964,"duration_ms":58354,"temperature":0.7,"pith_summary":"Tab-Shapley claims that the messy problem of locating concentrated anomalies in a table can be recast as a cooperative game over attributes, where each attribute is assigned the records that do not look anomalous in it. In this game the Shapley value of an attribute has a closed form: a sum over its evidence records of the reciprocal of how many attributes share that record. Lower scores flag attributes as more anomalous, and reordering rows and columns by these scores brings flagged cells into compact top-left blocks. Extracting the top-k such blocks yields prioritized data quality insights in time linear in the table size. A sympathetic reader would care because the same machinery could turn any binary cell-quality matrix into an actionable ranked list of where to look first.","feed_headline":"Closed-form Shapley scores expose top-k anomaly blocks","feed_subtitle":"Ranking attributes by a single efficient formula turns scattered flagged cells into prioritized data-quality clues.","key_machinery":"The load-bearing object is the evidence-set cooperative game $(A, V_a)$: players are the attributes, and $V_a(S) = |\\cup_{a_j \\in S} E_{a_j}|$, the count of distinct records covered by the evidence sets in the coalition. The closed-form identity of Lemma 1, $\\phi_a(a_j)=\\sum_{X_i\\in E_{a_j}} \\frac{1}{|\\{k : X_i \\in E_{a_k}\\}|}$, converts every record into a fractional non-anomaly vote split equally among the attributes that consider it healthy. This identity carries the whole argument: it turns an exponential coalition enumeration into a single O(mn) pass, and its inversion produces the anomaly rankings that feed the block-extraction step.","core_discovery":"The paper's central claim is that anomalies in a tabular dataset can be aggregated into ranked, human-consumable blocks by defining a cooperative game on the attributes and reading off closed-form Shapley values. In this game an attribute's evidence set is the records where that attribute was not flagged as a potential anomaly, and a coalition's payoff is the number of distinct records covered by the evidence sets of its members. Because this game is superadditive, the Shapley value of an attribute reduces to summing, over its own evidence records, the reciprocal of the number of attributes that share each record; inverting these scores ranks attributes and reorders rows and columns so that flagged cells accumulate in the top-left. The paper claims this closed form makes top-k block extraction correct and O(mn), and that on real-world labeled datasets the resulting blocks hold more anomalous cells than an unsupervised isolation-forest ranking.","pith_inferences":["The closed form is not specific to reconstruction errors: any binary matrix of flagged cells yields the same game, so the aggregation is a general block-localization layer that could sit on top of any detector.","The maximum-subarray extraction is one possible block aggregator; replacing it with connected-component or density-based block summaries would test whether the Shapley ranking or the block post-processing drives the reported concentration.","A direct human study of whether the returned insights match the actual source of data-quality failures would settle practical value better than counting flagged cells."],"forward_implications":["Ranking attributes by the inverted closed-form Shapley values requires only O(mn) time, so the method scales to wide and long tables.","The top-k blocks concentrate flagged cells better than the isolation-forest baseline on the evaluated datasets, and match a supervised feature-attribution baseline on many of them.","Because the game is superadditive and the closed form is exact, no coalition enumeration is needed for deployment.","Attribute scores correlate with both the size of evidence sets and the number of distinct records in them, supporting the two stated criteria."],"supporting_citations":[{"why":"Supplies the autoencoder whose per-cell reconstruction errors are thresholded into the PA/NA labels that define every evidence set.","marker":"(Arık and Pfister 2021)"},{"why":"Defines the Shapley value and its four axioms, the payoff concept the closed form evaluates.","marker":"(Shapley 1971)"},{"why":"Provides the cooperative-game and Shapley-value definitions the paper formalizes in its preliminaries.","marker":"(Myerson 1997)"},{"why":"Provides the DIFFI unsupervised feature-importance baseline that Tab-Shapley is compared against.","marker":"(Carletti et al. 2019)"},{"why":"Provides the SHAP supervised attribution baseline used as a second comparison point.","marker":"(Lundberg and Lee 2017)"},{"why":"Supplies the twelve real-world datasets with ground-truth record- and attribute-level anomaly labels used in evaluation.","marker":"(Xu et al. 2021)"},{"why":"Supplies the maximum-subarray routine used to extract the top-k disjoint insight blocks.","marker":"(Tamaki and Tokuyama 1998)"}],"fun_headline_variants":["Tab-Shapley: closed-form Shapley values rank data-quality blocks","Closed-form Shapley ranks top-k anomaly blocks in tables","Closed-form Shapley values pick top-k table anomalies","Tab-Shapley: one formula ranks data-quality blocks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The thresholded reconstruction-error labels that mark each cell PA or NA must actually track true anomalies, because every Shapley score and every top-k block on top of them inherits the accuracy of those labels.","fun_headline_variants_meta":{"raw":{"variants":["Tab-Shapley: closed-form Shapley values rank data-quality blocks","Closed-form Shapley ranks top-k anomaly blocks in tables","Closed-form Shapley values pick top-k table anomalies","Tab-Shapley: one formula ranks data-quality blocks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000675,"raw_usage":{"total_tokens":3042,"prompt_tokens":887,"completion_tokens":2155,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":2083}},"tokens_in":503,"tokens_out":2155,"duration_ms":14047,"temperature":1.0,"reasoning_tokens":2083,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:55:21.987698+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Tab-Shapley on a synthetic table with known injected anomalies, using cell labels from an independent detector different from the one in Section 4.1, and measure whether the top-k block still captures significantly more flagged cells than random row and column permutations; if the concentration disappears, the reported insight is an artifact of the label-generation step rather than of the Shapley aggregation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Shapley value and its four axioms, the payoff concept the closed form evaluates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the cooperative-game and Shapley-value definitions the paper formalizes in its preliminaries."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the DIFFI unsupervised feature-importance baseline that Tab-Shapley is compared against."},{"cited_title":"M.; and Lee, S.-I","cited_arxiv_id":null,"evidence_quote":"Provides the SHAP supervised attribution baseline used as a second comparison point."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the maximum-subarray routine used to extract the top-k disjoint insight blocks."}],"review_version":1}