{"id":"ba79cb5e-ce77-41ea-a634-8a1e0b79c2b7","arxiv_id":"2412.16684","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A multi-view graph-based two-sample test aggregates moment-specific Manhattan distances and has a chi-square null limit that enables fast p-values.","lead":"MATES is a new statistical test that combines several distance measures, each capturing a different moment of the data, to decide whether two samples come from the same distribution. It aims to catch differences in skewness, tail behavior, and other high-order features that classical tests often miss.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The default fourth-moment view is not well-defined for t_4-type heavy tails and can violate Theorem 4's no-hub graph conditions, contradicting the claimed heavy-tail robustness and threatening the central type-I error guarantee.","rationale":"The paper's central assertion is the distribution-free χ² limiting null (Theorem 4), which is what justifies analytic p-values. That assertion is conditional on graph conditions (1)–(5), which are not verified for the exact configuration used in the simulations and real data. The most concrete way the conditions can fail is a heavy-tailed null where the moment transforms used to define views have infinite population moments. The paper's own Section 2 claims robustness to heavy-tailed distributions, but the default s=4 view is built on fourth powers, which for t_4 have infinite expectation. In finite samples, one or a few extreme observations dominate the dissimilarity matrix, creating hub nodes in the k-NNG; Lemma 1's condition (1') is exactly a no-dominating-vertex condition, and condition (2') limits squares in the aggregated graph. If these fail, the chi-square p-value is not justified. The null simulations only include t_15, so this regime is untested. This is not a dispute with the asymptotics in the theorem; it is a scope mismatch between the advertised robustness and the default procedure. A single null simulation under t_4 would settle whether the practical type-I guarantee survives. If the size is controlled, the concern is resolved and the paper only needs a clarifying moment assumption; if not, the central claim needs qualification or the default views need robustification (e.g., rank or centering). Because the reader's verdict is already CONDITIONAL, our critique does not move it.","tokens_in":21883,"tokens_out":15712,"duration_ms":149321,"concrete_test":"Simulate the null X_i, Y_j iid t_4 in d=200 with m=n=50, using the paper's default S=4 Manhattan-power views, k=⌊N^0.8⌋, median-heuristic exponential kernel, and the Theorem 4 asymptotic χ²_8 p-value; run at least 1000 replications. If the empirical size is within the 95% binomial margin for α=0.05 (roughly 3.6%–6.4%), the moment-existence concern is benign for type-I control; if it deviates materially (e.g., >8% or <2%), the central claim fails for the advertised heavy-tailed regime. For a stronger check, repeat with t_3, where the second-moment view is also infinite.","verdict_should_be":"UNCHANGED","load_bearing_attack":"MATES's central claim is that TS is asymptotically χ²_{2S} under H0 (Theorem 4), giving straightforward type-I control. The default configuration (Section 3) uses S=4 views with Manhattan distances on elementwise powers, D^(s)_ij = Σ_r |Z^s_ir − Z^s_jr|, and a k-NNG with k=⌊N^0.8⌋. For a heavy-tailed null such as t_4, E|Z|^4=∞, so the 'fourth-moment' dissimilarity has no finite population analogue. Section 2 explicitly claims graph-based methods are robust to heavy-tailed distributions, but the chosen view concentrates on the largest observed fourth powers, creating hubs in the k-NNG. This can violate Lemma 1(1') (no dominating vertex: max_i |W̃_i|/√W_3 = O(N^{−γ}), γ > β/2) and the square-count condition (2'), so Theorem 4's conditions need not hold. The paper's null simulations only use t_15 (finite fourth moment), never t_4, so the claimed robustness and the 'distribution-free limiting distribution' are not demonstrated in the regime the Introduction advertises. The concern is not that the theorem is false as stated, but that the default view configuration can leave its scope.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MATES, a graph-based two-sample test that aggregates multiple dissimilarity views, each constructed from a moment of the data. The test statistic is a Mahalanobis-type combination of within-sample edge-count deviations across views, and the paper provides closed-form permutation-null moments and covariances (Theorem 1), a necessary and sufficient condition for the statistic to be well-defined (Theorem 3), and a claimed asymptotic chi-square null distribution with 2S degrees of freedom (Theorem 4). The method is evaluated in extensive simulations against 12 existing tests and in an S&P100 data application. The abstract and introduction advertise the distribution-free limiting distribution as enabling direct type-I error control, and the empirical results show high power in higher-moment alternatives.","tokens_in":22122,"tokens_out":5372,"duration_ms":43587,"significance":"If the asymptotic result is correct, MATES is a valuable addition to the nonparametric two-sample testing toolbox, providing a flexible multi-view framework with analytic p-values and strong empirical power for higher-order moment differences. The paper's closed-form moment formulas and careful simulation comparisons are strengths, as is the inclusion of a real data example. However, the central theoretical guarantee is not fully verifiable from the manuscript alone, and the default moment-based views conflict with the claimed robustness to heavy-tailed distributions.","major_comments":[{"comment":"The proof of Theorem 4 is deferred to Supplementary S.2.4, which was not available for review. Because the distribution-free chi-square limit is the basis for the paper's headline claim of straightforward type-I error control, this unverifiable proof is a load-bearing gap. The authors should either provide the supplement with the full proof or justify the conditions directly in the main text.","section":"Section 4, Theorem 4"},{"comment":"The default configuration uses S=4 views with D^(s)_ij = Σ_r |Z_ir^s − Z_jr^s| and k=⌊N^0.8⌋. For a heavy-tailed null such as t_4, the fourth-moment view has infinite expected dissimilarity and the k-NNG will be dominated by extreme observations. This contradicts the claim in Section 2 that graph-based methods offer robustness to heavy-tailed distributions. The null simulations in Table 3 only include t_15 (setting (d)), which has a finite fourth moment, so the heavy-tail robustness claim is not demonstrated. The authors should either restrict the robustness claim to distributions with finite moments of order 2S, or modify the default view construction to be robust to infinite moments.","section":"Section 3, default configuration"},{"comment":"The sufficient conditions (1') and (2') for Theorem 4 are not verified for the specific k-NNG with k=floor(N^0.8) and exponential kernel weights used in the simulations. Condition (1') requires no dominating vertex, and condition (2') restricts the number of squares in the aggregated graph, but the paper does not show that the default graph satisfies these conditions. Without such verification, the empirical type-I error rates in Table 3 do not establish the validity of the asymptotic approximation for the recommended configuration.","section":"Section 4, Lemma 1"},{"comment":"In the S&P100 application, the single-view p-values in Figure 1 are obtained from the asymptotic chi-square_2 approximation, but the sample sizes are only m=34 and n=32, and no evidence is given that the conditions of Theorem 4 hold for this data set. A permutation-based calibration for the single-view or aggregated statistic would strengthen the real-data conclusion.","section":"Section 5"}],"minor_comments":[{"comment":"The notation in D^(s)_ij = Σ_r |Z^s_ir − Z^s_jr| is ambiguous: it is unclear whether Z^s_ir means the s-th power of the r-th coordinate or the r-th coordinate of the s-th view; please clarify.","section":"Section 2, Similarity Measures"},{"comment":"The simulation settings table lists 't-distributions' for null setting (d) but does not specify the degrees of freedom in the table; the text later gives t_15. Please unify this information.","section":"Section 3, Table 2"},{"comment":"The phrase 'the S matrices cW^{(1)}, ..., cW^{(S)} are linearly independent' would be more precise if the ambient space (symmetric N×N matrices with zero diagonal) were specified.","section":"Section 4, Theorem 3"},{"comment":"The paper does not discuss the computational complexity of the proposed method; adding a brief statement would help practitioners assess its scalability.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript would be much easier to evaluate if the supplementary material, especially the proof of Theorem 4, were included in the submission. The heavy-tail concern is significant but addressable by either adjusting the default view construction or explicitly qualifying the robustness claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: MATES is a genuinely useful addition to the graph-based two-sample toolbox. It builds a Mahalanobis statistic from several weighted similarity graphs, each targeting a different moment, and proves a chi-square limit under the permutation null. The simulations are extensive and the power gains on higher-moment alternatives are real—0.91 where twelve competitors sit at nominal level. The proof is deferred to a supplement, and the regularity conditions are not checked for the exact default configuration. The biggest substantive gap is the heavy-tail claim: the default fourth-moment view uses Manhattan distances on fourth powers, which is undefined in population for t_4-type tails and will create hubs that plausibly violate the no-dominating-vertex condition (Lemma 1(1')). The paper doesn't run null simulations below t_15, so the 'robust to heavy tails' statement in Section 2 isn't backed by evidence for the default views.\n\nWhat's actually new: the specific aggregation of multiple moment-specific graphs into one chi-square test is a clean extension of Chen-Friedman and Zhou-Chen, and the closed-form permutation moments (Theorem 1) plus the w/diff orthogonal decomposition (Theorem 2) are well done. The linear-independence characterization of positive-definiteness (Theorem 3) is a nice touch—it tells the user when a view is redundant.\n\nThe soft spots, in proportion: (1) The heavy-tail issue above is real but fixable—use bounded or rank-transformed dissimilarities for higher moments, or state the finite-moment condition clearly. It's a caveat, not a fatal flaw. (2) The core proof lives in an unavailable supplement. That makes the chi-square claim unverifiable from the arXiv version; it needs a referee with the appendix in hand. (3) The power tables have no standard errors and no code, so independent replication is effort. (4) The S&P100 narrative is suggestive, not conclusive.\n\nWho's it for: anyone building or using nonparametric high-dimensional two-sample tests, especially when moment differences matter. It deserves a serious referee. The method is worth engaging with, and the heavy-tail gap is fixable in revision.\n\nRecommendation: send to peer review, with the supplement required. If the proof checks out and the moment-existence caveat is addressed, this is a solid paper.","headline":"A useful multi-view graph-based two-sample test with a clean chi-square null, but the heavy-tail robustness claim is not backed by the default moment-power views and the core proof sits in an unavailable supplement.","tokens_in":22669,"tokens_out":3613,"would_cite":true,"duration_ms":33148,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G10","62G20","62H15"],"pacs":[],"model":"deepseek-v4-flash","headline":"MATES aggregates multiple moment-specific similarity graphs into a single chi-square statistic that detects higher-order distributional differences where twelve existing tests remain at the nominal level.","keywords":["two-sample test","graph-based method","higher-order moments","multi-view aggregation","distribution-free test","chi-square limiting distribution","high-dimensional data"],"falsifier":"The decisive check is to simulate from a null distribution with infinite fourth moment, such as $t_3$, and attempt the four-view MATES: each pairwise fourth-moment dissimilarity is infinite, so the statistic is undefined. A separate check for the distribution-free claim is a size simulation under the Theorem 4 conditions, for example i.i.d. normal data with $k=\\lfloor N^{0.8}\\rfloor$; the empirical rejection rate must track the nominal level as $N$ grows.","tokens_in":21656,"feed_emoji":"📊","tokens_out":8727,"duration_ms":74080,"temperature":0.7,"pith_summary":"The paper proposes MATES, a two-sample test that builds one weighted similarity graph for each \"view\" of the data—commonly the first, second, third, and fourth moments—and combines the within-sample edge-weight sums from all views into a single Mahalanobis statistic. The aim is to detect distributional differences that leave means and covariances unchanged, such as differences in skewness or kurtosis, which the paper argues are invisible to most nonparametric tests in high dimensions. The central theoretical claim is that this statistic has a distribution-free limiting $\\chi^2_{2S}$ distribution under the null, so a practitioner can control type-I error with an analytic p-value rather than permutation. In the paper's motivating example, MATES reaches power 0.91 where twelve established tests remain near the 0.05 nominal level, and on S&P100 returns around the ChatGPT release it is the only method considered that rejects at level 0.01.","feed_headline":"MATES catches higher-moment differences twelve tests miss","feed_subtitle":"By pooling moment-specific graphs into one statistic, it detects skewness and kurtosis shifts with an analytic p-value.","key_machinery":"The central object is the multi-view weighted similarity graph $W^{(s)}$, a matrix of edge weights on the pooled sample's k-nearest-neighbor or minimum-spanning-tree graph built from the sth dissimilarity measure, such as $D^{(s)}_{ij}=\\sum_r |Z_{ir}^s - Z_{jr}^s|$. The machinery is the permutation-null moment calculus of Theorem 1, which gives closed-form means and covariances of within-sample weight sums in terms of aggregate weight sums; Theorem 2's decomposition into \"within\" and \"difference\" components, which isolates conditions for invertibility; and the Lindeberg-type hub conditions of Theorem 4 that yield the $\\chi^2_{2S}$ limit.","core_discovery":"The paper introduces MATES, a two-sample test built from S weighted similarity graphs on the pooled sample, one graph per \"view\" of the data. For each view s, the statistic collects $U_x^{(s)}$ and $U_y^{(s)}$, the sums of edge weights among observations within the first and second samples, centered by their permutation expectations. The aggregate statistic $T_S = v_S^\\top \\Sigma_S^{-1} v_S$ is a Mahalanobis combination of all views. The central discovery is that, under the permutation null, this statistic has a closed-form covariance (Theorem 1), splits into orthogonal within-sample and between-sample components (Theorem 2), is well defined exactly when the views' graph structures are linearly independent (Theorem 3), and converges to a $\\chi^2_{2S}$ distribution (Theorem 4), giving a distribution-free analytic p-value.","pith_inferences":["Going beyond the paper: the moment-based views require finite sth moments, so for heavy-tailed data such as $t$ distributions with fewer than four degrees of freedom, the fourth-moment graph cannot be built; the advertised ability to handle heavy-tailed data therefore holds only when the chosen views are finite.","Going beyond the paper: the same aggregation logic could be applied to change-point detection, classification, and clustering, where a shift or separation may appear only in variance or tail behavior.","Going beyond the paper: a practical pre-check for the chi-square limit is to inspect per-vertex degrees in each view and drop views with hubs; failure to control size in hub-heavy graphs would point directly at Condition (3) of Theorem 4."],"forward_implications":["A p-value can be read from a $\\chi^2_{2S}$ table, bypassing permutation or bootstrap, as long as the graph conditions hold.","MATES detects alternatives where two distributions share mean and covariance but differ in skewness or kurtosis; in the motivating example its power is 0.91 while twelve existing tests sit near 0.05.","Views are not limited to Euclidean moments: different Wasserstein orders for distributional data, cosine distance for images or networks, and rank or kernel weights can be mixed into the same statistic.","Views that are linearly redundant make the covariance matrix singular, so the framework supplies a check: drop redundant views before computing the test.","On S&P100 returns around the ChatGPT release, MATES rejects at level 0.01 while no comparator does, with the fourth-moment view alone giving a p-value of 0.004."],"supporting_citations":[{"why":"Supplies the graph-based two-sample testing paradigm and edge-count asymptotics that MATES generalizes to multiple views.","marker":"Chen and Friedman (2017)"},{"why":"Introduces the RISE ranking scheme whose single-view chi-square conditions MATES extends to S views.","marker":"Zhou and Chen (2023)"},{"why":"Establishes limiting distributions for graph-based statistics whose hub conditions appear as Conditions (1)-(3) of Theorem 4.","marker":"Zhu and Chen (2024)"},{"why":"Defines the generalized energy distance test, the strongest comparator in the motivating example and the main distance-based baseline MATES improves on.","marker":"Chakraborty and Zhang (2021)"},{"why":"Proposes mMMD multi-kernel aggregation; MATES contrasts its non-distribution-free null limit with its own chi-square limit.","marker":"Chatterjee and Bhattacharya (2024)"},{"why":"Provides the MMD test and the median bandwidth heuristic used in MATES simulations.","marker":"Gretton et al. (2012a)"},{"why":"Contains the orthogonal within/difference decomposition that Theorem 2 generalizes.","marker":"Chu and Chen (2019)"},{"why":"Gives generalized kernel two-sample tests with similar decomposition and limiting-distribution arguments.","marker":"Song and Chen (2023)"}],"fun_headline_variants":["MATES: one test for all moment differences","Graph-based test catches skewness and kurtosis","MATES merges views into a single two-sample test","Closed-form p-value for multi-view two-sample test","MATES sees beyond means and variances"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Each moment view is computed from absolute differences of the data raised to that power, so the method assumes the underlying distributions have finite moments up to the highest view used; with heavy-tailed data such as $t$ distributions on fewer than four degrees of freedom, the fourth-moment view is undefined.","fun_headline_variants_meta":{"raw":{"variants":["MATES: one test for all moment differences","Graph-based test catches skewness and kurtosis","MATES merges views into a single two-sample test","Closed-form p-value for multi-view two-sample test","MATES sees beyond means and variances"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000153,"raw_usage":{"total_tokens":1209,"prompt_tokens":948,"completion_tokens":261,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":188}},"tokens_in":564,"tokens_out":261,"duration_ms":2821,"temperature":1.0,"reasoning_tokens":188,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:20:54.953240+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The decisive check is to simulate from a null distribution with infinite fourth moment, such as $t_3$, and attempt the four-view MATES: each pairwise fourth-moment dissimilarity is infinite, so the statistic is undefined. A separate check for the distribution-free claim is a size simulation under the Theorem 4 conditions, for example i.i.d. normal data with $k=\\lfloor N^{0.8}\\rfloor$; the empirical rejection rate must track the nominal level as $N$ grows.","supporting_citations":[{"cited_title":"and Chen, H","cited_arxiv_id":null,"evidence_quote":"Introduces the RISE ranking scheme whose single-view chi-square conditions MATES extends to S views."},{"cited_title":"and Chen, H","cited_arxiv_id":null,"evidence_quote":"Establishes limiting distributions for graph-based statistics whose hub conditions appear as Conditions (1)-(3) of Theorem 4."},{"cited_title":"and Zhang, X","cited_arxiv_id":null,"evidence_quote":"Defines the generalized energy distance test, the strongest comparator in the motivating example and the main distance-based baseline MATES improves on."},{"cited_title":"and Bhattacharya, B","cited_arxiv_id":null,"evidence_quote":"Proposes mMMD multi-kernel aggregation; MATES contrasts its non-distribution-free null limit with its own chi-square limit."},{"cited_title":"and Chen, H","cited_arxiv_id":null,"evidence_quote":"Contains the orthogonal within/difference decomposition that Theorem 2 generalizes."},{"cited_title":"and Chen, H","cited_arxiv_id":null,"evidence_quote":"Gives generalized kernel two-sample tests with similar decomposition and limiting-distribution arguments."}],"review_version":1}